2024-02-16T19:03:38.865 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-16T19:03:38.865 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 0a 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687 -- /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687/orig.config.yaml 2024-02-16T19:03:38.921 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T19:03:39.008 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687 branch: wip-batrick-testing-20240215.160715 description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 0a 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7562687' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240215.160715 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f78a58c0ffd401d1493058a1022c35f011d65275 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f78a58c0ffd401d1493058a1022c35f011d65275 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: i64440 sha1: ed7ab725e1a43615009dfb74a42bf33d80081bc8 owner: scheduled_pdonnell@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2634 sha1: f78a58c0ffd401d1493058a1022c35f011d65275 sleep_before_teardown: 0 suite: fs:upgrade:mds_upgrade_sequence suite_branch: i64440 suite_path: /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: ed7ab725e1a43615009dfb74a42bf33d80081bc8 targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP5WBLJfQrY+S5hsuuLtCOUUFMdivkRktN3VAK5Zq0pceOWiEUs9klmdsq2JNtA2OBvjWYoz3hgO1eY+3UeTtNg= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBJoyXrV8epnYq1Lr3iMt/rZ9CyK+g/tfMX75McffgPPBagzOh2+ddwpgeioxNducTFjpJa/iW0MTZujfoViF5o= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - full_sequential_finally: - duration: 3600 sleep: null - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/pdonnell/src/github.com_batrick_ceph_i64440/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-16_18:26:20 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-16T19:03:39.009 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa; will attempt to use it 2024-02-16T19:03:39.009 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks 2024-02-16T19:03:39.010 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-16T19:03:39.011 INFO:teuthology.task.internal:Checking packages... 2024-02-16T19:03:39.038 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f78a58c0ffd401d1493058a1022c35f011d65275' 2024-02-16T19:03:39.038 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T19:03:39.038 INFO:teuthology.packaging:ref: None 2024-02-16T19:03:39.038 INFO:teuthology.packaging:tag: None 2024-02-16T19:03:39.038 INFO:teuthology.packaging:branch: wip-batrick-testing-20240215.160715 2024-02-16T19:03:39.038 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:03:39.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240215.160715 2024-02-16T19:03:39.185 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1450.gf78a58c0 2024-02-16T19:03:39.186 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-16T19:03:39.193 INFO:teuthology.task.internal:no buildpackages task found 2024-02-16T19:03:39.193 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-16T19:03:39.230 INFO:teuthology.task.internal:Saving configuration 2024-02-16T19:03:39.250 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-16T19:03:39.256 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T19:03:39.277 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 18:57:24.683060', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP5WBLJfQrY+S5hsuuLtCOUUFMdivkRktN3VAK5Zq0pceOWiEUs9klmdsq2JNtA2OBvjWYoz3hgO1eY+3UeTtNg='} 2024-02-16T19:03:39.292 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 18:57:24.685004', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBJoyXrV8epnYq1Lr3iMt/rZ9CyK+g/tfMX75McffgPPBagzOh2+ddwpgeioxNducTFjpJa/iW0MTZujfoViF5o='} 2024-02-16T19:03:39.292 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-16T19:03:39.298 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-16T19:03:39.298 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-16T19:03:39.298 INFO:teuthology.run_tasks:Running task console_log... 2024-02-16T19:03:39.382 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f614fcd50d0>, signals=[15]) 2024-02-16T19:03:39.382 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-16T19:03:39.389 INFO:teuthology.task.internal:Opening connections... 2024-02-16T19:03:39.389 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:03:39.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:03:39.542 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-02-16T19:03:39.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:03:39.634 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-16T19:03:39.641 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-02-16T19:03:39.680 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-02-16T19:03:39.681 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-02-16T19:03:39.736 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T19:03:39.737 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-02-16T19:03:39.737 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T19:03:39.737 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T19:03:39.737 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T19:03:39.737 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-02-16T19:03:39.768 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-02-16T19:03:39.784 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-02-16T19:03:39.784 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon" 2024-02-16T19:03:39.839 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T19:03:39.840 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2024-02-16T19:03:39.840 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T19:03:39.840 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T19:03:39.840 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T19:03:39.840 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-02-16T19:03:39.860 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-16T19:03:39.869 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-16T19:03:39.888 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-16T19:03:39.889 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-02-16T19:03:39.891 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-02-16T19:03:39.905 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-16T19:03:39.914 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-16T19:03:39.914 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-02-16T19:03:39.947 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-02-16T19:03:39.972 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-16T19:03:40.022 INFO:teuthology.run_tasks:Running task kernel... 2024-02-16T19:03:40.037 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-16T19:03:40.037 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-16T19:03:40.038 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-16T19:03:40.038 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-16T19:03:40.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:03:40.052 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-02-16T19:03:40.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:03:40.054 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-02-16T19:03:40.107 INFO:teuthology.orchestra.run.smithi163.stdout:5.14.0-361.el9.x86_64 2024-02-16T19:03:40.108 INFO:teuthology.task.kernel:Running kernel on smithi163: 5.14.0-361.el9.x86_64 2024-02-16T19:03:40.108 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum install -y kernel 2024-02-16T19:03:40.109 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-361.el9.x86_64 2024-02-16T19:03:40.109 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-361.el9.x86_64 2024-02-16T19:03:40.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-02-16T19:03:42.637 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - AppStream 7.4 MB/s | 6.7 MB 00:00 2024-02-16T19:03:42.825 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - AppStream 6.3 MB/s | 6.7 MB 00:01 2024-02-16T19:03:44.154 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2024-02-16T19:03:44.433 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-02-16T19:03:45.420 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-02-16T19:03:46.099 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-02-16T19:03:47.246 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-02-16T19:03:48.025 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-02-16T19:03:52.780 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.1 MB 00:01 2024-02-16T19:03:53.533 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.1 MB 00:01 2024-02-16T19:03:54.297 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 15 kB 00:00 2024-02-16T19:03:55.275 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 15 kB 00:00 2024-02-16T19:03:55.407 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2024-02-16T19:03:56.093 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 30 MB/s | 20 MB 00:00 2024-02-16T19:04:00.440 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-02-16T19:04:01.060 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-16T19:04:02.856 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-16T19:04:02.856 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-16T19:04:02.951 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:04:02.958 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:04:02.958 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:04:02.958 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:04:02.958 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:04:02.958 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-16T19:04:02.958 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:04:02.959 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-16T19:04:02.959 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-16T19:04:02.959 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-16T19:04:02.959 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:04:02.959 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:04:02.959 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:04:02.959 INFO:teuthology.orchestra.run.smithi040.stdout:Install 4 Packages 2024-02-16T19:04:02.959 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:04:02.960 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 100 M 2024-02-16T19:04:02.960 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 123 M 2024-02-16T19:04:02.960 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:04:03.350 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-16T19:04:03.350 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-16T19:04:03.358 INFO:teuthology.orchestra.run.smithi040.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 15 MB/s | 6.1 MB 00:00 2024-02-16T19:04:03.460 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-16T19:04:03.470 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:04:03.471 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:04:03.471 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:04:03.471 INFO:teuthology.orchestra.run.smithi163.stdout:Install 4 Packages 2024-02-16T19:04:03.471 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:04:03.471 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 100 M 2024-02-16T19:04:03.471 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 123 M 2024-02-16T19:04:03.472 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:04:04.000 INFO:teuthology.orchestra.run.smithi040.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 20 MB/s | 21 MB 00:01 2024-02-16T19:04:04.128 INFO:teuthology.orchestra.run.smithi163.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 9.2 MB/s | 6.1 MB 00:00 2024-02-16T19:04:05.103 INFO:teuthology.orchestra.run.smithi163.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 13 MB/s | 21 MB 00:01 2024-02-16T19:04:05.175 INFO:teuthology.orchestra.run.smithi040.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 18 MB/s | 33 MB 00:01 2024-02-16T19:04:05.525 INFO:teuthology.orchestra.run.smithi040.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 15 MB/s | 39 MB 00:02 2024-02-16T19:04:05.529 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:04:05.529 INFO:teuthology.orchestra.run.smithi040.stdout:Total 39 MB/s | 100 MB 00:02 2024-02-16T19:04:05.529 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:04:05.686 INFO:teuthology.orchestra.run.smithi163.stdout:(3/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 18 MB/s | 39 MB 00:02 2024-02-16T19:04:05.990 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:04:05.991 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:04:06.037 INFO:teuthology.orchestra.run.smithi163.stdout:(4/4): kernel-modules-core-5.14.0-421.el9.x86_6 18 MB/s | 33 MB 00:01 2024-02-16T19:04:06.040 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:04:06.040 INFO:teuthology.orchestra.run.smithi163.stdout:Total 39 MB/s | 100 MB 00:02 2024-02-16T19:04:06.040 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:04:06.434 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:04:06.434 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:04:07.030 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:04:07.031 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:04:07.477 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:04:07.477 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:04:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:04:09.892 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:04:10.250 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-16T19:04:10.401 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:04:10.553 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-16T19:04:10.687 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:04:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:04:13.468 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:04:13.520 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:04:13.648 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:04:18.367 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:04:18.569 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:18.583 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:04:18.692 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:23.310 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:52.510 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:52.510 INFO:teuthology.orchestra.run.smithi163.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-16T19:04:52.510 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:04:52.577 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:53.061 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:53.061 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-16T19:04:53.061 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:04:53.061 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:04:53.472 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:53.472 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:04:53.472 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:04:53.473 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-16T19:04:53.473 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-16T19:04:53.473 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-16T19:04:53.474 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-16T19:04:53.474 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:04:53.474 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:04:53.966 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T19:04:54.140 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T19:04:54.141 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T19:04:54.141 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-16T19:04:54.141 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-16T19:04:54.141 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-16T19:04:54.141 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-16T19:04:54.141 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi163.front.sepia.ceph.com, path=None, version=distro) 2024-02-16T19:04:54.141 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T19:04:54.249 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T19:04:54.249 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T19:04:54.249 DEBUG:teuthology.orchestra.run.smithi163:> sudo rpm -qi grub2-tools 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Name : grub2-tools 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Epoch : 1 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Version : 2.06 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Release : 68.el9 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: x86_64 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Group : Unspecified 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Size : 8264254 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:License : GPLv3+ 2024-02-16T19:04:54.290 INFO:teuthology.orchestra.run.smithi163.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:Packager : builder@centos.org 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:Vendor : CentOS 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:Summary : Support tools for GRUB. 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:Description : 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:hardware devices. 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:04:54.291 INFO:teuthology.orchestra.run.smithi163.stdout:This subpackage provides tools for support of all platforms. 2024-02-16T19:04:54.292 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-16T19:04:54.292 INFO:teuthology.task.kernel:Updating grub on smithi163 to boot 5.14.0-421.el9.x86_64 2024-02-16T19:04:54.293 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-16T19:04:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:54.298 INFO:teuthology.orchestra.run.smithi040.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-16T19:04:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:04:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:54.755 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:54.755 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-16T19:04:54.755 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-16T19:04:54.755 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-16T19:04:54.905 INFO:teuthology.orchestra.run.smithi163.stderr:Generating grub configuration file ... 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:04:55.192 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:04:55.721 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T19:04:55.858 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T19:04:55.858 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T19:04:55.858 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-16T19:04:55.858 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-16T19:04:55.858 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-16T19:04:55.859 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-16T19:04:55.859 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2024-02-16T19:04:55.859 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-02-16T19:04:55.971 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-16T19:04:55.971 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-16T19:04:55.971 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -qi grub2-tools 2024-02-16T19:04:56.018 INFO:teuthology.orchestra.run.smithi040.stdout:Name : grub2-tools 2024-02-16T19:04:56.018 INFO:teuthology.orchestra.run.smithi040.stdout:Epoch : 1 2024-02-16T19:04:56.018 INFO:teuthology.orchestra.run.smithi040.stdout:Version : 2.06 2024-02-16T19:04:56.018 INFO:teuthology.orchestra.run.smithi040.stdout:Release : 68.el9 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: x86_64 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Group : Unspecified 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Size : 8264254 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:License : GPLv3+ 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Packager : builder@centos.org 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Vendor : CentOS 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Summary : Support tools for GRUB. 2024-02-16T19:04:56.019 INFO:teuthology.orchestra.run.smithi040.stdout:Description : 2024-02-16T19:04:56.020 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:04:56.020 INFO:teuthology.orchestra.run.smithi040.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-16T19:04:56.020 INFO:teuthology.orchestra.run.smithi040.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-16T19:04:56.020 INFO:teuthology.orchestra.run.smithi040.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-16T19:04:56.020 INFO:teuthology.orchestra.run.smithi040.stdout:hardware devices. 2024-02-16T19:04:56.020 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:04:56.020 INFO:teuthology.orchestra.run.smithi040.stdout:This subpackage provides tools for support of all platforms. 2024-02-16T19:04:56.021 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-16T19:04:56.021 INFO:teuthology.task.kernel:Updating grub on smithi040 to boot 5.14.0-421.el9.x86_64 2024-02-16T19:04:56.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-16T19:04:56.299 INFO:teuthology.orchestra.run.smithi163.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-16T19:04:56.325 INFO:teuthology.orchestra.run.smithi163.stderr:done 2024-02-16T19:04:56.327 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-02-16T19:04:56.342 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.V7IGqwjA0q 2024-02-16T19:04:56.343 DEBUG:teuthology.orchestra.run.smithi163:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.V7IGqwjA0q 2024-02-16T19:04:56.420 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.V7IGqwjA0q 2024-02-16T19:04:56.564 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.V7IGqwjA0q is 6KB 2024-02-16T19:04:56.611 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.V7IGqwjA0q 2024-02-16T19:04:56.625 DEBUG:teuthology.orchestra.run.smithi163:> sudo /bin/ls /boot/loader/entries || true 2024-02-16T19:04:56.689 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-16T19:04:56.689 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-16T19:04:56.690 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-16T19:04:56.690 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-16T19:04:56.691 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-16T19:04:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2024-02-16T19:04:56.827 DEBUG:teuthology.orchestra.run.smithi163:> sudo shutdown -r now 2024-02-16T19:04:58.197 INFO:teuthology.orchestra.run.smithi040.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-16T19:04:58.233 INFO:teuthology.orchestra.run.smithi040.stderr:done 2024-02-16T19:04:58.235 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-02-16T19:04:58.250 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.5d1OcUwIHT 2024-02-16T19:04:58.250 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5d1OcUwIHT 2024-02-16T19:04:58.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.5d1OcUwIHT 2024-02-16T19:04:58.458 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.5d1OcUwIHT is 6KB 2024-02-16T19:04:58.507 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.5d1OcUwIHT 2024-02-16T19:04:58.522 DEBUG:teuthology.orchestra.run.smithi040:> sudo /bin/ls /boot/loader/entries || true 2024-02-16T19:04:58.590 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-16T19:04:58.590 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-16T19:04:58.590 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-16T19:04:58.590 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-16T19:04:58.592 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-16T19:04:58.779 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2024-02-16T19:05:26.858 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-16T19:05:26.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-02-16T19:05:26.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:05:28.783 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-16T19:05:28.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-02-16T19:05:28.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:05:45.276 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-02-16T19:05:47.324 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2024-02-16T19:05:54.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-02-16T19:05:54.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:05:56.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-02-16T19:05:56.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:05:57.340 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-02-16T19:06:09.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-02-16T19:06:09.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:06:12.412 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-02-16T19:06:27.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-02-16T19:06:27.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:06:45.948 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-02-16T19:06:56.340 DEBUG:teuthology.orchestra.remote:timed out 2024-02-16T19:07:03.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-02-16T19:07:03.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:07:04.316 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-02-16T19:07:04.694 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-02-16T19:07:04.695 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-421.el9.x86_64"... 2024-02-16T19:07:04.695 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-02-16T19:07:04.709 INFO:teuthology.orchestra.run.smithi163.stdout:5.14.0-421.el9.x86_64 2024-02-16T19:07:04.709 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-16T19:07:04.709 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-16T19:07:04.709 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-16T19:07:05.710 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-16T19:07:05.710 DEBUG:teuthology.orchestra.run.smithi163:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-16T19:07:05.769 INFO:teuthology.orchestra.run.smithi163.stdout:ttyS1 2024-02-16T19:07:05.810 DEBUG:teuthology.parallel:result is None 2024-02-16T19:07:08.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-02-16T19:07:08.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:07:08.707 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-02-16T19:07:09.097 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-02-16T19:07:09.098 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-421.el9.x86_64"... 2024-02-16T19:07:09.098 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-02-16T19:07:09.113 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-421.el9.x86_64 2024-02-16T19:07:09.113 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-16T19:07:09.113 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-16T19:07:09.113 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-16T19:07:10.114 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-16T19:07:10.115 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-16T19:07:10.204 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-02-16T19:07:10.241 DEBUG:teuthology.parallel:result is None 2024-02-16T19:07:10.241 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-16T19:07:10.248 INFO:teuthology.task.internal:Creating test directory... 2024-02-16T19:07:10.249 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-16T19:07:10.251 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-16T19:07:10.269 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-16T19:07:10.281 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-16T19:07:10.315 INFO:teuthology.task.internal:Creating archive directory... 2024-02-16T19:07:10.315 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-16T19:07:10.317 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-16T19:07:10.346 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-16T19:07:10.352 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-16T19:07:10.352 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-16T19:07:10.378 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-16T19:07:10.413 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T19:07:10.419 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T19:07:10.427 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T19:07:10.433 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-16T19:07:10.434 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-16T19:07:10.441 INFO:teuthology.task.internal:Configuring sudo... 2024-02-16T19:07:10.441 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-16T19:07:10.477 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-16T19:07:10.506 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-16T19:07:10.516 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-16T19:07:10.516 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-16T19:07:10.546 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-16T19:07:10.563 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T19:07:10.710 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T19:07:10.744 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T19:07:10.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T19:07:10.871 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:07:10.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-16T19:07:10.939 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T19:07:10.986 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-16T19:07:11.041 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T19:07:11.126 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-16T19:07:11.168 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:07:11.168 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-16T19:07:11.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-02-16T19:07:11.239 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-02-16T19:07:11.286 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T19:07:11.339 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T19:07:11.832 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-16T19:07:11.840 INFO:teuthology.task.internal:Starting timer... 2024-02-16T19:07:11.840 INFO:teuthology.run_tasks:Running task pcp... 2024-02-16T19:07:11.885 INFO:teuthology.run_tasks:Running task selinux... 2024-02-16T19:07:11.917 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-16T19:07:11.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-02-16T19:07:11.973 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-02-16T19:07:11.975 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-02-16T19:07:12.044 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-02-16T19:07:12.046 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-16T19:07:12.046 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-02-16T19:07:12.080 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-02-16T19:07:12.080 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-02-16T19:07:12.112 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-02-16T19:07:12.112 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2024-02-16T19:07:12.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T19:07:12.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:07:12.149 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T19:07:12.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:07:12.180 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-16T19:07:12.180 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-02-16T19:07:12.221 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-02-16T19:07:12.254 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-16T19:07:12.264 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-16T19:07:12.269 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-16T19:07:12.390 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-16T19:07:12.405 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-16T19:07:12.406 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-16T19:11:25.049 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2024-02-16T19:11:25.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-02-16T19:11:25.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:11:25.142 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-02-16T19:11:25.219 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-02-16T19:11:25.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-02-16T19:11:25.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T19:11:25.308 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-02-16T19:11:25.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-02-16T19:11:25.387 INFO:teuthology.run_tasks:Running task clock... 2024-02-16T19:11:25.399 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-16T19:11:25.400 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-16T19:11:25.400 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T19:11:25.402 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-16T19:11:25.403 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T19:11:25.431 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-16T19:11:25.448 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-16T19:11:25.458 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-16T19:11:25.475 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-16T19:11:25.477 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-02-16T19:11:25.490 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-02-16T19:11:25.504 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-02-16T19:11:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-16T19:11:25.518 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-02-16T19:11:25.523 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-16T19:11:25.534 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-16T19:11:25.551 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-16T19:11:25.576 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-02-16T19:11:25.580 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T19:11:25.580 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-02-16T19:11:25.580 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:11:25.580 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:11:25.580 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:11:25.580 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:11:25.603 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-02-16T19:11:25.606 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T19:11:25.606 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-02-16T19:11:25.606 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:11:25.606 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:11:25.606 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:11:25.606 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:11:25.607 INFO:teuthology.run_tasks:Running task install... 2024-02-16T19:11:25.616 DEBUG:teuthology.task.install:project ceph 2024-02-16T19:11:25.616 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'}} 2024-02-16T19:11:25.616 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-16T19:11:25.617 INFO:teuthology.task.install:Using flavor: default 2024-02-16T19:11:25.623 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-16T19:11:25.623 INFO:teuthology.task.install:extra packages: [] 2024-02-16T19:11:25.623 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': None, 'wait_for_package': False} 2024-02-16T19:11:25.623 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T19:11:25.624 INFO:teuthology.packaging:ref: None 2024-02-16T19:11:25.624 INFO:teuthology.packaging:tag: None 2024-02-16T19:11:25.624 INFO:teuthology.packaging:branch: quincy 2024-02-16T19:11:25.624 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:11:25.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-02-16T19:11:25.626 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': None, 'wait_for_package': False} 2024-02-16T19:11:25.626 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T19:11:25.626 INFO:teuthology.packaging:ref: None 2024-02-16T19:11:25.626 INFO:teuthology.packaging:tag: None 2024-02-16T19:11:25.626 INFO:teuthology.packaging:branch: quincy 2024-02-16T19:11:25.626 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:11:25.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-02-16T19:11:25.840 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/ 2024-02-16T19:11:25.840 INFO:teuthology.task.install.rpm:Package version is 17.2.7-513.gb88fae66 2024-02-16T19:11:25.846 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/ 2024-02-16T19:11:25.846 INFO:teuthology.task.install.rpm:Package version is 17.2.7-513.gb88fae66 2024-02-16T19:11:26.029 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-16T19:11:26.030 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:11:26.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-16T19:11:26.040 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/b88fae66e6ad896087939a6bde358f026d85f561/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-16T19:11:26.041 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:11:26.041 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-16T19:11:26.059 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-16T19:11:26.059 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T19:11:26.059 INFO:teuthology.packaging:ref: None 2024-02-16T19:11:26.059 INFO:teuthology.packaging:tag: None 2024-02-16T19:11:26.059 INFO:teuthology.packaging:branch: quincy 2024-02-16T19:11:26.059 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:11:26.060 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-16T19:11:26.070 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-16T19:11:26.070 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T19:11:26.071 INFO:teuthology.packaging:ref: None 2024-02-16T19:11:26.071 INFO:teuthology.packaging:tag: None 2024-02-16T19:11:26.071 INFO:teuthology.packaging:branch: quincy 2024-02-16T19:11:26.071 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:11:26.071 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-16T19:11:26.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-16T19:11:26.139 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-16T19:11:26.208 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-16T19:11:26.220 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-16T19:11:26.274 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-02-16T19:11:26.276 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-02-16T19:11:26.287 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-02-16T19:11:26.289 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-02-16T19:11:26.521 INFO:teuthology.orchestra.run.smithi040.stdout:74 files removed 2024-02-16T19:11:26.538 INFO:teuthology.orchestra.run.smithi163.stdout:74 files removed 2024-02-16T19:11:26.548 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw 2024-02-16T19:11:26.571 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw 2024-02-16T19:11:27.236 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 225 kB/s | 77 kB 00:00 2024-02-16T19:11:27.296 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 182 kB/s | 77 kB 00:00 2024-02-16T19:11:27.629 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 32 kB/s | 9.1 kB 00:00 2024-02-16T19:11:27.695 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 22 kB/s | 9.1 kB 00:00 2024-02-16T19:11:28.030 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 5.0 kB/s | 1.8 kB 00:00 2024-02-16T19:11:28.087 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 5.1 kB/s | 1.8 kB 00:00 2024-02-16T19:11:28.474 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-02-16T19:11:28.565 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-16T19:11:29.982 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-02-16T19:11:30.008 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-02-16T19:11:31.076 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-02-16T19:11:34.963 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-02-16T19:11:36.997 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-02-16T19:11:42.323 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.1 MB 00:01 2024-02-16T19:11:43.899 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 15 kB 00:00 2024-02-16T19:11:46.198 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 19 MB 00:13 2024-02-16T19:11:47.583 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 20 MB 00:03 2024-02-16T19:11:50.310 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-02-16T19:11:51.810 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 15 kB 00:00 2024-02-16T19:11:52.458 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-02-16T19:11:55.103 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:11:55.114 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:17.2.7-513.gb88fae66.el9 ceph 14 M 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 3.1 M 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.9 M 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:17.2.7-513.gb88fae66.el9 ceph 5.0 M 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:17.2.7-513.gb88fae66.el9 ceph 23 M 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:17.2.7-513.gb88fae66.el9 ceph 25 k 2024-02-16T19:11:55.115 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 635 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 458 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 4.9 M 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 45 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:17.2.7-513.gb88fae66.el9 ceph 116 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:17.2.7-513.gb88fae66.el9 ceph 156 k 2024-02-16T19:11:55.116 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:17.2.7-513.gb88fae66.el9 ceph 320 k 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 293 k 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:17.2.7-513.gb88fae66.el9 ceph 99 k 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout:Install 22 Packages 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 2 Packages 2024-02-16T19:11:55.117 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:11:55.118 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 58 M 2024-02-16T19:11:55.118 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:11:56.083 INFO:teuthology.orchestra.run.smithi163.stdout:(1/24): ceph-base-17.2.7-513.gb88fae66.el9.x86_ 6.2 MB/s | 5.0 MB 00:00 2024-02-16T19:11:56.159 INFO:teuthology.orchestra.run.smithi163.stdout:(2/24): ceph-selinux-17.2.7-513.gb88fae66.el9.x 335 kB/s | 25 kB 00:00 2024-02-16T19:11:56.267 INFO:teuthology.orchestra.run.smithi163.stdout:(3/24): libcephfs2-17.2.7-513.gb88fae66.el9.x86 5.7 MB/s | 635 kB 00:00 2024-02-16T19:11:56.367 INFO:teuthology.orchestra.run.smithi163.stdout:(4/24): libradosstriper1-17.2.7-513.gb88fae66.e 4.5 MB/s | 458 kB 00:00 2024-02-16T19:11:56.926 INFO:teuthology.orchestra.run.smithi163.stdout:(5/24): librgw2-17.2.7-513.gb88fae66.el9.x86_64 8.8 MB/s | 4.9 MB 00:00 2024-02-16T19:11:57.001 INFO:teuthology.orchestra.run.smithi163.stdout:(6/24): python3-ceph-argparse-17.2.7-513.gb88fa 602 kB/s | 45 kB 00:00 2024-02-16T19:11:57.076 INFO:teuthology.orchestra.run.smithi163.stdout:(7/24): python3-ceph-common-17.2.7-513.gb88fae6 1.5 MB/s | 116 kB 00:00 2024-02-16T19:11:57.177 INFO:teuthology.orchestra.run.smithi163.stdout:(8/24): python3-cephfs-17.2.7-513.gb88fae66.el9 1.5 MB/s | 156 kB 00:00 2024-02-16T19:11:57.402 INFO:teuthology.orchestra.run.smithi163.stdout:(9/24): ceph-common-17.2.7-513.gb88fae66.el9.x8 11 MB/s | 23 MB 00:02 2024-02-16T19:11:57.427 INFO:teuthology.orchestra.run.smithi163.stdout:(10/24): python3-rados-17.2.7-513.gb88fae66.el9 1.2 MB/s | 320 kB 00:00 2024-02-16T19:11:57.486 INFO:teuthology.orchestra.run.smithi163.stdout:(11/24): python3-rbd-17.2.7-513.gb88fae66.el9.x 3.4 MB/s | 293 kB 00:00 2024-02-16T19:11:57.511 INFO:teuthology.orchestra.run.smithi163.stdout:(12/24): python3-rgw-17.2.7-513.gb88fae66.el9.x 1.2 MB/s | 99 kB 00:00 2024-02-16T19:11:57.628 INFO:teuthology.orchestra.run.smithi163.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 483 kB/s | 44 kB 00:00 2024-02-16T19:11:57.654 INFO:teuthology.orchestra.run.smithi163.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 951 kB/s | 159 kB 00:00 2024-02-16T19:11:57.696 INFO:teuthology.orchestra.run.smithi163.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-02-16T19:11:57.721 INFO:teuthology.orchestra.run.smithi163.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-02-16T19:11:57.746 INFO:teuthology.orchestra.run.smithi163.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 830 kB/s | 41 kB 00:00 2024-02-16T19:11:58.030 INFO:teuthology.orchestra.run.smithi163.stdout:(18/24): ceph-radosgw-17.2.7-513.gb88fae66.el9. 5.1 MB/s | 14 MB 00:02 2024-02-16T19:11:58.405 INFO:teuthology.orchestra.run.smithi163.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 74 kB/s | 49 kB 00:00 2024-02-16T19:11:58.589 INFO:teuthology.orchestra.run.smithi163.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 121 kB/s | 67 kB 00:00 2024-02-16T19:11:58.722 INFO:teuthology.orchestra.run.smithi163.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 309 kB/s | 309 kB 00:01 2024-02-16T19:11:58.939 INFO:teuthology.orchestra.run.smithi163.stdout:(22/24): librados2-17.2.7-513.gb88fae66.el9.x86 8.8 MB/s | 3.1 MB 00:00 2024-02-16T19:11:59.156 INFO:teuthology.orchestra.run.smithi163.stdout:(23/24): librbd1-17.2.7-513.gb88fae66.el9.x86_6 6.8 MB/s | 2.9 MB 00:00 2024-02-16T19:12:00.431 INFO:teuthology.orchestra.run.smithi163.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 799 kB/s | 1.6 MB 00:02 2024-02-16T19:12:00.439 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:00.439 INFO:teuthology.orchestra.run.smithi163.stdout:Total 11 MB/s | 58 MB 00:05 2024-02-16T19:12:00.439 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:12:00.472 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:12:00.473 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:12:00.935 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:12:00.936 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:12:02.168 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:12:02.521 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-02-16T19:12:02.649 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T19:12:02.741 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T19:12:02.820 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 3/26 2024-02-16T19:12:02.842 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T19:12:02.940 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T19:12:02.984 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-02-16T19:12:03.014 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-02-16T19:12:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-02-16T19:12:03.478 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 8/26 2024-02-16T19:12:03.541 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 8/26 2024-02-16T19:12:03.569 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-02-16T19:12:03.612 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x 10/26 2024-02-16T19:12:03.650 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 11/26 2024-02-16T19:12:03.712 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 12/26 2024-02-16T19:12:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T19:12:03.807 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T19:12:03.845 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-02-16T19:12:03.880 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-02-16T19:12:03.913 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-02-16T19:12:04.208 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-02-16T19:12:04.243 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 18/26 2024-02-16T19:12:04.333 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 18/26 2024-02-16T19:12:04.405 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 19/26 2024-02-16T19:12:04.437 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86 20/26 2024-02-16T19:12:06.761 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 20 MB 00:14 2024-02-16T19:12:06.878 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T19:12:06.916 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T19:12:07.459 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T19:12:07.494 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 22/26 2024-02-16T19:12:07.592 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 22/26 2024-02-16T19:12:07.592 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-16T19:12:07.592 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-16T19:12:07.592 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:07.629 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T19:12:11.811 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-02-16T19:12:14.533 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:17.2.7-513.gb88fae66.el9 ceph 14 M 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 3.1 M 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.9 M 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:12:14.548 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:17.2.7-513.gb88fae66.el9 ceph 5.0 M 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:17.2.7-513.gb88fae66.el9 ceph 23 M 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:17.2.7-513.gb88fae66.el9 ceph 25 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 635 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 458 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:17.2.7-513.gb88fae66.el9 ceph 4.9 M 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 45 k 2024-02-16T19:12:14.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:17.2.7-513.gb88fae66.el9 ceph 116 k 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:17.2.7-513.gb88fae66.el9 ceph 156 k 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:17.2.7-513.gb88fae66.el9 ceph 320 k 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 293 k 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:17.2.7-513.gb88fae66.el9 ceph 99 k 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout:Install 22 Packages 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 2 Packages 2024-02-16T19:12:14.550 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:14.551 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 58 M 2024-02-16T19:12:14.551 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:12:15.811 INFO:teuthology.orchestra.run.smithi040.stdout:(1/24): ceph-radosgw-17.2.7-513.gb88fae66.el9.x 12 MB/s | 14 MB 00:01 2024-02-16T19:12:15.862 INFO:teuthology.orchestra.run.smithi040.stdout:(2/24): ceph-selinux-17.2.7-513.gb88fae66.el9.x 504 kB/s | 25 kB 00:00 2024-02-16T19:12:15.945 INFO:teuthology.orchestra.run.smithi040.stdout:(3/24): libcephfs2-17.2.7-513.gb88fae66.el9.x86 7.5 MB/s | 635 kB 00:00 2024-02-16T19:12:16.021 INFO:teuthology.orchestra.run.smithi040.stdout:(4/24): libradosstriper1-17.2.7-513.gb88fae66.e 6.0 MB/s | 458 kB 00:00 2024-02-16T19:12:16.396 INFO:teuthology.orchestra.run.smithi040.stdout:(5/24): librgw2-17.2.7-513.gb88fae66.el9.x86_64 13 MB/s | 4.9 MB 00:00 2024-02-16T19:12:16.463 INFO:teuthology.orchestra.run.smithi040.stdout:(6/24): python3-ceph-argparse-17.2.7-513.gb88fa 675 kB/s | 45 kB 00:00 2024-02-16T19:12:16.538 INFO:teuthology.orchestra.run.smithi040.stdout:(7/24): ceph-base-17.2.7-513.gb88fae66.el9.x86_ 2.6 MB/s | 5.0 MB 00:01 2024-02-16T19:12:16.572 INFO:teuthology.orchestra.run.smithi040.stdout:(8/24): python3-ceph-common-17.2.7-513.gb88fae6 1.0 MB/s | 116 kB 00:00 2024-02-16T19:12:16.606 INFO:teuthology.orchestra.run.smithi040.stdout:(9/24): python3-cephfs-17.2.7-513.gb88fae66.el9 2.3 MB/s | 156 kB 00:00 2024-02-16T19:12:16.639 INFO:teuthology.orchestra.run.smithi040.stdout:(10/24): python3-rados-17.2.7-513.gb88fae66.el9 4.7 MB/s | 320 kB 00:00 2024-02-16T19:12:16.698 INFO:teuthology.orchestra.run.smithi040.stdout:(11/24): python3-rgw-17.2.7-513.gb88fae66.el9.x 1.7 MB/s | 99 kB 00:00 2024-02-16T19:12:16.723 INFO:teuthology.orchestra.run.smithi040.stdout:(12/24): python3-rbd-17.2.7-513.gb88fae66.el9.x 2.4 MB/s | 293 kB 00:00 2024-02-16T19:12:16.840 INFO:teuthology.orchestra.run.smithi040.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 378 kB/s | 44 kB 00:00 2024-02-16T19:12:16.866 INFO:teuthology.orchestra.run.smithi040.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 952 kB/s | 159 kB 00:00 2024-02-16T19:12:17.082 INFO:teuthology.orchestra.run.smithi040.stdout:(15/24): ceph-common-17.2.7-513.gb88fae66.el9.x 9.2 MB/s | 23 MB 00:02 2024-02-16T19:12:17.116 INFO:teuthology.orchestra.run.smithi040.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 661 kB 00:00 2024-02-16T19:12:17.136 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T19:12:17.136 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-02-16T19:12:17.137 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-02-16T19:12:17.137 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-02-16T19:12:17.137 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-02-16T19:12:17.137 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-02-16T19:12:17.137 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /root 2024-02-16T19:12:17.137 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-02-16T19:12:17.137 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:17.141 INFO:teuthology.orchestra.run.smithi040.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 282 kB 00:00 2024-02-16T19:12:17.166 INFO:teuthology.orchestra.run.smithi040.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 499 kB/s | 41 kB 00:00 2024-02-16T19:12:17.233 INFO:teuthology.orchestra.run.smithi040.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 728 kB/s | 49 kB 00:00 2024-02-16T19:12:17.293 INFO:teuthology.orchestra.run.smithi040.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-02-16T19:12:17.317 INFO:teuthology.orchestra.run.smithi040.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.5 MB/s | 309 kB 00:00 2024-02-16T19:12:17.401 INFO:teuthology.orchestra.run.smithi040.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-02-16T19:12:17.543 INFO:teuthology.orchestra.run.smithi040.stdout:(23/24): librbd1-17.2.7-513.gb88fae66.el9.x86_6 13 MB/s | 2.9 MB 00:00 2024-02-16T19:12:17.943 INFO:teuthology.orchestra.run.smithi040.stdout:(24/24): librados2-17.2.7-513.gb88fae66.el9.x86 4.7 MB/s | 3.1 MB 00:00 2024-02-16T19:12:17.947 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:17.947 INFO:teuthology.orchestra.run.smithi040.stdout:Total 17 MB/s | 58 MB 00:03 2024-02-16T19:12:17.948 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:12:17.984 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:12:17.984 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:12:18.411 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 24/26 2024-02-16T19:12:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:12:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:12:18.481 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 24/26 2024-02-16T19:12:18.481 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:12:18.481 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-16T19:12:18.481 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T19:12:18.481 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T19:12:18.481 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:18.482 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-02-16T19:12:18.581 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-02-16T19:12:18.581 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T19:12:19.749 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 1/26 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 3/26 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 5/26 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 6/26 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 7/26 2024-02-16T19:12:19.879 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x 8/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86 9/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 10/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 11/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 12/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-02-16T19:12:19.880 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-02-16T19:12:19.881 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-02-16T19:12:19.881 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T19:12:19.881 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-02-16T19:12:19.881 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 25/26 2024-02-16T19:12:20.103 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-02-16T19:12:20.162 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T19:12:20.254 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T19:12:20.335 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 3/26 2024-02-16T19:12:20.363 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T19:12:20.460 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T19:12:20.506 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-02-16T19:12:20.535 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-02-16T19:12:20.813 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T19:12:20.813 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-16T19:12:20.814 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.815 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:20.816 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-16T19:12:20.816 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:20.816 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:12:20.992 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-02-16T19:12:21.025 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 8/26 2024-02-16T19:12:21.070 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 8/26 2024-02-16T19:12:21.099 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-02-16T19:12:21.141 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-test 2024-02-16T19:12:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x 10/26 2024-02-16T19:12:21.180 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 11/26 2024-02-16T19:12:21.243 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 12/26 2024-02-16T19:12:21.269 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T19:12:21.335 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T19:12:21.375 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-02-16T19:12:21.411 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-02-16T19:12:21.444 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-02-16T19:12:21.719 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:29 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:12:21.766 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-02-16T19:12:21.798 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 18/26 2024-02-16T19:12:21.872 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 18/26 2024-02-16T19:12:21.885 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:12:21.886 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:21.886 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:12:21.886 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:21.886 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:17.2.7-513.gb88fae66.el9 ceph 48 M 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:17.2.7-513.gb88fae66.el9 ceph 162 k 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout:Install 6 Packages 2024-02-16T19:12:21.887 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:21.888 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 49 M 2024-02-16T19:12:21.888 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 196 M 2024-02-16T19:12:21.888 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:12:21.947 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 19/26 2024-02-16T19:12:21.976 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86 20/26 2024-02-16T19:12:22.066 INFO:teuthology.orchestra.run.smithi163.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 300 kB 00:00 2024-02-16T19:12:22.100 INFO:teuthology.orchestra.run.smithi163.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-02-16T19:12:22.142 INFO:teuthology.orchestra.run.smithi163.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-02-16T19:12:22.167 INFO:teuthology.orchestra.run.smithi163.stdout:(4/6): libcephsqlite-17.2.7-513.gb88fae66.el9.x 584 kB/s | 162 kB 00:00 2024-02-16T19:12:22.192 INFO:teuthology.orchestra.run.smithi163.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-02-16T19:12:24.376 INFO:teuthology.orchestra.run.smithi163.stdout:(6/6): ceph-test-17.2.7-513.gb88fae66.el9.x86_6 19 MB/s | 48 MB 00:02 2024-02-16T19:12:24.379 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:24.379 INFO:teuthology.orchestra.run.smithi163.stdout:Total 20 MB/s | 49 MB 00:02 2024-02-16T19:12:24.379 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:12:24.400 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:12:24.400 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:12:24.491 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T19:12:24.523 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T19:12:24.787 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:12:24.787 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:12:25.083 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 21/26 2024-02-16T19:12:25.184 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 22/26 2024-02-16T19:12:25.276 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 22/26 2024-02-16T19:12:25.277 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-16T19:12:25.277 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-16T19:12:25.277 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:25.310 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T19:12:25.409 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:12:25.489 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-16T19:12:25.516 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-16T19:12:25.561 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-16T19:12:25.601 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-16T19:12:25.628 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 5/6 2024-02-16T19:12:30.080 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 5/6 2024-02-16T19:12:30.162 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 6/6 2024-02-16T19:12:32.384 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 6/6 2024-02-16T19:12:32.384 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 1/6 2024-02-16T19:12:32.384 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 2/6 2024-02-16T19:12:32.384 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-16T19:12:32.384 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-16T19:12:32.384 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-16T19:12:32.767 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout: jq-1.6-16.el9.x86_64 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:32.768 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:12:33.085 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph 2024-02-16T19:12:33.676 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:41 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:12:33.845 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================== 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================== 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:17.2.7-513.gb88fae66.el9 ceph 6.5 k 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.1 M 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:17.2.7-513.gb88fae66.el9 ceph 1.4 M 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 239 k 2024-02-16T19:12:33.848 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:17.2.7-513.gb88fae66.el9 ceph 3.7 M 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 15 M 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-16T19:12:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-16T19:12:33.850 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-16T19:12:33.851 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:33.851 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:12:33.851 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================== 2024-02-16T19:12:33.851 INFO:teuthology.orchestra.run.smithi163.stdout:Install 33 Packages 2024-02-16T19:12:33.851 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:33.851 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 27 M 2024-02-16T19:12:33.851 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 99 M 2024-02-16T19:12:33.852 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:12:34.388 INFO:teuthology.orchestra.run.smithi163.stdout:(1/33): ceph-17.2.7-513.gb88fae66.el9.x86_64.rp 38 kB/s | 6.5 kB 00:00 2024-02-16T19:12:34.722 INFO:teuthology.orchestra.run.smithi163.stdout:(2/33): ceph-mon-17.2.7-513.gb88fae66.el9.x86_6 11 MB/s | 3.7 MB 00:00 2024-02-16T19:12:35.039 INFO:teuthology.orchestra.run.smithi163.stdout:(3/33): ceph-mgr-17.2.7-513.gb88fae66.el9.x86_6 1.7 MB/s | 1.4 MB 00:00 2024-02-16T19:12:35.169 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T19:12:35.169 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-02-16T19:12:35.169 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-02-16T19:12:35.170 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-02-16T19:12:35.170 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-02-16T19:12:35.170 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-02-16T19:12:35.170 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-02-16T19:12:35.170 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-02-16T19:12:35.170 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:35.239 INFO:teuthology.orchestra.run.smithi163.stdout:(4/33): ceph-mgr-modules-core-17.2.7-513.gb88fa 1.2 MB/s | 239 kB 00:00 2024-02-16T19:12:35.498 INFO:teuthology.orchestra.run.smithi163.stdout:(5/33): ceph-osd-17.2.7-513.gb88fae66.el9.x86_6 20 MB/s | 15 MB 00:00 2024-02-16T19:12:35.523 INFO:teuthology.orchestra.run.smithi163.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 509 kB/s | 144 kB 00:00 2024-02-16T19:12:35.548 INFO:teuthology.orchestra.run.smithi163.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 631 kB/s | 32 kB 00:00 2024-02-16T19:12:35.574 INFO:teuthology.orchestra.run.smithi163.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 939 kB/s | 47 kB 00:00 2024-02-16T19:12:35.599 INFO:teuthology.orchestra.run.smithi163.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 771 kB/s | 39 kB 00:00 2024-02-16T19:12:35.641 INFO:teuthology.orchestra.run.smithi163.stdout:(10/33): ceph-mds-17.2.7-513.gb88fae66.el9.x86_ 1.5 MB/s | 2.1 MB 00:01 2024-02-16T19:12:35.666 INFO:teuthology.orchestra.run.smithi163.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-02-16T19:12:35.733 INFO:teuthology.orchestra.run.smithi163.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 8.5 MB/s | 1.1 MB 00:00 2024-02-16T19:12:35.759 INFO:teuthology.orchestra.run.smithi163.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 879 kB/s | 103 kB 00:00 2024-02-16T19:12:35.784 INFO:teuthology.orchestra.run.smithi163.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 1.0 MB/s | 124 kB 00:00 2024-02-16T19:12:35.809 INFO:teuthology.orchestra.run.smithi163.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 113 kB 00:00 2024-02-16T19:12:35.834 INFO:teuthology.orchestra.run.smithi163.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-16T19:12:36.193 INFO:teuthology.orchestra.run.smithi163.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 106 kB/s | 43 kB 00:00 2024-02-16T19:12:36.301 INFO:teuthology.orchestra.run.smithi163.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 99 kB/s | 11 kB 00:00 2024-02-16T19:12:36.369 INFO:teuthology.orchestra.run.smithi163.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 307 kB/s | 172 kB 00:00 2024-02-16T19:12:36.419 INFO:teuthology.orchestra.run.smithi163.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2024-02-16T19:12:36.477 INFO:teuthology.orchestra.run.smithi163.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 213 kB/s | 23 kB 00:00 2024-02-16T19:12:36.493 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 24/26 2024-02-16T19:12:36.511 INFO:teuthology.orchestra.run.smithi163.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 530 kB/s | 358 kB 00:00 2024-02-16T19:12:36.536 INFO:teuthology.orchestra.run.smithi163.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 166 kB/s | 19 kB 00:00 2024-02-16T19:12:36.570 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 24/26 2024-02-16T19:12:36.570 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:12:36.570 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-16T19:12:36.571 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T19:12:36.571 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-16T19:12:36.571 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:36.572 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-02-16T19:12:36.579 INFO:teuthology.orchestra.run.smithi163.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 195 kB/s | 20 kB 00:00 2024-02-16T19:12:36.612 INFO:teuthology.orchestra.run.smithi163.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 460 kB/s | 46 kB 00:00 2024-02-16T19:12:36.654 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-02-16T19:12:36.654 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T19:12:36.712 INFO:teuthology.orchestra.run.smithi163.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 164 kB/s | 16 kB 00:00 2024-02-16T19:12:36.737 INFO:teuthology.orchestra.run.smithi163.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 392 kB/s | 79 kB 00:00 2024-02-16T19:12:36.763 INFO:teuthology.orchestra.run.smithi163.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-02-16T19:12:36.838 INFO:teuthology.orchestra.run.smithi163.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 356 kB/s | 36 kB 00:00 2024-02-16T19:12:36.863 INFO:teuthology.orchestra.run.smithi163.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 596 kB/s | 90 kB 00:00 2024-02-16T19:12:36.889 INFO:teuthology.orchestra.run.smithi163.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-02-16T19:12:36.981 INFO:teuthology.orchestra.run.smithi163.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 170 kB/s | 20 kB 00:00 2024-02-16T19:12:37.006 INFO:teuthology.orchestra.run.smithi163.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-02-16T19:12:37.012 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:37.013 INFO:teuthology.orchestra.run.smithi163.stdout:Total 8.4 MB/s | 27 MB 00:03 2024-02-16T19:12:37.013 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:12:37.050 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:12:37.050 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:12:37.409 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:12:37.410 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:12:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 1/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 3/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 4/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 5/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 6/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 7/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x 8/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86 9/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 10/26 2024-02-16T19:12:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 11/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 12/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 13/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:17.2.7-513.gb88fae66.el9.x86_64 23/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-02-16T19:12:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 25/26 2024-02-16T19:12:38.731 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:12:38.741 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-02-16T19:12:38.741 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:38.741 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-02-16T19:12:38.741 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.741 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.741 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:12:38.741 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.741 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.742 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-16T19:12:38.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:38.743 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-16T19:12:38.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:38.743 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:12:38.785 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-16T19:12:38.816 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-16T19:12:38.858 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-16T19:12:38.958 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-16T19:12:39.046 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-16T19:12:39.081 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-test 2024-02-16T19:12:39.086 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-16T19:12:39.166 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-16T19:12:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-16T19:12:39.259 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-16T19:12:39.308 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-16T19:12:39.364 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-16T19:12:39.549 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-16T19:12:39.605 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-16T19:12:39.644 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-16T19:12:39.661 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:28 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:12:39.678 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-16T19:12:39.708 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-16T19:12:39.785 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-16T19:12:39.819 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T19:12:39.836 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:12:39.837 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:12:39.837 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:12:39.837 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:12:39.837 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:12:39.837 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:17.2.7-513.gb88fae66.el9 ceph 48 M 2024-02-16T19:12:39.837 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:17.2.7-513.gb88fae66.el9 ceph 162 k 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout:Install 6 Packages 2024-02-16T19:12:39.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:39.839 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 49 M 2024-02-16T19:12:39.839 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 196 M 2024-02-16T19:12:39.839 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:12:39.852 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-16T19:12:39.896 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-16T19:12:39.928 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-16T19:12:39.963 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-16T19:12:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 300 kB 00:00 2024-02-16T19:12:40.051 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-16T19:12:40.060 INFO:teuthology.orchestra.run.smithi040.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-02-16T19:12:40.102 INFO:teuthology.orchestra.run.smithi040.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-02-16T19:12:40.102 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-16T19:12:40.152 INFO:teuthology.orchestra.run.smithi040.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-02-16T19:12:40.175 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-16T19:12:40.285 INFO:teuthology.orchestra.run.smithi040.stdout:(5/6): libcephsqlite-17.2.7-513.gb88fae66.el9.x 364 kB/s | 162 kB 00:00 2024-02-16T19:12:40.313 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-16T19:12:40.437 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T19:12:40.585 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.n 28/33 2024-02-16T19:12:40.608 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 29/33 2024-02-16T19:12:40.634 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 29/33 2024-02-16T19:12:40.634 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:12:40.634 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-16T19:12:40.635 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T19:12:40.635 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T19:12:40.635 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:42.000 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 30/33 2024-02-16T19:12:42.027 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 30/33 2024-02-16T19:12:42.027 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:12:42.027 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-16T19:12:42.027 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T19:12:42.027 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T19:12:42.027 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:42.401 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 31/33 2024-02-16T19:12:42.424 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 31/33 2024-02-16T19:12:42.425 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:12:42.425 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-16T19:12:42.425 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T19:12:42.425 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T19:12:42.425 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:42.627 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 32/33 2024-02-16T19:12:42.650 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 32/33 2024-02-16T19:12:42.650 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:12:42.650 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-16T19:12:42.650 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T19:12:42.651 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T19:12:42.651 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:42.715 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:17.2.7-513.gb88fae66.el9.x86_64 33/33 2024-02-16T19:12:44.419 INFO:teuthology.orchestra.run.smithi040.stdout:(6/6): ceph-test-17.2.7-513.gb88fae66.el9.x86_6 10 MB/s | 48 MB 00:04 2024-02-16T19:12:44.422 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:44.422 INFO:teuthology.orchestra.run.smithi040.stdout:Total 11 MB/s | 49 MB 00:04 2024-02-16T19:12:44.422 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:12:44.443 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:12:44.444 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:12:44.468 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-2:17.2.7-513.gb88fae66.el9.x86_64 33/33 2024-02-16T19:12:44.468 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:17.2.7-513.gb88fae66.el9.x86_64 1/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 2/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 3/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 4/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 5/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.n 6/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-16T19:12:44.469 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-16T19:12:44.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-16T19:12:44.471 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-16T19:12:44.471 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-16T19:12:44.471 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-16T19:12:44.471 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T19:12:44.472 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-16T19:12:44.472 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-16T19:12:44.472 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-16T19:12:44.472 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-16T19:12:44.472 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-16T19:12:44.849 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:12:44.849 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-16T19:12:45.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-16T19:12:45.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply-3.11-14.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:45.347 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:12:45.444 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:12:45.504 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-16T19:12:45.536 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-16T19:12:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-16T19:12:45.628 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-16T19:12:45.655 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 5/6 2024-02-16T19:12:45.668 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-base 2024-02-16T19:12:46.242 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:54 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:12:46.355 INFO:teuthology.orchestra.run.smithi163.stdout:Package ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:12:46.406 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:12:46.407 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:12:46.407 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:12:46.477 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install cephadm 2024-02-16T19:12:47.058 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:55 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:12:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:12:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:47.224 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:12:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 80 k 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 80 k 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 364 k 2024-02-16T19:12:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:12:47.445 INFO:teuthology.orchestra.run.smithi163.stdout:cephadm-17.2.7-513.gb88fae66.el9.noarch.rpm 367 kB/s | 80 kB 00:00 2024-02-16T19:12:47.445 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:47.446 INFO:teuthology.orchestra.run.smithi163.stdout:Total 365 kB/s | 80 kB 00:00 2024-02-16T19:12:47.446 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:12:47.451 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:12:47.451 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:12:47.452 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:12:47.453 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:12:47.528 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:12:47.959 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T19:12:48.059 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T19:12:48.481 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T19:12:48.779 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T19:12:48.780 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:48.780 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:12:48.780 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:12:48.780 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:48.780 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:12:48.989 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-immutable-object-cache 2024-02-16T19:12:49.580 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:57 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:12:49.747 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:12:49.748 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:49.748 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repo Size 2024-02-16T19:12:49.748 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:49.748 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:12:49.748 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-513.gb88fae66.el9 ceph 137 k 2024-02-16T19:12:49.748 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:49.748 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:12:49.749 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:49.749 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2024-02-16T19:12:49.749 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:49.749 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 137 k 2024-02-16T19:12:49.749 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 379 k 2024-02-16T19:12:49.749 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:12:50.169 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-immutable-object-cache-17.2.7-513.gb88fae6 326 kB/s | 137 kB 00:00 2024-02-16T19:12:50.170 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:50.170 INFO:teuthology.orchestra.run.smithi163.stdout:Total 325 kB/s | 137 kB 00:00 2024-02-16T19:12:50.170 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:12:50.177 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:12:50.178 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:12:50.230 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:12:50.230 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:12:50.332 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 5/6 2024-02-16T19:12:50.369 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:12:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 6/6 2024-02-16T19:12:50.445 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T19:12:50.470 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T19:12:50.470 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:12:50.470 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-16T19:12:50.470 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:51.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T19:12:51.510 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:51.510 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:12:51.510 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:51.510 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:51.510 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:12:51.740 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr 2024-02-16T19:12:52.333 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:00 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:12:52.449 INFO:teuthology.orchestra.run.smithi163.stdout:Package ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:12:52.499 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:12:52.500 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:12:52.500 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:12:52.572 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-dashboard 2024-02-16T19:12:52.716 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 6/6 2024-02-16T19:12:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 1/6 2024-02-16T19:12:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 2/6 2024-02-16T19:12:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-16T19:12:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-16T19:12:52.717 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: jq-1.6-16.el9.x86_64 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-16T19:12:53.103 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-16T19:12:53.104 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:53.104 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:12:53.137 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:01 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:12:53.303 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:12:53.303 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:53.303 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 1.7 M 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 23 k 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 15 k 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout:Installing weak dependencies: 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-16T19:12:53.304 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:53.305 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:12:53.305 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:12:53.305 INFO:teuthology.orchestra.run.smithi163.stdout:Install 7 Packages 2024-02-16T19:12:53.305 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:53.305 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 2.0 M 2024-02-16T19:12:53.305 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 8.1 M 2024-02-16T19:12:53.306 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:12:53.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph 2024-02-16T19:12:53.790 INFO:teuthology.orchestra.run.smithi163.stdout:(1/7): ceph-prometheus-alerts-17.2.7-513.gb88fa 49 kB/s | 15 kB 00:00 2024-02-16T19:12:53.816 INFO:teuthology.orchestra.run.smithi163.stdout:(2/7): ceph-grafana-dashboards-17.2.7-513.gb88f 72 kB/s | 23 kB 00:00 2024-02-16T19:12:53.924 INFO:teuthology.orchestra.run.smithi163.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 68 kB/s | 9.0 kB 00:00 2024-02-16T19:12:53.957 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:42 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:12:54.033 INFO:teuthology.orchestra.run.smithi163.stdout:(4/7): ceph-mgr-dashboard-17.2.7-513.gb88fae66. 3.1 MB/s | 1.7 MB 00:00 2024-02-16T19:12:54.058 INFO:teuthology.orchestra.run.smithi163.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 169 kB/s | 41 kB 00:00 2024-02-16T19:12:54.083 INFO:teuthology.orchestra.run.smithi163.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 193 kB/s | 31 kB 00:00 2024-02-16T19:12:54.138 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:12:54.140 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================== 2024-02-16T19:12:54.140 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================== 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:17.2.7-513.gb88fae66.el9 ceph 6.5 k 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.1 M 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:17.2.7-513.gb88fae66.el9 ceph 1.4 M 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 239 k 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:17.2.7-513.gb88fae66.el9 ceph 3.7 M 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 15 M 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-16T19:12:54.141 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-16T19:12:54.142 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================== 2024-02-16T19:12:54.143 INFO:teuthology.orchestra.run.smithi040.stdout:Install 33 Packages 2024-02-16T19:12:54.144 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:12:54.144 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 27 M 2024-02-16T19:12:54.144 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 99 M 2024-02-16T19:12:54.144 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:12:54.233 INFO:teuthology.orchestra.run.smithi163.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 940 kB/s | 188 kB 00:00 2024-02-16T19:12:54.235 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:54.236 INFO:teuthology.orchestra.run.smithi163.stdout:Total 2.1 MB/s | 2.0 MB 00:00 2024-02-16T19:12:54.236 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:12:54.244 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:12:54.244 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:12:54.313 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:12:54.313 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:12:54.521 INFO:teuthology.orchestra.run.smithi040.stdout:(1/33): ceph-17.2.7-513.gb88fae66.el9.x86_64.rp 21 kB/s | 6.5 kB 00:00 2024-02-16T19:12:54.620 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:12:54.693 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-16T19:12:54.727 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-16T19:12:54.782 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-16T19:12:54.797 INFO:teuthology.orchestra.run.smithi040.stdout:(2/33): ceph-mds-17.2.7-513.gb88fae66.el9.x86_6 3.6 MB/s | 2.1 MB 00:00 2024-02-16T19:12:54.822 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-16T19:12:54.851 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.no 5/7 2024-02-16T19:12:55.084 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.n 6/7 2024-02-16T19:12:55.112 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 7/7 2024-02-16T19:12:55.664 INFO:teuthology.orchestra.run.smithi040.stdout:(3/33): ceph-mgr-17.2.7-513.gb88fae66.el9.x86_6 994 kB/s | 1.4 MB 00:01 2024-02-16T19:12:55.684 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 7/7 2024-02-16T19:12:55.684 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.n 1/7 2024-02-16T19:12:55.684 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 2/7 2024-02-16T19:12:55.684 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.no 3/7 2024-02-16T19:12:55.684 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-16T19:12:55.684 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-16T19:12:55.684 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-16T19:12:55.839 INFO:teuthology.orchestra.run.smithi040.stdout:(4/33): ceph-osd-17.2.7-513.gb88fae66.el9.x86_6 15 MB/s | 15 MB 00:01 2024-02-16T19:12:55.864 INFO:teuthology.orchestra.run.smithi040.stdout:(5/33): ceph-mgr-modules-core-17.2.7-513.gb88fa 1.2 MB/s | 239 kB 00:00 2024-02-16T19:12:56.026 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-16T19:12:56.026 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:56.026 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:12:56.026 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:12:56.026 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:12:56.026 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:12:56.026 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-16T19:12:56.027 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-16T19:12:56.027 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-16T19:12:56.027 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-16T19:12:56.027 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:56.027 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:12:56.048 INFO:teuthology.orchestra.run.smithi040.stdout:(6/33): ceph-mon-17.2.7-513.gb88fae66.el9.x86_6 2.4 MB/s | 3.7 MB 00:01 2024-02-16T19:12:56.073 INFO:teuthology.orchestra.run.smithi040.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 151 kB/s | 32 kB 00:00 2024-02-16T19:12:56.098 INFO:teuthology.orchestra.run.smithi040.stdout:(8/33): python3-mako-1.1.4-6.el9.noarch.rpm 558 kB/s | 144 kB 00:00 2024-02-16T19:12:56.126 INFO:teuthology.orchestra.run.smithi040.stdout:(9/33): python3-pytz-2021.1-5.el9.noarch.rpm 614 kB/s | 47 kB 00:00 2024-02-16T19:12:56.151 INFO:teuthology.orchestra.run.smithi040.stdout:(10/33): python3-toml-0.10.2-6.el9.noarch.rpm 496 kB/s | 39 kB 00:00 2024-02-16T19:12:56.209 INFO:teuthology.orchestra.run.smithi040.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 241 kB 00:00 2024-02-16T19:12:56.265 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-16T19:12:56.302 INFO:teuthology.orchestra.run.smithi040.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 685 kB/s | 103 kB 00:00 2024-02-16T19:12:56.335 INFO:teuthology.orchestra.run.smithi040.stdout:(13/33): python3-cryptography-36.0.1-4.el9.x86_ 5.5 MB/s | 1.1 MB 00:00 2024-02-16T19:12:56.360 INFO:teuthology.orchestra.run.smithi040.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 825 kB/s | 124 kB 00:00 2024-02-16T19:12:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 113 kB 00:00 2024-02-16T19:12:56.411 INFO:teuthology.orchestra.run.smithi040.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-16T19:12:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 88 kB/s | 43 kB 00:00 2024-02-16T19:12:56.853 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:04 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:12:56.936 INFO:teuthology.orchestra.run.smithi040.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-02-16T19:12:57.018 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================== 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================== 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 7.4 M 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-16T19:12:57.019 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================== 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout:Install 12 Packages 2024-02-16T19:12:57.020 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:12:57.021 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 37 M 2024-02-16T19:12:57.021 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 209 M 2024-02-16T19:12:57.021 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:12:57.045 INFO:teuthology.orchestra.run.smithi040.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-02-16T19:12:57.080 INFO:teuthology.orchestra.run.smithi040.stdout:[MIRROR] python3-jaraco-collections-3.0.0-8.el9.noarch.rpm: Status code: 500 for https://mirror.servaxnet.com/epel/9/Everything/x86_64/Packages/p/python3-jaraco-collections-3.0.0-8.el9.noarch.rpm (IP: 162.255.226.75) 2024-02-16T19:12:57.208 INFO:teuthology.orchestra.run.smithi163.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 232 kB/s | 32 kB 00:00 2024-02-16T19:12:57.250 INFO:teuthology.orchestra.run.smithi163.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 424 kB/s | 17 kB 00:00 2024-02-16T19:12:57.295 INFO:teuthology.orchestra.run.smithi040.stdout:(20/33): python3-jaraco-collections-3.0.0-8.el9 93 kB/s | 23 kB 00:00 2024-02-16T19:12:57.300 INFO:teuthology.orchestra.run.smithi163.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-02-16T19:12:57.345 INFO:teuthology.orchestra.run.smithi040.stdout:(21/33): python3-jaraco-functools-3.5.0-2.el9.n 390 kB/s | 19 kB 00:00 2024-02-16T19:12:57.392 INFO:teuthology.orchestra.run.smithi163.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.3 MB/s | 3.0 MB 00:00 2024-02-16T19:12:57.396 INFO:teuthology.orchestra.run.smithi040.stdout:(22/33): python3-jaraco-text-3.2.0-6.el9.noarch 394 kB/s | 20 kB 00:00 2024-02-16T19:12:57.421 INFO:teuthology.orchestra.run.smithi040.stdout:(23/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 165 kB/s | 172 kB 00:01 2024-02-16T19:12:57.471 INFO:teuthology.orchestra.run.smithi040.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 618 kB/s | 46 kB 00:00 2024-02-16T19:12:57.587 INFO:teuthology.orchestra.run.smithi040.stdout:(25/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-02-16T19:12:57.613 INFO:teuthology.orchestra.run.smithi040.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 410 kB/s | 79 kB 00:00 2024-02-16T19:12:57.639 INFO:teuthology.orchestra.run.smithi040.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 320 kB/s | 16 kB 00:00 2024-02-16T19:12:57.664 INFO:teuthology.orchestra.run.smithi040.stdout:(28/33): python3-cherrypy-18.6.1-2.el9.noarch.r 286 kB/s | 358 kB 00:01 2024-02-16T19:12:57.691 INFO:teuthology.orchestra.run.smithi040.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-02-16T19:12:57.714 INFO:teuthology.orchestra.run.smithi040.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 475 kB/s | 36 kB 00:00 2024-02-16T19:12:57.759 INFO:teuthology.orchestra.run.smithi163.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 10 MB/s | 4.7 MB 00:00 2024-02-16T19:12:57.764 INFO:teuthology.orchestra.run.smithi040.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.6 MB/s | 427 kB 00:00 2024-02-16T19:12:57.790 INFO:teuthology.orchestra.run.smithi040.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 266 kB/s | 20 kB 00:00 2024-02-16T19:12:57.840 INFO:teuthology.orchestra.run.smithi040.stdout:(33/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-02-16T19:12:57.846 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:57.847 INFO:teuthology.orchestra.run.smithi040.stdout:Total 7.2 MB/s | 27 MB 00:03 2024-02-16T19:12:57.847 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:12:57.859 INFO:teuthology.orchestra.run.smithi163.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 4.8 MB 00:00 2024-02-16T19:12:57.884 INFO:teuthology.orchestra.run.smithi163.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 217 kB 00:00 2024-02-16T19:12:57.889 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:12:57.889 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:12:58.185 INFO:teuthology.orchestra.run.smithi163.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 802 kB 00:00 2024-02-16T19:12:58.256 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:12:58.256 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:12:58.608 INFO:teuthology.orchestra.run.smithi163.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-02-16T19:12:58.635 INFO:teuthology.orchestra.run.smithi163.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 432 kB/s | 194 kB 00:00 2024-02-16T19:12:58.902 INFO:teuthology.orchestra.run.smithi163.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 836 kB/s | 245 kB 00:00 2024-02-16T19:12:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:12:59.593 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-16T19:12:59.626 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-16T19:12:59.659 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-16T19:12:59.756 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-16T19:12:59.832 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-16T19:12:59.879 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-16T19:12:59.886 INFO:teuthology.orchestra.run.smithi163.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-5 2.6 MB/s | 7.4 MB 00:02 2024-02-16T19:12:59.889 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:12:59.889 INFO:teuthology.orchestra.run.smithi163.stdout:Total 13 MB/s | 37 MB 00:02 2024-02-16T19:12:59.912 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:12:59.938 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:12:59.938 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:12:59.954 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-16T19:13:00.005 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-16T19:13:00.042 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-16T19:13:00.092 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-16T19:13:00.156 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-16T19:13:00.356 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-16T19:13:00.361 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:00.361 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:00.405 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-16T19:13:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-16T19:13:00.478 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-16T19:13:00.508 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-16T19:13:00.587 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-16T19:13:00.635 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T19:13:00.668 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-16T19:13:00.710 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-16T19:13:00.752 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-16T19:13:00.786 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-16T19:13:00.879 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-16T19:13:00.927 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-16T19:13:01.010 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-16T19:13:01.072 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:01.142 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-16T19:13:01.151 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-16T19:13:01.173 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-16T19:13:01.267 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-16T19:13:01.300 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T19:13:01.310 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-16T19:13:01.456 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.n 28/33 2024-02-16T19:13:01.480 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 29/33 2024-02-16T19:13:01.490 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T19:13:01.505 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 29/33 2024-02-16T19:13:01.505 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:13:01.505 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-16T19:13:01.506 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T19:13:01.506 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-16T19:13:01.506 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:01.526 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T19:13:01.603 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-16T19:13:02.297 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-16T19:13:02.407 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-16T19:13:02.931 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 30/33 2024-02-16T19:13:02.957 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 30/33 2024-02-16T19:13:02.957 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:13:02.957 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-16T19:13:02.957 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T19:13:02.957 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-16T19:13:02.957 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:03.357 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 31/33 2024-02-16T19:13:03.380 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 31/33 2024-02-16T19:13:03.380 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:13:03.380 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-16T19:13:03.380 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T19:13:03.380 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-16T19:13:03.380 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:03.599 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 32/33 2024-02-16T19:13:03.622 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 32/33 2024-02-16T19:13:03.622 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:13:03.622 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-16T19:13:03.622 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T19:13:03.622 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-16T19:13:03.622 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:03.671 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-16T19:13:03.690 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:17.2.7-513.gb88fae66.el9.x86_64 33/33 2024-02-16T19:13:04.715 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-16T19:13:04.752 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 12/12 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-2:17.2.7-513.gb88fae66.el9.x86_64 33/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:17.2.7-513.gb88fae66.el9.x86_64 1/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 2/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 3/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 4/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 5/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.n 6/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-16T19:13:05.379 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-16T19:13:05.380 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-16T19:13:05.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-16T19:13:05.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-16T19:13:05.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-16T19:13:05.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-16T19:13:05.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-16T19:13:05.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-16T19:13:05.381 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-16T19:13:06.211 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-16T19:13:06.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:06.211 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:06.211 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-16T19:13:06.212 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-16T19:13:06.213 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-16T19:13:06.214 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-16T19:13:06.214 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-16T19:13:06.214 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-16T19:13:06.214 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-16T19:13:06.214 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-16T19:13:06.214 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-16T19:13:06.214 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:06.214 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:06.538 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-base 2024-02-16T19:13:07.120 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:56 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:07.228 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 12/12 2024-02-16T19:13:07.228 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 1/12 2024-02-16T19:13:07.228 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-16T19:13:07.229 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-16T19:13:07.237 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-base-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:07.291 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:07.292 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:07.292 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:07.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install cephadm 2024-02-16T19:13:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-16T19:13:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:07.748 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-16T19:13:07.748 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:07.749 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:07.942 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:56 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:08.089 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-rook 2024-02-16T19:13:08.113 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:08.115 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:13:08.115 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:08.115 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 80 k 2024-02-16T19:13:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:08.116 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:08.116 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:08.116 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-02-16T19:13:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:08.116 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 80 k 2024-02-16T19:13:08.116 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 364 k 2024-02-16T19:13:08.116 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:08.331 INFO:teuthology.orchestra.run.smithi040.stdout:cephadm-17.2.7-513.gb88fae66.el9.noarch.rpm 373 kB/s | 80 kB 00:00 2024-02-16T19:13:08.332 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:08.332 INFO:teuthology.orchestra.run.smithi040.stdout:Total 370 kB/s | 80 kB 00:00 2024-02-16T19:13:08.332 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:08.337 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:08.338 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:08.339 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:08.340 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:08.692 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:16 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:08.871 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 47 k 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-16T19:13:08.872 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout:Install 13 Packages 2024-02-16T19:13:08.873 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:08.874 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 2.0 M 2024-02-16T19:13:08.874 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 26 M 2024-02-16T19:13:08.874 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:13:08.914 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T19:13:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T19:13:09.071 INFO:teuthology.orchestra.run.smithi163.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 122 kB/s | 18 kB 00:00 2024-02-16T19:13:09.096 INFO:teuthology.orchestra.run.smithi163.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 140 kB/s | 24 kB 00:00 2024-02-16T19:13:09.138 INFO:teuthology.orchestra.run.smithi163.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-02-16T19:13:09.163 INFO:teuthology.orchestra.run.smithi163.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-02-16T19:13:09.188 INFO:teuthology.orchestra.run.smithi163.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-16T19:13:09.214 INFO:teuthology.orchestra.run.smithi163.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 854 kB/s | 43 kB 00:00 2024-02-16T19:13:09.247 INFO:teuthology.orchestra.run.smithi163.stdout:(7/13): ceph-mgr-rook-17.2.7-513.gb88fae66.el9. 146 kB/s | 47 kB 00:00 2024-02-16T19:13:09.273 INFO:teuthology.orchestra.run.smithi163.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 255 kB/s | 15 kB 00:00 2024-02-16T19:13:09.298 INFO:teuthology.orchestra.run.smithi163.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 295 kB/s | 32 kB 00:00 2024-02-16T19:13:09.349 INFO:teuthology.orchestra.run.smithi163.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-02-16T19:13:09.374 INFO:teuthology.orchestra.run.smithi163.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 1.6 MB/s | 203 kB 00:00 2024-02-16T19:13:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T19:13:09.407 INFO:teuthology.orchestra.run.smithi163.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-02-16T19:13:09.566 INFO:teuthology.orchestra.run.smithi163.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.5 MB/s | 1.0 MB 00:00 2024-02-16T19:13:09.568 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:09.568 INFO:teuthology.orchestra.run.smithi163.stdout:Total 2.9 MB/s | 2.0 MB 00:00 2024-02-16T19:13:09.568 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:13:09.581 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:13:09.581 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:13:09.683 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:09.683 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:09.690 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/1 2024-02-16T19:13:09.690 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:09.690 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:09.690 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:09.690 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:09.690 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:09.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-immutable-object-cache 2024-02-16T19:13:10.222 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:10.389 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-16T19:13:10.442 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-16T19:13:10.464 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:59 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:10.498 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-16T19:13:10.535 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-16T19:13:10.576 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-16T19:13:10.635 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:10.635 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:10.635 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-513.gb88fae66.el9 ceph 137 k 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 137 k 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 379 k 2024-02-16T19:13:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:10.670 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-16T19:13:10.770 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-16T19:13:10.820 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-16T19:13:10.964 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-immutable-object-cache-17.2.7-513.gb88fae6 418 kB/s | 137 kB 00:00 2024-02-16T19:13:10.965 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:10.965 INFO:teuthology.orchestra.run.smithi040.stdout:Total 416 kB/s | 137 kB 00:00 2024-02-16T19:13:10.965 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:10.973 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:10.973 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:11.027 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:11.028 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:11.175 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T19:13:11.256 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T19:13:11.256 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:13:11.256 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-16T19:13:11.256 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:11.547 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-16T19:13:11.601 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-16T19:13:11.634 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-16T19:13:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-16T19:13:11.704 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 13/13 2024-02-16T19:13:12.264 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.e 1/1 2024-02-16T19:13:12.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:12.264 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:12.264 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:12.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:12.264 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:12.494 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr 2024-02-16T19:13:12.549 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 13/13 2024-02-16T19:13:12.549 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 1/13 2024-02-16T19:13:12.549 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-16T19:13:12.549 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-16T19:13:12.549 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-16T19:13:12.549 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-16T19:13:12.549 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-16T19:13:12.550 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-16T19:13:12.550 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-16T19:13:12.550 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-16T19:13:12.550 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-16T19:13:12.550 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-16T19:13:12.550 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-16T19:13:12.919 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-16T19:13:12.919 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:12.919 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:12.919 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:12.919 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-16T19:13:12.919 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:12.920 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:13.082 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:02 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:13.155 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-mgr-cephadm 2024-02-16T19:13:13.200 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-mgr-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:13.254 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:13.255 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:13.255 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:13.313 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-dashboard 2024-02-16T19:13:13.734 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:21 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:13.892 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:02 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:13.899 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 121 k 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-16T19:13:13.900 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-16T19:13:13.901 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-16T19:13:13.901 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-16T19:13:13.901 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:13.901 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:13:13.901 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-02-16T19:13:13.901 INFO:teuthology.orchestra.run.smithi163.stdout:Install 6 Packages 2024-02-16T19:13:13.901 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:13.902 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 6.8 M 2024-02-16T19:13:13.902 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 33 M 2024-02-16T19:13:13.902 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:13:14.066 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:14.066 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:14.066 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:13:14.066 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:14.066 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 1.7 M 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 23 k 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 15 k 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout:Install 7 Packages 2024-02-16T19:13:14.067 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:14.068 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.0 M 2024-02-16T19:13:14.068 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 8.1 M 2024-02-16T19:13:14.068 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:14.269 INFO:teuthology.orchestra.run.smithi163.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 227 kB 00:00 2024-02-16T19:13:14.470 INFO:teuthology.orchestra.run.smithi163.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-02-16T19:13:14.537 INFO:teuthology.orchestra.run.smithi163.stdout:(3/6): ceph-mgr-cephadm-17.2.7-513.gb88fae66.el 277 kB/s | 121 kB 00:00 2024-02-16T19:13:14.602 INFO:teuthology.orchestra.run.smithi040.stdout:(1/7): ceph-grafana-dashboards-17.2.7-513.gb88f 122 kB/s | 23 kB 00:00 2024-02-16T19:13:14.632 INFO:teuthology.orchestra.run.smithi163.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 357 kB/s | 58 kB 00:00 2024-02-16T19:13:14.662 INFO:teuthology.orchestra.run.smithi163.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.4 MB/s | 548 kB 00:00 2024-02-16T19:13:14.679 INFO:teuthology.orchestra.run.smithi163.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 358 kB/s | 51 kB 00:00 2024-02-16T19:13:14.681 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:14.681 INFO:teuthology.orchestra.run.smithi163.stdout:Total 8.7 MB/s | 6.8 MB 00:00 2024-02-16T19:13:14.681 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:13:14.690 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:13:14.691 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:13:14.727 INFO:teuthology.orchestra.run.smithi040.stdout:(2/7): ceph-prometheus-alerts-17.2.7-513.gb88fa 46 kB/s | 15 kB 00:00 2024-02-16T19:13:14.782 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:14.783 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:14.927 INFO:teuthology.orchestra.run.smithi040.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 28 kB/s | 9.0 kB 00:00 2024-02-16T19:13:15.061 INFO:teuthology.orchestra.run.smithi040.stdout:(4/7): ceph-mgr-dashboard-17.2.7-513.gb88fae66. 2.6 MB/s | 1.7 MB 00:00 2024-02-16T19:13:15.079 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:15.103 INFO:teuthology.orchestra.run.smithi040.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 176 kB/s | 31 kB 00:00 2024-02-16T19:13:15.153 INFO:teuthology.orchestra.run.smithi040.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 96 kB/s | 41 kB 00:00 2024-02-16T19:13:15.244 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-16T19:13:15.289 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-16T19:13:15.658 INFO:teuthology.orchestra.run.smithi040.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 316 kB/s | 188 kB 00:00 2024-02-16T19:13:15.662 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:15.662 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2024-02-16T19:13:15.663 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:15.672 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:15.672 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:15.745 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:15.745 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:15.809 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-16T19:13:15.883 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-16T19:13:15.938 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-16T19:13:15.968 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 6/6 2024-02-16T19:13:16.175 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:16.229 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-16T19:13:16.264 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-16T19:13:16.311 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-16T19:13:16.342 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-16T19:13:16.362 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.no 5/7 2024-02-16T19:13:16.596 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.n 6/7 2024-02-16T19:13:16.632 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 7/7 2024-02-16T19:13:16.777 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 6/6 2024-02-16T19:13:16.777 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/6 2024-02-16T19:13:16.777 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-16T19:13:16.777 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-16T19:13:16.777 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-16T19:13:16.777 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-16T19:13:17.101 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-16T19:13:17.101 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:17.101 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:17.101 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:17.101 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-16T19:13:17.102 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-16T19:13:17.102 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-16T19:13:17.102 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-16T19:13:17.102 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-16T19:13:17.102 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:17.102 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 7/7 2024-02-16T19:13:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.n 1/7 2024-02-16T19:13:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 2/7 2024-02-16T19:13:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.no 3/7 2024-02-16T19:13:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-16T19:13:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-16T19:13:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-16T19:13:17.372 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-fuse 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:17.608 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:17.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-16T19:13:17.950 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:25 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:18.117 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:18.117 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:18.117 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 812 k 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout:Install 2 Packages 2024-02-16T19:13:18.118 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:18.119 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 890 k 2024-02-16T19:13:18.119 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 2.6 M 2024-02-16T19:13:18.119 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:13:18.232 INFO:teuthology.orchestra.run.smithi163.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 698 kB/s | 79 kB 00:00 2024-02-16T19:13:18.421 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:07 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:18.595 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:=========================================================================================== 2024-02-16T19:13:18.596 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:13:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:=========================================================================================== 2024-02-16T19:13:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 7.4 M 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-16T19:13:18.597 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:=========================================================================================== 2024-02-16T19:13:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:Install 12 Packages 2024-02-16T19:13:18.598 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 37 M 2024-02-16T19:13:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 209 M 2024-02-16T19:13:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:18.793 INFO:teuthology.orchestra.run.smithi040.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 210 kB/s | 32 kB 00:00 2024-02-16T19:13:18.835 INFO:teuthology.orchestra.run.smithi040.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 414 kB/s | 17 kB 00:00 2024-02-16T19:13:18.877 INFO:teuthology.orchestra.run.smithi040.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 842 kB/s | 34 kB 00:00 2024-02-16T19:13:19.144 INFO:teuthology.orchestra.run.smithi040.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2024-02-16T19:13:19.182 INFO:teuthology.orchestra.run.smithi163.stdout:(2/2): ceph-fuse-17.2.7-513.gb88fae66.el9.x86_6 764 kB/s | 812 kB 00:01 2024-02-16T19:13:19.183 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:19.184 INFO:teuthology.orchestra.run.smithi163.stdout:Total 836 kB/s | 890 kB 00:01 2024-02-16T19:13:19.184 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:13:19.205 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:13:19.205 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:13:19.281 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:19.281 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:19.303 INFO:teuthology.orchestra.run.smithi040.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-02-16T19:13:19.370 INFO:teuthology.orchestra.run.smithi040.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.1 MB/s | 3.0 MB 00:00 2024-02-16T19:13:19.395 INFO:teuthology.orchestra.run.smithi040.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 217 kB 00:00 2024-02-16T19:13:19.459 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:19.585 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-16T19:13:19.612 INFO:teuthology.orchestra.run.smithi040.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 802 kB 00:00 2024-02-16T19:13:19.675 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2/2 2024-02-16T19:13:19.838 INFO:teuthology.orchestra.run.smithi040.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-02-16T19:13:19.862 INFO:teuthology.orchestra.run.smithi040.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 783 kB/s | 194 kB 00:00 2024-02-16T19:13:20.137 INFO:teuthology.orchestra.run.smithi040.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 819 kB/s | 245 kB 00:00 2024-02-16T19:13:20.417 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2/2 2024-02-16T19:13:20.417 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/2 2024-02-16T19:13:20.729 INFO:teuthology.orchestra.run.smithi040.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-5 3.5 MB/s | 7.4 MB 00:02 2024-02-16T19:13:20.732 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:20.732 INFO:teuthology.orchestra.run.smithi040.stdout:Total 17 MB/s | 37 MB 00:02 2024-02-16T19:13:20.733 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-16T19:13:20.733 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:20.733 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:20.733 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-16T19:13:20.733 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:20.733 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:20.753 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:20.779 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:20.779 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:21.002 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install librados-devel 2024-02-16T19:13:21.254 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:21.254 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:21.594 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:29 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:21.762 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repo Size 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:17.2.7-513.gb88fae66.el9 ceph 126 k 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2024-02-16T19:13:21.763 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:21.764 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 126 k 2024-02-16T19:13:21.764 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 448 k 2024-02-16T19:13:21.764 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:13:21.975 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:21.988 INFO:teuthology.orchestra.run.smithi163.stdout:librados-devel-17.2.7-513.gb88fae66.el9.x86_64. 564 kB/s | 126 kB 00:00 2024-02-16T19:13:21.988 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:21.989 INFO:teuthology.orchestra.run.smithi163.stdout:Total 560 kB/s | 126 kB 00:00 2024-02-16T19:13:21.989 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:13:21.994 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:13:21.994 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:13:22.049 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:22.050 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:22.052 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-16T19:13:22.084 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-16T19:13:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-16T19:13:22.205 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:22.229 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-16T19:13:22.303 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:22.417 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T19:13:22.452 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T19:13:22.531 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-16T19:13:22.786 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:23.112 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:23.112 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:23.112 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:23.112 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:23.113 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:23.113 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:23.227 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-16T19:13:23.335 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-16T19:13:23.351 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install libcephfs2 2024-02-16T19:13:23.942 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:31 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:24.059 INFO:teuthology.orchestra.run.smithi163.stdout:Package libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:24.111 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:24.112 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:13:24.112 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:24.178 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install libcephfs-devel 2024-02-16T19:13:24.649 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-16T19:13:24.760 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:32 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:24.924 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:24.926 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:24.926 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repo Size 2024-02-16T19:13:24.926 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:24.926 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:13:24.926 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:17.2.7-513.gb88fae66.el9 ceph 25 k 2024-02-16T19:13:24.927 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:24.927 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:13:24.927 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:24.927 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2024-02-16T19:13:24.927 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:24.927 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 25 k 2024-02-16T19:13:24.927 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 110 k 2024-02-16T19:13:24.927 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:13:25.178 INFO:teuthology.orchestra.run.smithi163.stdout:libcephfs-devel-17.2.7-513.gb88fae66.el9.x86_64 99 kB/s | 25 kB 00:00 2024-02-16T19:13:25.179 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:25.179 INFO:teuthology.orchestra.run.smithi163.stdout:Total 98 kB/s | 25 kB 00:00 2024-02-16T19:13:25.179 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:13:25.182 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:13:25.183 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:13:25.184 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:25.184 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:25.268 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:25.385 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:25.723 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-16T19:13:25.752 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 12/12 2024-02-16T19:13:25.757 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:26.030 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:26.030 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:26.030 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:26.030 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:26.030 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:26.030 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:26.193 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install librados2 2024-02-16T19:13:26.777 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:34 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:26.891 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:26.943 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:26.944 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:13:26.944 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:27.019 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install librbd1 2024-02-16T19:13:27.599 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:35 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:27.714 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:27.765 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:27.766 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:13:27.767 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:27.836 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install python3-rados 2024-02-16T19:13:28.369 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 12/12 2024-02-16T19:13:28.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae 1/12 2024-02-16T19:13:28.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-16T19:13:28.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-16T19:13:28.418 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:36 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:28.532 INFO:teuthology.orchestra.run.smithi163.stdout:Package python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:28.585 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:28.586 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:13:28.586 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:28.654 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install python3-rgw 2024-02-16T19:13:29.001 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-16T19:13:29.001 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:29.001 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:29.001 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:29.002 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:29.235 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:37 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:29.333 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-rook 2024-02-16T19:13:29.350 INFO:teuthology.orchestra.run.smithi163.stdout:Package python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:29.401 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:29.402 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:13:29.402 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:29.486 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install python3-cephfs 2024-02-16T19:13:29.920 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:18 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:30.066 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:38 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:30.095 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:30.096 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 47 k 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-16T19:13:30.097 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout:Install 13 Packages 2024-02-16T19:13:30.098 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:30.099 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.0 M 2024-02-16T19:13:30.099 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 26 M 2024-02-16T19:13:30.099 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:30.180 INFO:teuthology.orchestra.run.smithi163.stdout:Package python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:30.232 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:30.233 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:13:30.233 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:30.305 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install python3-rbd 2024-02-16T19:13:30.370 INFO:teuthology.orchestra.run.smithi040.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 198 kB/s | 24 kB 00:00 2024-02-16T19:13:30.395 INFO:teuthology.orchestra.run.smithi040.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 121 kB/s | 18 kB 00:00 2024-02-16T19:13:30.445 INFO:teuthology.orchestra.run.smithi040.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-02-16T19:13:30.471 INFO:teuthology.orchestra.run.smithi040.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-16T19:13:30.496 INFO:teuthology.orchestra.run.smithi040.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-16T19:13:30.521 INFO:teuthology.orchestra.run.smithi040.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 855 kB/s | 43 kB 00:00 2024-02-16T19:13:30.547 INFO:teuthology.orchestra.run.smithi040.stdout:(7/13): ceph-mgr-rook-17.2.7-513.gb88fae66.el9. 158 kB/s | 47 kB 00:00 2024-02-16T19:13:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:38 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:30.997 INFO:teuthology.orchestra.run.smithi163.stdout:Package python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:30.997 INFO:teuthology.orchestra.run.smithi040.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 64 kB/s | 32 kB 00:00 2024-02-16T19:13:31.022 INFO:teuthology.orchestra.run.smithi040.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 30 kB/s | 15 kB 00:00 2024-02-16T19:13:31.048 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:31.049 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2024-02-16T19:13:31.049 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:31.122 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install rbd-fuse 2024-02-16T19:13:31.230 INFO:teuthology.orchestra.run.smithi040.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 284 kB/s | 59 kB 00:00 2024-02-16T19:13:31.306 INFO:teuthology.orchestra.run.smithi040.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 267 kB/s | 203 kB 00:00 2024-02-16T19:13:31.431 INFO:teuthology.orchestra.run.smithi040.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 448 kB/s | 90 kB 00:00 2024-02-16T19:13:31.699 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:39 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:31.867 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout: Package Architecture Version Repository Size 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 81 k 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 81 k 2024-02-16T19:13:31.868 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 222 k 2024-02-16T19:13:31.869 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:13:32.098 INFO:teuthology.orchestra.run.smithi163.stdout:rbd-fuse-17.2.7-513.gb88fae66.el9.x86_64.rpm 354 kB/s | 81 kB 00:00 2024-02-16T19:13:32.099 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:32.099 INFO:teuthology.orchestra.run.smithi163.stdout:Total 353 kB/s | 81 kB 00:00 2024-02-16T19:13:32.099 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:13:32.106 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:13:32.106 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:13:32.160 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:32.161 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:32.299 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:32.413 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:32.537 INFO:teuthology.orchestra.run.smithi040.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 679 kB/s | 1.0 MB 00:01 2024-02-16T19:13:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:Total 856 kB/s | 2.0 MB 00:02 2024-02-16T19:13:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:32.559 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:32.559 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:32.669 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:32.671 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:32.837 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:33.128 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:33.128 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:33.128 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:33.128 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:33.128 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:33.128 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:33.213 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:33.387 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install rbd-mirror 2024-02-16T19:13:33.396 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-16T19:13:33.451 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-16T19:13:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-16T19:13:33.553 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-16T19:13:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-16T19:13:33.716 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-16T19:13:33.825 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-16T19:13:33.878 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-16T19:13:33.974 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:41 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:34.145 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repo Size 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.9 M 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2024-02-16T19:13:34.146 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:34.147 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 2.9 M 2024-02-16T19:13:34.147 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 12 M 2024-02-16T19:13:34.147 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:13:34.646 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-16T19:13:34.690 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-16T19:13:34.724 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-16T19:13:34.770 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-16T19:13:34.794 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 13/13 2024-02-16T19:13:35.253 INFO:teuthology.orchestra.run.smithi163.stdout:rbd-mirror-17.2.7-513.gb88fae66.el9.x86_64.rpm 2.7 MB/s | 2.9 MB 00:01 2024-02-16T19:13:35.253 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:35.254 INFO:teuthology.orchestra.run.smithi163.stdout:Total 2.7 MB/s | 2.9 MB 00:01 2024-02-16T19:13:35.254 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:13:35.263 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:13:35.263 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:13:35.337 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:35.337 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:35.629 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 13/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 1/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-16T19:13:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-16T19:13:35.749 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:35.803 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:35.827 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:35.827 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:13:35.827 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-16T19:13:35.827 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T19:13:35.828 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T19:13:35.828 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-16T19:13:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:36.025 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-16T19:13:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-16T19:13:36.025 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:36.026 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:36.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-cephadm 2024-02-16T19:13:36.808 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:25 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:36.944 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:36.944 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:36.945 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:36.945 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:36.945 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:36.945 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:36.982 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:36.983 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:17.2.7-513.gb88fae66.el9 ceph-noarch 121 k 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-02-16T19:13:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:Install 6 Packages 2024-02-16T19:13:36.985 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 6.8 M 2024-02-16T19:13:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 33 M 2024-02-16T19:13:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:37.201 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install rbd-nbd 2024-02-16T19:13:37.307 INFO:teuthology.orchestra.run.smithi040.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 227 kB 00:00 2024-02-16T19:13:37.508 INFO:teuthology.orchestra.run.smithi040.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-02-16T19:13:37.566 INFO:teuthology.orchestra.run.smithi040.stdout:(3/6): ceph-mgr-cephadm-17.2.7-513.gb88fae66.el 288 kB/s | 121 kB 00:00 2024-02-16T19:13:37.797 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:45 ago on Fri 16 Feb 2024 07:11:52 PM UTC. 2024-02-16T19:13:37.850 INFO:teuthology.orchestra.run.smithi040.stdout:(4/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-02-16T19:13:37.875 INFO:teuthology.orchestra.run.smithi040.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 158 kB/s | 58 kB 00:00 2024-02-16T19:13:37.900 INFO:teuthology.orchestra.run.smithi040.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 165 kB/s | 51 kB 00:00 2024-02-16T19:13:37.904 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:37.904 INFO:teuthology.orchestra.run.smithi040.stdout:Total 7.3 MB/s | 6.8 MB 00:00 2024-02-16T19:13:37.904 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:37.914 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:37.914 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:37.963 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-02-16T19:13:37.964 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:37.964 INFO:teuthology.orchestra.run.smithi163.stdout: Package Architecture Version Repository Size 2024-02-16T19:13:37.964 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:37.964 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 165 k 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout:Install 1 Package 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 165 k 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 477 k 2024-02-16T19:13:37.965 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-02-16T19:13:38.007 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:38.009 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:38.199 INFO:teuthology.orchestra.run.smithi163.stdout:rbd-nbd-17.2.7-513.gb88fae66.el9.x86_64.rpm 705 kB/s | 165 kB 00:00 2024-02-16T19:13:38.199 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:38.200 INFO:teuthology.orchestra.run.smithi163.stdout:Total 700 kB/s | 165 kB 00:00 2024-02-16T19:13:38.200 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-02-16T19:13:38.207 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-02-16T19:13:38.207 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-02-16T19:13:38.264 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-02-16T19:13:38.265 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-02-16T19:13:38.311 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:38.409 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-02-16T19:13:38.488 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-16T19:13:38.521 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:38.536 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-16T19:13:38.975 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:39.084 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-16T19:13:39.167 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-16T19:13:39.228 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-16T19:13:39.256 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 6/6 2024-02-16T19:13:39.301 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:39.301 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:39.301 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-02-16T19:13:39.301 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:39.301 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:13:39.301 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-02-16T19:13:39.527 DEBUG:teuthology.parallel:result is None 2024-02-16T19:13:40.062 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 6/6 2024-02-16T19:13:40.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 1/6 2024-02-16T19:13:40.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-16T19:13:40.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-16T19:13:40.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-16T19:13:40.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-16T19:13:40.397 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-16T19:13:40.397 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:17.2.7-513.gb88fae66.el9.noarch 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:40.398 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:40.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-fuse 2024-02-16T19:13:41.228 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:30 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:41.402 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:41.402 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:41.402 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 812 k 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout:Install 2 Packages 2024-02-16T19:13:41.403 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:41.404 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 890 k 2024-02-16T19:13:41.404 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 2.6 M 2024-02-16T19:13:41.404 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:41.527 INFO:teuthology.orchestra.run.smithi040.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 639 kB/s | 79 kB 00:00 2024-02-16T19:13:41.977 INFO:teuthology.orchestra.run.smithi040.stdout:(2/2): ceph-fuse-17.2.7-513.gb88fae66.el9.x86_6 1.4 MB/s | 812 kB 00:00 2024-02-16T19:13:41.979 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:41.979 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.5 MB/s | 890 kB 00:00 2024-02-16T19:13:41.979 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:42.001 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:42.001 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:42.080 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:42.080 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:42.265 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:42.402 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-16T19:13:42.471 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2/2 2024-02-16T19:13:43.203 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2/2 2024-02-16T19:13:43.203 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/2 2024-02-16T19:13:43.529 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-16T19:13:43.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:43.529 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:43.529 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-16T19:13:43.529 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:43.529 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:43.765 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados-devel 2024-02-16T19:13:44.353 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:33 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:44.526 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:44.527 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:44.527 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-02-16T19:13:44.527 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:17.2.7-513.gb88fae66.el9 ceph 126 k 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 126 k 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 448 k 2024-02-16T19:13:44.528 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:44.856 INFO:teuthology.orchestra.run.smithi040.stdout:librados-devel-17.2.7-513.gb88fae66.el9.x86_64. 385 kB/s | 126 kB 00:00 2024-02-16T19:13:44.857 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:44.857 INFO:teuthology.orchestra.run.smithi040.stdout:Total 383 kB/s | 126 kB 00:00 2024-02-16T19:13:44.857 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:44.863 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:44.863 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:44.921 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:44.921 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:45.079 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:45.604 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:45.879 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:45.880 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:45.880 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:45.880 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:46.117 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs2 2024-02-16T19:13:46.704 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:35 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:46.822 INFO:teuthology.orchestra.run.smithi040.stdout:Package libcephfs2-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:46.877 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:46.878 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:46.878 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:46.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs-devel 2024-02-16T19:13:47.530 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:36 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:47.702 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:47.702 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:17.2.7-513.gb88fae66.el9 ceph 25 k 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 25 k 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 110 k 2024-02-16T19:13:47.703 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:47.868 INFO:teuthology.orchestra.run.smithi040.stdout:libcephfs-devel-17.2.7-513.gb88fae66.el9.x86_64 151 kB/s | 25 kB 00:00 2024-02-16T19:13:47.870 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:47.870 INFO:teuthology.orchestra.run.smithi040.stdout:Total 150 kB/s | 25 kB 00:00 2024-02-16T19:13:47.870 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:47.873 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:47.873 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:47.875 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:47.875 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:47.964 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:48.095 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:48.473 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:48.744 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:48.744 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:48.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados2 2024-02-16T19:13:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:38 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:49.650 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:49.708 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:49.709 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:49.709 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:49.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librbd1 2024-02-16T19:13:50.362 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:39 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:50.481 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:50.537 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:50.538 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:50.538 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:50.597 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rados 2024-02-16T19:13:51.183 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:40 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:51.301 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rados-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:51.356 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:51.357 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:51.358 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:51.421 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rgw 2024-02-16T19:13:52.004 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:41 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:52.123 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rgw-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:52.179 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:52.180 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:52.180 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:52.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-cephfs 2024-02-16T19:13:52.830 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:41 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:52.947 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-cephfs-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:53.001 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:53.003 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:53.003 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:53.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rbd 2024-02-16T19:13:53.646 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:42 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:53.765 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rbd-2:17.2.7-513.gb88fae66.el9.x86_64 is already installed. 2024-02-16T19:13:53.820 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:53.821 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-02-16T19:13:53.821 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:53.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-fuse 2024-02-16T19:13:54.465 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:43 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:54.641 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:17.2.7-513.gb88fae66.el9 ceph 81 k 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:54.642 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-02-16T19:13:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:54.643 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 81 k 2024-02-16T19:13:54.643 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 222 k 2024-02-16T19:13:54.643 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:54.979 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-fuse-17.2.7-513.gb88fae66.el9.x86_64.rpm 241 kB/s | 81 kB 00:00 2024-02-16T19:13:54.980 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:54.980 INFO:teuthology.orchestra.run.smithi040.stdout:Total 241 kB/s | 81 kB 00:00 2024-02-16T19:13:54.980 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:54.988 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:54.988 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:55.046 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:55.046 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:55.187 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:55.289 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:55.691 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:56.051 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:56.051 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:56.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-mirror 2024-02-16T19:13:56.878 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:45 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:13:57.055 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:17.2.7-513.gb88fae66.el9 ceph 2.9 M 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:13:57.056 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-02-16T19:13:57.057 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:57.057 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.9 M 2024-02-16T19:13:57.057 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 12 M 2024-02-16T19:13:57.057 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:13:57.792 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-mirror-17.2.7-513.gb88fae66.el9.x86_64.rpm 4.0 MB/s | 2.9 MB 00:00 2024-02-16T19:13:57.792 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:13:57.793 INFO:teuthology.orchestra.run.smithi040.stdout:Total 4.0 MB/s | 2.9 MB 00:00 2024-02-16T19:13:57.793 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:13:57.802 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:13:57.803 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:13:57.876 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:13:57.877 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:13:58.305 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:13:58.359 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:58.385 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:58.385 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-16T19:13:58.385 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-16T19:13:58.385 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T19:13:58.385 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-16T19:13:58.385 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:59.558 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:13:59.558 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:59.559 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:13:59.559 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:13:59.559 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:13:59.559 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:13:59.769 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-nbd 2024-02-16T19:14:00.362 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:49 ago on Fri 16 Feb 2024 07:12:11 PM UTC. 2024-02-16T19:14:00.536 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-02-16T19:14:00.537 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:14:00.537 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-02-16T19:14:00.537 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:14:00.537 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-02-16T19:14:00.537 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:17.2.7-513.gb88fae66.el9 ceph 165 k 2024-02-16T19:14:00.538 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:14:00.538 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-02-16T19:14:00.538 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-02-16T19:14:00.538 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-02-16T19:14:00.538 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:14:00.538 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 165 k 2024-02-16T19:14:00.538 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 477 k 2024-02-16T19:14:00.538 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-02-16T19:14:00.764 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-nbd-17.2.7-513.gb88fae66.el9.x86_64.rpm 728 kB/s | 165 kB 00:00 2024-02-16T19:14:00.765 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:14:00.765 INFO:teuthology.orchestra.run.smithi040.stdout:Total 723 kB/s | 165 kB 00:00 2024-02-16T19:14:00.765 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-02-16T19:14:00.773 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-02-16T19:14:00.774 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-02-16T19:14:00.832 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-02-16T19:14:00.832 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-02-16T19:14:00.980 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-02-16T19:14:01.073 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:14:01.526 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:14:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 1/1 2024-02-16T19:14:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:14:01.867 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-02-16T19:14:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:17.2.7-513.gb88fae66.el9.x86_64 2024-02-16T19:14:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:14:01.868 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-02-16T19:14:02.102 DEBUG:teuthology.parallel:result is None 2024-02-16T19:14:02.103 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T19:14:02.103 INFO:teuthology.packaging:ref: None 2024-02-16T19:14:02.103 INFO:teuthology.packaging:tag: None 2024-02-16T19:14:02.103 INFO:teuthology.packaging:branch: quincy 2024-02-16T19:14:02.103 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:14:02.103 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-02-16T19:14:02.445 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-16T19:14:02.466 INFO:teuthology.orchestra.run.smithi040.stdout:17.2.7-513.gb88fae66.el9 2024-02-16T19:14:02.467 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-513.gb88fae66.el9 2024-02-16T19:14:02.467 INFO:teuthology.task.install:The correct ceph version 17.2.7-513.gb88fae66 is installed. 2024-02-16T19:14:02.468 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-16T19:14:02.469 INFO:teuthology.packaging:ref: None 2024-02-16T19:14:02.469 INFO:teuthology.packaging:tag: None 2024-02-16T19:14:02.469 INFO:teuthology.packaging:branch: quincy 2024-02-16T19:14:02.469 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:14:02.469 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-02-16T19:14:02.694 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-16T19:14:02.720 INFO:teuthology.orchestra.run.smithi163.stdout:17.2.7-513.gb88fae66.el9 2024-02-16T19:14:02.720 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-513.gb88fae66.el9 2024-02-16T19:14:02.720 INFO:teuthology.task.install:The correct ceph version 17.2.7-513.gb88fae66 is installed. 2024-02-16T19:14:02.722 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-16T19:14:02.722 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:14:02.722 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-16T19:14:02.751 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:14:02.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-16T19:14:02.793 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-16T19:14:02.793 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:14:02.793 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-02-16T19:14:02.820 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-16T19:14:02.886 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:14:02.886 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-02-16T19:14:02.917 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-16T19:14:02.986 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-16T19:14:02.986 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:14:02.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-16T19:14:03.013 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-16T19:14:03.079 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:14:03.079 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-16T19:14:03.108 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-16T19:14:03.178 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-16T19:14:03.179 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:14:03.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-02-16T19:14:03.204 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-16T19:14:03.271 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:14:03.271 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-02-16T19:14:03.300 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-16T19:14:03.368 INFO:teuthology.run_tasks:Running task print... 2024-02-16T19:14:03.378 INFO:teuthology.task.print:**** done install task... 2024-02-16T19:14:03.379 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-16T19:14:03.493 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-16T19:14:03.494 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-02-16T19:14:03.494 INFO:tasks.cephadm:Cluster fsid is 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:14:03.494 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-16T19:14:03.494 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-16T19:14:03.494 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi040': '172.21.15.40', 'mon.smithi163': '172.21.15.163'} 2024-02-16T19:14:03.494 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-16T19:14:03.495 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname $(hostname -s) 2024-02-16T19:14:03.521 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname $(hostname -s) 2024-02-16T19:14:03.550 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-02-16T19:14:03.550 DEBUG:teuthology.orchestra.run.smithi040:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-16T19:14:03.856 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r--. 1 ubuntu ubuntu 367151 Feb 16 19:14 /home/ubuntu/cephtest/cephadm 2024-02-16T19:14:03.857 DEBUG:teuthology.orchestra.run.smithi163:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-16T19:14:04.202 INFO:teuthology.orchestra.run.smithi163.stdout:-rw-r--r--. 1 ubuntu ubuntu 367151 Feb 16 19:14 /home/ubuntu/cephtest/cephadm 2024-02-16T19:14:04.202 DEBUG:teuthology.orchestra.run.smithi040:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-16T19:14:04.219 DEBUG:teuthology.orchestra.run.smithi163:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-16T19:14:04.245 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-02-16T19:14:04.246 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-02-16T19:14:04.261 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-02-16T19:14:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-16T19:14:05.153 INFO:teuthology.orchestra.run.smithi163.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-16T19:14:46.016 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-02-16T19:14:46.016 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_version": "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)", 2024-02-16T19:14:46.017 INFO:teuthology.orchestra.run.smithi163.stdout: "image_id": "870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0", 2024-02-16T19:14:46.017 INFO:teuthology.orchestra.run.smithi163.stdout: "repo_digests": [ 2024-02-16T19:14:46.017 INFO:teuthology.orchestra.run.smithi163.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c" 2024-02-16T19:14:46.017 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2024-02-16T19:14:46.017 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-02-16T19:14:47.001 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:14:47.001 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)", 2024-02-16T19:14:47.001 INFO:teuthology.orchestra.run.smithi040.stdout: "image_id": "870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0", 2024-02-16T19:14:47.001 INFO:teuthology.orchestra.run.smithi040.stdout: "repo_digests": [ 2024-02-16T19:14:47.001 INFO:teuthology.orchestra.run.smithi040.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c" 2024-02-16T19:14:47.001 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-02-16T19:14:47.001 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:14:47.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph 2024-02-16T19:14:47.067 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph 2024-02-16T19:14:47.102 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /etc/ceph 2024-02-16T19:14:47.134 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /etc/ceph 2024-02-16T19:14:47.173 INFO:tasks.cephadm:Writing seed config... 2024-02-16T19:14:47.175 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-02-16T19:14:47.175 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-02-16T19:14:47.175 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-16T19:14:47.175 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-16T19:14:47.175 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-16T19:14:47.176 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-16T19:14:47.176 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-16T19:14:47.176 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-16T19:14:47.176 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-16T19:14:47.176 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-16T19:14:47.176 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-16T19:14:47.177 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-16T19:14:47.177 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-16T19:14:47.177 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-16T19:14:47.177 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-16T19:14:47.177 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-16T19:14:47.177 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-16T19:14:47.177 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-16T19:14:47.178 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-16T19:14:47.178 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-16T19:14:47.178 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-16T19:14:47.178 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-16T19:14:47.178 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-16T19:14:47.178 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-16T19:14:47.178 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-16T19:14:47.179 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-16T19:14:47.179 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-16T19:14:47.179 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-16T19:14:47.179 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-16T19:14:47.179 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-16T19:14:47.179 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-16T19:14:47.179 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-16T19:14:47.179 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-16T19:14:47.180 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-16T19:14:47.180 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-16T19:14:47.180 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-16T19:14:47.180 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-16T19:14:47.180 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-16T19:14:47.180 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-16T19:14:47.180 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-16T19:14:47.182 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:14:47.182 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-16T19:14:47.197 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8c23bd6c-ccff-11ee-95ba-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-16T19:14:47.198 DEBUG:teuthology.orchestra.run.smithi040:mon.smithi040> sudo journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040.service 2024-02-16T19:14:47.240 INFO:tasks.cephadm:Bootstrapping... 2024-02-16T19:14:47.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.40 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:14:47.451 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-02-16T19:14:47.452 INFO:teuthology.orchestra.run.smithi040.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '8c23bd6c-ccff-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.40', '--skip-admin-label'] 2024-02-16T19:14:47.478 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T19:14:47.478 INFO:teuthology.orchestra.run.smithi040.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-16T19:14:47.478 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying podman|docker is present... 2024-02-16T19:14:47.502 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T19:14:47.502 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying lvm2 is present... 2024-02-16T19:14:47.502 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying time synchronization is in place... 2024-02-16T19:14:47.510 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-16T19:14:47.510 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-16T19:14:47.516 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-16T19:14:47.516 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-02-16T19:14:47.523 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout enabled 2024-02-16T19:14:47.529 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout active 2024-02-16T19:14:47.529 INFO:teuthology.orchestra.run.smithi040.stdout:Unit chronyd.service is enabled and running 2024-02-16T19:14:47.529 INFO:teuthology.orchestra.run.smithi040.stdout:Repeating the final host check... 2024-02-16T19:14:47.551 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-16T19:14:47.551 INFO:teuthology.orchestra.run.smithi040.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-16T19:14:47.552 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl is present 2024-02-16T19:14:47.552 INFO:teuthology.orchestra.run.smithi040.stdout:lvcreate is present 2024-02-16T19:14:47.558 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-16T19:14:47.558 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-16T19:14:47.564 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-16T19:14:47.564 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-02-16T19:14:47.571 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout enabled 2024-02-16T19:14:47.577 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout active 2024-02-16T19:14:47.577 INFO:teuthology.orchestra.run.smithi040.stdout:Unit chronyd.service is enabled and running 2024-02-16T19:14:47.577 INFO:teuthology.orchestra.run.smithi040.stdout:Host looks OK 2024-02-16T19:14:47.577 INFO:teuthology.orchestra.run.smithi040.stdout:Cluster fsid: 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:14:47.577 INFO:teuthology.orchestra.run.smithi040.stdout:Acquiring lock 140404843644576 on /run/cephadm/8c23bd6c-ccff-11ee-95ba-87774f69a715.lock 2024-02-16T19:14:47.577 INFO:teuthology.orchestra.run.smithi040.stdout:Lock 140404843644576 acquired on /run/cephadm/8c23bd6c-ccff-11ee-95ba-87774f69a715.lock 2024-02-16T19:14:47.578 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying IP 172.21.15.40 port 3300 ... 2024-02-16T19:14:47.578 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying IP 172.21.15.40 port 6789 ... 2024-02-16T19:14:47.578 INFO:teuthology.orchestra.run.smithi040.stdout:Base mon IP(s) is [172.21.15.40:3300, 172.21.15.40:6789], mon addrv is [v2:172.21.15.40:3300,v1:172.21.15.40:6789] 2024-02-16T19:14:47.582 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.40 metric 100 2024-02-16T19:14:47.582 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.40 metric 100 2024-02-16T19:14:47.584 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-16T19:14:47.584 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-16T19:14:47.585 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-16T19:14:47.587 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-16T19:14:47.587 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-16T19:14:47.587 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-16T19:14:47.587 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-16T19:14:47.587 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-02-16T19:14:47.587 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-16T19:14:47.588 INFO:teuthology.orchestra.run.smithi040.stdout:Mon IP `172.21.15.40` is in CIDR network `172.21.0.0/20` 2024-02-16T19:14:47.588 INFO:teuthology.orchestra.run.smithi040.stdout:Mon IP `172.21.15.40` is in CIDR network `172.21.0.0/20` 2024-02-16T19:14:47.588 INFO:teuthology.orchestra.run.smithi040.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-16T19:14:47.588 INFO:teuthology.orchestra.run.smithi040.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-16T19:14:47.589 INFO:teuthology.orchestra.run.smithi040.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-16T19:14:48.017 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0 2024-02-16T19:14:48.017 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-16T19:14:48.017 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Getting image source signatures 2024-02-16T19:14:48.017 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying blob sha256:5dfd87a714924b39df09034baf76a9bd7605cb8f0d34cc8ca301fad8be327611 2024-02-16T19:14:48.017 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-16T19:14:48.017 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying config sha256:870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0 2024-02-16T19:14:48.017 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-16T19:14:48.880 INFO:teuthology.orchestra.run.smithi040.stdout:ceph: stdout ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable) 2024-02-16T19:14:48.880 INFO:teuthology.orchestra.run.smithi040.stdout:Ceph version: ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable) 2024-02-16T19:14:48.880 INFO:teuthology.orchestra.run.smithi040.stdout:Extracting ceph user uid/gid from container image... 2024-02-16T19:14:49.811 INFO:teuthology.orchestra.run.smithi040.stdout:stat: stdout 167 167 2024-02-16T19:14:49.811 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial keys... 2024-02-16T19:14:50.717 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQAqtM9lv68XBxAAwfhGPZwl6EMr0TPjEu3ZVw== 2024-02-16T19:14:51.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQArtM9lqrrgARAAtc1KOfxyAUd1ZNqMvLYqiw== 2024-02-16T19:14:52.478 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQArtM9lH0BbNRAAwYMP32/Oz1Zrq2ckwBw2Og== 2024-02-16T19:14:52.479 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial monmap... 2024-02-16T19:14:53.393 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-16T19:14:53.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-02-16T19:14:53.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:14:53.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-16T19:14:53.394 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool for smithi040 [v2:172.21.15.40:3300,v1:172.21.15.40:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-16T19:14:53.394 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = octopus 2024-02-16T19:14:53.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: set fsid to 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:14:53.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-16T19:14:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:14:53.395 INFO:teuthology.orchestra.run.smithi040.stdout:Creating mon... 2024-02-16T19:14:54.190 INFO:teuthology.orchestra.run.smithi040.stdout:create mon.smithi040 on 2024-02-16T19:14:54.382 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-16T19:14:54.521 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-16T19:14:54.666 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715.target. 2024-02-16T19:14:54.667 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715.target. 2024-02-16T19:14:54.822 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040 2024-02-16T19:14:54.823 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040.service: Unit ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040.service not loaded. 2024-02-16T19:14:54.982 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715.target.wants/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040.service → /etc/systemd/system/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@.service. 2024-02-16T19:14:55.480 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T19:14:55.480 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2024-02-16T19:14:55.488 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T19:14:55.488 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-02-16T19:14:55.488 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2024-02-16T19:14:55.488 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to enable service . firewalld.service is not available 2024-02-16T19:14:55.488 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mon to start... 2024-02-16T19:14:55.488 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mon... 2024-02-16T19:14:56.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout cluster: 2024-02-16T19:14:56.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout id: 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:14:56.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-16T19:14:56.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:14:56.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout services: 2024-02-16T19:14:56.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi040 (age 0.587051s) 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout data: 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout pgs: 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.941+0000 7fdf684cc700 1 Processor -- start 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.941+0000 7fdf684cc700 1 -- start start 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.942+0000 7fdf684cc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.942+0000 7fdf684cc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf601030c0 con 0x7fdf60102770 2024-02-16T19:14:56.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.942+0000 7fdf66268700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.942+0000 7fdf66268700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41148/0 (socket says 172.21.15.40:41148) 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.942+0000 7fdf66268700 1 -- 172.21.15.40:0/3109222062 learned_addr learned my addr 172.21.15.40:0/3109222062 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.942+0000 7fdf66268700 1 -- 172.21.15.40:0/3109222062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60103200 con 0x7fdf60102770 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.942+0000 7fdf66268700 1 --2- 172.21.15.40:0/3109222062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e50 tx=0x7fdf50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ce1794b7a62545 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf65266700 1 -- 172.21.15.40:0/3109222062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf500092e0 con 0x7fdf60102770 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf65266700 1 -- 172.21.15.40:0/3109222062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5000ba20 con 0x7fdf60102770 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf65266700 1 -- 172.21.15.40:0/3109222062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf5000bd30 con 0x7fdf60102770 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf684cc700 1 -- 172.21.15.40:0/3109222062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 msgr2=0x7fdf60102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf684cc700 1 --2- 172.21.15.40:0/3109222062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e50 tx=0x7fdf50009840 comp rx=0 tx=0).stop 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf684cc700 1 -- 172.21.15.40:0/3109222062 shutdown_connections 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf684cc700 1 --2- 172.21.15.40:0/3109222062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60102b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf684cc700 1 -- 172.21.15.40:0/3109222062 >> 172.21.15.40:0/3109222062 conn(0x7fdf600fded0 msgr2=0x7fdf60100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf684cc700 1 -- 172.21.15.40:0/3109222062 shutdown_connections 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.943+0000 7fdf684cc700 1 -- 172.21.15.40:0/3109222062 wait complete. 2024-02-16T19:14:56.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.944+0000 7fdf684cc700 1 Processor -- start 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.944+0000 7fdf684cc700 1 -- start start 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.944+0000 7fdf684cc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60110e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.944+0000 7fdf684cc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60111370 con 0x7fdf60102770 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.944+0000 7fdf66268700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60110e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.944+0000 7fdf66268700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60110e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41156/0 (socket says 172.21.15.40:41156) 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.944+0000 7fdf66268700 1 -- 172.21.15.40:0/1431511772 learned_addr learned my addr 172.21.15.40:0/1431511772 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.944+0000 7fdf66268700 1 -- 172.21.15.40:0/1431511772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50009ae0 con 0x7fdf60102770 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.945+0000 7fdf66268700 1 --2- 172.21.15.40:0/1431511772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60110e30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf601034b0 tx=0x7fdf50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.945+0000 7fdf5effd700 1 -- 172.21.15.40:0/1431511772 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf50012b90 con 0x7fdf60102770 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.945+0000 7fdf5effd700 1 -- 172.21.15.40:0/1431511772 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50012d10 con 0x7fdf60102770 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.945+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf60111570 con 0x7fdf60102770 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.945+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf60112130 con 0x7fdf60102770 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.945+0000 7fdf5effd700 1 -- 172.21.15.40:0/1431511772 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf5001e400 con 0x7fdf60102770 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.945+0000 7fdf5effd700 1 -- 172.21.15.40:0/1431511772 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fdf50019030 con 0x7fdf60102770 2024-02-16T19:14:56.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.945+0000 7fdf5effd700 1 -- 172.21.15.40:0/1431511772 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdf5001ed50 con 0x7fdf60102770 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.946+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c0052e0 con 0x7fdf60102770 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:55.948+0000 7fdf5effd700 1 -- 172.21.15.40:0/1431511772 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fdf50017070 con 0x7fdf60102770 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.021+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fdf4c005860 con 0x7fdf60102770 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.021+0000 7fdf5effd700 1 -- 172.21.15.40:0/1431511772 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fdf5001e6d0 con 0x7fdf60102770 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.022+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 msgr2=0x7fdf60110e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.022+0000 7fdf684cc700 1 --2- 172.21.15.40:0/1431511772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60110e30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf601034b0 tx=0x7fdf50017000 comp rx=0 tx=0).stop 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.022+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 shutdown_connections 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.022+0000 7fdf684cc700 1 --2- 172.21.15.40:0/1431511772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf60102770 0x7fdf60110e30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.022+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 >> 172.21.15.40:0/1431511772 conn(0x7fdf600fded0 msgr2=0x7fdf601054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.022+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 shutdown_connections 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.022+0000 7fdf684cc700 1 -- 172.21.15.40:0/1431511772 wait complete. 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:mon is available 2024-02-16T19:14:56.623 INFO:teuthology.orchestra.run.smithi040.stdout:Assimilating anything we can from ceph.conf... 2024-02-16T19:14:57.613 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:57 smithi040 ceph-mon[28817]: from='client.? 172.21.15.40:0/1315661085' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [global] 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout fsid = 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.40:3300,v1:172.21.15.40:6789] 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-16T19:14:57.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [osd] 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.988+0000 7fba86f19700 1 Processor -- start 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.988+0000 7fba86f19700 1 -- start start 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.988+0000 7fba86f19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.988+0000 7fba86f19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80079130 con 0x7fba80078d20 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.989+0000 7fba84cb5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.989+0000 7fba84cb5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41168/0 (socket says 172.21.15.40:41168) 2024-02-16T19:14:57.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.989+0000 7fba84cb5700 1 -- 172.21.15.40:0/621145493 learned_addr learned my addr 172.21.15.40:0/621145493 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.989+0000 7fba84cb5700 1 -- 172.21.15.40:0/621145493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80077770 con 0x7fba80078d20 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.989+0000 7fba84cb5700 1 --2- 172.21.15.40:0/621145493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80077230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba78009e50 tx=0x7fba78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84a0d2eca29f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.989+0000 7fba7f7fe700 1 -- 172.21.15.40:0/621145493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba780092e0 con 0x7fba80078d20 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.989+0000 7fba7f7fe700 1 -- 172.21.15.40:0/621145493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7800ba20 con 0x7fba80078d20 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.989+0000 7fba7f7fe700 1 -- 172.21.15.40:0/621145493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba7800bd30 con 0x7fba80078d20 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.990+0000 7fba86f19700 1 -- 172.21.15.40:0/621145493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 msgr2=0x7fba80077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.990+0000 7fba86f19700 1 --2- 172.21.15.40:0/621145493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80077230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba78009e50 tx=0x7fba78009840 comp rx=0 tx=0).stop 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.990+0000 7fba86f19700 1 -- 172.21.15.40:0/621145493 shutdown_connections 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.990+0000 7fba86f19700 1 --2- 172.21.15.40:0/621145493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80077230 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.990+0000 7fba86f19700 1 -- 172.21.15.40:0/621145493 >> 172.21.15.40:0/621145493 conn(0x7fba800fded0 msgr2=0x7fba80100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.990+0000 7fba86f19700 1 -- 172.21.15.40:0/621145493 shutdown_connections 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.990+0000 7fba86f19700 1 -- 172.21.15.40:0/621145493 wait complete. 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba86f19700 1 Processor -- start 2024-02-16T19:14:57.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba86f19700 1 -- start start 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba86f19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba86f19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80113520 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba84cb5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba84cb5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41174/0 (socket says 172.21.15.40:41174) 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba84cb5700 1 -- 172.21.15.40:0/1315661085 learned_addr learned my addr 172.21.15.40:0/1315661085 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba84cb5700 1 -- 172.21.15.40:0/1315661085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78009ae0 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.991+0000 7fba84cb5700 1 --2- 172.21.15.40:0/1315661085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80112fe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba78003000 tx=0x7fba78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.992+0000 7fba7d7fa700 1 -- 172.21.15.40:0/1315661085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba78012b90 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.992+0000 7fba7d7fa700 1 -- 172.21.15.40:0/1315661085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba78012d10 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.992+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba80113720 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.992+0000 7fba7d7fa700 1 -- 172.21.15.40:0/1315661085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba7801e400 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.992+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba801143c0 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.992+0000 7fba7d7fa700 1 -- 172.21.15.40:0/1315661085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fba78019030 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.992+0000 7fba7d7fa700 1 -- 172.21.15.40:0/1315661085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fba7801eda0 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.993+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8005f5c0 con 0x7fba80078d20 2024-02-16T19:14:57.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:56.995+0000 7fba7d7fa700 1 -- 172.21.15.40:0/1315661085 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fba78017070 con 0x7fba80078d20 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.064+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fba80077c70 con 0x7fba80078d20 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.067+0000 7fba7d7fa700 1 -- 172.21.15.40:0/1315661085 <== mon.0 v2:172.21.15.40:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fba78039450 con 0x7fba80078d20 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.067+0000 7fba7d7fa700 1 -- 172.21.15.40:0/1315661085 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fba80077c70 con 0x7fba80078d20 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.068+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 msgr2=0x7fba80112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.068+0000 7fba86f19700 1 --2- 172.21.15.40:0/1315661085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80112fe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba78003000 tx=0x7fba78017000 comp rx=0 tx=0).stop 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.068+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 shutdown_connections 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.068+0000 7fba86f19700 1 --2- 172.21.15.40:0/1315661085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba80078d20 0x7fba80112fe0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.068+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 >> 172.21.15.40:0/1315661085 conn(0x7fba800fded0 msgr2=0x7fba80107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.068+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 shutdown_connections 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:57.069+0000 7fba86f19700 1 -- 172.21.15.40:0/1315661085 wait complete. 2024-02-16T19:14:57.627 INFO:teuthology.orchestra.run.smithi040.stdout:Generating new minimal ceph.conf... 2024-02-16T19:14:58.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:58 smithi040 ceph-mon[28817]: from='client.? 172.21.15.40:0/3436281215' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.031+0000 7fe75a0fe700 1 Processor -- start 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.031+0000 7fe75a0fe700 1 -- start start 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.031+0000 7fe75a0fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.031+0000 7fe75a0fe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7541014f0 con 0x7fe754100b80 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.032+0000 7fe7537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.032+0000 7fe7537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34714/0 (socket says 172.21.15.40:34714) 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.032+0000 7fe7537fe700 1 -- 172.21.15.40:0/1791267351 learned_addr learned my addr 172.21.15.40:0/1791267351 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.032+0000 7fe7537fe700 1 -- 172.21.15.40:0/1791267351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754101630 con 0x7fe754100b80 2024-02-16T19:14:58.669 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.032+0000 7fe7537fe700 1 --2- 172.21.15.40:0/1791267351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754100fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe7440098e0 tx=0x7fe744009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421b6c97247ff30e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.032+0000 7fe7527fc700 1 -- 172.21.15.40:0/1791267351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe74400b860 con 0x7fe754100b80 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.032+0000 7fe7527fc700 1 -- 172.21.15.40:0/1791267351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe74400be50 con 0x7fe754100b80 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.032+0000 7fe7527fc700 1 -- 172.21.15.40:0/1791267351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe744010380 con 0x7fe754100b80 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.033+0000 7fe75a0fe700 1 -- 172.21.15.40:0/1791267351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 msgr2=0x7fe754100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.033+0000 7fe75a0fe700 1 --2- 172.21.15.40:0/1791267351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754100fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe7440098e0 tx=0x7fe744009250 comp rx=0 tx=0).stop 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.033+0000 7fe75a0fe700 1 -- 172.21.15.40:0/1791267351 shutdown_connections 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.033+0000 7fe75a0fe700 1 --2- 172.21.15.40:0/1791267351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754100fb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.033+0000 7fe75a0fe700 1 -- 172.21.15.40:0/1791267351 >> 172.21.15.40:0/1791267351 conn(0x7fe7540fc580 msgr2=0x7fe7540fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.033+0000 7fe75a0fe700 1 -- 172.21.15.40:0/1791267351 shutdown_connections 2024-02-16T19:14:58.670 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.033+0000 7fe75a0fe700 1 -- 172.21.15.40:0/1791267351 wait complete. 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.033+0000 7fe75a0fe700 1 Processor -- start 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe75a0fe700 1 -- start start 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe75a0fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754111830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe75a0fe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754111d70 con 0x7fe754100b80 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe7537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754111830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe7537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754111830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34720/0 (socket says 172.21.15.40:34720) 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe7537fe700 1 -- 172.21.15.40:0/3436281215 learned_addr learned my addr 172.21.15.40:0/3436281215 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe7537fe700 1 -- 172.21.15.40:0/3436281215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440094f0 con 0x7fe754100b80 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe7537fe700 1 --2- 172.21.15.40:0/3436281215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754111830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe74400b440 tx=0x7fe744009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.034+0000 7fe73bfff700 1 -- 172.21.15.40:0/3436281215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7440107c0 con 0x7fe754100b80 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.035+0000 7fe73bfff700 1 -- 172.21.15.40:0/3436281215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe744010db0 con 0x7fe754100b80 2024-02-16T19:14:58.671 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.035+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe754111f70 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.035+0000 7fe73bfff700 1 -- 172.21.15.40:0/3436281215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe74401ad30 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.035+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe754112c10 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.035+0000 7fe73bfff700 1 -- 172.21.15.40:0/3436281215 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe74402e6c0 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.035+0000 7fe73bfff700 1 -- 172.21.15.40:0/3436281215 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe7440249b0 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.036+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75405f5c0 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.038+0000 7fe73bfff700 1 -- 172.21.15.40:0/3436281215 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe74401a430 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.107+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe7541067f0 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.107+0000 7fe73bfff700 1 -- 172.21.15.40:0/3436281215 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fe74402eba0 con 0x7fe754100b80 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.108+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 msgr2=0x7fe754111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.108+0000 7fe75a0fe700 1 --2- 172.21.15.40:0/3436281215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754111830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe74400b440 tx=0x7fe744009250 comp rx=0 tx=0).stop 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.108+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 shutdown_connections 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.108+0000 7fe75a0fe700 1 --2- 172.21.15.40:0/3436281215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe754100b80 0x7fe754111830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.108+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 >> 172.21.15.40:0/3436281215 conn(0x7fe7540fc580 msgr2=0x7fe7540fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:14:58.672 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.108+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 shutdown_connections 2024-02-16T19:14:58.673 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:14:58.108+0000 7fe75a0fe700 1 -- 172.21.15.40:0/3436281215 wait complete. 2024-02-16T19:14:58.673 INFO:teuthology.orchestra.run.smithi040.stdout:Restarting the monitor... 2024-02-16T19:14:58.924 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:58 smithi040 systemd[1]: Stopping Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:14:59.290 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[28812]: 2024-02-16T19:14:58.922+0000 7fe4c4ec0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi040 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:14:59.290 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[28812]: 2024-02-16T19:14:58.922+0000 7fe4c4ec0700 -1 mon.smithi040@0(leader) e1 *** Got Signal Terminated *** 2024-02-16T19:14:59.290 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 podman[29072]: 2024-02-16 19:14:59.086924879 +0000 UTC m=+0.263193979 container died e4bc8c33d79eddf5be20e820de01786ca01d9469cb8f5c00c34ebe97a5f6b48d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, RELEASE=quincy-b88fae6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:14:59.598 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 podman[29072]: 2024-02-16 19:14:59.38067883 +0000 UTC m=+0.556947998 container cleanup e4bc8c33d79eddf5be20e820de01786ca01d9469cb8f5c00c34ebe97a5f6b48d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-b88fae6, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-16T19:14:59.598 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 bash[29072]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040 2024-02-16T19:14:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 podman[29086]: 2024-02-16 19:14:59.59824372 +0000 UTC m=+0.513415342 container remove e4bc8c33d79eddf5be20e820de01786ca01d9469cb8f5c00c34ebe97a5f6b48d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=quincy-b88fae6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-16T19:14:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040.service: Deactivated successfully. 2024-02-16T19:14:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 systemd[1]: Stopped Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:14:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 systemd[1]: Starting Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:15:00.202 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 podman[29183]: 2024-02-16 19:14:59.915860381 +0000 UTC m=+0.104076684 container create e62bfb93d3286cbab019c5836b3134790e2c8cf4477c07a7a15c60bccb0405c5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-b88fae6, ceph=True, org.label-schema.vendor=CentOS) 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:14:59 smithi040 podman[29183]: 2024-02-16 19:14:59.829012796 +0000 UTC m=+0.017229106 image pull 870a0944ccb83fae8829e20e516a8aeec00e19c6a392cc9e32528216b9cb93f0 quay.ceph.io/ceph-ci/ceph:quincy 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 podman[29183]: 2024-02-16 19:15:00.118142514 +0000 UTC m=+0.306358817 container init e62bfb93d3286cbab019c5836b3134790e2c8cf4477c07a7a15c60bccb0405c5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=quincy-b88fae6) 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 podman[29183]: 2024-02-16 19:15:00.120891293 +0000 UTC m=+0.309107600 container start e62bfb93d3286cbab019c5836b3134790e2c8cf4477c07a7a15c60bccb0405c5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=quincy-b88fae6, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable), process ceph-mon, pid 2 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: pidfile_write: ignore empty --pid-file 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: load: jerasure load: lrc 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: RocksDB version: 6.15.5 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Compile date Feb 15 2024 2024-02-16T19:15:00.203 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: DB SUMMARY 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: DB Session ID: 9WZ1PC4O098GRA2QIOGH 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: CURRENT file: CURRENT 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi040/store.db dir, Total Num: 1, files: 000008.sst 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi040/store.db: 000010.log size: 97282 ; 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.error_if_exists: 0 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.create_if_missing: 0 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.paranoid_checks: 1 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.env: 0x56021e93ea80 2024-02-16T19:15:00.204 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.fs: Posix File System 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.info_log: 0x56021f50bcc0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.statistics: (nil) 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.use_fsync: 0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_log_file_size: 0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.allow_fallocate: 1 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.use_direct_reads: 0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T19:15:00.205 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.db_log_dir: 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi040/store.db 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.write_buffer_manager: 0x560220280b40 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-16T19:15:00.206 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T19:15:00.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T19:15:00.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T19:15:00.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T19:15:00.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T19:15:00.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T19:15:00.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T19:15:00.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.unordered_write: 0 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.row_cache: None 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.wal_filter: None 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.preserve_deletes: 0 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.two_write_queues: 0 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.atomic_flush: 0 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T19:15:00.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.log_readahead_size: 0 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_background_jobs: 2 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_background_compactions: -1 2024-02-16T19:15:00.210 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_subcompactions: 1 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_open_files: -1 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T19:15:00.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_background_flushes: -1 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Compression algorithms supported: 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: kZSTD supported: 0 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: kXpressCompression supported: 0 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: kZlibCompression supported: 1 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: kSnappyCompression supported: 1 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: kLZ4Compression supported: 1 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: kBZip2Compression supported: 0 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T19:15:00.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi040/store.db/MANIFEST-000009 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.merge_operator: 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_filter: None 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56021f560f58) 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: cache_index_and_filter_blocks: 1 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: pin_top_level_index_and_filter: 1 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_type: 0 2024-02-16T19:15:00.213 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_block_index_type: 0 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_shortening: 1 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: hash_index_allow_collision: 1 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: checksum: 1 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: no_block_cache: 0 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache: 0x56021f593090 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_name: BinnedLRUCache 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_options: 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: capacity : 536870912 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: num_shard_bits : 4 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: strict_capacity_limit : 0 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: high_pri_pool_ratio: 0.000 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_compressed: (nil) 2024-02-16T19:15:00.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: persistent_cache: (nil) 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_size: 4096 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_size_deviation: 10 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_restart_interval: 16 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_block_restart_interval: 1 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: metadata_block_size: 4096 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: partition_filters: 0 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: use_delta_encoding: 1 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: whole_key_filtering: 1 2024-02-16T19:15:00.215 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: verify_compression: 0 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: read_amp_bytes_per_bit: 0 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: format_version: 4 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: enable_index_compression: 1 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_align: 0 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compression: NoCompression 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.num_levels: 7 2024-02-16T19:15:00.216 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T19:15:00.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T19:15:00.218 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T19:15:00.219 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.arena_block_size: 4194304 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T19:15:00.220 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.table_properties_collectors: 2024-02-16T19:15:00.221 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.inplace_update_support: 0 2024-02-16T19:15:00.221 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T19:15:00.221 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T19:15:00.221 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T19:15:00.221 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T19:15:00.221 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.bloom_locality: 0 2024-02-16T19:15:00.221 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.max_successive_merges: 0 2024-02-16T19:15:00.221 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.ttl: 2592000 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.enable_blob_files: false 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.min_blob_size: 0 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T19:15:00.222 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi040/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708110900147178, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708110900147977, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92554, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90649, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83364, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708110900, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "fe8e5907-6f2f-4475-8992-38435b1b5eee", "db_session_id": "9WZ1PC4O098GRA2QIOGH"}} 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708110900148347, "job": 1, "event": "recovery_finished"} 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi040/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x56021f527180 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: DB pointer 0x56021f5fc000 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-02-16T19:15:00.223 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** DB Stats ** 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: L0 2/0 92.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.001 0 0 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Sum 2/0 92.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.001 0 0 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.001 0 0 2024-02-16T19:15:00.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 159.6 0.00 0.00 1 0.001 0 0 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative compaction: 0.00 GB write, 29.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval compaction: 0.00 GB write, 29.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:15:00.225 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: L0 2/0 92.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.001 0 0 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Sum 2/0 92.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.001 0 0 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 159.6 0.00 0.00 1 0.001 0 0 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:15:00.226 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative compaction: 0.00 GB write, 28.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: starting mon.smithi040 rank 0 at public addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] at bind addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi040 fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040@-1(???) e1 preinit fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040@-1(???).mds e1 new map 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040@-1(???).mds e1 print_map 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: e1 2024-02-16T19:15:00.227 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: legacy client fscid: -1 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: No filesystems configured 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta expand map: {default=false} 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta from 'false' to 'false' 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta expanded map: {default=false} 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta expand map: {default=info} 2024-02-16T19:15:00.228 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta from 'info' to 'info' 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta expanded map: {default=info} 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta expand map: {default=daemon} 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta from 'daemon' to 'daemon' 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta expanded map: {default=daemon} 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta expand map: {default=debug} 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta from 'debug' to 'debug' 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: expand_channel_meta expanded map: {default=debug} 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 bash[29183]: e62bfb93d3286cbab019c5836b3134790e2c8cf4477c07a7a15c60bccb0405c5 2024-02-16T19:15:00.229 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 systemd[1]: Started Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:15:00.229 INFO:teuthology.orchestra.run.smithi040.stdout:Setting mon public_network to 172.21.0.0/20 2024-02-16T19:15:00.457 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mon.smithi040 is new leader, mons smithi040 in quorum (ranks 0) 2024-02-16T19:15:00.457 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: monmap e1: 1 mons at {smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:15:00.457 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: fsmap 2024-02-16T19:15:00.457 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: osdmap e1: 0 total, 0 up, 0 in 2024-02-16T19:15:00.457 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:00 smithi040 ceph-mon[29201]: mgrmap e1: no daemons active 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.638+0000 7fe2cb61a700 1 Processor -- start 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.638+0000 7fe2cb61a700 1 -- start start 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.638+0000 7fe2cb61a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.638+0000 7fe2cb61a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c4102ff0 con 0x7fe2c41026a0 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.639+0000 7fe2c93b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.639+0000 7fe2c93b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34724/0 (socket says 172.21.15.40:34724) 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.639+0000 7fe2c93b6700 1 -- 172.21.15.40:0/1553659315 learned_addr learned my addr 172.21.15.40:0/1553659315 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.639+0000 7fe2c93b6700 1 -- 172.21.15.40:0/1553659315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c4103130 con 0x7fe2c41026a0 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.639+0000 7fe2c93b6700 1 --2- 172.21.15.40:0/1553659315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4102ab0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00098e0 tx=0x7fe2c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f48e52058544bd3 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.639+0000 7fe2bbfff700 1 -- 172.21.15.40:0/1553659315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2c000b860 con 0x7fe2c41026a0 2024-02-16T19:15:01.209 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.639+0000 7fe2bbfff700 1 -- 172.21.15.40:0/1553659315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe2c000be50 con 0x7fe2c41026a0 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.640+0000 7fe2bbfff700 1 -- 172.21.15.40:0/1553659315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2c0010380 con 0x7fe2c41026a0 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.640+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1553659315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 msgr2=0x7fe2c4102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.640+0000 7fe2cb61a700 1 --2- 172.21.15.40:0/1553659315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4102ab0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00098e0 tx=0x7fe2c0009250 comp rx=0 tx=0).stop 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.640+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1553659315 shutdown_connections 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.640+0000 7fe2cb61a700 1 --2- 172.21.15.40:0/1553659315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4102ab0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.640+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1553659315 >> 172.21.15.40:0/1553659315 conn(0x7fe2c40fde00 msgr2=0x7fe2c4100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.640+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1553659315 shutdown_connections 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.640+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1553659315 wait complete. 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2cb61a700 1 Processor -- start 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2cb61a700 1 -- start start 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2cb61a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2cb61a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c4077190 con 0x7fe2c41026a0 2024-02-16T19:15:01.210 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2c93b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2c93b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34732/0 (socket says 172.21.15.40:34732) 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2c93b6700 1 -- 172.21.15.40:0/1055363063 learned_addr learned my addr 172.21.15.40:0/1055363063 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2c93b6700 1 -- 172.21.15.40:0/1055363063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00094f0 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.641+0000 7fe2c93b6700 1 --2- 172.21.15.40:0/1055363063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4078d10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000b440 tx=0x7fe2c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.642+0000 7fe2b9ffb700 1 -- 172.21.15.40:0/1055363063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2c0010760 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.642+0000 7fe2b9ffb700 1 -- 172.21.15.40:0/1055363063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe2c0010d50 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.642+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c4077390 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.642+0000 7fe2b9ffb700 1 -- 172.21.15.40:0/1055363063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2c001ae20 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.642+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c4077820 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.642+0000 7fe2b9ffb700 1 -- 172.21.15.40:0/1055363063 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe2c002e6c0 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.642+0000 7fe2b9ffb700 1 -- 172.21.15.40:0/1055363063 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe2c00248c0 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.643+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c404bed0 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.645+0000 7fe2b9ffb700 1 -- 172.21.15.40:0/1055363063 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe2c0029070 con 0x7fe2c41026a0 2024-02-16T19:15:01.211 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.713+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fe2c4104580 con 0x7fe2c41026a0 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.718+0000 7fe2b9ffb700 1 -- 172.21.15.40:0/1055363063 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe2c001a6a0 con 0x7fe2c41026a0 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.719+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 msgr2=0x7fe2c4078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.719+0000 7fe2cb61a700 1 --2- 172.21.15.40:0/1055363063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4078d10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000b440 tx=0x7fe2c0009250 comp rx=0 tx=0).stop 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.719+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 shutdown_connections 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.719+0000 7fe2cb61a700 1 --2- 172.21.15.40:0/1055363063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2c41026a0 0x7fe2c4078d10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.719+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 >> 172.21.15.40:0/1055363063 conn(0x7fe2c40fde00 msgr2=0x7fe2c40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.719+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 shutdown_connections 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:00.719+0000 7fe2cb61a700 1 -- 172.21.15.40:0/1055363063 wait complete. 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:Creating mgr... 2024-02-16T19:15:01.212 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying port 9283 ... 2024-02-16T19:15:01.380 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mgr.smithi040.jzygzo 2024-02-16T19:15:01.380 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mgr.smithi040.jzygzo.service: Unit ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mgr.smithi040.jzygzo.service not loaded. 2024-02-16T19:15:01.527 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715.target.wants/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mgr.smithi040.jzygzo.service → /etc/systemd/system/ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@.service. 2024-02-16T19:15:01.843 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:01 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1055363063' entity='client.admin' 2024-02-16T19:15:02.045 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T19:15:02.045 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2024-02-16T19:15:02.053 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T19:15:02.053 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-02-16T19:15:02.053 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2024-02-16T19:15:02.053 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to enable service . firewalld.service is not available 2024-02-16T19:15:02.062 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T19:15:02.062 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2024-02-16T19:15:02.070 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T19:15:02.071 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-02-16T19:15:02.071 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2024-02-16T19:15:02.071 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-16T19:15:02.071 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr to start... 2024-02-16T19:15:02.071 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr... 2024-02-16T19:15:02.854 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:02 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2291866305' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T19:15:03.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:15:03.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:15:03.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "8c23bd6c-ccff-11ee-95ba-87774f69a715", 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "smithi040" 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T19:15:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T19:15:03.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T19:15:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T19:14:55.436594+0000", 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.489+0000 7f82e0d63700 1 Processor -- start 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.489+0000 7f82e0d63700 1 -- start start 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.489+0000 7f82e0d63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.489+0000 7f82e0d63700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc101fc0 con 0x7f82dc101670 2024-02-16T19:15:03.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.489+0000 7f82da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.489+0000 7f82da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34756/0 (socket says 172.21.15.40:34756) 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.489+0000 7f82da59c700 1 -- 172.21.15.40:0/819132421 learned_addr learned my addr 172.21.15.40:0/819132421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.490+0000 7f82da59c700 1 -- 172.21.15.40:0/819132421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc102100 con 0x7f82dc101670 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.490+0000 7f82da59c700 1 --2- 172.21.15.40:0/819132421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc101a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0098e0 tx=0x7f82cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96d75b1dce20e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.490+0000 7f82d959a700 1 -- 172.21.15.40:0/819132421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82cc00b860 con 0x7f82dc101670 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.490+0000 7f82d959a700 1 -- 172.21.15.40:0/819132421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f82cc00be50 con 0x7f82dc101670 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.490+0000 7f82d959a700 1 -- 172.21.15.40:0/819132421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82cc010380 con 0x7f82dc101670 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 -- 172.21.15.40:0/819132421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 msgr2=0x7f82dc101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 --2- 172.21.15.40:0/819132421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc101a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0098e0 tx=0x7f82cc009250 comp rx=0 tx=0).stop 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 -- 172.21.15.40:0/819132421 shutdown_connections 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 --2- 172.21.15.40:0/819132421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc101a80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 -- 172.21.15.40:0/819132421 >> 172.21.15.40:0/819132421 conn(0x7f82dc0fcdd0 msgr2=0x7f82dc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 -- 172.21.15.40:0/819132421 shutdown_connections 2024-02-16T19:15:03.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 -- 172.21.15.40:0/819132421 wait complete. 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 Processor -- start 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.491+0000 7f82e0d63700 1 -- start start 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82e0d63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc10f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82e0d63700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc10f880 con 0x7f82dc101670 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc10f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc10f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34762/0 (socket says 172.21.15.40:34762) 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82da59c700 1 -- 172.21.15.40:0/2291866305 learned_addr learned my addr 172.21.15.40:0/2291866305 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82da59c700 1 -- 172.21.15.40:0/2291866305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0094f0 con 0x7f82dc101670 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82da59c700 1 --2- 172.21.15.40:0/2291866305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc10f340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000c00 tx=0x7f82cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82caffd700 1 -- 172.21.15.40:0/2291866305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82cc0107c0 con 0x7f82dc101670 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82caffd700 1 -- 172.21.15.40:0/2291866305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f82cc010db0 con 0x7f82dc101670 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc10fa80 con 0x7f82dc101670 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82caffd700 1 -- 172.21.15.40:0/2291866305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82cc01ad00 con 0x7f82dc101670 2024-02-16T19:15:03.165 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.492+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc110720 con 0x7f82dc101670 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.493+0000 7f82caffd700 1 -- 172.21.15.40:0/2291866305 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f82cc024440 con 0x7f82dc101670 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.493+0000 7f82caffd700 1 -- 172.21.15.40:0/2291866305 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f82cc019750 con 0x7f82dc101670 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.493+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc05f5c0 con 0x7f82dc101670 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.496+0000 7f82caffd700 1 -- 172.21.15.40:0/2291866305 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f82cc0426d0 con 0x7f82dc101670 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.568+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f82dc1034e0 con 0x7f82dc101670 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.569+0000 7f82caffd700 1 -- 172.21.15.40:0/2291866305 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f82cc031020 con 0x7f82dc101670 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.570+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 msgr2=0x7f82dc10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.570+0000 7f82e0d63700 1 --2- 172.21.15.40:0/2291866305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc10f340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000c00 tx=0x7f82cc009250 comp rx=0 tx=0).stop 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.570+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 shutdown_connections 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.570+0000 7f82e0d63700 1 --2- 172.21.15.40:0/2291866305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82dc101670 0x7f82dc10f340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.570+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 >> 172.21.15.40:0/2291866305 conn(0x7f82dc0fcdd0 msgr2=0x7f82dc0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.570+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 shutdown_connections 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:02.570+0000 7f82e0d63700 1 -- 172.21.15.40:0/2291866305 wait complete. 2024-02-16T19:15:03.166 INFO:teuthology.orchestra.run.smithi040.stdout:mgr not available, waiting (1/15)... 2024-02-16T19:15:06.325 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:06 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1904667793' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T19:15:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: Activating manager daemon smithi040.jzygzo 2024-02-16T19:15:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: mgrmap e2: smithi040.jzygzo(active, starting, since 0.00283028s) 2024-02-16T19:15:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: Manager daemon smithi040.jzygzo is now available 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/trash_purge_schedule"}]: dispatch 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:08 smithi040 ceph-mon[29201]: from='mgr.14100 172.21.15.40:0/1031584549' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:11.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:10 smithi040 ceph-mon[29201]: mgrmap e3: smithi040.jzygzo(active, since 1.00725s) 2024-02-16T19:15:11.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:15:11.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:15:11.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "8c23bd6c-ccff-11ee-95ba-87774f69a715", 2024-02-16T19:15:11.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T19:15:11.162 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "smithi040" 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-16T19:15:11.163 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T19:15:11.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:11.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T19:15:11.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:11.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T19:15:11.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T19:15:11.164 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T19:15:11.171 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T19:15:11.171 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T19:15:11.171 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T19:15:11.171 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:11.171 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T19:15:11.171 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T19:15:11.172 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T19:14:55.436594+0000", 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:15:11.173 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.577+0000 7f519bf5c700 1 Processor -- start 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.577+0000 7f519bf5c700 1 -- start start 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.578+0000 7f519bf5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194105340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.578+0000 7f519bf5c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5194105880 con 0x7f5194076c60 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.578+0000 7f5199cf8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194105340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.578+0000 7f5199cf8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194105340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34774/0 (socket says 172.21.15.40:34774) 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.578+0000 7f5199cf8700 1 -- 172.21.15.40:0/2773027158 learned_addr learned my addr 172.21.15.40:0/2773027158 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.578+0000 7f5199cf8700 1 -- 172.21.15.40:0/2773027158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51941059c0 con 0x7f5194076c60 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.578+0000 7f5199cf8700 1 --2- 172.21.15.40:0/2773027158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194105340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f51880098e0 tx=0x7f5188009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cecc2b68415122b server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.578+0000 7f5198cf6700 1 -- 172.21.15.40:0/2773027158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f518800b860 con 0x7f5194076c60 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f5198cf6700 1 -- 172.21.15.40:0/2773027158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f518800be50 con 0x7f5194076c60 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f5198cf6700 1 -- 172.21.15.40:0/2773027158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5188010380 con 0x7f5194076c60 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f519bf5c700 1 -- 172.21.15.40:0/2773027158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 msgr2=0x7f5194105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f519bf5c700 1 --2- 172.21.15.40:0/2773027158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194105340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f51880098e0 tx=0x7f5188009250 comp rx=0 tx=0).stop 2024-02-16T19:15:11.174 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f519bf5c700 1 -- 172.21.15.40:0/2773027158 shutdown_connections 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f519bf5c700 1 --2- 172.21.15.40:0/2773027158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194105340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f519bf5c700 1 -- 172.21.15.40:0/2773027158 >> 172.21.15.40:0/2773027158 conn(0x7f51940fcdd0 msgr2=0x7f51940ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f519bf5c700 1 -- 172.21.15.40:0/2773027158 shutdown_connections 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.579+0000 7f519bf5c700 1 -- 172.21.15.40:0/2773027158 wait complete. 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f519bf5c700 1 Processor -- start 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f519bf5c700 1 -- start start 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f519bf5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f519bf5c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5194111a60 con 0x7f5194076c60 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f5199cf8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f5199cf8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194111520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34780/0 (socket says 172.21.15.40:34780) 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f5199cf8700 1 -- 172.21.15.40:0/1904667793 learned_addr learned my addr 172.21.15.40:0/1904667793 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f5199cf8700 1 -- 172.21.15.40:0/1904667793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880094f0 con 0x7f5194076c60 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.580+0000 7f5199cf8700 1 --2- 172.21.15.40:0/1904667793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194111520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f518800b440 tx=0x7f5188009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.581+0000 7f51867fc700 1 -- 172.21.15.40:0/1904667793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51880107c0 con 0x7f5194076c60 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.581+0000 7f51867fc700 1 -- 172.21.15.40:0/1904667793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5188010db0 con 0x7f5194076c60 2024-02-16T19:15:11.175 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.581+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5194111c60 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.581+0000 7f51867fc700 1 -- 172.21.15.40:0/1904667793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f518801ad30 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.581+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5194112900 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.581+0000 7f51867fc700 1 -- 172.21.15.40:0/1904667793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f518802e6c0 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.581+0000 7f51867fc700 1 -- 172.21.15.40:0/1904667793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5188024900 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.582+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519405f5c0 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.584+0000 7f51867fc700 1 -- 172.21.15.40:0/1904667793 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f518801a430 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.657+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5194106800 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.657+0000 7f51867fc700 1 -- 172.21.15.40:0/1904667793 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f5188010920 con 0x7f5194076c60 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.658+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 msgr2=0x7f5194111520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.658+0000 7f519bf5c700 1 --2- 172.21.15.40:0/1904667793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194111520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f518800b440 tx=0x7f5188009250 comp rx=0 tx=0).stop 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.658+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 shutdown_connections 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.658+0000 7f519bf5c700 1 --2- 172.21.15.40:0/1904667793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5194076c60 0x7f5194111520 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.658+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 >> 172.21.15.40:0/1904667793 conn(0x7f51940fcdd0 msgr2=0x7f51940fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:11.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.659+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 shutdown_connections 2024-02-16T19:15:11.177 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:05.659+0000 7f519bf5c700 1 -- 172.21.15.40:0/1904667793 wait complete. 2024-02-16T19:15:11.177 INFO:teuthology.orchestra.run.smithi040.stdout:mgr not available, waiting (2/15)... 2024-02-16T19:15:13.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:12 smithi040 ceph-mon[29201]: mgrmap e4: smithi040.jzygzo(active, since 3s) 2024-02-16T19:15:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:14 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3127516292' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-16T19:15:14.821 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:15:14.821 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:15:14.821 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "8c23bd6c-ccff-11ee-95ba-87774f69a715", 2024-02-16T19:15:14.821 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2024-02-16T19:15:14.821 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-16T19:15:14.821 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-16T19:15:14.821 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-16T19:15:14.821 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "smithi040" 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:14.822 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-16T19:15:14.823 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-16T19:15:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-16T19:15:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-16T19:15:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-16T19:15:14.830 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-16T19:15:14.831 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2024-02-16T19:14:55.436594+0000", 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2024-02-16T19:15:14.832 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.961+0000 7f76ad932700 1 Processor -- start 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.961+0000 7f76ad932700 1 -- start start 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.961+0000 7f76ad932700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a80771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.961+0000 7f76ad932700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80790b0 con 0x7f76a8078ca0 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.961+0000 7f76a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a80771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.961+0000 7f76a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a80771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44090/0 (socket says 172.21.15.40:44090) 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.961+0000 7f76a6ffd700 1 -- 172.21.15.40:0/1737326937 learned_addr learned my addr 172.21.15.40:0/1737326937 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.962+0000 7f76a6ffd700 1 -- 172.21.15.40:0/1737326937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80776f0 con 0x7f76a8078ca0 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.962+0000 7f76a6ffd700 1 --2- 172.21.15.40:0/1737326937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a80771b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f76a00098e0 tx=0x7f76a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7532497e9c514c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.962+0000 7f76a5ffb700 1 -- 172.21.15.40:0/1737326937 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76a000b860 con 0x7f76a8078ca0 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.962+0000 7f76a5ffb700 1 -- 172.21.15.40:0/1737326937 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f76a000be50 con 0x7f76a8078ca0 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.962+0000 7f76a5ffb700 1 -- 172.21.15.40:0/1737326937 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76a0010380 con 0x7f76a8078ca0 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 -- 172.21.15.40:0/1737326937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 msgr2=0x7f76a80771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:14.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 --2- 172.21.15.40:0/1737326937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a80771b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f76a00098e0 tx=0x7f76a0009250 comp rx=0 tx=0).stop 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 -- 172.21.15.40:0/1737326937 shutdown_connections 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 --2- 172.21.15.40:0/1737326937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a80771b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 -- 172.21.15.40:0/1737326937 >> 172.21.15.40:0/1737326937 conn(0x7f76a80fded0 msgr2=0x7f76a8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 -- 172.21.15.40:0/1737326937 shutdown_connections 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 -- 172.21.15.40:0/1737326937 wait complete. 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 Processor -- start 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.963+0000 7f76ad932700 1 -- start start 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76ad932700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a81137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76ad932700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8113d00 con 0x7f76a8078ca0 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a81137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a81137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44106/0 (socket says 172.21.15.40:44106) 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76a6ffd700 1 -- 172.21.15.40:0/3127516292 learned_addr learned my addr 172.21.15.40:0/3127516292 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76a6ffd700 1 -- 172.21.15.40:0/3127516292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00094f0 con 0x7f76a8078ca0 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76a6ffd700 1 --2- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a81137c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76a0003050 tx=0x7f76a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f769f7fe700 1 -- 172.21.15.40:0/3127516292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76a0010760 con 0x7f76a8078ca0 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f769f7fe700 1 -- 172.21.15.40:0/3127516292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f76a0010d50 con 0x7f76a8078ca0 2024-02-16T19:15:14.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a8113f00 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f769f7fe700 1 -- 172.21.15.40:0/3127516292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76a001ae20 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.964+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a8114ba0 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.965+0000 7f769f7fe700 1 -- 172.21.15.40:0/3127516292 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f76a002e440 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.965+0000 7f769f7fe700 1 --2- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7694036790 0x7f7694038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.965+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a805f5c0 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.966+0000 7f769f7fe700 1 -- 172.21.15.40:0/3127516292 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f76a004a5b0 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.966+0000 7f76a67fc700 1 --2- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7694036790 0x7f7694038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.966+0000 7f76a67fc700 1 --2- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7694036790 0x7f7694038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7690009990 tx=0x7f7690006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:13.971+0000 7f769f7fe700 1 -- 172.21.15.40:0/3127516292 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f76a0018710 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.237+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f76a804bed0 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.238+0000 7f769f7fe700 1 -- 172.21.15.40:0/3127516292 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f76a00334f0 con 0x7f76a8078ca0 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7694036790 msgr2=0x7f7694038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 --2- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7694036790 0x7f7694038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7690009990 tx=0x7f7690006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 msgr2=0x7f76a81137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 --2- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a81137c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76a0003050 tx=0x7f76a0009250 comp rx=0 tx=0).stop 2024-02-16T19:15:14.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 shutdown_connections 2024-02-16T19:15:14.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 --2- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7694036790 0x7f7694038c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:14.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 --2- 172.21.15.40:0/3127516292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a8078ca0 0x7f76a81137c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:14.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 >> 172.21.15.40:0/3127516292 conn(0x7f76a80fded0 msgr2=0x7f76a80ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:14.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 shutdown_connections 2024-02-16T19:15:14.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:14.240+0000 7f76ad932700 1 -- 172.21.15.40:0/3127516292 wait complete. 2024-02-16T19:15:14.836 INFO:teuthology.orchestra.run.smithi040.stdout:mgr is available 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [global] 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout fsid = 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-16T19:15:16.008 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [osd] 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.208+0000 7f3330379700 1 Processor -- start 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.208+0000 7f3330379700 1 -- start start 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.208+0000 7f3330379700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328103050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.208+0000 7f3330379700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3328103590 con 0x7f3328102c40 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.208+0000 7f332e115700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.208+0000 7f332e115700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44114/0 (socket says 172.21.15.40:44114) 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.208+0000 7f332e115700 1 -- 172.21.15.40:0/2308213715 learned_addr learned my addr 172.21.15.40:0/2308213715 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.208+0000 7f332e115700 1 -- 172.21.15.40:0/2308213715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33281036d0 con 0x7f3328102c40 2024-02-16T19:15:16.009 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f332e115700 1 --2- 172.21.15.40:0/2308213715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328103050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3320009e50 tx=0x7f3320009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662342ee1172d3cf server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f332d113700 1 -- 172.21.15.40:0/2308213715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33200092e0 con 0x7f3328102c40 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f332d113700 1 -- 172.21.15.40:0/2308213715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3320012430 con 0x7f3328102c40 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f332d113700 1 -- 172.21.15.40:0/2308213715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33200075c0 con 0x7f3328102c40 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f3330379700 1 -- 172.21.15.40:0/2308213715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 msgr2=0x7f3328103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f3330379700 1 --2- 172.21.15.40:0/2308213715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328103050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3320009e50 tx=0x7f3320009840 comp rx=0 tx=0).stop 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f3330379700 1 -- 172.21.15.40:0/2308213715 shutdown_connections 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f3330379700 1 --2- 172.21.15.40:0/2308213715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328103050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f3330379700 1 -- 172.21.15.40:0/2308213715 >> 172.21.15.40:0/2308213715 conn(0x7f33280fc580 msgr2=0x7f33280fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f3330379700 1 -- 172.21.15.40:0/2308213715 shutdown_connections 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.209+0000 7f3330379700 1 -- 172.21.15.40:0/2308213715 wait complete. 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.210+0000 7f3330379700 1 Processor -- start 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.210+0000 7f3330379700 1 -- start start 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.210+0000 7f3330379700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328111870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.210+0000 7f3330379700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3328111db0 con 0x7f3328102c40 2024-02-16T19:15:16.010 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.210+0000 7f332e115700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328111870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.210+0000 7f332e115700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328111870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44128/0 (socket says 172.21.15.40:44128) 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.210+0000 7f332e115700 1 -- 172.21.15.40:0/1134585073 learned_addr learned my addr 172.21.15.40:0/1134585073 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.210+0000 7f332e115700 1 -- 172.21.15.40:0/1134585073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320009ae0 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.211+0000 7f332e115700 1 --2- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328111870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33200127d0 tx=0x7f3320009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.211+0000 7f3326ffd700 1 -- 172.21.15.40:0/1134585073 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f332000bc30 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.211+0000 7f3326ffd700 1 -- 172.21.15.40:0/1134585073 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f332000bd90 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.211+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3328111fb0 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.211+0000 7f3326ffd700 1 -- 172.21.15.40:0/1134585073 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f332001c9d0 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.211+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3328112c50 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.211+0000 7f3326ffd700 1 -- 172.21.15.40:0/1134585073 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f3320019070 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.212+0000 7f3326ffd700 1 --2- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f3310036740 0x7f3310038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.212+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332805f5c0 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.212+0000 7f3326ffd700 1 -- 172.21.15.40:0/1134585073 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3320049de0 con 0x7f3328102c40 2024-02-16T19:15:16.011 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.212+0000 7f332d914700 1 --2- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f3310036740 0x7f3310038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.212+0000 7f332d914700 1 --2- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f3310036740 0x7f3310038bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f331c009990 tx=0x7f331c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.216+0000 7f3326ffd700 1 -- 172.21.15.40:0/1134585073 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3320017080 con 0x7f3328102c40 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.408+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f332804bed0 con 0x7f3328102c40 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.409+0000 7f3326ffd700 1 -- 172.21.15.40:0/1134585073 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f332002a020 con 0x7f3328102c40 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f3310036740 msgr2=0x7f3310038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 --2- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f3310036740 0x7f3310038bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f331c009990 tx=0x7f331c006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 msgr2=0x7f3328111870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 --2- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328111870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33200127d0 tx=0x7f3320009840 comp rx=0 tx=0).stop 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 shutdown_connections 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 --2- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f3310036740 0x7f3310038bf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 --2- 172.21.15.40:0/1134585073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3328102c40 0x7f3328111870 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 >> 172.21.15.40:0/1134585073 conn(0x7f33280fc580 msgr2=0x7f33280fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 shutdown_connections 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:15.411+0000 7f3330379700 1 -- 172.21.15.40:0/1134585073 wait complete. 2024-02-16T19:15:16.012 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling cephadm module... 2024-02-16T19:15:16.292 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:16 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1134585073' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-16T19:15:17.421 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:17 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2687170116' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-16T19:15:17.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.396+0000 7fb162115700 1 Processor -- start 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.396+0000 7fb162115700 1 -- start start 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.396+0000 7fb162115700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.396+0000 7fb162115700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c1030c0 con 0x7fb15c102770 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.396+0000 7fb15b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.396+0000 7fb15b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44142/0 (socket says 172.21.15.40:44142) 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.396+0000 7fb15b7fe700 1 -- 172.21.15.40:0/196681210 learned_addr learned my addr 172.21.15.40:0/196681210 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.397+0000 7fb15b7fe700 1 -- 172.21.15.40:0/196681210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c103200 con 0x7fb15c102770 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.397+0000 7fb15b7fe700 1 --2- 172.21.15.40:0/196681210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c102b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009e50 tx=0x7fb14c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b48151525a2317b server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.397+0000 7fb15a7fc700 1 -- 172.21.15.40:0/196681210 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb14c0092e0 con 0x7fb15c102770 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.397+0000 7fb15a7fc700 1 -- 172.21.15.40:0/196681210 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb14c012430 con 0x7fb15c102770 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.397+0000 7fb15a7fc700 1 -- 172.21.15.40:0/196681210 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb14c0075c0 con 0x7fb15c102770 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 -- 172.21.15.40:0/196681210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 msgr2=0x7fb15c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:17.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 --2- 172.21.15.40:0/196681210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c102b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009e50 tx=0x7fb14c009840 comp rx=0 tx=0).stop 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 -- 172.21.15.40:0/196681210 shutdown_connections 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 --2- 172.21.15.40:0/196681210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c102b80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 -- 172.21.15.40:0/196681210 >> 172.21.15.40:0/196681210 conn(0x7fb15c0fded0 msgr2=0x7fb15c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 -- 172.21.15.40:0/196681210 shutdown_connections 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 -- 172.21.15.40:0/196681210 wait complete. 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 Processor -- start 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.398+0000 7fb162115700 1 -- start start 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb162115700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c1115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb162115700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c111b10 con 0x7fb15c102770 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb15b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c1115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb15b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c1115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44152/0 (socket says 172.21.15.40:44152) 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb15b7fe700 1 -- 172.21.15.40:0/2687170116 learned_addr learned my addr 172.21.15.40:0/2687170116 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb15b7fe700 1 -- 172.21.15.40:0/2687170116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c009ae0 con 0x7fb15c102770 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb15b7fe700 1 --2- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c1115d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009ab0 tx=0x7fb14c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb153fff700 1 -- 172.21.15.40:0/2687170116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb14c0092e0 con 0x7fb15c102770 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb153fff700 1 -- 172.21.15.40:0/2687170116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb14c007d10 con 0x7fb15c102770 2024-02-16T19:15:17.697 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c111d10 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb153fff700 1 -- 172.21.15.40:0/2687170116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb14c01c990 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.399+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c1129b0 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.400+0000 7fb153fff700 1 -- 172.21.15.40:0/2687170116 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fb14c019070 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.400+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c05f5c0 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.400+0000 7fb153fff700 1 --2- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fb148036740 0x7fb148038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.400+0000 7fb153fff700 1 -- 172.21.15.40:0/2687170116 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb14c049f50 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.400+0000 7fb15affd700 1 --2- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fb148036740 0x7fb148038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.401+0000 7fb15affd700 1 --2- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fb148036740 0x7fb148038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb144009990 tx=0x7fb144006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.405+0000 7fb153fff700 1 -- 172.21.15.40:0/2687170116 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb14c019320 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:16.649+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb15c112d30 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.160+0000 7fb153fff700 1 -- 172.21.15.40:0/2687170116 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb14c0243b0 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.161+0000 7fb153fff700 1 -- 172.21.15.40:0/2687170116 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb14c049960 con 0x7fb15c102770 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.163+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fb148036740 msgr2=0x7fb148038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:17.698 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.163+0000 7fb162115700 1 --2- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fb148036740 0x7fb148038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb144009990 tx=0x7fb144006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:17.699 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.163+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 msgr2=0x7fb15c1115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:17.699 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.163+0000 7fb162115700 1 --2- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c1115d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009ab0 tx=0x7fb14c009840 comp rx=0 tx=0).stop 2024-02-16T19:15:17.699 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.163+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 shutdown_connections 2024-02-16T19:15:17.699 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.163+0000 7fb162115700 1 --2- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fb148036740 0x7fb148038bf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:17.699 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.163+0000 7fb162115700 1 --2- 172.21.15.40:0/2687170116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb15c102770 0x7fb15c1115d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:17.699 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.163+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 >> 172.21.15.40:0/2687170116 conn(0x7fb15c0fded0 msgr2=0x7fb15c0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:17.699 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.164+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 shutdown_connections 2024-02-16T19:15:17.699 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:17.164+0000 7fb162115700 1 -- 172.21.15.40:0/2687170116 wait complete. 2024-02-16T19:15:18.305 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:18 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2687170116' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-16T19:15:18.305 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:18 smithi040 ceph-mon[29201]: mgrmap e5: smithi040.jzygzo(active, since 8s) 2024-02-16T19:15:19.463 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:19 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3876194436' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-16T19:15:20.581 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:15:20.581 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-16T19:15:20.581 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T19:15:20.581 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "active_name": "smithi040.jzygzo", 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.096+0000 7f7a25338700 1 Processor -- start 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.096+0000 7f7a25338700 1 -- start start 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.096+0000 7f7a25338700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a200771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.096+0000 7f7a25338700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200790d0 con 0x7f7a20078cc0 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.097+0000 7f7a1effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a200771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.097+0000 7f7a1effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a200771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44826/0 (socket says 172.21.15.40:44826) 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.097+0000 7f7a1effd700 1 -- 172.21.15.40:0/229914048 learned_addr learned my addr 172.21.15.40:0/229914048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.097+0000 7f7a1effd700 1 -- 172.21.15.40:0/229914048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a20077710 con 0x7f7a20078cc0 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.097+0000 7f7a1effd700 1 --2- 172.21.15.40:0/229914048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a200771d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a100098e0 tx=0x7f7a10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4535fdb8a2a723b server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a1dffb700 1 -- 172.21.15.40:0/229914048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a1000b860 con 0x7f7a20078cc0 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a1dffb700 1 -- 172.21.15.40:0/229914048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7a1000be50 con 0x7f7a20078cc0 2024-02-16T19:15:20.582 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a1dffb700 1 -- 172.21.15.40:0/229914048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a10010380 con 0x7f7a20078cc0 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a25338700 1 -- 172.21.15.40:0/229914048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 msgr2=0x7f7a200771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a25338700 1 --2- 172.21.15.40:0/229914048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a200771d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a100098e0 tx=0x7f7a10009250 comp rx=0 tx=0).stop 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a25338700 1 -- 172.21.15.40:0/229914048 shutdown_connections 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a25338700 1 --2- 172.21.15.40:0/229914048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a200771d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a25338700 1 -- 172.21.15.40:0/229914048 >> 172.21.15.40:0/229914048 conn(0x7f7a200fdf00 msgr2=0x7f7a20100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a25338700 1 -- 172.21.15.40:0/229914048 shutdown_connections 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.098+0000 7f7a25338700 1 -- 172.21.15.40:0/229914048 wait complete. 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.099+0000 7f7a25338700 1 Processor -- start 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.099+0000 7f7a25338700 1 -- start start 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a25338700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a201137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a25338700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a20113d20 con 0x7f7a20078cc0 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a1effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a201137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a1effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a201137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44828/0 (socket says 172.21.15.40:44828) 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a1effd700 1 -- 172.21.15.40:0/3876194436 learned_addr learned my addr 172.21.15.40:0/3876194436 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:20.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a1effd700 1 -- 172.21.15.40:0/3876194436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a100094f0 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a1effd700 1 --2- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a201137e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000b440 tx=0x7f7a10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a077fe700 1 -- 172.21.15.40:0/3876194436 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a10010760 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a077fe700 1 -- 172.21.15.40:0/3876194436 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7a10010d50 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a077fe700 1 -- 172.21.15.40:0/3876194436 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a1001adc0 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a20113f20 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.100+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a20114bc0 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.101+0000 7f7a077fe700 1 -- 172.21.15.40:0/3876194436 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f7a100108c0 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.101+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2005f5c0 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.102+0000 7f7a077fe700 1 --2- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7a080367e0 0x7f7a08038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.102+0000 7f7a1e7fc700 1 -- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7a080367e0 msgr2=0x7f7a08038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1383032518 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.102+0000 7f7a077fe700 1 -- 172.21.15.40:0/3876194436 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7a10049f20 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.102+0000 7f7a1e7fc700 1 --2- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7a080367e0 0x7f7a08038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.106+0000 7f7a077fe700 1 -- 172.21.15.40:0/3876194436 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7a10022770 con 0x7f7a20078cc0 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.303+0000 7f7a1e7fc700 1 -- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7a080367e0 msgr2=0x7f7a08038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1383032518 2024-02-16T19:15:20.584 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.303+0000 7f7a1e7fc700 1 --2- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7a080367e0 0x7f7a08038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.354+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7a20114f00 con 0x7f7a20078cc0 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.355+0000 7f7a077fe700 1 -- 172.21.15.40:0/3876194436 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f7a10018e40 con 0x7f7a20078cc0 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7a080367e0 msgr2=0x7f7a08038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 --2- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7a080367e0 0x7f7a08038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 msgr2=0x7f7a201137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 --2- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a201137e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000b440 tx=0x7f7a10009250 comp rx=0 tx=0).stop 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 shutdown_connections 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 --2- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7f7a080367e0 0x7f7a08038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 --2- 172.21.15.40:0/3876194436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a20078cc0 0x7f7a201137e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 >> 172.21.15.40:0/3876194436 conn(0x7f7a200fdf00 msgr2=0x7f7a200ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 shutdown_connections 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:18.357+0000 7f7a25338700 1 -- 172.21.15.40:0/3876194436 wait complete. 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for the mgr to restart... 2024-02-16T19:15:20.585 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr epoch 5... 2024-02-16T19:15:24.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: Active manager daemon smithi040.jzygzo restarted 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: Activating manager daemon smithi040.jzygzo 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: osdmap e2: 0 total, 0 up, 0 in 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: mgrmap e6: smithi040.jzygzo(active, starting, since 0.00311882s) 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: Manager daemon smithi040.jzygzo is now available 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:15:24.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:15:24.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:23 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:15:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:24 smithi040 ceph-mon[29201]: Found migration_current of "None". Setting to last migration. 2024-02-16T19:15:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:24 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:15:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:24 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/trash_purge_schedule"}]: dispatch 2024-02-16T19:15:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:24 smithi040 ceph-mon[29201]: mgrmap e7: smithi040.jzygzo(active, since 1.00506s) 2024-02-16T19:15:25.456 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:15:25.456 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.050+0000 7fbf8531b700 1 Processor -- start 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.051+0000 7fbf8531b700 1 -- start start 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.051+0000 7fbf8531b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.051+0000 7fbf8531b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80101fc0 con 0x7fbf80101670 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.051+0000 7fbf7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.051+0000 7fbf7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44834/0 (socket says 172.21.15.40:44834) 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.051+0000 7fbf7effd700 1 -- 172.21.15.40:0/4034165448 learned_addr learned my addr 172.21.15.40:0/4034165448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.051+0000 7fbf7effd700 1 -- 172.21.15.40:0/4034165448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf80102100 con 0x7fbf80101670 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.051+0000 7fbf7effd700 1 --2- 172.21.15.40:0/4034165448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80101a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbf740098e0 tx=0x7fbf74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7268281a5ed08a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf7dffb700 1 -- 172.21.15.40:0/4034165448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf7400b860 con 0x7fbf80101670 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf7dffb700 1 -- 172.21.15.40:0/4034165448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbf7400be50 con 0x7fbf80101670 2024-02-16T19:15:25.457 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf7dffb700 1 -- 172.21.15.40:0/4034165448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf74010380 con 0x7fbf80101670 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf8531b700 1 -- 172.21.15.40:0/4034165448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 msgr2=0x7fbf80101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf8531b700 1 --2- 172.21.15.40:0/4034165448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80101a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbf740098e0 tx=0x7fbf74009250 comp rx=0 tx=0).stop 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf8531b700 1 -- 172.21.15.40:0/4034165448 shutdown_connections 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf8531b700 1 --2- 172.21.15.40:0/4034165448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80101a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf8531b700 1 -- 172.21.15.40:0/4034165448 >> 172.21.15.40:0/4034165448 conn(0x7fbf800fcdd0 msgr2=0x7fbf800ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.052+0000 7fbf8531b700 1 -- 172.21.15.40:0/4034165448 shutdown_connections 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.053+0000 7fbf8531b700 1 -- 172.21.15.40:0/4034165448 wait complete. 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.053+0000 7fbf8531b700 1 Processor -- start 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.053+0000 7fbf8531b700 1 -- start start 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.053+0000 7fbf8531b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80107a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.053+0000 7fbf8531b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80105f10 con 0x7fbf80101670 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.053+0000 7fbf7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:25.458 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.053+0000 7fbf7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80107a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44840/0 (socket says 172.21.15.40:44840) 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.054+0000 7fbf7effd700 1 -- 172.21.15.40:0/2221411649 learned_addr learned my addr 172.21.15.40:0/2221411649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.054+0000 7fbf7effd700 1 -- 172.21.15.40:0/2221411649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf740094f0 con 0x7fbf80101670 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.054+0000 7fbf7effd700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80107a90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbf74000c00 tx=0x7fbf74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.054+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf740107c0 con 0x7fbf80101670 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.054+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf80106110 con 0x7fbf80101670 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.054+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbf74010db0 con 0x7fbf80101670 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.054+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf7401ad00 con 0x7fbf80101670 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.054+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf801065a0 con 0x7fbf80101670 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.055+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbf740193c0 con 0x7fbf80101670 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.055+0000 7fbf677fe700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 0x7fbf68038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.055+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 --> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf68039350 con 0x7fbf68036790 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.055+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbf7404a650 con 0x7fbf80101670 2024-02-16T19:15:25.459 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.055+0000 7fbf7e7fc700 1 -- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 msgr2=0x7fbf68038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1383032518 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.055+0000 7fbf7e7fc700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 0x7fbf68038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.256+0000 7fbf7e7fc700 1 -- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 msgr2=0x7fbf68038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1383032518 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.256+0000 7fbf7e7fc700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 0x7fbf68038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.657+0000 7fbf7e7fc700 1 -- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 msgr2=0x7fbf68038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1383032518 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:21.657+0000 7fbf7e7fc700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 0x7fbf68038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:22.458+0000 7fbf7e7fc700 1 -- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 msgr2=0x7fbf68038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1383032518 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:22.458+0000 7fbf7e7fc700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 0x7fbf68038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:23.874+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7fbf740198a0 con 0x7fbf80101670 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:23.875+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 msgr2=0x7fbf68038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:23.875+0000 7fbf677fe700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 0x7fbf68038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.876+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7fbf74018c70 con 0x7fbf80101670 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.876+0000 7fbf677fe700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fbf68039fc0 0x7fbf6803c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:25.460 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.876+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf68039350 con 0x7fbf68039fc0 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.876+0000 7fbf7e7fc700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fbf68039fc0 0x7fbf6803c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.877+0000 7fbf7e7fc700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fbf68039fc0 0x7fbf6803c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbf700039d0 tx=0x7fbf70009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.878+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fbf68039350 con 0x7fbf68039fc0 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbf8005f5c0 con 0x7fbf68039fc0 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf677fe700 1 -- 172.21.15.40:0/2221411649 <== mgr.14118 v2:172.21.15.40:6800/2140771334 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbf8005f5c0 con 0x7fbf68039fc0 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fbf68039fc0 msgr2=0x7fbf6803c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fbf68039fc0 0x7fbf6803c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbf700039d0 tx=0x7fbf70009350 comp rx=0 tx=0).stop 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 msgr2=0x7fbf80107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80107a90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbf74000c00 tx=0x7fbf74009250 comp rx=0 tx=0).stop 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 shutdown_connections 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fbf68039fc0 0x7fbf6803c3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf80101670 0x7fbf80107a90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 --2- 172.21.15.40:0/2221411649 >> [v2:172.21.15.40:6800/1383032518,v1:172.21.15.40:6801/1383032518] conn(0x7fbf68036790 0x7fbf68038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 >> 172.21.15.40:0/2221411649 conn(0x7fbf800fcdd0 msgr2=0x7fbf800710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:25.461 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 shutdown_connections 2024-02-16T19:15:25.462 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:24.884+0000 7fbf8531b700 1 -- 172.21.15.40:0/2221411649 wait complete. 2024-02-16T19:15:25.462 INFO:teuthology.orchestra.run.smithi040.stdout:mgr epoch 5 is available 2024-02-16T19:15:25.462 INFO:teuthology.orchestra.run.smithi040.stdout:Setting orchestrator backend to cephadm... 2024-02-16T19:15:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:26 smithi040 ceph-mon[29201]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-16T19:15:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:26 smithi040 ceph-mon[29201]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-16T19:15:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:26 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:26 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:15:26.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.861+0000 7f53a1e6c700 1 Processor -- start 2024-02-16T19:15:26.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f53a1e6c700 1 -- start start 2024-02-16T19:15:26.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f53a1e6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:26.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f53a1e6c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c103190 con 0x7f539c071460 2024-02-16T19:15:26.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:26.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44906/0 (socket says 172.21.15.40:44906) 2024-02-16T19:15:26.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f539b7fe700 1 -- 172.21.15.40:0/2641164444 learned_addr learned my addr 172.21.15.40:0/2641164444 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f539b7fe700 1 -- 172.21.15.40:0/2641164444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c107ce0 con 0x7f539c071460 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f539b7fe700 1 --2- 172.21.15.40:0/2641164444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1077a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f538c0098e0 tx=0x7f538c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c103faeed491eaea server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.862+0000 7f539a7fc700 1 -- 172.21.15.40:0/2641164444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f538c00b860 con 0x7f539c071460 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f539a7fc700 1 -- 172.21.15.40:0/2641164444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f538c00be50 con 0x7f539c071460 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f539a7fc700 1 -- 172.21.15.40:0/2641164444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f538c010380 con 0x7f539c071460 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2641164444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 msgr2=0x7f539c1077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f53a1e6c700 1 --2- 172.21.15.40:0/2641164444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1077a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f538c0098e0 tx=0x7f538c009250 comp rx=0 tx=0).stop 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2641164444 shutdown_connections 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f53a1e6c700 1 --2- 172.21.15.40:0/2641164444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1077a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2641164444 >> 172.21.15.40:0/2641164444 conn(0x7f539c0fcdd0 msgr2=0x7f539c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2641164444 shutdown_connections 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.863+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2641164444 wait complete. 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.864+0000 7f53a1e6c700 1 Processor -- start 2024-02-16T19:15:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.864+0000 7f53a1e6c700 1 -- start start 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.864+0000 7f53a1e6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.864+0000 7f53a1e6c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c113ce0 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.864+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.864+0000 7f539b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44914/0 (socket says 172.21.15.40:44914) 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.864+0000 7f539b7fe700 1 -- 172.21.15.40:0/2887519326 learned_addr learned my addr 172.21.15.40:0/2887519326 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.864+0000 7f539b7fe700 1 -- 172.21.15.40:0/2887519326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c0094f0 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.865+0000 7f539b7fe700 1 --2- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1137a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f538c003050 tx=0x7f538c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.865+0000 7f5383fff700 1 -- 172.21.15.40:0/2887519326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f538c010760 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.865+0000 7f5383fff700 1 -- 172.21.15.40:0/2887519326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f538c010d50 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.865+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c113ee0 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.865+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c114b80 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.865+0000 7f5383fff700 1 -- 172.21.15.40:0/2887519326 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f538c01ae20 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.866+0000 7f5383fff700 1 -- 172.21.15.40:0/2887519326 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f538c02e440 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.866+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53880052e0 con 0x7f539c071460 2024-02-16T19:15:26.682 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.866+0000 7f5383fff700 1 --2- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f5384036660 0x7f5384038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.866+0000 7f5383fff700 1 -- 172.21.15.40:0/2887519326 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f538c04a490 con 0x7f539c071460 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.866+0000 7f539affd700 1 --2- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f5384036660 0x7f5384038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.867+0000 7f539affd700 1 --2- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f5384036660 0x7f5384038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5390009990 tx=0x7f5390006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:25.871+0000 7f5383fff700 1 -- 172.21.15.40:0/2887519326 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f538c018940 con 0x7f539c071460 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.077+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f5388000bf0 con 0x7f5384036660 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.083+0000 7f5383fff700 1 -- 172.21.15.40:0/2887519326 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5388000bf0 con 0x7f5384036660 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.085+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f5384036660 msgr2=0x7f5384038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.085+0000 7f53a1e6c700 1 --2- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f5384036660 0x7f5384038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5390009990 tx=0x7f5390006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.085+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 msgr2=0x7f539c1137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.085+0000 7f53a1e6c700 1 --2- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1137a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f538c003050 tx=0x7f538c009250 comp rx=0 tx=0).stop 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.085+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 shutdown_connections 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.085+0000 7f53a1e6c700 1 --2- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f5384036660 0x7f5384038b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.085+0000 7f53a1e6c700 1 --2- 172.21.15.40:0/2887519326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539c071460 0x7f539c1137a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.085+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 >> 172.21.15.40:0/2887519326 conn(0x7f539c0fcdd0 msgr2=0x7f539c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:26.683 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.086+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 shutdown_connections 2024-02-16T19:15:26.684 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:26.086+0000 7f53a1e6c700 1 -- 172.21.15.40:0/2887519326 wait complete. 2024-02-16T19:15:27.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:27 smithi040 ceph-mon[29201]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:27 smithi040 ceph-mon[29201]: mgrmap e8: smithi040.jzygzo(active, since 2s) 2024-02-16T19:15:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:27 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:27 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:15:27.825 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.068+0000 7face14b3700 1 Processor -- start 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.068+0000 7face14b3700 1 -- start start 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.068+0000 7face14b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc105380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.068+0000 7face14b3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc1058c0 con 0x7facdc076c20 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.069+0000 7facdaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.069+0000 7facdaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44918/0 (socket says 172.21.15.40:44918) 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.069+0000 7facdaffd700 1 -- 172.21.15.40:0/3074879011 learned_addr learned my addr 172.21.15.40:0/3074879011 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.069+0000 7facdaffd700 1 -- 172.21.15.40:0/3074879011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc105a00 con 0x7facdc076c20 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.069+0000 7facdaffd700 1 --2- 172.21.15.40:0/3074879011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc105380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7facd4009e50 tx=0x7facd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17be924561f03299 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.069+0000 7facd9ffb700 1 -- 172.21.15.40:0/3074879011 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd40092e0 con 0x7facdc076c20 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.069+0000 7facd9ffb700 1 -- 172.21.15.40:0/3074879011 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7facd4012430 con 0x7facdc076c20 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.069+0000 7facd9ffb700 1 -- 172.21.15.40:0/3074879011 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd40075c0 con 0x7facdc076c20 2024-02-16T19:15:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.070+0000 7face14b3700 1 -- 172.21.15.40:0/3074879011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 msgr2=0x7facdc105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.070+0000 7face14b3700 1 --2- 172.21.15.40:0/3074879011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc105380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7facd4009e50 tx=0x7facd4009840 comp rx=0 tx=0).stop 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.070+0000 7face14b3700 1 -- 172.21.15.40:0/3074879011 shutdown_connections 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.070+0000 7face14b3700 1 --2- 172.21.15.40:0/3074879011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc105380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.070+0000 7face14b3700 1 -- 172.21.15.40:0/3074879011 >> 172.21.15.40:0/3074879011 conn(0x7facdc0fcdd0 msgr2=0x7facdc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.070+0000 7face14b3700 1 -- 172.21.15.40:0/3074879011 shutdown_connections 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.070+0000 7face14b3700 1 -- 172.21.15.40:0/3074879011 wait complete. 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7face14b3700 1 Processor -- start 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7face14b3700 1 -- start start 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7face14b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc111560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7face14b3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc111aa0 con 0x7facdc076c20 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7facdaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc111560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7facdaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc111560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44926/0 (socket says 172.21.15.40:44926) 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7facdaffd700 1 -- 172.21.15.40:0/513088837 learned_addr learned my addr 172.21.15.40:0/513088837 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7facdaffd700 1 -- 172.21.15.40:0/513088837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd4009ae0 con 0x7facdc076c20 2024-02-16T19:15:27.827 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7facdaffd700 1 --2- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc111560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facd4009ab0 tx=0x7facd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7facd37fe700 1 -- 172.21.15.40:0/513088837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd40092e0 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7facd37fe700 1 -- 172.21.15.40:0/513088837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7facd4007d10 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.071+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc111ca0 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.072+0000 7facd37fe700 1 -- 172.21.15.40:0/513088837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd401c9a0 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.072+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc112940 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.072+0000 7facd37fe700 1 -- 172.21.15.40:0/513088837 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7facd4019070 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.072+0000 7facd37fe700 1 --2- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7facc80367e0 0x7facc8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.072+0000 7facd37fe700 1 -- 172.21.15.40:0/513088837 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7facd4049f30 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.072+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc05f5c0 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.073+0000 7facda7fc700 1 --2- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7facc80367e0 0x7facc8038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.073+0000 7facda7fc700 1 --2- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7facc80367e0 0x7facc8038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7facc4009990 tx=0x7facc4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.077+0000 7facd37fe700 1 -- 172.21.15.40:0/513088837 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7facd4017020 con 0x7facdc076c20 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.270+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7facdc0ff0c0 con 0x7facc80367e0 2024-02-16T19:15:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.271+0000 7facd37fe700 1 -- 172.21.15.40:0/513088837 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7facdc0ff0c0 con 0x7facc80367e0 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7facc80367e0 msgr2=0x7facc8038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 --2- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7facc80367e0 0x7facc8038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7facc4009990 tx=0x7facc4006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 msgr2=0x7facdc111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 --2- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc111560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facd4009ab0 tx=0x7facd4009840 comp rx=0 tx=0).stop 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 shutdown_connections 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 --2- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7facc80367e0 0x7facc8038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 --2- 172.21.15.40:0/513088837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc076c20 0x7facdc111560 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 >> 172.21.15.40:0/513088837 conn(0x7facdc0fcdd0 msgr2=0x7facdc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 shutdown_connections 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:27.273+0000 7face14b3700 1 -- 172.21.15.40:0/513088837 wait complete. 2024-02-16T19:15:27.829 INFO:teuthology.orchestra.run.smithi040.stdout:Generating ssh key... 2024-02-16T19:15:28.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:28 smithi040 ceph-mon[29201]: [16/Feb/2024:19:15:26] ENGINE Bus STARTING 2024-02-16T19:15:28.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:28 smithi040 ceph-mon[29201]: [16/Feb/2024:19:15:27] ENGINE Serving on https://172.21.15.40:7150 2024-02-16T19:15:28.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:28 smithi040 ceph-mon[29201]: [16/Feb/2024:19:15:27] ENGINE Bus STARTED 2024-02-16T19:15:29.425 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:29 smithi040 ceph-mon[29201]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:29.425 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:29 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:29.425 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:29 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:29.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.251+0000 7f604c886700 1 Processor -- start 2024-02-16T19:15:29.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.251+0000 7f604c886700 1 -- start start 2024-02-16T19:15:29.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.251+0000 7f604c886700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60480771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.251+0000 7f604c886700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60480790f0 con 0x7f6048078ce0 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.251+0000 7f604659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60480771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.251+0000 7f604659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60480771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59254/0 (socket says 172.21.15.40:59254) 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.251+0000 7f604659c700 1 -- 172.21.15.40:0/328109073 learned_addr learned my addr 172.21.15.40:0/328109073 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.252+0000 7f604659c700 1 -- 172.21.15.40:0/328109073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6048077730 con 0x7f6048078ce0 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.252+0000 7f604659c700 1 --2- 172.21.15.40:0/328109073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60480771f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f60380098e0 tx=0x7f6038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fb8c57878ab56a server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.252+0000 7f604559a700 1 -- 172.21.15.40:0/328109073 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f603800b860 con 0x7f6048078ce0 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.252+0000 7f604559a700 1 -- 172.21.15.40:0/328109073 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f603800be50 con 0x7f6048078ce0 2024-02-16T19:15:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.252+0000 7f604559a700 1 -- 172.21.15.40:0/328109073 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6038010380 con 0x7f6048078ce0 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.252+0000 7f604c886700 1 -- 172.21.15.40:0/328109073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 msgr2=0x7f60480771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.252+0000 7f604c886700 1 --2- 172.21.15.40:0/328109073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60480771f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f60380098e0 tx=0x7f6038009250 comp rx=0 tx=0).stop 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 -- 172.21.15.40:0/328109073 shutdown_connections 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 --2- 172.21.15.40:0/328109073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60480771f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 -- 172.21.15.40:0/328109073 >> 172.21.15.40:0/328109073 conn(0x7f60480fded0 msgr2=0x7f6048100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 -- 172.21.15.40:0/328109073 shutdown_connections 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 -- 172.21.15.40:0/328109073 wait complete. 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 Processor -- start 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 -- start start 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60481137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604c886700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6048113d20 con 0x7f6048078ce0 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60481137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60481137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59262/0 (socket says 172.21.15.40:59262) 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.253+0000 7f604659c700 1 -- 172.21.15.40:0/3333888287 learned_addr learned my addr 172.21.15.40:0/3333888287 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.254+0000 7f604659c700 1 -- 172.21.15.40:0/3333888287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60380094f0 con 0x7f6048078ce0 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.254+0000 7f604659c700 1 --2- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60481137e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6038006fd0 tx=0x7f6038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.254+0000 7f6036ffd700 1 -- 172.21.15.40:0/3333888287 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60380107c0 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.254+0000 7f6036ffd700 1 -- 172.21.15.40:0/3333888287 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6038010db0 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.254+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6048113f20 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.254+0000 7f6036ffd700 1 -- 172.21.15.40:0/3333888287 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f603801ada0 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.254+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6048114bc0 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.255+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604805f5c0 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.255+0000 7f6036ffd700 1 -- 172.21.15.40:0/3333888287 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f6038010920 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.256+0000 7f6036ffd700 1 --2- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f60300367e0 0x7f6030038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.256+0000 7f6036ffd700 1 -- 172.21.15.40:0/3333888287 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6038049e30 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.256+0000 7f6045d9b700 1 --2- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f60300367e0 0x7f6030038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.256+0000 7f6045d9b700 1 --2- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f60300367e0 0x7f6030038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f603c009990 tx=0x7f603c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.260+0000 7f6036ffd700 1 -- 172.21.15.40:0/3333888287 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6038023770 con 0x7f6048078ce0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.451+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f60481001f0 con 0x7f60300367e0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.862+0000 7f6036ffd700 1 -- 172.21.15.40:0/3333888287 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f60481001f0 con 0x7f60300367e0 2024-02-16T19:15:29.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f60300367e0 msgr2=0x7f6030038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 --2- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f60300367e0 0x7f6030038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f603c009990 tx=0x7f603c006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 msgr2=0x7f60481137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 --2- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60481137e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6038006fd0 tx=0x7f6038009250 comp rx=0 tx=0).stop 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 shutdown_connections 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 --2- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f60300367e0 0x7f6030038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 --2- 172.21.15.40:0/3333888287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048078ce0 0x7f60481137e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 >> 172.21.15.40:0/3333888287 conn(0x7f60480fded0 msgr2=0x7f60480ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 shutdown_connections 2024-02-16T19:15:29.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:28.864+0000 7f604c886700 1 -- 172.21.15.40:0/3333888287 wait complete. 2024-02-16T19:15:30.305 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:30 smithi040 ceph-mon[29201]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:30.305 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:30 smithi040 ceph-mon[29201]: Generating ssh key... 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI2GDFuSLO4krAncSHFFBZv3EQfFZ+9j2mjloFepe/I89arzS9Z+5z2CAjMjRlc54//8K7a3MUXCNXGI5TpyapjavrAUEDL4cc9kMLKsF6KhrOHCWwwDWYUl4W0yKoVBaeXfCz2qiv9427HMak0842KLyBXQS1eWhTVSRiwzElLNc4OjAazZ6a6cvnJL8Ifr4TA1B56spp6RAzHag5ziPStUQG2tUtapLoyYEzt2kMX0DVdduDBNkTYvK3VuC99K7Sk5AuN6qPRlcXYAYXJS4U7jW15PoWedAugrBYkqSgPDOLhBPqaAGK/8WUmlwAr4NIzsQ1RAiQ9jeaxtpyUjHkFDSW+osU9ioHujsBsxzwyouIz8BEQH6ecNmhT1HYrdJv47+3xDZxltSPy0SGUuGVj9wGLaxBl/AyolO9TGdK+KWqzlKuO15xkem+Pwrd1fzxzok5JBaEnvX5z/HXuW2bkrO82pfU2GL6dVnOwd/M09ssazbpHsjI0pwcGkzwa08= ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.801+0000 7f07c637f700 1 Processor -- start 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.801+0000 7f07c637f700 1 -- start start 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.801+0000 7f07c637f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0104190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07c637f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c01046d0 con 0x7f07c0103d80 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0104190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59274/0 (socket says 172.21.15.40:59274) 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07bffff700 1 -- 172.21.15.40:0/2559698121 learned_addr learned my addr 172.21.15.40:0/2559698121 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:30.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07bffff700 1 -- 172.21.15.40:0/2559698121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0104810 con 0x7f07c0103d80 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07bffff700 1 --2- 172.21.15.40:0/2559698121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0104190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f07b00098e0 tx=0x7f07b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a757955830d9912 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07beffd700 1 -- 172.21.15.40:0/2559698121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07b000b860 con 0x7f07c0103d80 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07beffd700 1 -- 172.21.15.40:0/2559698121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f07b000be50 con 0x7f07c0103d80 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.802+0000 7f07beffd700 1 -- 172.21.15.40:0/2559698121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07b0010380 con 0x7f07c0103d80 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.803+0000 7f07c637f700 1 -- 172.21.15.40:0/2559698121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 msgr2=0x7f07c0104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.803+0000 7f07c637f700 1 --2- 172.21.15.40:0/2559698121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0104190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f07b00098e0 tx=0x7f07b0009250 comp rx=0 tx=0).stop 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.803+0000 7f07c637f700 1 -- 172.21.15.40:0/2559698121 shutdown_connections 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.803+0000 7f07c637f700 1 --2- 172.21.15.40:0/2559698121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0104190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.803+0000 7f07c637f700 1 -- 172.21.15.40:0/2559698121 >> 172.21.15.40:0/2559698121 conn(0x7f07c00fc580 msgr2=0x7f07c00fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.803+0000 7f07c637f700 1 -- 172.21.15.40:0/2559698121 shutdown_connections 2024-02-16T19:15:30.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.803+0000 7f07c637f700 1 -- 172.21.15.40:0/2559698121 wait complete. 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.803+0000 7f07c637f700 1 Processor -- start 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.804+0000 7f07c637f700 1 -- start start 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.804+0000 7f07c637f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0111800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.804+0000 7f07c637f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c0111d40 con 0x7f07c0103d80 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.804+0000 7f07bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0111800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.804+0000 7f07bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0111800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59278/0 (socket says 172.21.15.40:59278) 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.804+0000 7f07bffff700 1 -- 172.21.15.40:0/2471729838 learned_addr learned my addr 172.21.15.40:0/2471729838 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.804+0000 7f07bffff700 1 -- 172.21.15.40:0/2471729838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b00094f0 con 0x7f07c0103d80 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.804+0000 7f07bffff700 1 --2- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0111800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b440 tx=0x7f07b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.805+0000 7f07bcff9700 1 -- 172.21.15.40:0/2471729838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07b0010760 con 0x7f07c0103d80 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.805+0000 7f07bcff9700 1 -- 172.21.15.40:0/2471729838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f07b0010d50 con 0x7f07c0103d80 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.805+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c0111f40 con 0x7f07c0103d80 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.805+0000 7f07bcff9700 1 -- 172.21.15.40:0/2471729838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07b0024d80 con 0x7f07c0103d80 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.805+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c0112be0 con 0x7f07c0103d80 2024-02-16T19:15:30.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.805+0000 7f07bcff9700 1 -- 172.21.15.40:0/2471729838 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f07b00108c0 con 0x7f07c0103d80 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.806+0000 7f07bcff9700 1 --2- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f07a80367e0 0x7f07a8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.806+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c005f5c0 con 0x7f07c0103d80 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.806+0000 7f07bcff9700 1 -- 172.21.15.40:0/2471729838 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f07b004a570 con 0x7f07c0103d80 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.806+0000 7f07bf7fe700 1 --2- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f07a80367e0 0x7f07a8038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.806+0000 7f07bf7fe700 1 --2- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f07a80367e0 0x7f07a8038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009990 tx=0x7f07b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:29.810+0000 7f07bcff9700 1 -- 172.21.15.40:0/2471729838 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f07b002d3f0 con 0x7f07c0103d80 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.003+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f07c0101430 con 0x7f07a80367e0 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.003+0000 7f07bcff9700 1 -- 172.21.15.40:0/2471729838 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f07c0101430 con 0x7f07a80367e0 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f07a80367e0 msgr2=0x7f07a8038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 --2- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f07a80367e0 0x7f07a8038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009990 tx=0x7f07b4006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 msgr2=0x7f07c0111800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 --2- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0111800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b440 tx=0x7f07b0009250 comp rx=0 tx=0).stop 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 shutdown_connections 2024-02-16T19:15:30.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 --2- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f07a80367e0 0x7f07a8038c90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:30.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 --2- 172.21.15.40:0/2471729838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07c0103d80 0x7f07c0111800 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:30.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 >> 172.21.15.40:0/2471729838 conn(0x7f07c00fc580 msgr2=0x7f07c00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:30.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 shutdown_connections 2024-02-16T19:15:30.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:30.006+0000 7f07c637f700 1 -- 172.21.15.40:0/2471729838 wait complete. 2024-02-16T19:15:30.619 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-16T19:15:30.619 INFO:teuthology.orchestra.run.smithi040.stdout:Adding key to root@localhost authorized_keys... 2024-02-16T19:15:30.619 INFO:teuthology.orchestra.run.smithi040.stdout:Adding host smithi040... 2024-02-16T19:15:31.233 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:31 smithi040 ceph-mon[29201]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:32.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:32 smithi040 ceph-mon[29201]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi040", "addr": "172.21.15.40", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:32.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:32 smithi040 ceph-mon[29201]: Deploying cephadm binary to smithi040 2024-02-16T19:15:33.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Added host 'smithi040' with addr '172.21.15.40' 2024-02-16T19:15:33.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.026+0000 7f65f40b5700 1 Processor -- start 2024-02-16T19:15:33.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.027+0000 7f65f40b5700 1 -- start start 2024-02-16T19:15:33.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.027+0000 7f65f40b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec103020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:33.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.027+0000 7f65f40b5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec103560 con 0x7f65ec102c10 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.027+0000 7f65f1e51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.027+0000 7f65f1e51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59286/0 (socket says 172.21.15.40:59286) 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.027+0000 7f65f1e51700 1 -- 172.21.15.40:0/1532920466 learned_addr learned my addr 172.21.15.40:0/1532920466 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.027+0000 7f65f1e51700 1 -- 172.21.15.40:0/1532920466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec1036a0 con 0x7f65ec102c10 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f1e51700 1 --2- 172.21.15.40:0/1532920466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec103020 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65e80098e0 tx=0x7f65e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31923d4029c8bc5 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f0e4f700 1 -- 172.21.15.40:0/1532920466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65e800b860 con 0x7f65ec102c10 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f0e4f700 1 -- 172.21.15.40:0/1532920466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f65e800be50 con 0x7f65ec102c10 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f0e4f700 1 -- 172.21.15.40:0/1532920466 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65e8010380 con 0x7f65ec102c10 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f40b5700 1 -- 172.21.15.40:0/1532920466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 msgr2=0x7f65ec103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f40b5700 1 --2- 172.21.15.40:0/1532920466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec103020 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65e80098e0 tx=0x7f65e8009250 comp rx=0 tx=0).stop 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f40b5700 1 -- 172.21.15.40:0/1532920466 shutdown_connections 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f40b5700 1 --2- 172.21.15.40:0/1532920466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec103020 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:33.933 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f40b5700 1 -- 172.21.15.40:0/1532920466 >> 172.21.15.40:0/1532920466 conn(0x7f65ec0fc580 msgr2=0x7f65ec0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f40b5700 1 -- 172.21.15.40:0/1532920466 shutdown_connections 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.028+0000 7f65f40b5700 1 -- 172.21.15.40:0/1532920466 wait complete. 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.029+0000 7f65f40b5700 1 Processor -- start 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.029+0000 7f65f40b5700 1 -- start start 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.029+0000 7f65f40b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec111830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.029+0000 7f65f40b5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec111d70 con 0x7f65ec102c10 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.029+0000 7f65f1e51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec111830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.029+0000 7f65f1e51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec111830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59290/0 (socket says 172.21.15.40:59290) 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.029+0000 7f65f1e51700 1 -- 172.21.15.40:0/2217955402 learned_addr learned my addr 172.21.15.40:0/2217955402 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.029+0000 7f65f1e51700 1 -- 172.21.15.40:0/2217955402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e80094f0 con 0x7f65ec102c10 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.030+0000 7f65f1e51700 1 --2- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec111830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f65e800b440 tx=0x7f65e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.030+0000 7f65e27fc700 1 -- 172.21.15.40:0/2217955402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65e80107c0 con 0x7f65ec102c10 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.030+0000 7f65e27fc700 1 -- 172.21.15.40:0/2217955402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f65e8010db0 con 0x7f65ec102c10 2024-02-16T19:15:33.934 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.030+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec111f70 con 0x7f65ec102c10 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.030+0000 7f65e27fc700 1 -- 172.21.15.40:0/2217955402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65e801ad30 con 0x7f65ec102c10 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.030+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec112c10 con 0x7f65ec102c10 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.031+0000 7f65e27fc700 1 -- 172.21.15.40:0/2217955402 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f65e8010920 con 0x7f65ec102c10 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.031+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec05f5c0 con 0x7f65ec102c10 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.031+0000 7f65e27fc700 1 --2- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f65d80367e0 0x7f65d8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.031+0000 7f65e27fc700 1 -- 172.21.15.40:0/2217955402 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f65e804ac10 con 0x7f65ec102c10 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.031+0000 7f65f1650700 1 --2- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f65d80367e0 0x7f65d8038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.031+0000 7f65f1650700 1 --2- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f65d80367e0 0x7f65d8038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009990 tx=0x7f65dc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.036+0000 7f65e27fc700 1 -- 172.21.15.40:0/2217955402 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f65e801a430 con 0x7f65ec102c10 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:31.231+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi040", "addr": "172.21.15.40", "target": ["mon-mgr", ""]}) v1 -- 0x7f65ec102660 con 0x7f65d80367e0 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.363+0000 7f65e27fc700 1 -- 172.21.15.40:0/2217955402 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f65ec102660 con 0x7f65d80367e0 2024-02-16T19:15:33.935 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f65d80367e0 msgr2=0x7f65d8038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 --2- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f65d80367e0 0x7f65d8038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009990 tx=0x7f65dc006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 msgr2=0x7f65ec111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 --2- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec111830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f65e800b440 tx=0x7f65e8009250 comp rx=0 tx=0).stop 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 shutdown_connections 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 --2- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f65d80367e0 0x7f65d8038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 --2- 172.21.15.40:0/2217955402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65ec102c10 0x7f65ec111830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 >> 172.21.15.40:0/2217955402 conn(0x7f65ec0fc580 msgr2=0x7f65ec0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 shutdown_connections 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:33.366+0000 7f65f40b5700 1 -- 172.21.15.40:0/2217955402 wait complete. 2024-02-16T19:15:33.936 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying mon service with default placement... 2024-02-16T19:15:34.536 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:34 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:34.536 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:34 smithi040 ceph-mon[29201]: Added host smithi040 2024-02-16T19:15:34.536 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:34 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.306+0000 7f33b4fe6700 1 Processor -- start 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.306+0000 7f33b4fe6700 1 -- start start 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.307+0000 7f33b4fe6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.307+0000 7f33b4fe6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0101fc0 con 0x7f33b0101670 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.307+0000 7f33ae59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.307+0000 7f33ae59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59302/0 (socket says 172.21.15.40:59302) 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.307+0000 7f33ae59c700 1 -- 172.21.15.40:0/4202931847 learned_addr learned my addr 172.21.15.40:0/4202931847 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:35.266 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.307+0000 7f33ae59c700 1 -- 172.21.15.40:0/4202931847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0102100 con 0x7f33b0101670 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.307+0000 7f33ae59c700 1 --2- 172.21.15.40:0/4202931847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0101a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f33a00098e0 tx=0x7f33a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15b80f3979a21e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33ad59a700 1 -- 172.21.15.40:0/4202931847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33a000b860 con 0x7f33b0101670 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33ad59a700 1 -- 172.21.15.40:0/4202931847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f33a000be50 con 0x7f33b0101670 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33ad59a700 1 -- 172.21.15.40:0/4202931847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33a0010380 con 0x7f33b0101670 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33b4fe6700 1 -- 172.21.15.40:0/4202931847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 msgr2=0x7f33b0101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33b4fe6700 1 --2- 172.21.15.40:0/4202931847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0101a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f33a00098e0 tx=0x7f33a0009250 comp rx=0 tx=0).stop 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33b4fe6700 1 -- 172.21.15.40:0/4202931847 shutdown_connections 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33b4fe6700 1 --2- 172.21.15.40:0/4202931847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0101a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33b4fe6700 1 -- 172.21.15.40:0/4202931847 >> 172.21.15.40:0/4202931847 conn(0x7f33b00fcdd0 msgr2=0x7f33b00ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33b4fe6700 1 -- 172.21.15.40:0/4202931847 shutdown_connections 2024-02-16T19:15:35.267 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.308+0000 7f33b4fe6700 1 -- 172.21.15.40:0/4202931847 wait complete. 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.309+0000 7f33b4fe6700 1 Processor -- start 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.309+0000 7f33b4fe6700 1 -- start start 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.309+0000 7f33b4fe6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0114220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.309+0000 7f33b4fe6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0076bf0 con 0x7f33b0101670 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.309+0000 7f33ae59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0114220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.309+0000 7f33ae59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0114220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59318/0 (socket says 172.21.15.40:59318) 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.309+0000 7f33ae59c700 1 -- 172.21.15.40:0/2323837727 learned_addr learned my addr 172.21.15.40:0/2323837727 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.310+0000 7f33ae59c700 1 -- 172.21.15.40:0/2323837727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a00094f0 con 0x7f33b0101670 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.310+0000 7f33ae59c700 1 --2- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0114220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33a000b440 tx=0x7f33a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.310+0000 7f339effd700 1 -- 172.21.15.40:0/2323837727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33a00107c0 con 0x7f33b0101670 2024-02-16T19:15:35.268 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.310+0000 7f339effd700 1 -- 172.21.15.40:0/2323837727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f33a0010db0 con 0x7f33b0101670 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.310+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b0076df0 con 0x7f33b0101670 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.310+0000 7f339effd700 1 -- 172.21.15.40:0/2323837727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33a001ada0 con 0x7f33b0101670 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.310+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b0114920 con 0x7f33b0101670 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.311+0000 7f339effd700 1 -- 172.21.15.40:0/2323837727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f33a0010920 con 0x7f33b0101670 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.311+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b004bed0 con 0x7f33b0101670 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.311+0000 7f339effd700 1 --2- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f33980367e0 0x7f3398038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.311+0000 7f339effd700 1 -- 172.21.15.40:0/2323837727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f33a004acc0 con 0x7f33b0101670 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.311+0000 7f33add9b700 1 --2- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f33980367e0 0x7f3398038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.312+0000 7f33add9b700 1 --2- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f33980367e0 0x7f3398038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009990 tx=0x7f33a4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.316+0000 7f339effd700 1 -- 172.21.15.40:0/2323837727 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f33a0018940 con 0x7f33b0101670 2024-02-16T19:15:35.269 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.534+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f33b004bed0 con 0x7f33980367e0 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.541+0000 7f339effd700 1 -- 172.21.15.40:0/2323837727 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f33b004bed0 con 0x7f33980367e0 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f33980367e0 msgr2=0x7f3398038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 --2- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f33980367e0 0x7f3398038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009990 tx=0x7f33a4006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 msgr2=0x7f33b0114220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 --2- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0114220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33a000b440 tx=0x7f33a0009250 comp rx=0 tx=0).stop 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 shutdown_connections 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 --2- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f33980367e0 0x7f3398038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 --2- 172.21.15.40:0/2323837727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0101670 0x7f33b0114220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 >> 172.21.15.40:0/2323837727 conn(0x7f33b00fcdd0 msgr2=0x7f33b00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 shutdown_connections 2024-02-16T19:15:35.270 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:34.544+0000 7f33b4fe6700 1 -- 172.21.15.40:0/2323837727 wait complete. 2024-02-16T19:15:35.271 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying mgr service with default placement... 2024-02-16T19:15:35.846 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:35 smithi040 ceph-mon[29201]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:35.846 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:35 smithi040 ceph-mon[29201]: Saving service mon spec with placement count:5 2024-02-16T19:15:35.846 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:35 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.649+0000 7fabf1a1e700 1 Processor -- start 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.649+0000 7fabf1a1e700 1 -- start start 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.650+0000 7fabf1a1e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.650+0000 7fabf1a1e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec105920 con 0x7fabec071420 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.650+0000 7fabeaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.650+0000 7fabeaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59326/0 (socket says 172.21.15.40:59326) 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.650+0000 7fabeaffd700 1 -- 172.21.15.40:0/3690490740 learned_addr learned my addr 172.21.15.40:0/3690490740 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.650+0000 7fabeaffd700 1 -- 172.21.15.40:0/3690490740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec105a60 con 0x7fabec071420 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.650+0000 7fabeaffd700 1 --2- 172.21.15.40:0/3690490740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1053e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0098e0 tx=0x7fabdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88bb2b594d7f793 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.650+0000 7fabe9ffb700 1 -- 172.21.15.40:0/3690490740 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fabdc00b860 con 0x7fabec071420 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabe9ffb700 1 -- 172.21.15.40:0/3690490740 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabdc00be90 con 0x7fabec071420 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabe9ffb700 1 -- 172.21.15.40:0/3690490740 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fabdc0103c0 con 0x7fabec071420 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabf1a1e700 1 -- 172.21.15.40:0/3690490740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 msgr2=0x7fabec1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabf1a1e700 1 --2- 172.21.15.40:0/3690490740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1053e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0098e0 tx=0x7fabdc009250 comp rx=0 tx=0).stop 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabf1a1e700 1 -- 172.21.15.40:0/3690490740 shutdown_connections 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabf1a1e700 1 --2- 172.21.15.40:0/3690490740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1053e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabf1a1e700 1 -- 172.21.15.40:0/3690490740 >> 172.21.15.40:0/3690490740 conn(0x7fabec0fcdd0 msgr2=0x7fabec0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabf1a1e700 1 -- 172.21.15.40:0/3690490740 shutdown_connections 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.651+0000 7fabf1a1e700 1 -- 172.21.15.40:0/3690490740 wait complete. 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.652+0000 7fabf1a1e700 1 Processor -- start 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.652+0000 7fabf1a1e700 1 -- start start 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.652+0000 7fabf1a1e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.652+0000 7fabf1a1e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec111af0 con 0x7fabec071420 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabeaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:37.405 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabeaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59336/0 (socket says 172.21.15.40:59336) 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabeaffd700 1 -- 172.21.15.40:0/2982147476 learned_addr learned my addr 172.21.15.40:0/2982147476 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabeaffd700 1 -- 172.21.15.40:0/2982147476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0094f0 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabeaffd700 1 --2- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1115b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006d40 tx=0x7fabdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabe37fe700 1 -- 172.21.15.40:0/2982147476 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fabdc010840 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabe37fe700 1 -- 172.21.15.40:0/2982147476 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabdc010e70 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec111cf0 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabe37fe700 1 -- 172.21.15.40:0/2982147476 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fabdc024ea0 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.653+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec112990 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.654+0000 7fabe37fe700 1 -- 172.21.15.40:0/2982147476 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fabdc01a4c0 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.654+0000 7fabe37fe700 1 --2- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fabd8036790 0x7fabd8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.654+0000 7fabe37fe700 1 -- 172.21.15.40:0/2982147476 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fabdc04b740 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.654+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec112180 con 0x7fabec071420 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.654+0000 7fabea7fc700 1 --2- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fabd8036790 0x7fabd8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:37.406 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.655+0000 7fabea7fc700 1 --2- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fabd8036790 0x7fabd8038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009990 tx=0x7fabd4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.659+0000 7fabe37fe700 1 -- 172.21.15.40:0/2982147476 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fabdc018940 con 0x7fabec071420 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.870+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fabec0ff0d0 con 0x7fabd8036790 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.877+0000 7fabe37fe700 1 -- 172.21.15.40:0/2982147476 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fabec0ff0d0 con 0x7fabd8036790 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fabd8036790 msgr2=0x7fabd8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 --2- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fabd8036790 0x7fabd8038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009990 tx=0x7fabd4006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 msgr2=0x7fabec1115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 --2- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1115b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006d40 tx=0x7fabdc009250 comp rx=0 tx=0).stop 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 shutdown_connections 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 --2- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fabd8036790 0x7fabd8038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 --2- 172.21.15.40:0/2982147476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabec071420 0x7fabec1115b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 >> 172.21.15.40:0/2982147476 conn(0x7fabec0fcdd0 msgr2=0x7fabec0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 shutdown_connections 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:36.879+0000 7fabf1a1e700 1 -- 172.21.15.40:0/2982147476 wait complete. 2024-02-16T19:15:37.407 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying crash service with default placement... 2024-02-16T19:15:37.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:37 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:37.648 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:37 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:37.648 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:37 smithi040 ceph-mon[29201]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:37.648 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:37 smithi040 ceph-mon[29201]: Saving service mgr spec with placement count:2 2024-02-16T19:15:37.648 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:37 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:39.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-02-16T19:15:39.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.823+0000 7fca8bf46700 1 Processor -- start 2024-02-16T19:15:39.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.823+0000 7fca8bf46700 1 -- start start 2024-02-16T19:15:39.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.823+0000 7fca8bf46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca84101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:39.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.823+0000 7fca8bf46700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca84101fc0 con 0x7fca84101670 2024-02-16T19:15:39.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.823+0000 7fca89ce2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca84101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.823+0000 7fca89ce2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca84101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53264/0 (socket says 172.21.15.40:53264) 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.823+0000 7fca89ce2700 1 -- 172.21.15.40:0/3387337293 learned_addr learned my addr 172.21.15.40:0/3387337293 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca89ce2700 1 -- 172.21.15.40:0/3387337293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca84102100 con 0x7fca84101670 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca89ce2700 1 --2- 172.21.15.40:0/3387337293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca84101a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca780098e0 tx=0x7fca78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473689f57e5e12d server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca88ce0700 1 -- 172.21.15.40:0/3387337293 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca7800b860 con 0x7fca84101670 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca88ce0700 1 -- 172.21.15.40:0/3387337293 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca7800be90 con 0x7fca84101670 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca88ce0700 1 -- 172.21.15.40:0/3387337293 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca780103c0 con 0x7fca84101670 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca8bf46700 1 -- 172.21.15.40:0/3387337293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 msgr2=0x7fca84101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca8bf46700 1 --2- 172.21.15.40:0/3387337293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca84101a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca780098e0 tx=0x7fca78009250 comp rx=0 tx=0).stop 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca8bf46700 1 -- 172.21.15.40:0/3387337293 shutdown_connections 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca8bf46700 1 --2- 172.21.15.40:0/3387337293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca84101a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.824+0000 7fca8bf46700 1 -- 172.21.15.40:0/3387337293 >> 172.21.15.40:0/3387337293 conn(0x7fca840fcdd0 msgr2=0x7fca840ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca8bf46700 1 -- 172.21.15.40:0/3387337293 shutdown_connections 2024-02-16T19:15:39.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca8bf46700 1 -- 172.21.15.40:0/3387337293 wait complete. 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca8bf46700 1 Processor -- start 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca8bf46700 1 -- start start 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca8bf46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca8410f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca8bf46700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8410f8a0 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca89ce2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca8410f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca89ce2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca8410f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53268/0 (socket says 172.21.15.40:53268) 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.825+0000 7fca89ce2700 1 -- 172.21.15.40:0/845465836 learned_addr learned my addr 172.21.15.40:0/845465836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.826+0000 7fca89ce2700 1 -- 172.21.15.40:0/845465836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780094f0 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.826+0000 7fca89ce2700 1 --2- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca8410f360 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fca780090b0 tx=0x7fca78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.826+0000 7fca767fc700 1 -- 172.21.15.40:0/845465836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca78010840 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.826+0000 7fca767fc700 1 -- 172.21.15.40:0/845465836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca78010e70 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.826+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8410faa0 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.826+0000 7fca767fc700 1 -- 172.21.15.40:0/845465836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca78024e40 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.826+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca84110740 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.827+0000 7fca767fc700 1 -- 172.21.15.40:0/845465836 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fca7801a4c0 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.827+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8405f5c0 con 0x7fca84101670 2024-02-16T19:15:39.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.827+0000 7fca767fc700 1 --2- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fca700389c0 0x7fca7003ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.827+0000 7fca767fc700 1 -- 172.21.15.40:0/845465836 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fca7804a4a0 con 0x7fca84101670 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.827+0000 7fca894e1700 1 --2- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fca700389c0 0x7fca7003ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.828+0000 7fca894e1700 1 --2- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fca700389c0 0x7fca7003ae70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca80009990 tx=0x7fca80006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:37.832+0000 7fca767fc700 1 -- 172.21.15.40:0/845465836 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fca78024430 con 0x7fca84101670 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.050+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fca84111320 con 0x7fca700389c0 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.054+0000 7fca767fc700 1 -- 172.21.15.40:0/845465836 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fca84111320 con 0x7fca700389c0 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fca700389c0 msgr2=0x7fca7003ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 --2- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fca700389c0 0x7fca7003ae70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca80009990 tx=0x7fca80006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 msgr2=0x7fca8410f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 --2- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca8410f360 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fca780090b0 tx=0x7fca78009250 comp rx=0 tx=0).stop 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 shutdown_connections 2024-02-16T19:15:39.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 --2- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fca700389c0 0x7fca7003ae70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:39.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 --2- 172.21.15.40:0/845465836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca84101670 0x7fca8410f360 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:39.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 >> 172.21.15.40:0/845465836 conn(0x7fca840fcdd0 msgr2=0x7fca840ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:39.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 shutdown_connections 2024-02-16T19:15:39.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:38.057+0000 7fca8bf46700 1 -- 172.21.15.40:0/845465836 wait complete. 2024-02-16T19:15:39.059 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying prometheus service with default placement... 2024-02-16T19:15:39.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:39 smithi040 ceph-mon[29201]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:39.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:39 smithi040 ceph-mon[29201]: Saving service crash spec with placement * 2024-02-16T19:15:39.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:39 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:40.288 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:40 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:40.288 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:40 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:40.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-02-16T19:15:40.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.487+0000 7fe627dce700 1 Processor -- start 2024-02-16T19:15:40.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.487+0000 7fe627dce700 1 -- start start 2024-02-16T19:15:40.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe627dce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe620101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:40.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe627dce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe620101fc0 con 0x7fe620101670 2024-02-16T19:15:40.295 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe625b6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe620101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe625b6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe620101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53274/0 (socket says 172.21.15.40:53274) 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe625b6a700 1 -- 172.21.15.40:0/44279551 learned_addr learned my addr 172.21.15.40:0/44279551 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe625b6a700 1 -- 172.21.15.40:0/44279551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe620102100 con 0x7fe620101670 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe625b6a700 1 --2- 172.21.15.40:0/44279551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe620101a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009e50 tx=0x7fe61c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3728b8f14869c630 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe624b68700 1 -- 172.21.15.40:0/44279551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe61c0092e0 con 0x7fe620101670 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe624b68700 1 -- 172.21.15.40:0/44279551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe61c012430 con 0x7fe620101670 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.488+0000 7fe624b68700 1 -- 172.21.15.40:0/44279551 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe61c0075a0 con 0x7fe620101670 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.489+0000 7fe627dce700 1 -- 172.21.15.40:0/44279551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 msgr2=0x7fe620101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.489+0000 7fe627dce700 1 --2- 172.21.15.40:0/44279551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe620101a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009e50 tx=0x7fe61c009840 comp rx=0 tx=0).stop 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.489+0000 7fe627dce700 1 -- 172.21.15.40:0/44279551 shutdown_connections 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.489+0000 7fe627dce700 1 --2- 172.21.15.40:0/44279551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe620101a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.489+0000 7fe627dce700 1 -- 172.21.15.40:0/44279551 >> 172.21.15.40:0/44279551 conn(0x7fe6200fcdd0 msgr2=0x7fe6200ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.489+0000 7fe627dce700 1 -- 172.21.15.40:0/44279551 shutdown_connections 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.489+0000 7fe627dce700 1 -- 172.21.15.40:0/44279551 wait complete. 2024-02-16T19:15:40.296 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe627dce700 1 Processor -- start 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe627dce700 1 -- start start 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe627dce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe6201169c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe627dce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201123b0 con 0x7fe620101670 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe625b6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe6201169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe625b6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe6201169c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53284/0 (socket says 172.21.15.40:53284) 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe625b6a700 1 -- 172.21.15.40:0/4008008663 learned_addr learned my addr 172.21.15.40:0/4008008663 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe625b6a700 1 -- 172.21.15.40:0/4008008663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c009ae0 con 0x7fe620101670 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe625b6a700 1 --2- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe6201169c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe61c012800 tx=0x7fe61c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe6167fc700 1 -- 172.21.15.40:0/4008008663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe61c0092e0 con 0x7fe620101670 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe620076bf0 con 0x7fe620101670 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe6167fc700 1 -- 172.21.15.40:0/4008008663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe61c007d30 con 0x7fe620101670 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.490+0000 7fe6167fc700 1 -- 172.21.15.40:0/4008008663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe61c0259a0 con 0x7fe620101670 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.491+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe620116f00 con 0x7fe620101670 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.491+0000 7fe6167fc700 1 -- 172.21.15.40:0/4008008663 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fe61c019050 con 0x7fe620101670 2024-02-16T19:15:40.297 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.491+0000 7fe6167fc700 1 --2- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fe60c036790 0x7fe60c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.491+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62010a590 con 0x7fe620101670 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.491+0000 7fe6167fc700 1 -- 172.21.15.40:0/4008008663 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe61c04a220 con 0x7fe620101670 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.492+0000 7fe625369700 1 --2- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fe60c036790 0x7fe60c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.492+0000 7fe625369700 1 --2- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fe60c036790 0x7fe60c038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe610009990 tx=0x7fe610006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.496+0000 7fe6167fc700 1 -- 172.21.15.40:0/4008008663 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe61c020020 con 0x7fe620101670 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.696+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fe62005e3d0 con 0x7fe60c036790 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.700+0000 7fe6167fc700 1 -- 172.21.15.40:0/4008008663 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fe62005e3d0 con 0x7fe60c036790 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fe60c036790 msgr2=0x7fe60c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 --2- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fe60c036790 0x7fe60c038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe610009990 tx=0x7fe610006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 msgr2=0x7fe6201169c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 --2- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe6201169c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe61c012800 tx=0x7fe61c009840 comp rx=0 tx=0).stop 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 shutdown_connections 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 --2- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fe60c036790 0x7fe60c038c40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 --2- 172.21.15.40:0/4008008663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620101670 0x7fe6201169c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:40.298 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 >> 172.21.15.40:0/4008008663 conn(0x7fe6200fcdd0 msgr2=0x7fe6200fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:40.299 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 shutdown_connections 2024-02-16T19:15:40.299 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:39.702+0000 7fe627dce700 1 -- 172.21.15.40:0/4008008663 wait complete. 2024-02-16T19:15:40.299 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying grafana service with default placement... 2024-02-16T19:15:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:41 smithi040 ceph-mon[29201]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:41 smithi040 ceph-mon[29201]: Saving service prometheus spec with placement count:1 2024-02-16T19:15:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:41 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:41 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:41.583 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-02-16T19:15:41.588 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.721+0000 7f9b217fe700 1 Processor -- start 2024-02-16T19:15:41.588 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.721+0000 7f9b217fe700 1 -- start start 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.721+0000 7f9b217fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.721+0000 7f9b217fe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1030c0 con 0x7f9b1c102770 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.721+0000 7f9b1affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.721+0000 7f9b1affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53286/0 (socket says 172.21.15.40:53286) 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.721+0000 7f9b1affd700 1 -- 172.21.15.40:0/3819715262 learned_addr learned my addr 172.21.15.40:0/3819715262 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.722+0000 7f9b1affd700 1 -- 172.21.15.40:0/3819715262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c103200 con 0x7f9b1c102770 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.722+0000 7f9b1affd700 1 --2- 172.21.15.40:0/3819715262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c102b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0098e0 tx=0x7f9b0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c382fb788a8eb35e server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.722+0000 7f9b19ffb700 1 -- 172.21.15.40:0/3819715262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b0c00b860 con 0x7f9b1c102770 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.722+0000 7f9b19ffb700 1 -- 172.21.15.40:0/3819715262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b0c00be90 con 0x7f9b1c102770 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.722+0000 7f9b19ffb700 1 -- 172.21.15.40:0/3819715262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b0c0103c0 con 0x7f9b1c102770 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.722+0000 7f9b217fe700 1 -- 172.21.15.40:0/3819715262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 msgr2=0x7f9b1c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.722+0000 7f9b217fe700 1 --2- 172.21.15.40:0/3819715262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c102b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0098e0 tx=0x7f9b0c009250 comp rx=0 tx=0).stop 2024-02-16T19:15:41.589 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 -- 172.21.15.40:0/3819715262 shutdown_connections 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 --2- 172.21.15.40:0/3819715262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c102b80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 -- 172.21.15.40:0/3819715262 >> 172.21.15.40:0/3819715262 conn(0x7f9b1c0fded0 msgr2=0x7f9b1c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 -- 172.21.15.40:0/3819715262 shutdown_connections 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 -- 172.21.15.40:0/3819715262 wait complete. 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 Processor -- start 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 -- start start 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c1115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.723+0000 7f9b217fe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c111af0 con 0x7f9b1c102770 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b1affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c1115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b1affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c1115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53288/0 (socket says 172.21.15.40:53288) 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b1affd700 1 -- 172.21.15.40:0/952695280 learned_addr learned my addr 172.21.15.40:0/952695280 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b1affd700 1 -- 172.21.15.40:0/952695280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0094f0 con 0x7f9b1c102770 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b1affd700 1 --2- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c1115b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0090b0 tx=0x7f9b0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b037fe700 1 -- 172.21.15.40:0/952695280 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b0c010840 con 0x7f9b1c102770 2024-02-16T19:15:41.590 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b037fe700 1 -- 172.21.15.40:0/952695280 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b0c010e70 con 0x7f9b1c102770 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c111cf0 con 0x7f9b1c102770 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b037fe700 1 -- 172.21.15.40:0/952695280 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b0c024e90 con 0x7f9b1c102770 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.724+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c112990 con 0x7f9b1c102770 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.725+0000 7f9b037fe700 1 -- 172.21.15.40:0/952695280 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f9b0c024430 con 0x7f9b1c102770 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.725+0000 7f9b037fe700 1 --2- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f9b04036790 0x7f9b04038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.725+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c05f5c0 con 0x7f9b1c102770 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.725+0000 7f9b037fe700 1 -- 172.21.15.40:0/952695280 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9b0c04a710 con 0x7f9b1c102770 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.726+0000 7f9b1a7fc700 1 --2- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f9b04036790 0x7f9b04038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.726+0000 7f9b1a7fc700 1 --2- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f9b04036790 0x7f9b04038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009990 tx=0x7f9b10006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.730+0000 7f9b037fe700 1 -- 172.21.15.40:0/952695280 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9b0c02c930 con 0x7f9b1c102770 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.944+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b1c1001b0 con 0x7f9b04036790 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.948+0000 7f9b037fe700 1 -- 172.21.15.40:0/952695280 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f9b1c1001b0 con 0x7f9b04036790 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f9b04036790 msgr2=0x7f9b04038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 --2- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f9b04036790 0x7f9b04038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009990 tx=0x7f9b10006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 msgr2=0x7f9b1c1115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 --2- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c1115b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0090b0 tx=0x7f9b0c009250 comp rx=0 tx=0).stop 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 shutdown_connections 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 --2- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f9b04036790 0x7f9b04038c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 --2- 172.21.15.40:0/952695280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c102770 0x7f9b1c1115b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 >> 172.21.15.40:0/952695280 conn(0x7f9b1c0fded0 msgr2=0x7f9b1c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 shutdown_connections 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:40.950+0000 7f9b217fe700 1 -- 172.21.15.40:0/952695280 wait complete. 2024-02-16T19:15:41.592 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying node-exporter service with default placement... 2024-02-16T19:15:42.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:42 smithi040 ceph-mon[29201]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:42.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:42 smithi040 ceph-mon[29201]: Saving service grafana spec with placement count:1 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.963+0000 7f8e436b4700 1 Processor -- start 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e436b4700 1 -- start start 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e436b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e436b4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1030c0 con 0x7f8e3c102770 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e41450700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e41450700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53292/0 (socket says 172.21.15.40:53292) 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e41450700 1 -- 172.21.15.40:0/749228319 learned_addr learned my addr 172.21.15.40:0/749228319 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e41450700 1 -- 172.21.15.40:0/749228319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c103200 con 0x7f8e3c102770 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e41450700 1 --2- 172.21.15.40:0/749228319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c102b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e380098e0 tx=0x7f8e38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6ebc125843a8e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.964+0000 7f8e33fff700 1 -- 172.21.15.40:0/749228319 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e3800b860 con 0x7f8e3c102770 2024-02-16T19:15:42.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e33fff700 1 -- 172.21.15.40:0/749228319 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e3800be90 con 0x7f8e3c102770 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e33fff700 1 -- 172.21.15.40:0/749228319 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e380103c0 con 0x7f8e3c102770 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e436b4700 1 -- 172.21.15.40:0/749228319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 msgr2=0x7f8e3c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e436b4700 1 --2- 172.21.15.40:0/749228319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c102b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e380098e0 tx=0x7f8e38009250 comp rx=0 tx=0).stop 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e436b4700 1 -- 172.21.15.40:0/749228319 shutdown_connections 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e436b4700 1 --2- 172.21.15.40:0/749228319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c102b80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e436b4700 1 -- 172.21.15.40:0/749228319 >> 172.21.15.40:0/749228319 conn(0x7f8e3c0fded0 msgr2=0x7f8e3c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e436b4700 1 -- 172.21.15.40:0/749228319 shutdown_connections 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.965+0000 7f8e436b4700 1 -- 172.21.15.40:0/749228319 wait complete. 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e436b4700 1 Processor -- start 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e436b4700 1 -- start start 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e436b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c1115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e436b4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c111af0 con 0x7f8e3c102770 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e41450700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c1115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e41450700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c1115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53298/0 (socket says 172.21.15.40:53298) 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e41450700 1 -- 172.21.15.40:0/185544477 learned_addr learned my addr 172.21.15.40:0/185544477 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e41450700 1 -- 172.21.15.40:0/185544477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380094f0 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e41450700 1 --2- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c1115b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e380090b0 tx=0x7f8e38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.966+0000 7f8e31ffb700 1 -- 172.21.15.40:0/185544477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e38010840 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.967+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c111cf0 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.967+0000 7f8e31ffb700 1 -- 172.21.15.40:0/185544477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e38010e70 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.967+0000 7f8e31ffb700 1 -- 172.21.15.40:0/185544477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e38024e40 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.967+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c112990 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.967+0000 7f8e31ffb700 1 -- 172.21.15.40:0/185544477 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8e3801a530 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.968+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c05f5c0 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.968+0000 7f8e31ffb700 1 --2- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f8e28036740 0x7f8e28038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.968+0000 7f8e31ffb700 1 -- 172.21.15.40:0/185544477 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8e3804a960 con 0x7f8e3c102770 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.968+0000 7f8e40c4f700 1 --2- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f8e28036740 0x7f8e28038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.968+0000 7f8e40c4f700 1 --2- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f8e28036740 0x7f8e28038bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009990 tx=0x7f8e2c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:42.900 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:41.972+0000 7f8e31ffb700 1 -- 172.21.15.40:0/185544477 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8e38018350 con 0x7f8e3c102770 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.177+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e3c1001b0 con 0x7f8e28036740 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.181+0000 7f8e31ffb700 1 -- 172.21.15.40:0/185544477 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f8e3c1001b0 con 0x7f8e28036740 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f8e28036740 msgr2=0x7f8e28038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 --2- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f8e28036740 0x7f8e28038bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009990 tx=0x7f8e2c006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 msgr2=0x7f8e3c1115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 --2- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c1115b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e380090b0 tx=0x7f8e38009250 comp rx=0 tx=0).stop 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 shutdown_connections 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 --2- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f8e28036740 0x7f8e28038bf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 --2- 172.21.15.40:0/185544477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e3c102770 0x7f8e3c1115b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 >> 172.21.15.40:0/185544477 conn(0x7f8e3c0fded0 msgr2=0x7f8e3c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 shutdown_connections 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:42.183+0000 7f8e436b4700 1 -- 172.21.15.40:0/185544477 wait complete. 2024-02-16T19:15:42.901 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying alertmanager service with default placement... 2024-02-16T19:15:43.317 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:43 smithi040 ceph-mon[29201]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:43.318 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:43 smithi040 ceph-mon[29201]: Saving service node-exporter spec with placement * 2024-02-16T19:15:43.318 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:43 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:44.115 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-02-16T19:15:44.115 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.341+0000 7f1af4656700 1 Processor -- start 2024-02-16T19:15:44.115 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.341+0000 7f1af4656700 1 -- start start 2024-02-16T19:15:44.115 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.341+0000 7f1af4656700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:44.115 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.341+0000 7f1af4656700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec101fc0 con 0x7f1aec101670 2024-02-16T19:15:44.115 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.341+0000 7f1af23f2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:44.115 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.341+0000 7f1af23f2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53302/0 (socket says 172.21.15.40:53302) 2024-02-16T19:15:44.115 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.341+0000 7f1af23f2700 1 -- 172.21.15.40:0/1705446738 learned_addr learned my addr 172.21.15.40:0/1705446738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.342+0000 7f1af23f2700 1 -- 172.21.15.40:0/1705446738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec102100 con 0x7f1aec101670 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.342+0000 7f1af23f2700 1 --2- 172.21.15.40:0/1705446738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec101a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80098e0 tx=0x7f1ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40aa63f3e29f6d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.342+0000 7f1af13f0700 1 -- 172.21.15.40:0/1705446738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ae800b860 con 0x7f1aec101670 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.342+0000 7f1af13f0700 1 -- 172.21.15.40:0/1705446738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ae800be90 con 0x7f1aec101670 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.342+0000 7f1af13f0700 1 -- 172.21.15.40:0/1705446738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ae80103c0 con 0x7f1aec101670 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 -- 172.21.15.40:0/1705446738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 msgr2=0x7f1aec101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 --2- 172.21.15.40:0/1705446738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec101a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80098e0 tx=0x7f1ae8009250 comp rx=0 tx=0).stop 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 -- 172.21.15.40:0/1705446738 shutdown_connections 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 --2- 172.21.15.40:0/1705446738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec101a80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 -- 172.21.15.40:0/1705446738 >> 172.21.15.40:0/1705446738 conn(0x7f1aec0fcdd0 msgr2=0x7f1aec0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 -- 172.21.15.40:0/1705446738 shutdown_connections 2024-02-16T19:15:44.116 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 -- 172.21.15.40:0/1705446738 wait complete. 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 Processor -- start 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 -- start start 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.343+0000 7f1af4656700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1af4656700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec110980 con 0x7f1aec101670 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1af23f2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1af23f2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec110440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53308/0 (socket says 172.21.15.40:53308) 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1af23f2700 1 -- 172.21.15.40:0/3047980421 learned_addr learned my addr 172.21.15.40:0/3047980421 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1af23f2700 1 -- 172.21.15.40:0/3047980421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80094f0 con 0x7f1aec101670 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1af23f2700 1 --2- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec110440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8006b00 tx=0x7f1ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1ae2ffd700 1 -- 172.21.15.40:0/3047980421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ae80109a0 con 0x7f1aec101670 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1ae2ffd700 1 -- 172.21.15.40:0/3047980421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ae801a430 con 0x7f1aec101670 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec110b80 con 0x7f1aec101670 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1ae2ffd700 1 -- 172.21.15.40:0/3047980421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ae80245b0 con 0x7f1aec101670 2024-02-16T19:15:44.117 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.344+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec111820 con 0x7f1aec101670 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.345+0000 7f1ae2ffd700 1 -- 172.21.15.40:0/3047980421 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1ae801aaa0 con 0x7f1aec101670 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.345+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec05f5c0 con 0x7f1aec101670 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.345+0000 7f1ae2ffd700 1 --2- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1ad8036790 0x7f1ad8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.346+0000 7f1ae2ffd700 1 -- 172.21.15.40:0/3047980421 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1ae804a6d0 con 0x7f1aec101670 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.346+0000 7f1af1bf1700 1 --2- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1ad8036790 0x7f1ad8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.346+0000 7f1af1bf1700 1 --2- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1ad8036790 0x7f1ad8038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009990 tx=0x7f1adc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.350+0000 7f1ae2ffd700 1 -- 172.21.15.40:0/3047980421 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1ae8006d70 con 0x7f1aec101670 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.557+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f1aec0fe2a0 con 0x7f1ad8036790 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.563+0000 7f1ae2ffd700 1 -- 172.21.15.40:0/3047980421 <== mgr.14118 v2:172.21.15.40:6800/2140771334 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f1aec0fe2a0 con 0x7f1ad8036790 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.565+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1ad8036790 msgr2=0x7f1ad8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.565+0000 7f1af4656700 1 --2- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1ad8036790 0x7f1ad8038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009990 tx=0x7f1adc006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.565+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 msgr2=0x7f1aec110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:44.118 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.565+0000 7f1af4656700 1 --2- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec110440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8006b00 tx=0x7f1ae8009250 comp rx=0 tx=0).stop 2024-02-16T19:15:44.119 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.565+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 shutdown_connections 2024-02-16T19:15:44.119 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.565+0000 7f1af4656700 1 --2- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1ad8036790 0x7f1ad8038c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:44.119 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.565+0000 7f1af4656700 1 --2- 172.21.15.40:0/3047980421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aec101670 0x7f1aec110440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:44.119 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.565+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 >> 172.21.15.40:0/3047980421 conn(0x7f1aec0fcdd0 msgr2=0x7f1aec0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:44.119 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.566+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 shutdown_connections 2024-02-16T19:15:44.119 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:43.566+0000 7f1af4656700 1 -- 172.21.15.40:0/3047980421 wait complete. 2024-02-16T19:15:44.705 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:44 smithi040 ceph-mon[29201]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:44.705 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:44 smithi040 ceph-mon[29201]: Saving service alertmanager spec with placement count:1 2024-02-16T19:15:44.705 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:44 smithi040 ceph-mon[29201]: from='mgr.14118 172.21.15.40:0/3396411591' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:44.706 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:44 smithi040 ceph-mon[29201]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.503+0000 7ff085c67700 1 Processor -- start 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.503+0000 7ff085c67700 1 -- start start 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.503+0000 7ff085c67700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.503+0000 7ff085c67700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080105960 con 0x7ff080076b80 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.503+0000 7ff07f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.503+0000 7ff07f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53318/0 (socket says 172.21.15.40:53318) 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.503+0000 7ff07f7fe700 1 -- 172.21.15.40:0/45924141 learned_addr learned my addr 172.21.15.40:0/45924141 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.503+0000 7ff07f7fe700 1 -- 172.21.15.40:0/45924141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080105aa0 con 0x7ff080076b80 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff07f7fe700 1 --2- 172.21.15.40:0/45924141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080105420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0700098e0 tx=0x7ff070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec2fc6c2b89e4aa server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:45.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff07e7fc700 1 -- 172.21.15.40:0/45924141 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff07000b860 con 0x7ff080076b80 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff07e7fc700 1 -- 172.21.15.40:0/45924141 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff07000be90 con 0x7ff080076b80 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff07e7fc700 1 -- 172.21.15.40:0/45924141 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0700103c0 con 0x7ff080076b80 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff085c67700 1 -- 172.21.15.40:0/45924141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 msgr2=0x7ff080105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff085c67700 1 --2- 172.21.15.40:0/45924141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080105420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0700098e0 tx=0x7ff070009250 comp rx=0 tx=0).stop 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff085c67700 1 -- 172.21.15.40:0/45924141 shutdown_connections 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff085c67700 1 --2- 172.21.15.40:0/45924141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080105420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff085c67700 1 -- 172.21.15.40:0/45924141 >> 172.21.15.40:0/45924141 conn(0x7ff0800fcdd0 msgr2=0x7ff0800ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff085c67700 1 -- 172.21.15.40:0/45924141 shutdown_connections 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.504+0000 7ff085c67700 1 -- 172.21.15.40:0/45924141 wait complete. 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.505+0000 7ff085c67700 1 Processor -- start 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.505+0000 7ff085c67700 1 -- start start 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.505+0000 7ff085c67700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.505+0000 7ff085c67700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080101320 con 0x7ff080076b80 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.505+0000 7ff07f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.505+0000 7ff07f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53328/0 (socket says 172.21.15.40:53328) 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.505+0000 7ff07f7fe700 1 -- 172.21.15.40:0/3963448417 learned_addr learned my addr 172.21.15.40:0/3963448417 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:45.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.506+0000 7ff07f7fe700 1 -- 172.21.15.40:0/3963448417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0700094f0 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.506+0000 7ff07f7fe700 1 --2- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080102ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0700090b0 tx=0x7ff070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.506+0000 7ff067fff700 1 -- 172.21.15.40:0/3963448417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0700118f0 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.506+0000 7ff067fff700 1 -- 172.21.15.40:0/3963448417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff070010d10 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.506+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080101520 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.506+0000 7ff067fff700 1 -- 172.21.15.40:0/3963448417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff07001aea0 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.506+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0801019b0 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.507+0000 7ff067fff700 1 -- 172.21.15.40:0/3963448417 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ff070010840 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.507+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08010b6b0 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.507+0000 7ff067fff700 1 --2- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7ff068036740 0x7ff068038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.507+0000 7ff067fff700 1 -- 172.21.15.40:0/3963448417 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff07004a5d0 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.507+0000 7ff07effd700 1 --2- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7ff068036740 0x7ff068038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.507+0000 7ff07effd700 1 --2- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7ff068036740 0x7ff068038bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff074009990 tx=0x7ff074006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.512+0000 7ff067fff700 1 -- 172.21.15.40:0/3963448417 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff070022cc0 con 0x7ff080076b80 2024-02-16T19:15:45.448 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.704+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ff08004bed0 con 0x7ff080076b80 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.707+0000 7ff067fff700 1 -- 172.21.15.40:0/3963448417 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff070022ae0 con 0x7ff080076b80 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.709+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7ff068036740 msgr2=0x7ff068038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 --2- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7ff068036740 0x7ff068038bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff074009990 tx=0x7ff074006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 msgr2=0x7ff080102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 --2- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080102ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0700090b0 tx=0x7ff070009250 comp rx=0 tx=0).stop 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 shutdown_connections 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 --2- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7ff068036740 0x7ff068038bf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 --2- 172.21.15.40:0/3963448417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff080076b80 0x7ff080102ea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 >> 172.21.15.40:0/3963448417 conn(0x7ff0800fcdd0 msgr2=0x7ff0800ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 shutdown_connections 2024-02-16T19:15:45.449 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:44.710+0000 7ff085c67700 1 -- 172.21.15.40:0/3963448417 wait complete. 2024-02-16T19:15:46.050 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:45 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3963448417' entity='client.admin' 2024-02-16T19:15:46.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.849+0000 7fd33a538700 1 Processor -- start 2024-02-16T19:15:46.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.849+0000 7fd33a538700 1 -- start start 2024-02-16T19:15:46.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.849+0000 7fd33a538700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd334077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:46.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.849+0000 7fd33a538700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd334079110 con 0x7fd334078d00 2024-02-16T19:15:46.660 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.849+0000 7fd333fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd334077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.849+0000 7fd333fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd334077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53330/0 (socket says 172.21.15.40:53330) 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.849+0000 7fd333fff700 1 -- 172.21.15.40:0/142648177 learned_addr learned my addr 172.21.15.40:0/142648177 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.849+0000 7fd333fff700 1 -- 172.21.15.40:0/142648177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd334077750 con 0x7fd334078d00 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd333fff700 1 --2- 172.21.15.40:0/142648177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd334077210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd3240098e0 tx=0x7fd324009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fcccef0bf55a98 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd332ffd700 1 -- 172.21.15.40:0/142648177 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd32400b860 con 0x7fd334078d00 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd332ffd700 1 -- 172.21.15.40:0/142648177 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd32400be90 con 0x7fd334078d00 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd332ffd700 1 -- 172.21.15.40:0/142648177 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3240103c0 con 0x7fd334078d00 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd33a538700 1 -- 172.21.15.40:0/142648177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 msgr2=0x7fd334077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd33a538700 1 --2- 172.21.15.40:0/142648177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd334077210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd3240098e0 tx=0x7fd324009250 comp rx=0 tx=0).stop 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd33a538700 1 -- 172.21.15.40:0/142648177 shutdown_connections 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd33a538700 1 --2- 172.21.15.40:0/142648177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd334077210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.850+0000 7fd33a538700 1 -- 172.21.15.40:0/142648177 >> 172.21.15.40:0/142648177 conn(0x7fd3340fded0 msgr2=0x7fd334100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.851+0000 7fd33a538700 1 -- 172.21.15.40:0/142648177 shutdown_connections 2024-02-16T19:15:46.661 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.851+0000 7fd33a538700 1 -- 172.21.15.40:0/142648177 wait complete. 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.851+0000 7fd33a538700 1 Processor -- start 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.851+0000 7fd33a538700 1 -- start start 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.851+0000 7fd33a538700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd3341137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd33a538700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd334113d00 con 0x7fd334078d00 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd333fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd3341137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd333fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd3341137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53346/0 (socket says 172.21.15.40:53346) 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd333fff700 1 -- 172.21.15.40:0/2644429590 learned_addr learned my addr 172.21.15.40:0/2644429590 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd333fff700 1 -- 172.21.15.40:0/2644429590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240094f0 con 0x7fd334078d00 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd333fff700 1 --2- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd3341137c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd3240090b0 tx=0x7fd324009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd330ff9700 1 -- 172.21.15.40:0/2644429590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3240118f0 con 0x7fd334078d00 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd334113f00 con 0x7fd334078d00 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd330ff9700 1 -- 172.21.15.40:0/2644429590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd324010d10 con 0x7fd334078d00 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd330ff9700 1 -- 172.21.15.40:0/2644429590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd324024e70 con 0x7fd334078d00 2024-02-16T19:15:46.662 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.852+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd334114ba0 con 0x7fd334078d00 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.853+0000 7fd330ff9700 1 -- 172.21.15.40:0/2644429590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd32401a550 con 0x7fd334078d00 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.853+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33405f5c0 con 0x7fd334078d00 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.853+0000 7fd330ff9700 1 --2- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fd31c036740 0x7fd31c038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.853+0000 7fd330ff9700 1 -- 172.21.15.40:0/2644429590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd32404a6f0 con 0x7fd334078d00 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.854+0000 7fd3337fe700 1 --2- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fd31c036740 0x7fd31c038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.854+0000 7fd3337fe700 1 --2- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fd31c036740 0x7fd31c038bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd328009990 tx=0x7fd328006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:45.858+0000 7fd330ff9700 1 -- 172.21.15.40:0/2644429590 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd32401a800 con 0x7fd334078d00 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.048+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fd33404bed0 con 0x7fd334078d00 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.052+0000 7fd330ff9700 1 -- 172.21.15.40:0/2644429590 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fd324018350 con 0x7fd334078d00 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fd31c036740 msgr2=0x7fd31c038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 --2- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fd31c036740 0x7fd31c038bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd328009990 tx=0x7fd328006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 msgr2=0x7fd3341137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 --2- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd3341137c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd3240090b0 tx=0x7fd324009250 comp rx=0 tx=0).stop 2024-02-16T19:15:46.663 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 shutdown_connections 2024-02-16T19:15:46.664 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 --2- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7fd31c036740 0x7fd31c038bf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:46.664 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 --2- 172.21.15.40:0/2644429590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd334078d00 0x7fd3341137c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:46.664 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 >> 172.21.15.40:0/2644429590 conn(0x7fd3340fded0 msgr2=0x7fd3340ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:46.664 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 shutdown_connections 2024-02-16T19:15:46.664 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:46.055+0000 7fd33a538700 1 -- 172.21.15.40:0/2644429590 wait complete. 2024-02-16T19:15:46.664 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling the dashboard module... 2024-02-16T19:15:47.278 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:47 smithi040 ceph-mon[29201]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:15:47.278 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:47 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2644429590' entity='client.admin' 2024-02-16T19:15:48.376 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:48 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/270626665' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.046+0000 7f38adaa7700 1 Processor -- start 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.046+0000 7f38adaa7700 1 -- start start 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.046+0000 7f38adaa7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.046+0000 7f38adaa7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a81030c0 con 0x7f38a8102770 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.047+0000 7f38a77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.047+0000 7f38a77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53352/0 (socket says 172.21.15.40:53352) 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.047+0000 7f38a77fe700 1 -- 172.21.15.40:0/3340357565 learned_addr learned my addr 172.21.15.40:0/3340357565 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.047+0000 7f38a77fe700 1 -- 172.21.15.40:0/3340357565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8103200 con 0x7f38a8102770 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.047+0000 7f38a77fe700 1 --2- 172.21.15.40:0/3340357565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8102b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38980098e0 tx=0x7f3898009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8208ad9d875c18f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.047+0000 7f38a67fc700 1 -- 172.21.15.40:0/3340357565 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f389800b860 con 0x7f38a8102770 2024-02-16T19:15:48.693 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.047+0000 7f38a67fc700 1 -- 172.21.15.40:0/3340357565 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f389800be90 con 0x7f38a8102770 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.047+0000 7f38a67fc700 1 -- 172.21.15.40:0/3340357565 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38980103c0 con 0x7f38a8102770 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.048+0000 7f38adaa7700 1 -- 172.21.15.40:0/3340357565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 msgr2=0x7f38a8102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.048+0000 7f38adaa7700 1 --2- 172.21.15.40:0/3340357565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8102b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38980098e0 tx=0x7f3898009250 comp rx=0 tx=0).stop 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.048+0000 7f38adaa7700 1 -- 172.21.15.40:0/3340357565 shutdown_connections 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.048+0000 7f38adaa7700 1 --2- 172.21.15.40:0/3340357565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8102b80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.048+0000 7f38adaa7700 1 -- 172.21.15.40:0/3340357565 >> 172.21.15.40:0/3340357565 conn(0x7f38a80fded0 msgr2=0x7f38a8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.048+0000 7f38adaa7700 1 -- 172.21.15.40:0/3340357565 shutdown_connections 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.048+0000 7f38adaa7700 1 -- 172.21.15.40:0/3340357565 wait complete. 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.049+0000 7f38adaa7700 1 Processor -- start 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.049+0000 7f38adaa7700 1 -- start start 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.049+0000 7f38adaa7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.049+0000 7f38adaa7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a8077190 con 0x7f38a8102770 2024-02-16T19:15:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.049+0000 7f38a77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.049+0000 7f38a77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53358/0 (socket says 172.21.15.40:53358) 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.049+0000 7f38a77fe700 1 -- 172.21.15.40:0/270626665 learned_addr learned my addr 172.21.15.40:0/270626665 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.049+0000 7f38a77fe700 1 -- 172.21.15.40:0/270626665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980094f0 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.050+0000 7f38a77fe700 1 --2- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8078d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3898011650 tx=0x7f3898009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.050+0000 7f389ffff700 1 -- 172.21.15.40:0/270626665 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3898010890 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.050+0000 7f389ffff700 1 -- 172.21.15.40:0/270626665 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3898010ec0 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.050+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a8077390 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.050+0000 7f389ffff700 1 -- 172.21.15.40:0/270626665 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f389801a430 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.050+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a8077820 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.051+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a804bed0 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.051+0000 7f389ffff700 1 -- 172.21.15.40:0/270626665 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f38980247a0 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.051+0000 7f389ffff700 1 --2- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f3894036790 0x7f3894038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.051+0000 7f389ffff700 1 -- 172.21.15.40:0/270626665 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f389804a510 con 0x7f38a8102770 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.051+0000 7f38a6ffd700 1 --2- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f3894036790 0x7f3894038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:48.695 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.052+0000 7f38a6ffd700 1 --2- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f3894036790 0x7f3894038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3890009990 tx=0x7f3890006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.058+0000 7f389ffff700 1 -- 172.21.15.40:0/270626665 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3898022510 con 0x7f38a8102770 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:47.303+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f38a8077f30 con 0x7f38a8102770 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.057+0000 7f389ffff700 1 -- 172.21.15.40:0/270626665 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f389800fda0 con 0x7f38a8102770 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.059+0000 7f389ffff700 1 -- 172.21.15.40:0/270626665 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f389804aa10 con 0x7f38a8102770 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.061+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f3894036790 msgr2=0x7f3894038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.061+0000 7f38adaa7700 1 --2- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f3894036790 0x7f3894038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3890009990 tx=0x7f3890006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.062+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 msgr2=0x7f38a8078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.062+0000 7f38adaa7700 1 --2- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8078d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3898011650 tx=0x7f3898009250 comp rx=0 tx=0).stop 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.062+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 shutdown_connections 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.062+0000 7f38adaa7700 1 --2- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f3894036790 0x7f3894038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.062+0000 7f38adaa7700 1 --2- 172.21.15.40:0/270626665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38a8102770 0x7f38a8078d10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.062+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 >> 172.21.15.40:0/270626665 conn(0x7f38a80fded0 msgr2=0x7f38a80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.062+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 shutdown_connections 2024-02-16T19:15:48.696 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:48.062+0000 7f38adaa7700 1 -- 172.21.15.40:0/270626665 wait complete. 2024-02-16T19:15:49.273 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:49 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/270626665' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-16T19:15:49.273 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:49 smithi040 ceph-mon[29201]: mgrmap e9: smithi040.jzygzo(active, since 24s) 2024-02-16T19:15:49.861 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:15:49.866 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "active_name": "smithi040.jzygzo", 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.066+0000 7f62b8fc3700 1 Processor -- start 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.066+0000 7f62b8fc3700 1 -- start start 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.067+0000 7f62b8fc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b40771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.067+0000 7f62b8fc3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40790d0 con 0x7f62b4078cc0 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.067+0000 7f62b259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b40771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.067+0000 7f62b259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b40771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45058/0 (socket says 172.21.15.40:45058) 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.067+0000 7f62b259c700 1 -- 172.21.15.40:0/1423496788 learned_addr learned my addr 172.21.15.40:0/1423496788 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.067+0000 7f62b259c700 1 -- 172.21.15.40:0/1423496788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b4077710 con 0x7f62b4078cc0 2024-02-16T19:15:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.067+0000 7f62b259c700 1 --2- 172.21.15.40:0/1423496788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b40771d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098e0 tx=0x7f629c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8882937d4739f0b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b159a700 1 -- 172.21.15.40:0/1423496788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f629c00b860 con 0x7f62b4078cc0 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b159a700 1 -- 172.21.15.40:0/1423496788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f629c00be90 con 0x7f62b4078cc0 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b159a700 1 -- 172.21.15.40:0/1423496788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f629c0103c0 con 0x7f62b4078cc0 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1423496788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 msgr2=0x7f62b40771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b8fc3700 1 --2- 172.21.15.40:0/1423496788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b40771d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098e0 tx=0x7f629c009250 comp rx=0 tx=0).stop 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1423496788 shutdown_connections 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b8fc3700 1 --2- 172.21.15.40:0/1423496788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b40771d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1423496788 >> 172.21.15.40:0/1423496788 conn(0x7f62b40fded0 msgr2=0x7f62b4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1423496788 shutdown_connections 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.068+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1423496788 wait complete. 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.069+0000 7f62b8fc3700 1 Processor -- start 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.069+0000 7f62b8fc3700 1 -- start start 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.069+0000 7f62b8fc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b4113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:49.868 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.069+0000 7f62b8fc3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4113cc0 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.069+0000 7f62b259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b4113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.069+0000 7f62b259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b4113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45060/0 (socket says 172.21.15.40:45060) 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.069+0000 7f62b259c700 1 -- 172.21.15.40:0/1206586030 learned_addr learned my addr 172.21.15.40:0/1206586030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.069+0000 7f62b259c700 1 -- 172.21.15.40:0/1206586030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0094f0 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.070+0000 7f62b259c700 1 --2- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b4113780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f629c0090b0 tx=0x7f629c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.070+0000 7f62aaffd700 1 -- 172.21.15.40:0/1206586030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f629c0118f0 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.070+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b4113ec0 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.070+0000 7f62aaffd700 1 -- 172.21.15.40:0/1206586030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f629c010d10 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.070+0000 7f62aaffd700 1 -- 172.21.15.40:0/1206586030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f629c024e70 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.070+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b4114b60 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.071+0000 7f62aaffd700 1 -- 172.21.15.40:0/1206586030 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f629c01a550 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.071+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b405f5c0 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.071+0000 7f62aaffd700 1 --2- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f62a0036790 0x7f62a0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.071+0000 7f62aaffd700 1 -- 172.21.15.40:0/1206586030 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f629c04a590 con 0x7f62b4078cc0 2024-02-16T19:15:49.869 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.071+0000 7f62b1d9b700 1 -- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f62a0036790 msgr2=0x7f62a0038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/2140771334 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.071+0000 7f62b1d9b700 1 --2- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f62a0036790 0x7f62a0038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.075+0000 7f62aaffd700 1 -- 172.21.15.40:0/1206586030 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f629c018db0 con 0x7f62b4078cc0 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.272+0000 7f62b1d9b700 1 -- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f62a0036790 msgr2=0x7f62a0038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/2140771334 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.272+0000 7f62b1d9b700 1 --2- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f62a0036790 0x7f62a0038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.326+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f62b4114ea0 con 0x7f62b4078cc0 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.326+0000 7f62aaffd700 1 -- 172.21.15.40:0/1206586030 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f629c020e90 con 0x7f62b4078cc0 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f62a0036790 msgr2=0x7f62a0038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 --2- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f62a0036790 0x7f62a0038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 msgr2=0x7f62b4113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 --2- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b4113780 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f629c0090b0 tx=0x7f629c009250 comp rx=0 tx=0).stop 2024-02-16T19:15:49.870 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 shutdown_connections 2024-02-16T19:15:49.871 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 --2- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f62a0036790 0x7f62a0038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:49.871 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 --2- 172.21.15.40:0/1206586030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62b4078cc0 0x7f62b4113780 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:49.871 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 >> 172.21.15.40:0/1206586030 conn(0x7f62b40fded0 msgr2=0x7f62b40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:49.871 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 shutdown_connections 2024-02-16T19:15:49.871 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:49.329+0000 7f62b8fc3700 1 -- 172.21.15.40:0/1206586030 wait complete. 2024-02-16T19:15:49.871 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for the mgr to restart... 2024-02-16T19:15:49.871 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr epoch 9... 2024-02-16T19:15:50.140 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:50 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1206586030' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-16T19:15:55.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: Active manager daemon smithi040.jzygzo restarted 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: Activating manager daemon smithi040.jzygzo 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: osdmap e3: 0 total, 0 up, 0 in 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: mgrmap e10: smithi040.jzygzo(active, starting, since 0.00378035s) 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: Manager daemon smithi040.jzygzo is now available 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:15:55.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:54 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:15:56.125 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:55 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:15:56.125 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:55 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/trash_purge_schedule"}]: dispatch 2024-02-16T19:15:56.126 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:55 smithi040 ceph-mon[29201]: mgrmap e11: smithi040.jzygzo(active, since 1.00571s) 2024-02-16T19:15:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2024-02-16T19:15:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-16T19:15:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-16T19:15:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2024-02-16T19:15:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.326+0000 7f13606d4700 1 Processor -- start 2024-02-16T19:15:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.326+0000 7f13606d4700 1 -- start start 2024-02-16T19:15:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.326+0000 7f13606d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f1358101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:56.377 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.326+0000 7f13606d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1358101fc0 con 0x7f1358101670 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.327+0000 7f135e470700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f1358101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.327+0000 7f135e470700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f1358101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45062/0 (socket says 172.21.15.40:45062) 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.327+0000 7f135e470700 1 -- 172.21.15.40:0/2700991249 learned_addr learned my addr 172.21.15.40:0/2700991249 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.327+0000 7f135e470700 1 -- 172.21.15.40:0/2700991249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358102100 con 0x7f1358101670 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.327+0000 7f135e470700 1 --2- 172.21.15.40:0/2700991249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f1358101a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f13480098e0 tx=0x7f1348009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7087348636f77c67 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.327+0000 7f135d46e700 1 -- 172.21.15.40:0/2700991249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f134800b860 con 0x7f1358101670 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.327+0000 7f135d46e700 1 -- 172.21.15.40:0/2700991249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f134800be90 con 0x7f1358101670 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.327+0000 7f135d46e700 1 -- 172.21.15.40:0/2700991249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13480103c0 con 0x7f1358101670 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.328+0000 7f13606d4700 1 -- 172.21.15.40:0/2700991249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 msgr2=0x7f1358101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.328+0000 7f13606d4700 1 --2- 172.21.15.40:0/2700991249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f1358101a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f13480098e0 tx=0x7f1348009250 comp rx=0 tx=0).stop 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.328+0000 7f13606d4700 1 -- 172.21.15.40:0/2700991249 shutdown_connections 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.328+0000 7f13606d4700 1 --2- 172.21.15.40:0/2700991249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f1358101a80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.328+0000 7f13606d4700 1 -- 172.21.15.40:0/2700991249 >> 172.21.15.40:0/2700991249 conn(0x7f13580fcdd0 msgr2=0x7f13580ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.328+0000 7f13606d4700 1 -- 172.21.15.40:0/2700991249 shutdown_connections 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.328+0000 7f13606d4700 1 -- 172.21.15.40:0/2700991249 wait complete. 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f13606d4700 1 Processor -- start 2024-02-16T19:15:56.378 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f13606d4700 1 -- start start 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f13606d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f135810f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f13606d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135810f8d0 con 0x7f1358101670 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f135e470700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f135810f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f135e470700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f135810f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45070/0 (socket says 172.21.15.40:45070) 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f135e470700 1 -- 172.21.15.40:0/2960474197 learned_addr learned my addr 172.21.15.40:0/2960474197 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f135e470700 1 -- 172.21.15.40:0/2960474197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13480094f0 con 0x7f1358101670 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f135e470700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f135810f390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1348011680 tx=0x7f1348009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.329+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1348010890 con 0x7f1358101670 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.330+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1348010ec0 con 0x7f1358101670 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.330+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135810fad0 con 0x7f1358101670 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.330+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13480243f0 con 0x7f1358101670 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.330+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1358110770 con 0x7f1358101670 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.330+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f134802d3f0 con 0x7f1358101670 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.331+0000 7f134effd700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 0x7f134403aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:56.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.331+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 --> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f134403b5f0 con 0x7f1344038a30 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.331+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f134804a3e0 con 0x7f1358101670 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.331+0000 7f135dc6f700 1 -- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 msgr2=0x7f134403aee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/2140771334 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.331+0000 7f135dc6f700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 0x7f134403aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.532+0000 7f135dc6f700 1 -- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 msgr2=0x7f134403aee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/2140771334 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.532+0000 7f135dc6f700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 0x7f134403aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.933+0000 7f135dc6f700 1 -- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 msgr2=0x7f134403aee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/2140771334 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:50.933+0000 7f135dc6f700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 0x7f134403aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:51.734+0000 7f135dc6f700 1 -- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 msgr2=0x7f134403aee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/2140771334 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:51.734+0000 7f135dc6f700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 0x7f134403aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:53.335+0000 7f135dc6f700 1 -- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 msgr2=0x7f134403aee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/2140771334 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:53.335+0000 7f135dc6f700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 0x7f134403aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:54.762+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f134802dd00 con 0x7f1358101670 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:54.762+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 msgr2=0x7f134403aee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:54.762+0000 7f134effd700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 0x7f134403aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.763+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f134801a8c0 con 0x7f1358101670 2024-02-16T19:15:56.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.763+0000 7f134effd700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f134403c490 0x7f134403e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.763+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 --> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f134403b5f0 con 0x7f134403c490 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.764+0000 7f135dc6f700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f134403c490 0x7f134403e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.764+0000 7f135dc6f700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f134403c490 0x7f134403e870 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13540039d0 tx=0x7f13540092f0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.765+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mgr.14160 v2:172.21.15.40:6800/277905681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f134403b5f0 con 0x7f134403c490 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.775+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 --> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f135810ff60 con 0x7f134403c490 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.775+0000 7f134effd700 1 -- 172.21.15.40:0/2960474197 <== mgr.14160 v2:172.21.15.40:6800/277905681 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f135810ff60 con 0x7f134403c490 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f134403c490 msgr2=0x7f134403e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f134403c490 0x7f134403e870 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13540039d0 tx=0x7f13540092f0 comp rx=0 tx=0).stop 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 msgr2=0x7f135810f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f135810f390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1348011680 tx=0x7f1348009250 comp rx=0 tx=0).stop 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 shutdown_connections 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f134403c490 0x7f134403e870 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358101670 0x7f135810f390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 --2- 172.21.15.40:0/2960474197 >> [v2:172.21.15.40:6800/2140771334,v1:172.21.15.40:6801/2140771334] conn(0x7f1344038a30 0x7f134403aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 >> 172.21.15.40:0/2960474197 conn(0x7f13580fcdd0 msgr2=0x7f13581063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:56.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 shutdown_connections 2024-02-16T19:15:56.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:55.776+0000 7f13606d4700 1 -- 172.21.15.40:0/2960474197 wait complete. 2024-02-16T19:15:56.382 INFO:teuthology.orchestra.run.smithi040.stdout:mgr epoch 9 is available 2024-02-16T19:15:56.382 INFO:teuthology.orchestra.run.smithi040.stdout:Generating a dashboard self-signed certificate... 2024-02-16T19:15:57.108 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:56 smithi040 ceph-mon[29201]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-16T19:15:57.108 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:56 smithi040 ceph-mon[29201]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-16T19:15:57.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-16T19:15:57.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.892+0000 7f2a1cd6c700 1 Processor -- start 2024-02-16T19:15:57.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.892+0000 7f2a1cd6c700 1 -- start start 2024-02-16T19:15:57.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.893+0000 7f2a1cd6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:57.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.893+0000 7f2a1cd6c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a18101fc0 con 0x7f2a18101670 2024-02-16T19:15:57.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.893+0000 7f2a1659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:57.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.893+0000 7f2a1659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45136/0 (socket says 172.21.15.40:45136) 2024-02-16T19:15:57.833 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.893+0000 7f2a1659c700 1 -- 172.21.15.40:0/320076401 learned_addr learned my addr 172.21.15.40:0/320076401 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.893+0000 7f2a1659c700 1 -- 172.21.15.40:0/320076401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a18102100 con 0x7f2a18101670 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.893+0000 7f2a1659c700 1 --2- 172.21.15.40:0/320076401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18101a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800bb80 tx=0x7f2a0800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f4e448699cd7b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1559a700 1 -- 172.21.15.40:0/320076401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a0800dd00 con 0x7f2a18101670 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1559a700 1 -- 172.21.15.40:0/320076401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a0800de60 con 0x7f2a18101670 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1559a700 1 -- 172.21.15.40:0/320076401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a08018920 con 0x7f2a18101670 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/320076401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 msgr2=0x7f2a18101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1cd6c700 1 --2- 172.21.15.40:0/320076401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18101a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800bb80 tx=0x7f2a0800b4f0 comp rx=0 tx=0).stop 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/320076401 shutdown_connections 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1cd6c700 1 --2- 172.21.15.40:0/320076401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18101a80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/320076401 >> 172.21.15.40:0/320076401 conn(0x7f2a180fcdd0 msgr2=0x7f2a180ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/320076401 shutdown_connections 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.894+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/320076401 wait complete. 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1cd6c700 1 Processor -- start 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1cd6c700 1 -- start start 2024-02-16T19:15:57.834 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1cd6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1cd6c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a18113c90 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18113750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45138/0 (socket says 172.21.15.40:45138) 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1659c700 1 -- 172.21.15.40:0/1226212568 learned_addr learned my addr 172.21.15.40:0/1226212568 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1659c700 1 -- 172.21.15.40:0/1226212568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0800b790 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.895+0000 7f2a1659c700 1 --2- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18113750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a080106a0 tx=0x7f2a0800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.896+0000 7f2a0effd700 1 -- 172.21.15.40:0/1226212568 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a0800dc70 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.896+0000 7f2a0effd700 1 -- 172.21.15.40:0/1226212568 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a08010c30 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.896+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a18113e90 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.896+0000 7f2a0effd700 1 -- 172.21.15.40:0/1226212568 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a08021550 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.896+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a18114b30 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.897+0000 7f2a0effd700 1 -- 172.21.15.40:0/1226212568 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f2a080223f0 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.897+0000 7f2a0effd700 1 --2- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f2a04036790 0x7f2a04038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.897+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1805f5c0 con 0x7f2a18101670 2024-02-16T19:15:57.835 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.897+0000 7f2a0effd700 1 -- 172.21.15.40:0/1226212568 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2a08014070 con 0x7f2a18101670 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.897+0000 7f2a15d9b700 1 --2- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f2a04036790 0x7f2a04038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.898+0000 7f2a15d9b700 1 --2- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f2a04036790 0x7f2a04038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009990 tx=0x7f2a00006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:56.902+0000 7f2a0effd700 1 -- 172.21.15.40:0/1226212568 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2a08026030 con 0x7f2a18101670 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.107+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 --> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a181156b0 con 0x7f2a04036790 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.275+0000 7f2a0effd700 1 -- 172.21.15.40:0/1226212568 <== mgr.14160 v2:172.21.15.40:6800/277905681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f2a181156b0 con 0x7f2a04036790 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f2a04036790 msgr2=0x7f2a04038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 --2- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f2a04036790 0x7f2a04038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009990 tx=0x7f2a00006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 msgr2=0x7f2a18113750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 --2- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18113750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a080106a0 tx=0x7f2a0800b4f0 comp rx=0 tx=0).stop 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 shutdown_connections 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 --2- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f2a04036790 0x7f2a04038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 --2- 172.21.15.40:0/1226212568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18101670 0x7f2a18113750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:57.836 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 >> 172.21.15.40:0/1226212568 conn(0x7f2a180fcdd0 msgr2=0x7f2a180ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:57.837 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 shutdown_connections 2024-02-16T19:15:57.837 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:57.278+0000 7f2a1cd6c700 1 -- 172.21.15.40:0/1226212568 wait complete. 2024-02-16T19:15:57.837 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial admin user... 2024-02-16T19:15:57.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:57 smithi040 ceph-mon[29201]: mgrmap e12: smithi040.jzygzo(active, since 2s) 2024-02-16T19:15:57.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:57 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:57.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:57 smithi040 ceph-mon[29201]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:15:57.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:57 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:57.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:57 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:59.084 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:58 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:59.084 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:58 smithi040 ceph-mon[29201]: [16/Feb/2024:19:15:58] ENGINE Bus STARTING 2024-02-16T19:15:59.084 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:58 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:59.085 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:58 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:15:59.752 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$DZV497SPWgXNgiKJzszBsutNu/VE8GxwScJOSYmUeMrWiW9PTbI5u", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708110958, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-16T19:15:59.752 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.261+0000 7fad3995a700 1 Processor -- start 2024-02-16T19:15:59.752 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.261+0000 7fad3995a700 1 -- start start 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.261+0000 7fad3995a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad34104290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.261+0000 7fad3995a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad341047d0 con 0x7fad34103e80 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.262+0000 7fad32ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad34104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.262+0000 7fad32ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad34104290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43188/0 (socket says 172.21.15.40:43188) 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.262+0000 7fad32ffd700 1 -- 172.21.15.40:0/2136277627 learned_addr learned my addr 172.21.15.40:0/2136277627 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.262+0000 7fad32ffd700 1 -- 172.21.15.40:0/2136277627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad34104910 con 0x7fad34103e80 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.262+0000 7fad32ffd700 1 --2- 172.21.15.40:0/2136277627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad34104290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009e50 tx=0x7fad2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbc33fd5b041ec2 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.262+0000 7fad31ffb700 1 -- 172.21.15.40:0/2136277627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad2c0092e0 con 0x7fad34103e80 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.262+0000 7fad31ffb700 1 -- 172.21.15.40:0/2136277627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad2c012430 con 0x7fad34103e80 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.262+0000 7fad31ffb700 1 -- 172.21.15.40:0/2136277627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad2c0075a0 con 0x7fad34103e80 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.263+0000 7fad3995a700 1 -- 172.21.15.40:0/2136277627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 msgr2=0x7fad34104290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.263+0000 7fad3995a700 1 --2- 172.21.15.40:0/2136277627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad34104290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009e50 tx=0x7fad2c009840 comp rx=0 tx=0).stop 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.263+0000 7fad3995a700 1 -- 172.21.15.40:0/2136277627 shutdown_connections 2024-02-16T19:15:59.753 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.263+0000 7fad3995a700 1 --2- 172.21.15.40:0/2136277627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad34104290 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.263+0000 7fad3995a700 1 -- 172.21.15.40:0/2136277627 >> 172.21.15.40:0/2136277627 conn(0x7fad340fc580 msgr2=0x7fad340fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.263+0000 7fad3995a700 1 -- 172.21.15.40:0/2136277627 shutdown_connections 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.263+0000 7fad3995a700 1 -- 172.21.15.40:0/2136277627 wait complete. 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.264+0000 7fad3995a700 1 Processor -- start 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.264+0000 7fad3995a700 1 -- start start 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.264+0000 7fad3995a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad341115a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.264+0000 7fad3995a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad34111ae0 con 0x7fad34103e80 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.264+0000 7fad32ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad341115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.264+0000 7fad32ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad341115a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43190/0 (socket says 172.21.15.40:43190) 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.264+0000 7fad32ffd700 1 -- 172.21.15.40:0/2820282084 learned_addr learned my addr 172.21.15.40:0/2820282084 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.265+0000 7fad32ffd700 1 -- 172.21.15.40:0/2820282084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c009ae0 con 0x7fad34103e80 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.265+0000 7fad32ffd700 1 --2- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad341115a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0127d0 tx=0x7fad2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.265+0000 7fad2b7fe700 1 -- 172.21.15.40:0/2820282084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad2c0092e0 con 0x7fad34103e80 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.265+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad34111ce0 con 0x7fad34103e80 2024-02-16T19:15:59.754 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.265+0000 7fad2b7fe700 1 -- 172.21.15.40:0/2820282084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad2c007d30 con 0x7fad34103e80 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.265+0000 7fad2b7fe700 1 -- 172.21.15.40:0/2820282084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad2c025960 con 0x7fad34103e80 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.265+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad34112980 con 0x7fad34103e80 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.266+0000 7fad2b7fe700 1 -- 172.21.15.40:0/2820282084 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fad2c019050 con 0x7fad34103e80 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.266+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3405f5c0 con 0x7fad34103e80 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.266+0000 7fad2b7fe700 1 --2- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fad200367e0 0x7fad20038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.266+0000 7fad2b7fe700 1 -- 172.21.15.40:0/2820282084 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fad2c04a320 con 0x7fad34103e80 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.266+0000 7fad327fc700 1 --2- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fad200367e0 0x7fad20038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.267+0000 7fad327fc700 1 --2- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fad200367e0 0x7fad20038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009990 tx=0x7fad1c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.271+0000 7fad2b7fe700 1 -- 172.21.15.40:0/2820282084 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fad2c023320 con 0x7fad34103e80 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.495+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 --> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fad3405f3e0 con 0x7fad200367e0 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.741+0000 7fad2b7fe700 1 -- 172.21.15.40:0/2820282084 <== mgr.14160 v2:172.21.15.40:6800/277905681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fad3405f3e0 con 0x7fad200367e0 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fad200367e0 msgr2=0x7fad20038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 --2- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fad200367e0 0x7fad20038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009990 tx=0x7fad1c006e40 comp rx=0 tx=0).stop 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 msgr2=0x7fad341115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 --2- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad341115a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0127d0 tx=0x7fad2c009840 comp rx=0 tx=0).stop 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 shutdown_connections 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 --2- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fad200367e0 0x7fad20038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 --2- 172.21.15.40:0/2820282084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34103e80 0x7fad341115a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 >> 172.21.15.40:0/2820282084 conn(0x7fad340fc580 msgr2=0x7fad340fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 shutdown_connections 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:15:58.743+0000 7fad3995a700 1 -- 172.21.15.40:0/2820282084 wait complete. 2024-02-16T19:15:59.756 INFO:teuthology.orchestra.run.smithi040.stdout:Fetching dashboard port number... 2024-02-16T19:16:00.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:59 smithi040 ceph-mon[29201]: [16/Feb/2024:19:15:58] ENGINE Serving on https://172.21.15.40:7150 2024-02-16T19:16:00.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:59 smithi040 ceph-mon[29201]: [16/Feb/2024:19:15:58] ENGINE Bus STARTED 2024-02-16T19:16:00.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:15:59 smithi040 ceph-mon[29201]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:16:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 8443 2024-02-16T19:16:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.161+0000 7fb371504700 1 Processor -- start 2024-02-16T19:16:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.161+0000 7fb371504700 1 -- start start 2024-02-16T19:16:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.161+0000 7fb371504700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c0771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb371504700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0790d0 con 0x7fb36c078cc0 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb36affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c0771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb36affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c0771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43194/0 (socket says 172.21.15.40:43194) 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb36affd700 1 -- 172.21.15.40:0/897378553 learned_addr learned my addr 172.21.15.40:0/897378553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb36affd700 1 -- 172.21.15.40:0/897378553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c077710 con 0x7fb36c078cc0 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb36affd700 1 --2- 172.21.15.40:0/897378553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c0771d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb364009e50 tx=0x7fb364009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d054701e6dcb6a80 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb369ffb700 1 -- 172.21.15.40:0/897378553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3640092e0 con 0x7fb36c078cc0 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb369ffb700 1 -- 172.21.15.40:0/897378553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb364012430 con 0x7fb36c078cc0 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.162+0000 7fb369ffb700 1 -- 172.21.15.40:0/897378553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3640075a0 con 0x7fb36c078cc0 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.163+0000 7fb371504700 1 -- 172.21.15.40:0/897378553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 msgr2=0x7fb36c0771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.163+0000 7fb371504700 1 --2- 172.21.15.40:0/897378553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c0771d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb364009e50 tx=0x7fb364009840 comp rx=0 tx=0).stop 2024-02-16T19:16:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.163+0000 7fb371504700 1 -- 172.21.15.40:0/897378553 shutdown_connections 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.163+0000 7fb371504700 1 --2- 172.21.15.40:0/897378553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c0771d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.163+0000 7fb371504700 1 -- 172.21.15.40:0/897378553 >> 172.21.15.40:0/897378553 conn(0x7fb36c0fdf00 msgr2=0x7fb36c100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.163+0000 7fb371504700 1 -- 172.21.15.40:0/897378553 shutdown_connections 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.163+0000 7fb371504700 1 -- 172.21.15.40:0/897378553 wait complete. 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb371504700 1 Processor -- start 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb371504700 1 -- start start 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb371504700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb371504700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c113d70 con 0x7fb36c078cc0 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb36affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb36affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c113830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43200/0 (socket says 172.21.15.40:43200) 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb36affd700 1 -- 172.21.15.40:0/866856984 learned_addr learned my addr 172.21.15.40:0/866856984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb36affd700 1 -- 172.21.15.40:0/866856984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364009ae0 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb36affd700 1 --2- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c113830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb364009fd0 tx=0x7fb364009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb3637fe700 1 -- 172.21.15.40:0/866856984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3640092e0 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb3637fe700 1 -- 172.21.15.40:0/866856984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb364007ce0 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c113f70 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb3637fe700 1 -- 172.21.15.40:0/866856984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3640259c0 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.164+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c114c10 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.165+0000 7fb3637fe700 1 -- 172.21.15.40:0/866856984 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fb364019050 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.165+0000 7fb3637fe700 1 --2- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb358036830 0x7fb358038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.165+0000 7fb3637fe700 1 -- 172.21.15.40:0/866856984 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb36404a290 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.165+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c05f5c0 con 0x7fb36c078cc0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.165+0000 7fb36a7fc700 1 --2- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb358036830 0x7fb358038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.166+0000 7fb36a7fc700 1 --2- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb358036830 0x7fb358038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb354009990 tx=0x7fb354006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.170+0000 7fb3637fe700 1 -- 172.21.15.40:0/866856984 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb364017030 con 0x7fb36c078cc0 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.366+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb36c04bed0 con 0x7fb36c078cc0 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.367+0000 7fb3637fe700 1 -- 172.21.15.40:0/866856984 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fb364007810 con 0x7fb36c078cc0 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.368+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb358036830 msgr2=0x7fb358038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.368+0000 7fb371504700 1 --2- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb358036830 0x7fb358038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb354009990 tx=0x7fb354006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.368+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 msgr2=0x7fb36c113830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.368+0000 7fb371504700 1 --2- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c113830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb364009fd0 tx=0x7fb364009840 comp rx=0 tx=0).stop 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.369+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 shutdown_connections 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.369+0000 7fb371504700 1 --2- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb358036830 0x7fb358038ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.369+0000 7fb371504700 1 --2- 172.21.15.40:0/866856984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb36c078cc0 0x7fb36c113830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.369+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 >> 172.21.15.40:0/866856984 conn(0x7fb36c0fdf00 msgr2=0x7fb36c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.369+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 shutdown_connections 2024-02-16T19:16:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:00.369+0000 7fb371504700 1 -- 172.21.15.40:0/866856984 wait complete. 2024-02-16T19:16:01.110 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-16T19:16:01.110 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2024-02-16T19:16:01.117 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-16T19:16:01.117 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2024-02-16T19:16:01.118 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2024-02-16T19:16:01.118 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-16T19:16:01.119 INFO:teuthology.orchestra.run.smithi040.stdout:Ceph Dashboard is now available at: 2024-02-16T19:16:01.120 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:01.120 INFO:teuthology.orchestra.run.smithi040.stdout: URL: https://smithi040.front.sepia.ceph.com:8443/ 2024-02-16T19:16:01.120 INFO:teuthology.orchestra.run.smithi040.stdout: User: admin 2024-02-16T19:16:01.120 INFO:teuthology.orchestra.run.smithi040.stdout: Password: vizvdyiybm 2024-02-16T19:16:01.120 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:01.121 INFO:teuthology.orchestra.run.smithi040.stdout:Saving cluster configuration to /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config directory 2024-02-16T19:16:01.121 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling autotune for osd_memory_target 2024-02-16T19:16:01.423 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:01 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/866856984' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-16T19:16:02.393 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.522+0000 7f1547c18700 1 Processor -- start 2024-02-16T19:16:02.393 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.522+0000 7f1547c18700 1 -- start start 2024-02-16T19:16:02.393 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.522+0000 7f1547c18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f1540101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:02.393 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.522+0000 7f1547c18700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1540101fc0 con 0x7f1540101670 2024-02-16T19:16:02.393 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.522+0000 7f15459b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f1540101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:02.393 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.522+0000 7f15459b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f1540101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43202/0 (socket says 172.21.15.40:43202) 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.522+0000 7f15459b4700 1 -- 172.21.15.40:0/3961554211 learned_addr learned my addr 172.21.15.40:0/3961554211 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.523+0000 7f15459b4700 1 -- 172.21.15.40:0/3961554211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540102100 con 0x7f1540101670 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.523+0000 7f15459b4700 1 --2- 172.21.15.40:0/3961554211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f1540101a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f153c0098e0 tx=0x7f153c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1c78054fa64620 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.523+0000 7f15449b2700 1 -- 172.21.15.40:0/3961554211 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f153c00b860 con 0x7f1540101670 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.523+0000 7f15449b2700 1 -- 172.21.15.40:0/3961554211 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f153c00be90 con 0x7f1540101670 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.523+0000 7f15449b2700 1 -- 172.21.15.40:0/3961554211 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f153c0103c0 con 0x7f1540101670 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.523+0000 7f1547c18700 1 -- 172.21.15.40:0/3961554211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 msgr2=0x7f1540101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.523+0000 7f1547c18700 1 --2- 172.21.15.40:0/3961554211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f1540101a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f153c0098e0 tx=0x7f153c009250 comp rx=0 tx=0).stop 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 -- 172.21.15.40:0/3961554211 shutdown_connections 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 --2- 172.21.15.40:0/3961554211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f1540101a80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 -- 172.21.15.40:0/3961554211 >> 172.21.15.40:0/3961554211 conn(0x7f15400fcdd0 msgr2=0x7f15400ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 -- 172.21.15.40:0/3961554211 shutdown_connections 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 -- 172.21.15.40:0/3961554211 wait complete. 2024-02-16T19:16:02.394 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 Processor -- start 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 -- start start 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f15401137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f1547c18700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1540113ce0 con 0x7f1540101670 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f15459b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f15401137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f15459b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f15401137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43212/0 (socket says 172.21.15.40:43212) 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.524+0000 7f15459b4700 1 -- 172.21.15.40:0/1945750087 learned_addr learned my addr 172.21.15.40:0/1945750087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.525+0000 7f15459b4700 1 -- 172.21.15.40:0/1945750087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c0094f0 con 0x7f1540101670 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.525+0000 7f15459b4700 1 --2- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f15401137a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f153c0090b0 tx=0x7f153c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.525+0000 7f15367fc700 1 -- 172.21.15.40:0/1945750087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f153c010840 con 0x7f1540101670 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.525+0000 7f15367fc700 1 -- 172.21.15.40:0/1945750087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f153c010e70 con 0x7f1540101670 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.525+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1540113ee0 con 0x7f1540101670 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.525+0000 7f15367fc700 1 -- 172.21.15.40:0/1945750087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f153c024e40 con 0x7f1540101670 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.525+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1540114b80 con 0x7f1540101670 2024-02-16T19:16:02.395 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.526+0000 7f15367fc700 1 -- 172.21.15.40:0/1945750087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f153c01a510 con 0x7f1540101670 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.526+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154005f5c0 con 0x7f1540101670 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.526+0000 7f15367fc700 1 --2- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f152c036830 0x7f152c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.526+0000 7f15367fc700 1 -- 172.21.15.40:0/1945750087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f153c04a970 con 0x7f1540101670 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.526+0000 7f15451b3700 1 --2- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f152c036830 0x7f152c038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.526+0000 7f15451b3700 1 --2- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f152c036830 0x7f152c038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1530009990 tx=0x7f1530006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.531+0000 7f15367fc700 1 -- 172.21.15.40:0/1945750087 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f153c024430 con 0x7f1540101670 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.721+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f154004bed0 con 0x7f1540101670 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.722+0000 7f15367fc700 1 -- 172.21.15.40:0/1945750087 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f153c018c50 con 0x7f1540101670 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.723+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f152c036830 msgr2=0x7f152c038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.723+0000 7f1547c18700 1 --2- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f152c036830 0x7f152c038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1530009990 tx=0x7f1530006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.724+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 msgr2=0x7f15401137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:02.396 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.724+0000 7f1547c18700 1 --2- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f15401137a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f153c0090b0 tx=0x7f153c009250 comp rx=0 tx=0).stop 2024-02-16T19:16:02.397 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.724+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 shutdown_connections 2024-02-16T19:16:02.397 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.724+0000 7f1547c18700 1 --2- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f152c036830 0x7f152c038ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:02.397 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.724+0000 7f1547c18700 1 --2- 172.21.15.40:0/1945750087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1540101670 0x7f15401137a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:02.397 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.724+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 >> 172.21.15.40:0/1945750087 conn(0x7f15400fcdd0 msgr2=0x7f1540076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:02.397 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.724+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 shutdown_connections 2024-02-16T19:16:02.397 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:01.724+0000 7f1547c18700 1 -- 172.21.15.40:0/1945750087 wait complete. 2024-02-16T19:16:02.511 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:02 smithi040 ceph-mon[29201]: mgrmap e13: smithi040.jzygzo(active, since 6s) 2024-02-16T19:16:03.677 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.833+0000 7fd95aa5b700 1 Processor -- start 2024-02-16T19:16:03.677 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.833+0000 7fd95aa5b700 1 -- start start 2024-02-16T19:16:03.677 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.833+0000 7fd95aa5b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954102b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:03.677 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.833+0000 7fd95aa5b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541030b0 con 0x7fd954102760 2024-02-16T19:16:03.677 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.833+0000 7fd953fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:03.677 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.833+0000 7fd953fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43226/0 (socket says 172.21.15.40:43226) 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.833+0000 7fd953fff700 1 -- 172.21.15.40:0/2735062277 learned_addr learned my addr 172.21.15.40:0/2735062277 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.834+0000 7fd953fff700 1 -- 172.21.15.40:0/2735062277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9541031f0 con 0x7fd954102760 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.834+0000 7fd953fff700 1 --2- 172.21.15.40:0/2735062277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954102b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9440098e0 tx=0x7fd944009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700af222fd13a24a server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.834+0000 7fd952ffd700 1 -- 172.21.15.40:0/2735062277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd94400b860 con 0x7fd954102760 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.834+0000 7fd952ffd700 1 -- 172.21.15.40:0/2735062277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd94400be90 con 0x7fd954102760 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.834+0000 7fd952ffd700 1 -- 172.21.15.40:0/2735062277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9440103c0 con 0x7fd954102760 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.834+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2735062277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 msgr2=0x7fd954102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.835+0000 7fd95aa5b700 1 --2- 172.21.15.40:0/2735062277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954102b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9440098e0 tx=0x7fd944009250 comp rx=0 tx=0).stop 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.835+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2735062277 shutdown_connections 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.835+0000 7fd95aa5b700 1 --2- 172.21.15.40:0/2735062277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954102b70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.835+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2735062277 >> 172.21.15.40:0/2735062277 conn(0x7fd9540fdf00 msgr2=0x7fd954100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.835+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2735062277 shutdown_connections 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.835+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2735062277 wait complete. 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.835+0000 7fd95aa5b700 1 Processor -- start 2024-02-16T19:16:03.678 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd95aa5b700 1 -- start start 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd95aa5b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954111530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd95aa5b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd954111a70 con 0x7fd954102760 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd953fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954111530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd953fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954111530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43240/0 (socket says 172.21.15.40:43240) 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd953fff700 1 -- 172.21.15.40:0/2485953329 learned_addr learned my addr 172.21.15.40:0/2485953329 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd953fff700 1 -- 172.21.15.40:0/2485953329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9440094f0 con 0x7fd954102760 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd953fff700 1 --2- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954111530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd944006b00 tx=0x7fd944009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd950ff9700 1 -- 172.21.15.40:0/2485953329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9440109a0 con 0x7fd954102760 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd954111c70 con 0x7fd954102760 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd950ff9700 1 -- 172.21.15.40:0/2485953329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd94401a430 con 0x7fd954102760 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.836+0000 7fd950ff9700 1 -- 172.21.15.40:0/2485953329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9440245b0 con 0x7fd954102760 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.837+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd954112910 con 0x7fd954102760 2024-02-16T19:16:03.679 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.837+0000 7fd950ff9700 1 -- 172.21.15.40:0/2485953329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fd944010b00 con 0x7fd954102760 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.838+0000 7fd950ff9700 1 --2- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd940036920 0x7fd940038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.838+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95405f5c0 con 0x7fd954102760 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.838+0000 7fd950ff9700 1 -- 172.21.15.40:0/2485953329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd94404a8a0 con 0x7fd954102760 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.838+0000 7fd9537fe700 1 --2- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd940036920 0x7fd940038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.838+0000 7fd9537fe700 1 --2- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd940036920 0x7fd940038dd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009990 tx=0x7fd93c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:02.842+0000 7fd950ff9700 1 -- 172.21.15.40:0/2485953329 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd94401a5a0 con 0x7fd954102760 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.139+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fd95404bed0 con 0x7fd954102760 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.142+0000 7fd950ff9700 1 -- 172.21.15.40:0/2485953329 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fd94400f2e0 con 0x7fd954102760 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.143+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd940036920 msgr2=0x7fd940038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 --2- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd940036920 0x7fd940038dd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009990 tx=0x7fd93c006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 msgr2=0x7fd954111530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 --2- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954111530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd944006b00 tx=0x7fd944009250 comp rx=0 tx=0).stop 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 shutdown_connections 2024-02-16T19:16:03.680 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 --2- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd940036920 0x7fd940038dd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 --2- 172.21.15.40:0/2485953329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd954102760 0x7fd954111530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 >> 172.21.15.40:0/2485953329 conn(0x7fd9540fdf00 msgr2=0x7fd9540fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 shutdown_connections 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2024-02-16T19:16:03.144+0000 7fd95aa5b700 1 -- 172.21.15.40:0/2485953329 wait complete. 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:Or, if you are only running a single cluster on this host: 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-16T19:16:03.682 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:03.682 INFO:teuthology.orchestra.run.smithi040.stdout: ceph telemetry on 2024-02-16T19:16:03.682 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:03.682 INFO:teuthology.orchestra.run.smithi040.stdout:For more information see: 2024-02-16T19:16:03.682 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:03.682 INFO:teuthology.orchestra.run.smithi040.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-02-16T19:16:03.682 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:03.682 INFO:teuthology.orchestra.run.smithi040.stdout:Bootstrap complete. 2024-02-16T19:16:03.715 INFO:tasks.cephadm:Fetching config... 2024-02-16T19:16:03.716 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:16:03.716 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-16T19:16:03.732 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-16T19:16:03.732 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:16:03.732 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-16T19:16:03.787 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-16T19:16:03.787 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:16:03.787 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/keyring of=/dev/stdout 2024-02-16T19:16:03.854 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-16T19:16:03.854 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:16:03.854 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-16T19:16:03.908 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-16T19:16:03.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI2GDFuSLO4krAncSHFFBZv3EQfFZ+9j2mjloFepe/I89arzS9Z+5z2CAjMjRlc54//8K7a3MUXCNXGI5TpyapjavrAUEDL4cc9kMLKsF6KhrOHCWwwDWYUl4W0yKoVBaeXfCz2qiv9427HMak0842KLyBXQS1eWhTVSRiwzElLNc4OjAazZ6a6cvnJL8Ifr4TA1B56spp6RAzHag5ziPStUQG2tUtapLoyYEzt2kMX0DVdduDBNkTYvK3VuC99K7Sk5AuN6qPRlcXYAYXJS4U7jW15PoWedAugrBYkqSgPDOLhBPqaAGK/8WUmlwAr4NIzsQ1RAiQ9jeaxtpyUjHkFDSW+osU9ioHujsBsxzwyouIz8BEQH6ecNmhT1HYrdJv47+3xDZxltSPy0SGUuGVj9wGLaxBl/AyolO9TGdK+KWqzlKuO15xkem+Pwrd1fzxzok5JBaEnvX5z/HXuW2bkrO82pfU2GL6dVnOwd/M09ssazbpHsjI0pwcGkzwa08= ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-16T19:16:03.989 INFO:teuthology.orchestra.run.smithi040.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI2GDFuSLO4krAncSHFFBZv3EQfFZ+9j2mjloFepe/I89arzS9Z+5z2CAjMjRlc54//8K7a3MUXCNXGI5TpyapjavrAUEDL4cc9kMLKsF6KhrOHCWwwDWYUl4W0yKoVBaeXfCz2qiv9427HMak0842KLyBXQS1eWhTVSRiwzElLNc4OjAazZ6a6cvnJL8Ifr4TA1B56spp6RAzHag5ziPStUQG2tUtapLoyYEzt2kMX0DVdduDBNkTYvK3VuC99K7Sk5AuN6qPRlcXYAYXJS4U7jW15PoWedAugrBYkqSgPDOLhBPqaAGK/8WUmlwAr4NIzsQ1RAiQ9jeaxtpyUjHkFDSW+osU9ioHujsBsxzwyouIz8BEQH6ecNmhT1HYrdJv47+3xDZxltSPy0SGUuGVj9wGLaxBl/AyolO9TGdK+KWqzlKuO15xkem+Pwrd1fzxzok5JBaEnvX5z/HXuW2bkrO82pfU2GL6dVnOwd/M09ssazbpHsjI0pwcGkzwa08= ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:16:04.002 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI2GDFuSLO4krAncSHFFBZv3EQfFZ+9j2mjloFepe/I89arzS9Z+5z2CAjMjRlc54//8K7a3MUXCNXGI5TpyapjavrAUEDL4cc9kMLKsF6KhrOHCWwwDWYUl4W0yKoVBaeXfCz2qiv9427HMak0842KLyBXQS1eWhTVSRiwzElLNc4OjAazZ6a6cvnJL8Ifr4TA1B56spp6RAzHag5ziPStUQG2tUtapLoyYEzt2kMX0DVdduDBNkTYvK3VuC99K7Sk5AuN6qPRlcXYAYXJS4U7jW15PoWedAugrBYkqSgPDOLhBPqaAGK/8WUmlwAr4NIzsQ1RAiQ9jeaxtpyUjHkFDSW+osU9ioHujsBsxzwyouIz8BEQH6ecNmhT1HYrdJv47+3xDZxltSPy0SGUuGVj9wGLaxBl/AyolO9TGdK+KWqzlKuO15xkem+Pwrd1fzxzok5JBaEnvX5z/HXuW2bkrO82pfU2GL6dVnOwd/M09ssazbpHsjI0pwcGkzwa08= ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-16T19:16:04.049 INFO:teuthology.orchestra.run.smithi163.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI2GDFuSLO4krAncSHFFBZv3EQfFZ+9j2mjloFepe/I89arzS9Z+5z2CAjMjRlc54//8K7a3MUXCNXGI5TpyapjavrAUEDL4cc9kMLKsF6KhrOHCWwwDWYUl4W0yKoVBaeXfCz2qiv9427HMak0842KLyBXQS1eWhTVSRiwzElLNc4OjAazZ6a6cvnJL8Ifr4TA1B56spp6RAzHag5ziPStUQG2tUtapLoyYEzt2kMX0DVdduDBNkTYvK3VuC99K7Sk5AuN6qPRlcXYAYXJS4U7jW15PoWedAugrBYkqSgPDOLhBPqaAGK/8WUmlwAr4NIzsQ1RAiQ9jeaxtpyUjHkFDSW+osU9ioHujsBsxzwyouIz8BEQH6ecNmhT1HYrdJv47+3xDZxltSPy0SGUuGVj9wGLaxBl/AyolO9TGdK+KWqzlKuO15xkem+Pwrd1fzxzok5JBaEnvX5z/HXuW2bkrO82pfU2GL6dVnOwd/M09ssazbpHsjI0pwcGkzwa08= ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:16:04.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-16T19:16:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:16:04.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:04 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2485953329' entity='client.admin' 2024-02-16T19:16:06.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.497+0000 7f4ce38a0700 1 -- 172.21.15.40:0/2639716176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 msgr2=0x7f4cdc0fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:06.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.497+0000 7f4ce38a0700 1 --2- 172.21.15.40:0/2639716176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 0x7f4cdc0fcfe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009c60 tx=0x7f4ccc004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:06.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.497+0000 7f4ce38a0700 1 -- 172.21.15.40:0/2639716176 shutdown_connections 2024-02-16T19:16:06.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.497+0000 7f4ce38a0700 1 --2- 172.21.15.40:0/2639716176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 0x7f4cdc0fcfe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:06.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.497+0000 7f4ce38a0700 1 -- 172.21.15.40:0/2639716176 >> 172.21.15.40:0/2639716176 conn(0x7f4cdc0f8270 msgr2=0x7f4cdc0fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:06.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.497+0000 7f4ce38a0700 1 -- 172.21.15.40:0/2639716176 shutdown_connections 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.497+0000 7f4ce38a0700 1 -- 172.21.15.40:0/2639716176 wait complete. 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.497+0000 7f4ce38a0700 1 Processor -- start 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce38a0700 1 -- start start 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce38a0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 0x7f4cdc10cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce38a0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc10d4a0 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce163c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 0x7f4cdc10cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce163c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 0x7f4cdc10cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43260/0 (socket says 172.21.15.40:43260) 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce163c700 1 -- 172.21.15.40:0/244529283 learned_addr learned my addr 172.21.15.40:0/244529283 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce163c700 1 -- 172.21.15.40:0/244529283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc009940 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce163c700 1 --2- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 0x7f4cdc10cf60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc015040 tx=0x7f4ccc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4cd9ffb700 1 -- 172.21.15.40:0/244529283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ccc007c80 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc10d6a0 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4cd9ffb700 1 -- 172.21.15.40:0/244529283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ccc01d910 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.498+0000 7f4cd9ffb700 1 -- 172.21.15.40:0/244529283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ccc0129f0 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.499+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc10e2f0 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.499+0000 7f4cd9ffb700 1 -- 172.21.15.40:0/244529283 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f4ccc019040 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.500+0000 7f4cd9ffb700 1 --2- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f4cc4036870 0x7f4cc4038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:06.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.500+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc05f5c0 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.500+0000 7f4cd9ffb700 1 -- 172.21.15.40:0/244529283 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4ccc04a4a0 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.500+0000 7f4ce0e3b700 1 --2- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f4cc4036870 0x7f4cc4038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:06.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.500+0000 7f4ce0e3b700 1 --2- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f4cc4036870 0x7f4cc4038d20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009990 tx=0x7f4cd0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:06.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.504+0000 7f4cd9ffb700 1 -- 172.21.15.40:0/244529283 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4ccc021070 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.702+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f4cdc04bed0 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.706+0000 7f4cd9ffb700 1 -- 172.21.15.40:0/244529283 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4ccc0490c0 con 0x7f4cdc0fcbd0 2024-02-16T19:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f4cc4036870 msgr2=0x7f4cc4038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 --2- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f4cc4036870 0x7f4cc4038d20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009990 tx=0x7f4cd0006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 msgr2=0x7f4cdc10cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 --2- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 0x7f4cdc10cf60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc015040 tx=0x7f4ccc004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 shutdown_connections 2024-02-16T19:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 --2- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f4cc4036870 0x7f4cc4038d20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 --2- 172.21.15.40:0/244529283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cdc0fcbd0 0x7f4cdc10cf60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 >> 172.21.15.40:0/244529283 conn(0x7f4cdc0f8270 msgr2=0x7f4cdc0f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 shutdown_connections 2024-02-16T19:16:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:06.708+0000 7f4ce38a0700 1 -- 172.21.15.40:0/244529283 wait complete. 2024-02-16T19:16:06.799 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:06 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:06.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:06 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:06.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:06 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:16:06.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:06 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:06.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:06 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:16:06.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:06 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-16T19:16:06.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:06 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:16:06.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:06 smithi040 ceph-mon[29201]: Deploying daemon crash.smithi040 on smithi040 2024-02-16T19:16:07.611 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-16T19:16:07.611 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-16T19:16:07.875 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:07 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/244529283' entity='client.admin' 2024-02-16T19:16:08.209 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:16:10.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.139+0000 7f7578fe1700 1 -- 172.21.15.40:0/1008968313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 msgr2=0x7f75740fbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:10.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.139+0000 7f7578fe1700 1 --2- 172.21.15.40:0/1008968313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 0x7f75740fbec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f75640048c0 tx=0x7f7564009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.139+0000 7f7578fe1700 1 -- 172.21.15.40:0/1008968313 shutdown_connections 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.139+0000 7f7578fe1700 1 --2- 172.21.15.40:0/1008968313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 0x7f75740fbec0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.139+0000 7f7578fe1700 1 -- 172.21.15.40:0/1008968313 >> 172.21.15.40:0/1008968313 conn(0x7f75740f7210 msgr2=0x7f75740f9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.139+0000 7f7578fe1700 1 -- 172.21.15.40:0/1008968313 shutdown_connections 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.139+0000 7f7578fe1700 1 -- 172.21.15.40:0/1008968313 wait complete. 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.140+0000 7f7578fe1700 1 Processor -- start 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.140+0000 7f7578fe1700 1 -- start start 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.140+0000 7f7578fe1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 0x7f757410f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.140+0000 7f7578fe1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757410f600 con 0x7f75740fbab0 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.140+0000 7f757259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 0x7f757410f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.140+0000 7f757259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 0x7f757410f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39920/0 (socket says 172.21.15.40:39920) 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.140+0000 7f757259c700 1 -- 172.21.15.40:0/3688804578 learned_addr learned my addr 172.21.15.40:0/3688804578 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:10.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.141+0000 7f757259c700 1 -- 172.21.15.40:0/3688804578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7564005040 con 0x7f75740fbab0 2024-02-16T19:16:10.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.141+0000 7f757259c700 1 --2- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 0x7f757410f0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7564009680 tx=0x7f7564009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:10.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.141+0000 7f7562ffd700 1 -- 172.21.15.40:0/3688804578 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f756400b5c0 con 0x7f75740fbab0 2024-02-16T19:16:10.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.141+0000 7f7562ffd700 1 -- 172.21.15.40:0/3688804578 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7564004e60 con 0x7f75740fbab0 2024-02-16T19:16:10.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.141+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757410f800 con 0x7f75740fbab0 2024-02-16T19:16:10.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.141+0000 7f7562ffd700 1 -- 172.21.15.40:0/3688804578 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f756401c590 con 0x7f75740fbab0 2024-02-16T19:16:10.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.141+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75741104a0 con 0x7f75740fbab0 2024-02-16T19:16:10.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.142+0000 7f7562ffd700 1 -- 172.21.15.40:0/3688804578 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f7564013a80 con 0x7f75740fbab0 2024-02-16T19:16:10.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.142+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757405f5c0 con 0x7f75740fbab0 2024-02-16T19:16:10.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.142+0000 7f7562ffd700 1 --2- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f755c0368c0 0x7f755c038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:10.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.142+0000 7f7562ffd700 1 -- 172.21.15.40:0/3688804578 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7564049ad0 con 0x7f75740fbab0 2024-02-16T19:16:10.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.142+0000 7f7571d9b700 1 --2- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f755c0368c0 0x7f755c038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:10.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.143+0000 7f7571d9b700 1 --2- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f755c0368c0 0x7f755c038d70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7568009990 tx=0x7f7568006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:10.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.147+0000 7f7562ffd700 1 -- 172.21.15.40:0/3688804578 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7564025400 con 0x7f75740fbab0 2024-02-16T19:16:10.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.344+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 --> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f757405f5c0 con 0x7f755c0368c0 2024-02-16T19:16:10.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.347+0000 7f7562ffd700 1 -- 172.21.15.40:0/3688804578 <== mgr.14160 v2:172.21.15.40:6800/277905681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f757405f5c0 con 0x7f755c0368c0 2024-02-16T19:16:10.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f755c0368c0 msgr2=0x7f755c038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 --2- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f755c0368c0 0x7f755c038d70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7568009990 tx=0x7f7568006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 msgr2=0x7f757410f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 --2- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 0x7f757410f0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7564009680 tx=0x7f7564009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 shutdown_connections 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 --2- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f755c0368c0 0x7f755c038d70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 --2- 172.21.15.40:0/3688804578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75740fbab0 0x7f757410f0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 >> 172.21.15.40:0/3688804578 conn(0x7f75740f7210 msgr2=0x7f75740f9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 shutdown_connections 2024-02-16T19:16:10.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:10.349+0000 7f7578fe1700 1 -- 172.21.15.40:0/3688804578 wait complete. 2024-02-16T19:16:11.253 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi163 2024-02-16T19:16:11.253 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:16:11.253 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.conf 2024-02-16T19:16:11.272 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:16:11.272 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:16:11.330 INFO:tasks.cephadm:Adding host smithi163 to orchestrator... 2024-02-16T19:16:11.331 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch host add smithi163 2024-02-16T19:16:11.567 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:11 smithi040 ceph-mon[29201]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:16:11.568 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:11 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:11.780 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:16:13.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:12 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:13.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:12 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:13.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:12 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:13.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:12 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:13.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:12 smithi040 ceph-mon[29201]: Deploying daemon node-exporter.smithi040 on smithi040 2024-02-16T19:16:13.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.517+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1872084295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 msgr2=0x7fdb540fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.517+0000 7fdb5bf6c700 1 --2- 172.21.15.40:0/1872084295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 0x7fdb540fcfe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009c60 tx=0x7fdb44004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.517+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1872084295 shutdown_connections 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.517+0000 7fdb5bf6c700 1 --2- 172.21.15.40:0/1872084295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 0x7fdb540fcfe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.517+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1872084295 >> 172.21.15.40:0/1872084295 conn(0x7fdb540f8290 msgr2=0x7fdb540fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.517+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1872084295 shutdown_connections 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.517+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1872084295 wait complete. 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.518+0000 7fdb5bf6c700 1 Processor -- start 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.518+0000 7fdb5bf6c700 1 -- start start 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.518+0000 7fdb5bf6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 0x7fdb54071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.518+0000 7fdb5bf6c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb540702e0 con 0x7fdb540fcbd0 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.518+0000 7fdb59d08700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 0x7fdb54071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.518+0000 7fdb59d08700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 0x7fdb54071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39956/0 (socket says 172.21.15.40:39956) 2024-02-16T19:16:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.518+0000 7fdb59d08700 1 -- 172.21.15.40:0/1214738739 learned_addr learned my addr 172.21.15.40:0/1214738739 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.519+0000 7fdb59d08700 1 -- 172.21.15.40:0/1214738739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44009940 con 0x7fdb540fcbd0 2024-02-16T19:16:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.519+0000 7fdb59d08700 1 --2- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 0x7fdb54071e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb44015040 tx=0x7fdb44004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.519+0000 7fdb527fc700 1 -- 172.21.15.40:0/1214738739 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb44007c80 con 0x7fdb540fcbd0 2024-02-16T19:16:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.519+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb540704e0 con 0x7fdb540fcbd0 2024-02-16T19:16:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.519+0000 7fdb527fc700 1 -- 172.21.15.40:0/1214738739 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb4401d910 con 0x7fdb540fcbd0 2024-02-16T19:16:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.519+0000 7fdb527fc700 1 -- 172.21.15.40:0/1214738739 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb440129f0 con 0x7fdb540fcbd0 2024-02-16T19:16:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.519+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb540709d0 con 0x7fdb540fcbd0 2024-02-16T19:16:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.520+0000 7fdb527fc700 1 -- 172.21.15.40:0/1214738739 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fdb44019040 con 0x7fdb540fcbd0 2024-02-16T19:16:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.520+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb541069a0 con 0x7fdb540fcbd0 2024-02-16T19:16:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.520+0000 7fdb527fc700 1 --2- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fdb3c036870 0x7fdb3c038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.520+0000 7fdb527fc700 1 -- 172.21.15.40:0/1214738739 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdb4404a3b0 con 0x7fdb540fcbd0 2024-02-16T19:16:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.520+0000 7fdb59507700 1 --2- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fdb3c036870 0x7fdb3c038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.520+0000 7fdb59507700 1 --2- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fdb3c036870 0x7fdb3c038d20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009990 tx=0x7fdb48006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.525+0000 7fdb527fc700 1 -- 172.21.15.40:0/1214738739 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fdb4402eb40 con 0x7fdb540fcbd0 2024-02-16T19:16:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:13.719+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 --> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb5402b410 con 0x7fdb3c036870 2024-02-16T19:16:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:14 smithi040 ceph-mon[29201]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:16:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.901+0000 7fdb527fc700 1 -- 172.21.15.40:0/1214738739 <== mgr.14160 v2:172.21.15.40:6800/277905681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdb5402b410 con 0x7fdb3c036870 2024-02-16T19:16:15.903 INFO:teuthology.orchestra.run.smithi040.stdout:Added host 'smithi163' with addr '172.21.15.163' 2024-02-16T19:16:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fdb3c036870 msgr2=0x7fdb3c038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 --2- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fdb3c036870 0x7fdb3c038d20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009990 tx=0x7fdb48006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 msgr2=0x7fdb54071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 --2- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 0x7fdb54071e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb44015040 tx=0x7fdb44004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 shutdown_connections 2024-02-16T19:16:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 --2- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fdb3c036870 0x7fdb3c038d20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 --2- 172.21.15.40:0/1214738739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb540fcbd0 0x7fdb54071e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 >> 172.21.15.40:0/1214738739 conn(0x7fdb540f8290 msgr2=0x7fdb540f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 shutdown_connections 2024-02-16T19:16:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:15.904+0000 7fdb5bf6c700 1 -- 172.21.15.40:0/1214738739 wait complete. 2024-02-16T19:16:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:15 smithi040 ceph-mon[29201]: Deploying cephadm binary to smithi163 2024-02-16T19:16:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:15 smithi040 ceph-mon[29201]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:15 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:16.757 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-16T19:16:17.048 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:16:17.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:16 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:17.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:16 smithi040 ceph-mon[29201]: Added host smithi163 2024-02-16T19:16:17.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:16 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:17.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:16 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:17.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:16 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:17.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:16 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:18.169 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:17 smithi040 ceph-mon[29201]: Deploying daemon alertmanager.smithi040 on smithi040 2024-02-16T19:16:18.169 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:17 smithi040 ceph-mon[29201]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.582+0000 7efe93435700 1 -- 172.21.15.40:0/2164572291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 msgr2=0x7efe8c0fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:19.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.582+0000 7efe93435700 1 --2- 172.21.15.40:0/2164572291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 0x7efe8c0fbee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe880048c0 tx=0x7efe88009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.582+0000 7efe93435700 1 -- 172.21.15.40:0/2164572291 shutdown_connections 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.582+0000 7efe93435700 1 --2- 172.21.15.40:0/2164572291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 0x7efe8c0fbee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.582+0000 7efe93435700 1 -- 172.21.15.40:0/2164572291 >> 172.21.15.40:0/2164572291 conn(0x7efe8c0f7190 msgr2=0x7efe8c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.582+0000 7efe93435700 1 -- 172.21.15.40:0/2164572291 shutdown_connections 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.582+0000 7efe93435700 1 -- 172.21.15.40:0/2164572291 wait complete. 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.583+0000 7efe93435700 1 Processor -- start 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.583+0000 7efe93435700 1 -- start start 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.583+0000 7efe93435700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 0x7efe8c10be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.583+0000 7efe93435700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c10c360 con 0x7efe8c0fbad0 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.583+0000 7efe911d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 0x7efe8c10be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.583+0000 7efe911d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 0x7efe8c10be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60630/0 (socket says 172.21.15.40:60630) 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.583+0000 7efe911d1700 1 -- 172.21.15.40:0/1716790782 learned_addr learned my addr 172.21.15.40:0/1716790782 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:19.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.584+0000 7efe911d1700 1 -- 172.21.15.40:0/1716790782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe88005040 con 0x7efe8c0fbad0 2024-02-16T19:16:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.584+0000 7efe911d1700 1 --2- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 0x7efe8c10be20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe88009680 tx=0x7efe88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.584+0000 7efe81ffb700 1 -- 172.21.15.40:0/1716790782 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe8800bed0 con 0x7efe8c0fbad0 2024-02-16T19:16:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.584+0000 7efe81ffb700 1 -- 172.21.15.40:0/1716790782 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe8800b5c0 con 0x7efe8c0fbad0 2024-02-16T19:16:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.584+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c10c560 con 0x7efe8c0fbad0 2024-02-16T19:16:19.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.584+0000 7efe81ffb700 1 -- 172.21.15.40:0/1716790782 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe8801c5b0 con 0x7efe8c0fbad0 2024-02-16T19:16:19.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.584+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c10d120 con 0x7efe8c0fbad0 2024-02-16T19:16:19.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.585+0000 7efe81ffb700 1 -- 172.21.15.40:0/1716790782 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7efe8801c820 con 0x7efe8c0fbad0 2024-02-16T19:16:19.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.585+0000 7efe81ffb700 1 --2- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7efe78036820 0x7efe78038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:19.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.585+0000 7efe81ffb700 1 -- 172.21.15.40:0/1716790782 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7efe8804ab00 con 0x7efe8c0fbad0 2024-02-16T19:16:19.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.585+0000 7efe909d0700 1 --2- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7efe78036820 0x7efe78038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:19.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.585+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8c05f5c0 con 0x7efe8c0fbad0 2024-02-16T19:16:19.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.586+0000 7efe909d0700 1 --2- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7efe78036820 0x7efe78038cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009990 tx=0x7efe7c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.590+0000 7efe81ffb700 1 -- 172.21.15.40:0/1716790782 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7efe88011dd0 con 0x7efe8c0fbad0 2024-02-16T19:16:19.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.801+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 --> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe8c10dc70 con 0x7efe78036820 2024-02-16T19:16:19.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.802+0000 7efe81ffb700 1 -- 172.21.15.40:0/1716790782 <== mgr.14160 v2:172.21.15.40:6800/277905681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7efe8c10dc70 con 0x7efe78036820 2024-02-16T19:16:19.804 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:16:19.804 INFO:teuthology.orchestra.run.smithi040.stdout:[{"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}, {"addr": "172.21.15.163", "hostname": "smithi163", "labels": [], "status": ""}] 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.804+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7efe78036820 msgr2=0x7efe78038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.804+0000 7efe93435700 1 --2- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7efe78036820 0x7efe78038cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009990 tx=0x7efe7c006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.804+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 msgr2=0x7efe8c10be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.804+0000 7efe93435700 1 --2- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 0x7efe8c10be20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe88009680 tx=0x7efe88009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.804+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 shutdown_connections 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.805+0000 7efe93435700 1 --2- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7efe78036820 0x7efe78038cd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.805+0000 7efe93435700 1 --2- 172.21.15.40:0/1716790782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe8c0fbad0 0x7efe8c10be20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.805+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 >> 172.21.15.40:0/1716790782 conn(0x7efe8c0f7190 msgr2=0x7efe8c0f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.805+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 shutdown_connections 2024-02-16T19:16:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:19.805+0000 7efe93435700 1 -- 172.21.15.40:0/1716790782 wait complete. 2024-02-16T19:16:20.174 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:19 smithi040 ceph-mon[29201]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:20.174 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:19 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:20.648 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-16T19:16:20.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-16T19:16:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:20 smithi040 ceph-mon[29201]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:16:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:16:21.980 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:21 smithi040 ceph-mon[29201]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:24.185 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:23 smithi040 ceph-mon[29201]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:26.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:25 smithi040 ceph-mon[29201]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:26.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.194+0000 7fb15db51700 1 -- 172.21.15.40:0/3449336189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 msgr2=0x7fb1580fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:26.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.194+0000 7fb15db51700 1 --2- 172.21.15.40:0/3449336189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 0x7fb1580fbee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb1500048c0 tx=0x7fb150009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:26.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.194+0000 7fb15db51700 1 -- 172.21.15.40:0/3449336189 shutdown_connections 2024-02-16T19:16:26.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.194+0000 7fb15db51700 1 --2- 172.21.15.40:0/3449336189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 0x7fb1580fbee0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.194+0000 7fb15db51700 1 -- 172.21.15.40:0/3449336189 >> 172.21.15.40:0/3449336189 conn(0x7fb1580f7190 msgr2=0x7fb1580f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.194+0000 7fb15db51700 1 -- 172.21.15.40:0/3449336189 shutdown_connections 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.194+0000 7fb15db51700 1 -- 172.21.15.40:0/3449336189 wait complete. 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb15db51700 1 Processor -- start 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb15db51700 1 -- start start 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb15db51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 0x7fb15810cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb15db51700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15810d400 con 0x7fb1580fbad0 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb1577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 0x7fb15810cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb1577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 0x7fb15810cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60638/0 (socket says 172.21.15.40:60638) 2024-02-16T19:16:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb1577fe700 1 -- 172.21.15.40:0/770190445 learned_addr learned my addr 172.21.15.40:0/770190445 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:16:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb1577fe700 1 -- 172.21.15.40:0/770190445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb150005040 con 0x7fb1580fbad0 2024-02-16T19:16:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.195+0000 7fb1577fe700 1 --2- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 0x7fb15810cec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb150009b80 tx=0x7fb150009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.196+0000 7fb14ffff700 1 -- 172.21.15.40:0/770190445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb15000b5c0 con 0x7fb1580fbad0 2024-02-16T19:16:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.196+0000 7fb14ffff700 1 -- 172.21.15.40:0/770190445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb15000be70 con 0x7fb1580fbad0 2024-02-16T19:16:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.196+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15810d600 con 0x7fb1580fbad0 2024-02-16T19:16:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.196+0000 7fb14ffff700 1 -- 172.21.15.40:0/770190445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb15001c660 con 0x7fb1580fbad0 2024-02-16T19:16:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.196+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15810e2a0 con 0x7fb1580fbad0 2024-02-16T19:16:26.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.197+0000 7fb14ffff700 1 -- 172.21.15.40:0/770190445 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb15001c8d0 con 0x7fb1580fbad0 2024-02-16T19:16:26.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.197+0000 7fb14ffff700 1 --2- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb144036870 0x7fb144038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:26.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.197+0000 7fb14ffff700 1 -- 172.21.15.40:0/770190445 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb150049990 con 0x7fb1580fbad0 2024-02-16T19:16:26.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.197+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15805f5c0 con 0x7fb1580fbad0 2024-02-16T19:16:26.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.197+0000 7fb156ffd700 1 --2- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb144036870 0x7fb144038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:26.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.197+0000 7fb156ffd700 1 --2- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb144036870 0x7fb144038d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb140009990 tx=0x7fb140006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:26.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.202+0000 7fb14ffff700 1 -- 172.21.15.40:0/770190445 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb150011db0 con 0x7fb1580fbad0 2024-02-16T19:16:26.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.400+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb15810ed70 con 0x7fb1580fbad0 2024-02-16T19:16:26.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.911+0000 7fb14ffff700 1 -- 172.21.15.40:0/770190445 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb150023990 con 0x7fb1580fbad0 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.913+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb144036870 msgr2=0x7fb144038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.913+0000 7fb15db51700 1 --2- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb144036870 0x7fb144038d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb140009990 tx=0x7fb140006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.913+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 msgr2=0x7fb15810cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.913+0000 7fb15db51700 1 --2- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 0x7fb15810cec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb150009b80 tx=0x7fb150009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.913+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 shutdown_connections 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.913+0000 7fb15db51700 1 --2- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fb144036870 0x7fb144038d20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.914+0000 7fb15db51700 1 --2- 172.21.15.40:0/770190445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1580fbad0 0x7fb15810cec0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.914+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 >> 172.21.15.40:0/770190445 conn(0x7fb1580f7190 msgr2=0x7fb158065960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.914+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 shutdown_connections 2024-02-16T19:16:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:16:26.914+0000 7fb15db51700 1 -- 172.21.15.40:0/770190445 wait complete. 2024-02-16T19:16:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-02-16T19:16:26.962 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:26 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/770190445' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-16T19:16:26.962 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:26 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:26.962 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:26 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:26.962 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:26 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:26.962 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:26 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:27.531 INFO:tasks.cephadm:Adding mon.smithi040 on smithi040 2024-02-16T19:16:27.531 INFO:tasks.cephadm:Adding mon.smithi163 on smithi163 2024-02-16T19:16:27.532 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch apply mon '2;smithi040:172.21.15.40=smithi040;smithi163:172.21.15.163=smithi163' 2024-02-16T19:16:27.779 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:27.827 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/770190445' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-16T19:16:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: osdmap e4: 0 total, 0 up, 0 in 2024-02-16T19:16:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:28.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:28.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-16T19:16:28.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-16T19:16:28.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:28.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:27 smithi040 ceph-mon[29201]: Deploying daemon grafana.smithi040 on smithi040 2024-02-16T19:16:29.394 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.391+0000 7f0fc5baf700 1 -- 172.21.15.163:0/218143165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 msgr2=0x7f0fc00fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.391+0000 7f0fc5baf700 1 --2- 172.21.15.163:0/218143165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 0x7f0fc00fcfd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40048c0 tx=0x7f0fb4009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.391+0000 7f0fc5baf700 1 -- 172.21.15.163:0/218143165 shutdown_connections 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.391+0000 7f0fc5baf700 1 --2- 172.21.15.163:0/218143165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 0x7f0fc00fcfd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.391+0000 7f0fc5baf700 1 -- 172.21.15.163:0/218143165 >> 172.21.15.163:0/218143165 conn(0x7f0fc00f82a0 msgr2=0x7f0fc00fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.391+0000 7f0fc5baf700 1 -- 172.21.15.163:0/218143165 shutdown_connections 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.391+0000 7f0fc5baf700 1 -- 172.21.15.163:0/218143165 wait complete. 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.392+0000 7f0fc5baf700 1 Processor -- start 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.392+0000 7f0fc5baf700 1 -- start start 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.392+0000 7f0fc5baf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 0x7f0fc010ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.392+0000 7f0fc5baf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc010d3d0 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.395 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.392+0000 7f0fbf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 0x7f0fc010ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.392+0000 7f0fbf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 0x7f0fc010ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:44804/0 (socket says 172.21.15.163:44804) 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.392+0000 7f0fbf7fe700 1 -- 172.21.15.163:0/1972283513 learned_addr learned my addr 172.21.15.163:0/1972283513 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.392+0000 7f0fbf7fe700 1 -- 172.21.15.163:0/1972283513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4005040 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.393+0000 7f0fbf7fe700 1 --2- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 0x7f0fc010ce90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009680 tx=0x7f0fb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.393+0000 7f0fa7fff700 1 -- 172.21.15.163:0/1972283513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fb400bed0 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.393+0000 7f0fa7fff700 1 -- 172.21.15.163:0/1972283513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fb400b5c0 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.393+0000 7f0fa7fff700 1 -- 172.21.15.163:0/1972283513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fb401c5b0 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.393+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc010d5d0 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.396 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.393+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc010e270 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.397 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.394+0000 7f0fa7fff700 1 -- 172.21.15.163:0/1972283513 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f0fb4013420 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.397 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.394+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc005f5c0 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.397 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.394+0000 7f0fa7fff700 1 --2- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f0fa8036820 0x7f0fa8038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:29.397 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.394+0000 7f0fa7fff700 1 -- 172.21.15.163:0/1972283513 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0fb4049c10 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.397 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.394+0000 7f0fbeffd700 1 --2- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f0fa8036820 0x7f0fa8038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:29.397 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.395+0000 7f0fbeffd700 1 --2- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f0fa8036820 0x7f0fa8038cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009990 tx=0x7f0fb0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:29.400 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.399+0000 7f0fa7fff700 1 -- 172.21.15.163:0/1972283513 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0fb4013a80 con 0x7f0fc00fcbc0 2024-02-16T19:16:29.592 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.590+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 --> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi040:172.21.15.40=smithi040;smithi163:172.21.15.163=smithi163", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fc010ee30 con 0x7f0fa8036820 2024-02-16T19:16:29.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.596+0000 7f0fa7fff700 1 -- 172.21.15.163:0/1972283513 <== mgr.14160 v2:172.21.15.40:6800/277905681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0fc010ee30 con 0x7f0fa8036820 2024-02-16T19:16:29.597 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mon update... 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.597+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f0fa8036820 msgr2=0x7f0fa8038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.597+0000 7f0fc5baf700 1 --2- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f0fa8036820 0x7f0fa8038cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009990 tx=0x7f0fb0006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.597+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 msgr2=0x7f0fc010ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.597+0000 7f0fc5baf700 1 --2- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 0x7f0fc010ce90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009680 tx=0x7f0fb4009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.598+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 shutdown_connections 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.598+0000 7f0fc5baf700 1 --2- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f0fa8036820 0x7f0fa8038cd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.598+0000 7f0fc5baf700 1 --2- 172.21.15.163:0/1972283513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc00fcbc0 0x7f0fc010ce90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.598+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 >> 172.21.15.163:0/1972283513 conn(0x7f0fc00f82a0 msgr2=0x7f0fc00f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:29.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.598+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 shutdown_connections 2024-02-16T19:16:29.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:29.598+0000 7f0fc5baf700 1 -- 172.21.15.163:0/1972283513 wait complete. 2024-02-16T19:16:30.165 DEBUG:teuthology.orchestra.run.smithi163:mon.smithi163> sudo journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi163.service 2024-02-16T19:16:30.167 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:30.168 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:30.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:29 smithi040 ceph-mon[29201]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:30.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:29 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:30.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:29 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:30.447 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:30.494 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:31.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:30 smithi040 ceph-mon[29201]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi040:172.21.15.40=smithi040;smithi163:172.21.15.163=smithi163", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:16:31.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:30 smithi040 ceph-mon[29201]: Saving service mon spec with placement smithi040:172.21.15.40=smithi040;smithi163:172.21.15.163=smithi163;count:2 2024-02-16T19:16:31.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.928+0000 7fca10699700 1 -- 172.21.15.163:0/977689450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 msgr2=0x7fca080fbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:31.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.928+0000 7fca10699700 1 --2- 172.21.15.163:0/977689450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 0x7fca080fbec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca04009c60 tx=0x7fca04004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:31.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.928+0000 7fca10699700 1 -- 172.21.15.163:0/977689450 shutdown_connections 2024-02-16T19:16:31.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.928+0000 7fca10699700 1 --2- 172.21.15.163:0/977689450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 0x7fca080fbec0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:31.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.928+0000 7fca10699700 1 -- 172.21.15.163:0/977689450 >> 172.21.15.163:0/977689450 conn(0x7fca080f7210 msgr2=0x7fca080f9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:31.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.928+0000 7fca10699700 1 -- 172.21.15.163:0/977689450 shutdown_connections 2024-02-16T19:16:31.931 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.928+0000 7fca10699700 1 -- 172.21.15.163:0/977689450 wait complete. 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.929+0000 7fca10699700 1 Processor -- start 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.929+0000 7fca10699700 1 -- start start 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.929+0000 7fca10699700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 0x7fca0810cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.929+0000 7fca10699700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0810d400 con 0x7fca080fbab0 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.929+0000 7fca0e435700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 0x7fca0810cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.929+0000 7fca0e435700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 0x7fca0810cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:44824/0 (socket says 172.21.15.163:44824) 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.929+0000 7fca0e435700 1 -- 172.21.15.163:0/2926427904 learned_addr learned my addr 172.21.15.163:0/2926427904 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.929+0000 7fca0e435700 1 -- 172.21.15.163:0/2926427904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca04009940 con 0x7fca080fbab0 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.930+0000 7fca0e435700 1 --2- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 0x7fca0810cec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fca04004e60 tx=0x7fca04004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.930+0000 7fc9feffd700 1 -- 172.21.15.163:0/2926427904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca0401d440 con 0x7fca080fbab0 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.930+0000 7fc9feffd700 1 -- 172.21.15.163:0/2926427904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca0401da70 con 0x7fca080fbab0 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.930+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0810d600 con 0x7fca080fbab0 2024-02-16T19:16:31.932 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.930+0000 7fc9feffd700 1 -- 172.21.15.163:0/2926427904 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca04012aa0 con 0x7fca080fbab0 2024-02-16T19:16:31.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.930+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0810e2a0 con 0x7fca080fbab0 2024-02-16T19:16:31.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.931+0000 7fc9feffd700 1 -- 172.21.15.163:0/2926427904 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fca04019040 con 0x7fca080fbab0 2024-02-16T19:16:31.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.931+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0805f5c0 con 0x7fca080fbab0 2024-02-16T19:16:31.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.931+0000 7fc9feffd700 1 --2- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fc9f4036870 0x7fc9f4038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:31.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.931+0000 7fc9feffd700 1 -- 172.21.15.163:0/2926427904 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fca0404a3d0 con 0x7fca080fbab0 2024-02-16T19:16:31.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.931+0000 7fca0dc34700 1 --2- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fc9f4036870 0x7fc9f4038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:31.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.931+0000 7fca0dc34700 1 --2- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fc9f4036870 0x7fc9f4038d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009990 tx=0x7fc9f8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:31.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:31.936+0000 7fc9feffd700 1 -- 172.21.15.163:0/2926427904 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fca04012410 con 0x7fca080fbab0 2024-02-16T19:16:32.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.194+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fca0810ed90 con 0x7fca080fbab0 2024-02-16T19:16:32.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.194+0000 7fc9feffd700 1 -- 172.21.15.163:0/2926427904 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fca04017020 con 0x7fca080fbab0 2024-02-16T19:16:32.196 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:16:32.196 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fc9f4036870 msgr2=0x7fc9f4038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 --2- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fc9f4036870 0x7fc9f4038d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009990 tx=0x7fc9f8006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 msgr2=0x7fca0810cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 --2- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 0x7fca0810cec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fca04004e60 tx=0x7fca04004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 shutdown_connections 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 --2- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fc9f4036870 0x7fc9f4038d20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 --2- 172.21.15.163:0/2926427904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca080fbab0 0x7fca0810cec0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 >> 172.21.15.163:0/2926427904 conn(0x7fca080f7210 msgr2=0x7fca08066430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 shutdown_connections 2024-02-16T19:16:32.198 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:32.196+0000 7fca10699700 1 -- 172.21.15.163:0/2926427904 wait complete. 2024-02-16T19:16:32.199 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:16:32.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:31 smithi040 ceph-mon[29201]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:33.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:32 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/2926427904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:16:33.745 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:33.745 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:33.998 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:34.048 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:34.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:33 smithi040 ceph-mon[29201]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:35.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.448+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/1556440459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 msgr2=0x7fd2b40fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:35.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.448+0000 7fd2bb0c7700 1 --2- 172.21.15.163:0/1556440459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 0x7fd2b40fbee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80048c0 tx=0x7fd2a8009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:35.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.448+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/1556440459 shutdown_connections 2024-02-16T19:16:35.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.448+0000 7fd2bb0c7700 1 --2- 172.21.15.163:0/1556440459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 0x7fd2b40fbee0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:35.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.448+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/1556440459 >> 172.21.15.163:0/1556440459 conn(0x7fd2b40f7190 msgr2=0x7fd2b40f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:35.450 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.448+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/1556440459 shutdown_connections 2024-02-16T19:16:35.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.448+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/1556440459 wait complete. 2024-02-16T19:16:35.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.449+0000 7fd2bb0c7700 1 Processor -- start 2024-02-16T19:16:35.451 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.449+0000 7fd2bb0c7700 1 -- start start 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.449+0000 7fd2bb0c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 0x7fd2b410cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.449+0000 7fd2bb0c7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b410d400 con 0x7fd2b40fbad0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.449+0000 7fd2b8e63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 0x7fd2b410cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.449+0000 7fd2b8e63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 0x7fd2b410cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:43214/0 (socket says 172.21.15.163:43214) 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.449+0000 7fd2b8e63700 1 -- 172.21.15.163:0/2027282946 learned_addr learned my addr 172.21.15.163:0/2027282946 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.450+0000 7fd2b8e63700 1 -- 172.21.15.163:0/2027282946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8005040 con 0x7fd2b40fbad0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.450+0000 7fd2b8e63700 1 --2- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 0x7fd2b410cec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009680 tx=0x7fd2a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.450+0000 7fd2b17fa700 1 -- 172.21.15.163:0/2027282946 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2a800bed0 con 0x7fd2b40fbad0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.450+0000 7fd2b17fa700 1 -- 172.21.15.163:0/2027282946 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2a800b5c0 con 0x7fd2b40fbad0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.450+0000 7fd2b17fa700 1 -- 172.21.15.163:0/2027282946 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2a801c5b0 con 0x7fd2b40fbad0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.450+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b410d600 con 0x7fd2b40fbad0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.450+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b410e2a0 con 0x7fd2b40fbad0 2024-02-16T19:16:35.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.451+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b405f5c0 con 0x7fd2b40fbad0 2024-02-16T19:16:35.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.451+0000 7fd2b17fa700 1 -- 172.21.15.163:0/2027282946 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fd2a8013a80 con 0x7fd2b40fbad0 2024-02-16T19:16:35.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.451+0000 7fd2b17fa700 1 --2- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd2a4036820 0x7fd2a4038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:35.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.451+0000 7fd2b17fa700 1 -- 172.21.15.163:0/2027282946 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd2a8049ab0 con 0x7fd2b40fbad0 2024-02-16T19:16:35.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.452+0000 7fd2b3fff700 1 --2- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd2a4036820 0x7fd2a4038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:35.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.452+0000 7fd2b3fff700 1 --2- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd2a4036820 0x7fd2a4038cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009990 tx=0x7fd2a0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:35.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.457+0000 7fd2b17fa700 1 -- 172.21.15.163:0/2027282946 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd2a8011960 con 0x7fd2b40fbad0 2024-02-16T19:16:35.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.713+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd2b410ee10 con 0x7fd2b40fbad0 2024-02-16T19:16:35.714 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.713+0000 7fd2b17fa700 1 -- 172.21.15.163:0/2027282946 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd2a8011370 con 0x7fd2b40fbad0 2024-02-16T19:16:35.715 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:16:35.715 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.715+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd2a4036820 msgr2=0x7fd2a4038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.715+0000 7fd2bb0c7700 1 --2- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd2a4036820 0x7fd2a4038cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009990 tx=0x7fd2a0006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.715+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 msgr2=0x7fd2b410cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.715+0000 7fd2bb0c7700 1 --2- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 0x7fd2b410cec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009680 tx=0x7fd2a8009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.715+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 shutdown_connections 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.715+0000 7fd2bb0c7700 1 --2- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fd2a4036820 0x7fd2a4038cd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.715+0000 7fd2bb0c7700 1 --2- 172.21.15.163:0/2027282946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2b40fbad0 0x7fd2b410cec0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.716+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 >> 172.21.15.163:0/2027282946 conn(0x7fd2b40f7190 msgr2=0x7fd2b4065960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.716+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 shutdown_connections 2024-02-16T19:16:35.718 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:35.716+0000 7fd2bb0c7700 1 -- 172.21.15.163:0/2027282946 wait complete. 2024-02-16T19:16:35.718 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:16:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:35 smithi040 ceph-mon[29201]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:35 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/2027282946' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:16:37.248 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:37.249 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:37.497 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:37.543 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:38.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:37 smithi040 ceph-mon[29201]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.026+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/383199386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 msgr2=0x7fbbe00fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.026+0000 7fbbe4ff0700 1 --2- 172.21.15.163:0/383199386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 0x7fbbe00fcfd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009c60 tx=0x7fbbd8004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.026+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/383199386 shutdown_connections 2024-02-16T19:16:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.026+0000 7fbbe4ff0700 1 --2- 172.21.15.163:0/383199386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 0x7fbbe00fcfd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.026+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/383199386 >> 172.21.15.163:0/383199386 conn(0x7fbbe00f82a0 msgr2=0x7fbbe00fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.026+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/383199386 shutdown_connections 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.026+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/383199386 wait complete. 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbe4ff0700 1 Processor -- start 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbe4ff0700 1 -- start start 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbe4ff0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 0x7fbbe010ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbe4ff0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe010d3d0 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbde59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 0x7fbbe010ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbde59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 0x7fbbe010ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:43228/0 (socket says 172.21.15.163:43228) 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbde59c700 1 -- 172.21.15.163:0/26324424 learned_addr learned my addr 172.21.15.163:0/26324424 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbde59c700 1 -- 172.21.15.163:0/26324424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009940 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.027+0000 7fbbde59c700 1 --2- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 0x7fbbe010ce90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8015040 tx=0x7fbbd8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.028+0000 7fbbd6ffd700 1 -- 172.21.15.163:0/26324424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbd8007c80 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.028+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe010d5d0 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.028+0000 7fbbd6ffd700 1 -- 172.21.15.163:0/26324424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbd801d910 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.028+0000 7fbbd6ffd700 1 -- 172.21.15.163:0/26324424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbd80129f0 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.028+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe010e270 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.028+0000 7fbbd6ffd700 1 -- 172.21.15.163:0/26324424 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fbbd8019040 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.028+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe005f5c0 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.029+0000 7fbbd6ffd700 1 --2- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fbbcc036870 0x7fbbcc038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.029+0000 7fbbd6ffd700 1 -- 172.21.15.163:0/26324424 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbbd804a450 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.029+0000 7fbbddd9b700 1 --2- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fbbcc036870 0x7fbbcc038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.029+0000 7fbbddd9b700 1 --2- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fbbcc036870 0x7fbbcc038d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009990 tx=0x7fbbc8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:39.036 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.035+0000 7fbbd6ffd700 1 -- 172.21.15.163:0/26324424 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbbd802eb40 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.291+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbbe010ed60 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.291+0000 7fbbd6ffd700 1 -- 172.21.15.163:0/26324424 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fbbd8017070 con 0x7fbbe00fcbc0 2024-02-16T19:16:39.293 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:16:39.293 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fbbcc036870 msgr2=0x7fbbcc038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 --2- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fbbcc036870 0x7fbbcc038d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009990 tx=0x7fbbc8006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 msgr2=0x7fbbe010ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 --2- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 0x7fbbe010ce90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8015040 tx=0x7fbbd8004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 shutdown_connections 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 --2- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fbbcc036870 0x7fbbcc038d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 --2- 172.21.15.163:0/26324424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbe00fcbc0 0x7fbbe010ce90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 >> 172.21.15.163:0/26324424 conn(0x7fbbe00f82a0 msgr2=0x7fbbe00f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 shutdown_connections 2024-02-16T19:16:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:39.293+0000 7fbbe4ff0700 1 -- 172.21.15.163:0/26324424 wait complete. 2024-02-16T19:16:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:16:40.272 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:39 smithi040 ceph-mon[29201]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:40.272 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:39 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/26324424' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:16:40.884 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:40.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:41.132 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:41.178 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:41.212 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:41 smithi040 ceph-mon[29201]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:42.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.633+0000 7f89495e9700 1 -- 172.21.15.163:0/3561477331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 msgr2=0x7f89440fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:42.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.633+0000 7f89495e9700 1 --2- 172.21.15.163:0/3561477331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 0x7f89440fcfe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89340048c0 tx=0x7f8934009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:42.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.633+0000 7f89495e9700 1 -- 172.21.15.163:0/3561477331 shutdown_connections 2024-02-16T19:16:42.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.633+0000 7f89495e9700 1 --2- 172.21.15.163:0/3561477331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 0x7f89440fcfe0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:42.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.633+0000 7f89495e9700 1 -- 172.21.15.163:0/3561477331 >> 172.21.15.163:0/3561477331 conn(0x7f89440f8270 msgr2=0x7f89440fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:42.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.633+0000 7f89495e9700 1 -- 172.21.15.163:0/3561477331 shutdown_connections 2024-02-16T19:16:42.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.633+0000 7f89495e9700 1 -- 172.21.15.163:0/3561477331 wait complete. 2024-02-16T19:16:42.636 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.634+0000 7f89495e9700 1 Processor -- start 2024-02-16T19:16:42.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.634+0000 7f89495e9700 1 -- start start 2024-02-16T19:16:42.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.634+0000 7f89495e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 0x7f894410cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:42.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.634+0000 7f89495e9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894410d4a0 con 0x7f89440fcbd0 2024-02-16T19:16:42.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.634+0000 7f8942ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 0x7f894410cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:42.637 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.634+0000 7f8942ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 0x7f894410cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:39156/0 (socket says 172.21.15.163:39156) 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.634+0000 7f8942ffd700 1 -- 172.21.15.163:0/3584904742 learned_addr learned my addr 172.21.15.163:0/3584904742 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.635+0000 7f8942ffd700 1 -- 172.21.15.163:0/3584904742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934005040 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.635+0000 7f8942ffd700 1 --2- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 0x7f894410cf60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8934009b80 tx=0x7f8934009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.635+0000 7f893bfff700 1 -- 172.21.15.163:0/3584904742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f893400b5c0 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.635+0000 7f893bfff700 1 -- 172.21.15.163:0/3584904742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f893400be70 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.635+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894410d6a0 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.635+0000 7f893bfff700 1 -- 172.21.15.163:0/3584904742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f893401c660 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.635+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894410e2f0 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.636+0000 7f893bfff700 1 -- 172.21.15.163:0/3584904742 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f8934013420 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.636+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894405f5c0 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.636+0000 7f893bfff700 1 --2- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f8930036870 0x7f8930038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.636+0000 7f893bfff700 1 -- 172.21.15.163:0/3584904742 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8934049a80 con 0x7f89440fcbd0 2024-02-16T19:16:42.638 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.636+0000 7f893a7fc700 1 --2- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f8930036870 0x7f8930038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:42.639 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.637+0000 7f893a7fc700 1 --2- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f8930036870 0x7f8930038d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f892c009990 tx=0x7f892c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:42.642 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.641+0000 7f893bfff700 1 -- 172.21.15.163:0/3584904742 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8934025400 con 0x7f89440fcbd0 2024-02-16T19:16:42.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.894+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f894410ee60 con 0x7f89440fcbd0 2024-02-16T19:16:42.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.895+0000 7f893bfff700 1 -- 172.21.15.163:0/3584904742 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8934011370 con 0x7f89440fcbd0 2024-02-16T19:16:42.897 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:16:42.897 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f8930036870 msgr2=0x7f8930038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 --2- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f8930036870 0x7f8930038d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f892c009990 tx=0x7f892c006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 msgr2=0x7f894410cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 --2- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 0x7f894410cf60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8934009b80 tx=0x7f8934009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 shutdown_connections 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 --2- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f8930036870 0x7f8930038d20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 --2- 172.21.15.163:0/3584904742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440fcbd0 0x7f894410cf60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 >> 172.21.15.163:0/3584904742 conn(0x7f89440f8270 msgr2=0x7f89440f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 shutdown_connections 2024-02-16T19:16:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:42.897+0000 7f89495e9700 1 -- 172.21.15.163:0/3584904742 wait complete. 2024-02-16T19:16:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:16:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:43 smithi040 ceph-mon[29201]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:43 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/3584904742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:16:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:43 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:43 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:43 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:43.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:43 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:43.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:43 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:43.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:43 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:44.548 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:44.548 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:44.844 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:45.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:44 smithi040 ceph-mon[29201]: Deploying daemon prometheus.smithi040 on smithi040 2024-02-16T19:16:45.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:44 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:16:46.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:45 smithi040 ceph-mon[29201]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.225+0000 7f1225416700 1 -- 172.21.15.163:0/397039921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 msgr2=0x7f12200ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.225+0000 7f1225416700 1 --2- 172.21.15.163:0/397039921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 0x7f12200ff790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f12140048c0 tx=0x7f1214009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.225+0000 7f1225416700 1 -- 172.21.15.163:0/397039921 shutdown_connections 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.225+0000 7f1225416700 1 --2- 172.21.15.163:0/397039921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 0x7f12200ff790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.225+0000 7f1225416700 1 -- 172.21.15.163:0/397039921 >> 172.21.15.163:0/397039921 conn(0x7f12200f71a0 msgr2=0x7f12200f95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.225+0000 7f1225416700 1 -- 172.21.15.163:0/397039921 shutdown_connections 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.225+0000 7f1225416700 1 -- 172.21.15.163:0/397039921 wait complete. 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.226+0000 7f1225416700 1 Processor -- start 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.226+0000 7f1225416700 1 -- start start 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.226+0000 7f1225416700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 0x7f122010ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.226+0000 7f1225416700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122010d3c0 con 0x7f1220066880 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.226+0000 7f121effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 0x7f122010ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.226+0000 7f121effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 0x7f122010ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:39176/0 (socket says 172.21.15.163:39176) 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.227+0000 7f121effd700 1 -- 172.21.15.163:0/1976296768 learned_addr learned my addr 172.21.15.163:0/1976296768 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.227+0000 7f121effd700 1 -- 172.21.15.163:0/1976296768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1214005040 con 0x7f1220066880 2024-02-16T19:16:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.227+0000 7f121effd700 1 --2- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 0x7f122010ce80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1214009680 tx=0x7f1214009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.227+0000 7f12077fe700 1 -- 172.21.15.163:0/1976296768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f121400b5c0 con 0x7f1220066880 2024-02-16T19:16:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.227+0000 7f12077fe700 1 -- 172.21.15.163:0/1976296768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1214004e60 con 0x7f1220066880 2024-02-16T19:16:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.227+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122010d5c0 con 0x7f1220066880 2024-02-16T19:16:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.227+0000 7f12077fe700 1 -- 172.21.15.163:0/1976296768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f121401c590 con 0x7f1220066880 2024-02-16T19:16:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.227+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122010e260 con 0x7f1220066880 2024-02-16T19:16:46.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.228+0000 7f12077fe700 1 -- 172.21.15.163:0/1976296768 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f1214013420 con 0x7f1220066880 2024-02-16T19:16:46.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.228+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122005f5c0 con 0x7f1220066880 2024-02-16T19:16:46.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.228+0000 7f12077fe700 1 --2- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f12080368c0 0x7f1208038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:46.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.228+0000 7f12077fe700 1 -- 172.21.15.163:0/1976296768 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1214049b70 con 0x7f1220066880 2024-02-16T19:16:46.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.228+0000 7f121e7fc700 1 --2- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f12080368c0 0x7f1208038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:46.231 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.229+0000 7f121e7fc700 1 --2- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f12080368c0 0x7f1208038d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1210009990 tx=0x7f1210006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:46.234 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.233+0000 7f12077fe700 1 -- 172.21.15.163:0/1976296768 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f12140119c0 con 0x7f1220066880 2024-02-16T19:16:46.487 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.485+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f122010edb0 con 0x7f1220066880 2024-02-16T19:16:46.487 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.486+0000 7f12077fe700 1 -- 172.21.15.163:0/1976296768 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1214011370 con 0x7f1220066880 2024-02-16T19:16:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:16:46.487 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:16:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.487+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f12080368c0 msgr2=0x7f1208038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.487+0000 7f1225416700 1 --2- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f12080368c0 0x7f1208038d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1210009990 tx=0x7f1210006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.487+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 msgr2=0x7f122010ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.487+0000 7f1225416700 1 --2- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 0x7f122010ce80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1214009680 tx=0x7f1214009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.487+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 shutdown_connections 2024-02-16T19:16:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.487+0000 7f1225416700 1 --2- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f12080368c0 0x7f1208038d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.487+0000 7f1225416700 1 --2- 172.21.15.163:0/1976296768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1220066880 0x7f122010ce80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:46.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.488+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 >> 172.21.15.163:0/1976296768 conn(0x7f12200f71a0 msgr2=0x7f12200f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:46.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.488+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 shutdown_connections 2024-02-16T19:16:46.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:46.488+0000 7f1225416700 1 -- 172.21.15.163:0/1976296768 wait complete. 2024-02-16T19:16:46.490 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:16:47.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:46 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/1976296768' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:16:47.991 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:47.991 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:48.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:47 smithi040 ceph-mon[29201]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:48.288 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.817+0000 7f3dc65db700 1 -- 172.21.15.163:0/3074090157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 msgr2=0x7f3dc00fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.817+0000 7f3dc65db700 1 --2- 172.21.15.163:0/3074090157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 0x7f3dc00fcfe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3db00048c0 tx=0x7f3db0009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.817+0000 7f3dc65db700 1 -- 172.21.15.163:0/3074090157 shutdown_connections 2024-02-16T19:16:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.817+0000 7f3dc65db700 1 --2- 172.21.15.163:0/3074090157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 0x7f3dc00fcfe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.817+0000 7f3dc65db700 1 -- 172.21.15.163:0/3074090157 >> 172.21.15.163:0/3074090157 conn(0x7f3dc00f8290 msgr2=0x7f3dc00fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.817+0000 7f3dc65db700 1 -- 172.21.15.163:0/3074090157 shutdown_connections 2024-02-16T19:16:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.817+0000 7f3dc65db700 1 -- 172.21.15.163:0/3074090157 wait complete. 2024-02-16T19:16:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.817+0000 7f3dc65db700 1 Processor -- start 2024-02-16T19:16:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.818+0000 7f3dc65db700 1 -- start start 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.818+0000 7f3dc65db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 0x7f3dc010f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.818+0000 7f3dc65db700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc010f600 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.818+0000 7f3dbffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 0x7f3dc010f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.818+0000 7f3dbffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 0x7f3dc010f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:39196/0 (socket says 172.21.15.163:39196) 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.818+0000 7f3dbffff700 1 -- 172.21.15.163:0/3251599467 learned_addr learned my addr 172.21.15.163:0/3251599467 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.818+0000 7f3dbffff700 1 -- 172.21.15.163:0/3251599467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0005040 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.819+0000 7f3dbffff700 1 --2- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 0x7f3dc010f0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009680 tx=0x7f3db0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.819+0000 7f3dbcff9700 1 -- 172.21.15.163:0/3251599467 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3db000bed0 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.819+0000 7f3dbcff9700 1 -- 172.21.15.163:0/3251599467 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3db000b5c0 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.819+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc010f800 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.819+0000 7f3dbcff9700 1 -- 172.21.15.163:0/3251599467 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3db001c5b0 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.819+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc01104a0 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.820+0000 7f3dbcff9700 1 -- 172.21.15.163:0/3251599467 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f3db0013420 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.820+0000 7f3dbcff9700 1 --2- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f3da8036820 0x7f3da8038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.820+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc005f5c0 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.820+0000 7f3dbcff9700 1 -- 172.21.15.163:0/3251599467 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3db0049c10 con 0x7f3dc00fcbd0 2024-02-16T19:16:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.820+0000 7f3dbf7fe700 1 --2- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f3da8036820 0x7f3da8038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:49.823 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.820+0000 7f3dbf7fe700 1 --2- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f3da8036820 0x7f3da8038cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009990 tx=0x7f3db4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:49.826 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:49.825+0000 7f3dbcff9700 1 -- 172.21.15.163:0/3251599467 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3db0011370 con 0x7f3dc00fcbd0 2024-02-16T19:16:50.083 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.082+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3dc004bed0 con 0x7f3dc00fcbd0 2024-02-16T19:16:50.083 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.082+0000 7f3dbcff9700 1 -- 172.21.15.163:0/3251599467 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3db0011730 con 0x7f3dc00fcbd0 2024-02-16T19:16:50.084 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:16:50.084 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:16:50.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f3da8036820 msgr2=0x7f3da8038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:50.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 --2- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f3da8036820 0x7f3da8038cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009990 tx=0x7f3db4006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:50.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 msgr2=0x7f3dc010f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:50.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 --2- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 0x7f3dc010f0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009680 tx=0x7f3db0009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:50.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 shutdown_connections 2024-02-16T19:16:50.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 --2- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f3da8036820 0x7f3da8038cd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:50.086 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 --2- 172.21.15.163:0/3251599467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00fcbd0 0x7f3dc010f0c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 >> 172.21.15.163:0/3251599467 conn(0x7f3dc00f8290 msgr2=0x7f3dc00701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 shutdown_connections 2024-02-16T19:16:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:50.085+0000 7f3dc65db700 1 -- 172.21.15.163:0/3251599467 wait complete. 2024-02-16T19:16:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:16:50.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:49 smithi040 ceph-mon[29201]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:51.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:50 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/3251599467' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:16:51.696 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:51.696 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:51.948 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:51.993 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:52.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:51 smithi040 ceph-mon[29201]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:53.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 -- 172.21.15.163:0/4112670254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 msgr2=0x7f7d5c0fbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:53.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 --2- 172.21.15.163:0/4112670254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 0x7f7d5c0fbed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d500048c0 tx=0x7f7d50009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:53.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 -- 172.21.15.163:0/4112670254 shutdown_connections 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 --2- 172.21.15.163:0/4112670254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 0x7f7d5c0fbed0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 -- 172.21.15.163:0/4112670254 >> 172.21.15.163:0/4112670254 conn(0x7f7d5c0f71a0 msgr2=0x7f7d5c0f95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 -- 172.21.15.163:0/4112670254 shutdown_connections 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 -- 172.21.15.163:0/4112670254 wait complete. 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 Processor -- start 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.264+0000 7f7d62f28700 1 -- start start 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d62f28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 0x7f7d5c10f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d62f28700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c10f620 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d60cc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 0x7f7d5c10f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d60cc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 0x7f7d5c10f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:56290/0 (socket says 172.21.15.163:56290) 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d60cc4700 1 -- 172.21.15.163:0/2814828286 learned_addr learned my addr 172.21.15.163:0/2814828286 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d60cc4700 1 -- 172.21.15.163:0/2814828286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d50005040 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.267 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d60cc4700 1 --2- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 0x7f7d5c10f0e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009b80 tx=0x7f7d50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:53.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d597fa700 1 -- 172.21.15.163:0/2814828286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d5000b5c0 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d597fa700 1 -- 172.21.15.163:0/2814828286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d5000be70 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.265+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c10f820 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.266+0000 7f7d597fa700 1 -- 172.21.15.163:0/2814828286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d5001c660 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.266+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c1104c0 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.266+0000 7f7d597fa700 1 -- 172.21.15.163:0/2814828286 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f7d50013420 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.266+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c05f5c0 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.267+0000 7f7d597fa700 1 --2- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f7d4c036870 0x7f7d4c038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:53.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.267+0000 7f7d597fa700 1 -- 172.21.15.163:0/2814828286 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7d50049a80 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.267+0000 7f7d5bfff700 1 --2- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f7d4c036870 0x7f7d4c038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:53.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.267+0000 7f7d5bfff700 1 --2- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f7d4c036870 0x7f7d4c038d20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009990 tx=0x7f7d48006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:53.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.272+0000 7f7d597fa700 1 -- 172.21.15.163:0/2814828286 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7d50011550 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.526+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7d5c111010 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.527+0000 7f7d597fa700 1 -- 172.21.15.163:0/2814828286 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7d50011370 con 0x7f7d5c0fbac0 2024-02-16T19:16:53.528 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:16:53.528 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.528+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f7d4c036870 msgr2=0x7f7d4c038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.528+0000 7f7d62f28700 1 --2- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f7d4c036870 0x7f7d4c038d20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009990 tx=0x7f7d48006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.529+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 msgr2=0x7f7d5c10f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.529+0000 7f7d62f28700 1 --2- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 0x7f7d5c10f0e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009b80 tx=0x7f7d50009e30 comp rx=0 tx=0).stop 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.529+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 shutdown_connections 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.529+0000 7f7d62f28700 1 --2- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f7d4c036870 0x7f7d4c038d20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.529+0000 7f7d62f28700 1 --2- 172.21.15.163:0/2814828286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d5c0fbac0 0x7f7d5c10f0e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.529+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 >> 172.21.15.163:0/2814828286 conn(0x7f7d5c0f71a0 msgr2=0x7f7d5c066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:53.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.529+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 shutdown_connections 2024-02-16T19:16:53.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:53.529+0000 7f7d62f28700 1 -- 172.21.15.163:0/2814828286 wait complete. 2024-02-16T19:16:53.531 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:16:54.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:53 smithi040 ceph-mon[29201]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:54.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:53 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/2814828286' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:16:55.146 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:55.146 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:55.394 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:55.440 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:56.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:55 smithi040 ceph-mon[29201]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:56.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.831+0000 7fe80cdc0700 1 -- 172.21.15.163:0/3852666956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 msgr2=0x7fe808101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:56.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.831+0000 7fe80cdc0700 1 --2- 172.21.15.163:0/3852666956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 0x7fe808101ba0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe800009c60 tx=0x7fe800004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:56.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.831+0000 7fe80cdc0700 1 -- 172.21.15.163:0/3852666956 shutdown_connections 2024-02-16T19:16:56.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.831+0000 7fe80cdc0700 1 --2- 172.21.15.163:0/3852666956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 0x7fe808101ba0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:56.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.831+0000 7fe80cdc0700 1 -- 172.21.15.163:0/3852666956 >> 172.21.15.163:0/3852666956 conn(0x7fe8080f7190 msgr2=0x7fe8080f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:56.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.831+0000 7fe80cdc0700 1 -- 172.21.15.163:0/3852666956 shutdown_connections 2024-02-16T19:16:56.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.831+0000 7fe80cdc0700 1 -- 172.21.15.163:0/3852666956 wait complete. 2024-02-16T19:16:56.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.832+0000 7fe80cdc0700 1 Processor -- start 2024-02-16T19:16:56.834 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.832+0000 7fe80cdc0700 1 -- start start 2024-02-16T19:16:56.834 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.832+0000 7fe80cdc0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 0x7fe80810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:56.834 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.832+0000 7fe80cdc0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80810f5c0 con 0x7fe808065dd0 2024-02-16T19:16:56.835 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.832+0000 7fe80659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 0x7fe80810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:56.835 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.832+0000 7fe80659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 0x7fe80810f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:56316/0 (socket says 172.21.15.163:56316) 2024-02-16T19:16:56.835 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.832+0000 7fe80659c700 1 -- 172.21.15.163:0/567919383 learned_addr learned my addr 172.21.15.163:0/567919383 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.832+0000 7fe80659c700 1 -- 172.21.15.163:0/567919383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009940 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.833+0000 7fe80659c700 1 --2- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 0x7fe80810f080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe800004e60 tx=0x7fe800004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.833+0000 7fe7f6ffd700 1 -- 172.21.15.163:0/567919383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe80001d440 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.833+0000 7fe7f6ffd700 1 -- 172.21.15.163:0/567919383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe80001da70 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.833+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80810f7c0 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.833+0000 7fe7f6ffd700 1 -- 172.21.15.163:0/567919383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe800012aa0 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.833+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe808110460 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.834+0000 7fe7f6ffd700 1 -- 172.21.15.163:0/567919383 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe800019040 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.834+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80805f5c0 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.834+0000 7fe7f6ffd700 1 --2- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fe7f0036870 0x7fe7f0038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.834+0000 7fe7f6ffd700 1 -- 172.21.15.163:0/567919383 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe80004a3d0 con 0x7fe808065dd0 2024-02-16T19:16:56.836 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.834+0000 7fe805d9b700 1 --2- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fe7f0036870 0x7fe7f0038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:16:56.837 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.835+0000 7fe805d9b700 1 --2- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fe7f0036870 0x7fe7f0038d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009990 tx=0x7fe7fc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:16:56.840 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:56.839+0000 7fe7f6ffd700 1 -- 172.21.15.163:0/567919383 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe800012410 con 0x7fe808065dd0 2024-02-16T19:16:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.094+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe808110f50 con 0x7fe808065dd0 2024-02-16T19:16:57.095 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.094+0000 7fe7f6ffd700 1 -- 172.21.15.163:0/567919383 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe800017020 con 0x7fe808065dd0 2024-02-16T19:16:57.096 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:16:57.096 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:16:57.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fe7f0036870 msgr2=0x7fe7f0038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:57.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 --2- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fe7f0036870 0x7fe7f0038d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009990 tx=0x7fe7fc006e40 comp rx=0 tx=0).stop 2024-02-16T19:16:57.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 msgr2=0x7fe80810f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:16:57.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 --2- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 0x7fe80810f080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe800004e60 tx=0x7fe800004d10 comp rx=0 tx=0).stop 2024-02-16T19:16:57.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 shutdown_connections 2024-02-16T19:16:57.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 --2- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7fe7f0036870 0x7fe7f0038d20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:57.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 --2- 172.21.15.163:0/567919383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808065dd0 0x7fe80810f080 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:16:57.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 >> 172.21.15.163:0/567919383 conn(0x7fe8080f7190 msgr2=0x7fe8080f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:16:57.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 shutdown_connections 2024-02-16T19:16:57.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:16:57.096+0000 7fe80cdc0700 1 -- 172.21.15.163:0/567919383 wait complete. 2024-02-16T19:16:57.099 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:16:58.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:57 smithi040 ceph-mon[29201]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:16:58.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:57 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/567919383' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:16:58.709 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:16:58.710 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:16:58.959 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:16:59.007 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:00.082 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:16:59 smithi040 ceph-mon[29201]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:00.489 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 -- 172.21.15.163:0/2022672439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 msgr2=0x7ff65c0fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 --2- 172.21.15.163:0/2022672439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 0x7ff65c0fcfd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff6500048c0 tx=0x7ff650009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 -- 172.21.15.163:0/2022672439 shutdown_connections 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 --2- 172.21.15.163:0/2022672439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 0x7ff65c0fcfd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 -- 172.21.15.163:0/2022672439 >> 172.21.15.163:0/2022672439 conn(0x7ff65c0f82a0 msgr2=0x7ff65c0fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 -- 172.21.15.163:0/2022672439 shutdown_connections 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 -- 172.21.15.163:0/2022672439 wait complete. 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 Processor -- start 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 -- start start 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.487+0000 7ff662c8f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 0x7ff65c10ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff662c8f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c10d3d0 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff660a2b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 0x7ff65c10ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff660a2b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 0x7ff65c10ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:56326/0 (socket says 172.21.15.163:56326) 2024-02-16T19:17:00.490 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff660a2b700 1 -- 172.21.15.163:0/1970236051 learned_addr learned my addr 172.21.15.163:0/1970236051 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:00.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff660a2b700 1 -- 172.21.15.163:0/1970236051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff650005040 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff660a2b700 1 --2- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 0x7ff65c10ce90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff650009b80 tx=0x7ff650009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:00.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff6597fa700 1 -- 172.21.15.163:0/1970236051 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff65000b5c0 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c10d5d0 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff6597fa700 1 -- 172.21.15.163:0/1970236051 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff65000be70 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff6597fa700 1 -- 172.21.15.163:0/1970236051 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff65001c660 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.491 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.488+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c10e270 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.492 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.489+0000 7ff6597fa700 1 -- 172.21.15.163:0/1970236051 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ff650013420 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.492 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.489+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c05f5c0 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.492 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.489+0000 7ff6597fa700 1 --2- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7ff64c036870 0x7ff64c038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:00.492 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.489+0000 7ff6597fa700 1 -- 172.21.15.163:0/1970236051 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff650049a80 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.492 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.490+0000 7ff65bfff700 1 --2- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7ff64c036870 0x7ff64c038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:00.492 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.490+0000 7ff65bfff700 1 --2- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7ff64c036870 0x7ff64c038d20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff648009990 tx=0x7ff648006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:00.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.495+0000 7ff6597fa700 1 -- 172.21.15.163:0/1970236051 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff650011550 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.752 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.751+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff65c10edc0 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.753 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.752+0000 7ff6597fa700 1 -- 172.21.15.163:0/1970236051 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff650011370 con 0x7ff65c0fcbc0 2024-02-16T19:17:00.753 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:00.754 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:17:00.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7ff64c036870 msgr2=0x7ff64c038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:00.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 --2- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7ff64c036870 0x7ff64c038d20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff648009990 tx=0x7ff648006e40 comp rx=0 tx=0).stop 2024-02-16T19:17:00.755 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 msgr2=0x7ff65c10ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:00.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 --2- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 0x7ff65c10ce90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff650009b80 tx=0x7ff650009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:00.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 shutdown_connections 2024-02-16T19:17:00.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 --2- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7ff64c036870 0x7ff64c038d20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:00.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 --2- 172.21.15.163:0/1970236051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff65c0fcbc0 0x7ff65c10ce90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:00.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 >> 172.21.15.163:0/1970236051 conn(0x7ff65c0f82a0 msgr2=0x7ff65c0f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:00.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 shutdown_connections 2024-02-16T19:17:00.756 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:00.754+0000 7ff662c8f700 1 -- 172.21.15.163:0/1970236051 wait complete. 2024-02-16T19:17:00.756 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:17:01.529 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:01 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/1970236051' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:01.529 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:01 smithi040 ceph-mon[29201]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:02.320 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:17:02.320 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:17:02.568 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:02.615 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:03 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:03 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:03 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:03 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-16T19:17:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:03 smithi040 ceph-mon[29201]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:04.026 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.024+0000 7f9e1ef03700 1 -- 172.21.15.163:0/981446341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 msgr2=0x7f9e180fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.024+0000 7f9e1ef03700 1 --2- 172.21.15.163:0/981446341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 0x7f9e180fcfd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009c60 tx=0x7f9e10004d10 comp rx=0 tx=0).stop 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.024+0000 7f9e1ef03700 1 -- 172.21.15.163:0/981446341 shutdown_connections 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.024+0000 7f9e1ef03700 1 --2- 172.21.15.163:0/981446341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 0x7f9e180fcfd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.024+0000 7f9e1ef03700 1 -- 172.21.15.163:0/981446341 >> 172.21.15.163:0/981446341 conn(0x7f9e180f82a0 msgr2=0x7f9e180fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.024+0000 7f9e1ef03700 1 -- 172.21.15.163:0/981446341 shutdown_connections 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.024+0000 7f9e1ef03700 1 -- 172.21.15.163:0/981446341 wait complete. 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.025+0000 7f9e1ef03700 1 Processor -- start 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.025+0000 7f9e1ef03700 1 -- start start 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.025+0000 7f9e1ef03700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 0x7f9e1810ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.025+0000 7f9e1ef03700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1810d3d0 con 0x7f9e180fcbc0 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.025+0000 7f9e1cc9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 0x7f9e1810ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.025+0000 7f9e1cc9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 0x7f9e1810ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:35374/0 (socket says 172.21.15.163:35374) 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.025+0000 7f9e1cc9f700 1 -- 172.21.15.163:0/852084814 learned_addr learned my addr 172.21.15.163:0/852084814 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:04.027 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.025+0000 7f9e1cc9f700 1 -- 172.21.15.163:0/852084814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e10009940 con 0x7f9e180fcbc0 2024-02-16T19:17:04.028 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.026+0000 7f9e1cc9f700 1 --2- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 0x7f9e1810ce90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e10015040 tx=0x7f9e10004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.026+0000 7f9e157fa700 1 -- 172.21.15.163:0/852084814 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e10007c80 con 0x7f9e180fcbc0 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.026+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1810d5d0 con 0x7f9e180fcbc0 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.026+0000 7f9e157fa700 1 -- 172.21.15.163:0/852084814 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e1001d910 con 0x7f9e180fcbc0 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.026+0000 7f9e157fa700 1 -- 172.21.15.163:0/852084814 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e100129f0 con 0x7f9e180fcbc0 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.026+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1810e270 con 0x7f9e180fcbc0 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.027+0000 7f9e157fa700 1 -- 172.21.15.163:0/852084814 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f9e10019040 con 0x7f9e180fcbc0 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.027+0000 7f9e157fa700 1 --2- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f9e040368c0 0x7f9e04038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.027+0000 7f9e157fa700 1 -- 172.21.15.163:0/852084814 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9e1004b3f0 con 0x7f9e180fcbc0 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.027+0000 7f9e17fff700 1 -- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f9e040368c0 msgr2=0x7f9e04038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/277905681 2024-02-16T19:17:04.029 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.027+0000 7f9e17fff700 1 --2- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f9e040368c0 0x7f9e04038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:17:04.030 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.027+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1805f5c0 con 0x7f9e180fcbc0 2024-02-16T19:17:04.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.032+0000 7f9e157fa700 1 -- 172.21.15.163:0/852084814 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9e10017020 con 0x7f9e180fcbc0 2024-02-16T19:17:04.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.228+0000 7f9e17fff700 1 -- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f9e040368c0 msgr2=0x7f9e04038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/277905681 2024-02-16T19:17:04.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.228+0000 7f9e17fff700 1 --2- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f9e040368c0 0x7f9e04038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:17:04.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.286+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9e1804bed0 con 0x7f9e180fcbc0 2024-02-16T19:17:04.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.286+0000 7f9e157fa700 1 -- 172.21.15.163:0/852084814 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9e10035090 con 0x7f9e180fcbc0 2024-02-16T19:17:04.288 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:04.288 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.288+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f9e040368c0 msgr2=0x7f9e04038d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.288+0000 7f9e1ef03700 1 --2- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f9e040368c0 0x7f9e04038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.288+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 msgr2=0x7f9e1810ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.288+0000 7f9e1ef03700 1 --2- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 0x7f9e1810ce90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e10015040 tx=0x7f9e10004d10 comp rx=0 tx=0).stop 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.289+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 shutdown_connections 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.289+0000 7f9e1ef03700 1 --2- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f9e040368c0 0x7f9e04038d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.289+0000 7f9e1ef03700 1 --2- 172.21.15.163:0/852084814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e180fcbc0 0x7f9e1810ce90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.289+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 >> 172.21.15.163:0/852084814 conn(0x7f9e180f82a0 msgr2=0x7f9e180f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:04.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.289+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 shutdown_connections 2024-02-16T19:17:04.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:04.289+0000 7f9e1ef03700 1 -- 172.21.15.163:0/852084814 wait complete. 2024-02-16T19:17:04.291 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:17:04.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:04 smithi040 ceph-mon[29201]: from='mgr.14160 172.21.15.40:0/2897559828' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-16T19:17:04.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:04 smithi040 ceph-mon[29201]: mgrmap e14: smithi040.jzygzo(active, since 68s) 2024-02-16T19:17:04.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:04 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/852084814' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:05.915 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:17:05.915 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:17:06.156 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:06.204 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.609+0000 7f90554c9700 1 -- 172.21.15.163:0/3900816413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 msgr2=0x7f90500fcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.609+0000 7f90554c9700 1 --2- 172.21.15.163:0/3900816413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 0x7f90500fcfc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f90440048c0 tx=0x7f9044009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.609+0000 7f90554c9700 1 -- 172.21.15.163:0/3900816413 shutdown_connections 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.609+0000 7f90554c9700 1 --2- 172.21.15.163:0/3900816413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 0x7f90500fcfc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.609+0000 7f90554c9700 1 -- 172.21.15.163:0/3900816413 >> 172.21.15.163:0/3900816413 conn(0x7f90500f8330 msgr2=0x7f90500fa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.609+0000 7f90554c9700 1 -- 172.21.15.163:0/3900816413 shutdown_connections 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.609+0000 7f90554c9700 1 -- 172.21.15.163:0/3900816413 wait complete. 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.610+0000 7f90554c9700 1 Processor -- start 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.610+0000 7f90554c9700 1 -- start start 2024-02-16T19:17:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.610+0000 7f90554c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 0x7f905010ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:07.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.610+0000 7f90554c9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905010d3d0 con 0x7f90500fcbb0 2024-02-16T19:17:07.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.610+0000 7f904effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 0x7f905010ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:07.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.610+0000 7f904effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 0x7f905010ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:35390/0 (socket says 172.21.15.163:35390) 2024-02-16T19:17:07.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.610+0000 7f904effd700 1 -- 172.21.15.163:0/2460195351 learned_addr learned my addr 172.21.15.163:0/2460195351 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:07.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.610+0000 7f904effd700 1 -- 172.21.15.163:0/2460195351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044005040 con 0x7f90500fcbb0 2024-02-16T19:17:07.613 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.611+0000 7f904effd700 1 --2- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 0x7f905010ce90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9044009680 tx=0x7f9044009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:07.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.611+0000 7f90377fe700 1 -- 172.21.15.163:0/2460195351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f904400b5c0 con 0x7f90500fcbb0 2024-02-16T19:17:07.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.611+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905010d5d0 con 0x7f90500fcbb0 2024-02-16T19:17:07.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.611+0000 7f90377fe700 1 -- 172.21.15.163:0/2460195351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9044004e60 con 0x7f90500fcbb0 2024-02-16T19:17:07.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.611+0000 7f90377fe700 1 -- 172.21.15.163:0/2460195351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f904401c590 con 0x7f90500fcbb0 2024-02-16T19:17:07.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.611+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905010e270 con 0x7f90500fcbb0 2024-02-16T19:17:07.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.612+0000 7f90377fe700 1 -- 172.21.15.163:0/2460195351 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f904401c810 con 0x7f90500fcbb0 2024-02-16T19:17:07.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.612+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905005f5c0 con 0x7f90500fcbb0 2024-02-16T19:17:07.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.612+0000 7f90377fe700 1 --2- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f90380368c0 0x7f9038038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:07.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.612+0000 7f90377fe700 1 -- 172.21.15.163:0/2460195351 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9044049e70 con 0x7f90500fcbb0 2024-02-16T19:17:07.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.612+0000 7f904e7fc700 1 -- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f90380368c0 msgr2=0x7f9038038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/277905681 2024-02-16T19:17:07.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.612+0000 7f904e7fc700 1 --2- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f90380368c0 0x7f9038038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:17:07.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.617+0000 7f90377fe700 1 -- 172.21.15.163:0/2460195351 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9044023300 con 0x7f90500fcbb0 2024-02-16T19:17:07.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.813+0000 7f904e7fc700 1 -- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f90380368c0 msgr2=0x7f9038038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/277905681 2024-02-16T19:17:07.814 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.813+0000 7f904e7fc700 1 --2- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f90380368c0 0x7f9038038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-16T19:17:07.882 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.881+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f905010edc0 con 0x7f90500fcbb0 2024-02-16T19:17:07.883 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.882+0000 7f90377fe700 1 -- 172.21.15.163:0/2460195351 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9044011b60 con 0x7f90500fcbb0 2024-02-16T19:17:07.883 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:07.883 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:17:07.885 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f90380368c0 msgr2=0x7f9038038d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 --2- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f90380368c0 0x7f9038038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 msgr2=0x7f905010ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 --2- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 0x7f905010ce90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9044009680 tx=0x7f9044009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 shutdown_connections 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 --2- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:6800/277905681,v1:172.21.15.40:6801/277905681] conn(0x7f90380368c0 0x7f9038038d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 --2- 172.21.15.163:0/2460195351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90500fcbb0 0x7f905010ce90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 >> 172.21.15.163:0/2460195351 conn(0x7f90500f8330 msgr2=0x7f90500f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 shutdown_connections 2024-02-16T19:17:07.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:07.884+0000 7f90554c9700 1 -- 172.21.15.163:0/2460195351 wait complete. 2024-02-16T19:17:07.887 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:17:08.382 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:07 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/2460195351' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:09.450 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:17:09.451 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:17:09.697 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:09.743 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: Active manager daemon smithi040.jzygzo restarted 2024-02-16T19:17:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: Activating manager daemon smithi040.jzygzo 2024-02-16T19:17:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: mgrmap e15: smithi040.jzygzo(active, starting, since 0.00321094s) 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: Manager daemon smithi040.jzygzo is now available 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:17:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:17:11.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.260+0000 7f43b7ea9700 1 -- 172.21.15.163:0/1840606417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 msgr2=0x7f43b00fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.260+0000 7f43b7ea9700 1 --2- 172.21.15.163:0/1840606417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 0x7f43b00fcfe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009c60 tx=0x7f43a8004d10 comp rx=0 tx=0).stop 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.260+0000 7f43b7ea9700 1 -- 172.21.15.163:0/1840606417 shutdown_connections 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.260+0000 7f43b7ea9700 1 --2- 172.21.15.163:0/1840606417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 0x7f43b00fcfe0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.260+0000 7f43b7ea9700 1 -- 172.21.15.163:0/1840606417 >> 172.21.15.163:0/1840606417 conn(0x7f43b00f8270 msgr2=0x7f43b00fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.260+0000 7f43b7ea9700 1 -- 172.21.15.163:0/1840606417 shutdown_connections 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.260+0000 7f43b7ea9700 1 -- 172.21.15.163:0/1840606417 wait complete. 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.261+0000 7f43b7ea9700 1 Processor -- start 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.261+0000 7f43b7ea9700 1 -- start start 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.261+0000 7f43b7ea9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 0x7f43b010cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.261+0000 7f43b7ea9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b010d430 con 0x7f43b00fcbd0 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.261+0000 7f43b5c45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 0x7f43b010cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.261+0000 7f43b5c45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 0x7f43b010cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:35406/0 (socket says 172.21.15.163:35406) 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.261+0000 7f43b5c45700 1 -- 172.21.15.163:0/188466455 learned_addr learned my addr 172.21.15.163:0/188466455 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:11.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.261+0000 7f43b5c45700 1 -- 172.21.15.163:0/188466455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8009940 con 0x7f43b00fcbd0 2024-02-16T19:17:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.262+0000 7f43b5c45700 1 --2- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 0x7f43b010cef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f43a8015040 tx=0x7f43a8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.262+0000 7f43ae7fc700 1 -- 172.21.15.163:0/188466455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43a8007c80 con 0x7f43b00fcbd0 2024-02-16T19:17:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.262+0000 7f43ae7fc700 1 -- 172.21.15.163:0/188466455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43a801d910 con 0x7f43b00fcbd0 2024-02-16T19:17:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.262+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b010d630 con 0x7f43b00fcbd0 2024-02-16T19:17:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.262+0000 7f43ae7fc700 1 -- 172.21.15.163:0/188466455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43a80129f0 con 0x7f43b00fcbd0 2024-02-16T19:17:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.262+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b010e2d0 con 0x7f43b00fcbd0 2024-02-16T19:17:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.263+0000 7f43ae7fc700 1 -- 172.21.15.163:0/188466455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7f43a8019040 con 0x7f43b00fcbd0 2024-02-16T19:17:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.263+0000 7f43ae7fc700 1 --2- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4398036700 0x7f4398038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:11.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.263+0000 7f43ae7fc700 1 -- 172.21.15.163:0/188466455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f43a804a290 con 0x7f43b00fcbd0 2024-02-16T19:17:11.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.263+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b005f5c0 con 0x7f43b00fcbd0 2024-02-16T19:17:11.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.263+0000 7f43b5444700 1 --2- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4398036700 0x7f4398038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:11.266 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.264+0000 7f43b5444700 1 --2- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4398036700 0x7f4398038bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009990 tx=0x7f43a4006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.272+0000 7f43ae7fc700 1 -- 172.21.15.163:0/188466455 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f43a8017080 con 0x7f43b00fcbd0 2024-02-16T19:17:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:17:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:17:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/trash_purge_schedule"}]: dispatch 2024-02-16T19:17:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:11 smithi040 ceph-mon[29201]: mgrmap e16: smithi040.jzygzo(active, since 1.00552s) 2024-02-16T19:17:11.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.548+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f43b010edc0 con 0x7f43b00fcbd0 2024-02-16T19:17:11.549 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.548+0000 7f43ae7fc700 1 -- 172.21.15.163:0/188466455 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f43a8021020 con 0x7f43b00fcbd0 2024-02-16T19:17:11.550 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:11.550 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4398036700 msgr2=0x7f4398038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 --2- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4398036700 0x7f4398038bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009990 tx=0x7f43a4006e40 comp rx=0 tx=0).stop 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 msgr2=0x7f43b010cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 --2- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 0x7f43b010cef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f43a8015040 tx=0x7f43a8004d10 comp rx=0 tx=0).stop 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 shutdown_connections 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 --2- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4398036700 0x7f4398038bb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 --2- 172.21.15.163:0/188466455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b00fcbd0 0x7f43b010cef0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 >> 172.21.15.163:0/188466455 conn(0x7f43b00f8270 msgr2=0x7f43b00f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 shutdown_connections 2024-02-16T19:17:11.552 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:11.550+0000 7f43b7ea9700 1 -- 172.21.15.163:0/188466455 wait complete. 2024-02-16T19:17:11.553 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:17:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:12 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/188466455' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:13.194 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:17:13.195 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:17:13.512 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:13.640 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:13 smithi040 ceph-mon[29201]: [16/Feb/2024:19:17:12] ENGINE Bus STARTING 2024-02-16T19:17:13.641 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:13 smithi040 ceph-mon[29201]: mgrmap e17: smithi040.jzygzo(active, since 2s) 2024-02-16T19:17:13.641 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:13 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:13.907 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-16T19:17:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:14 smithi040 ceph-mon[29201]: [16/Feb/2024:19:17:12] ENGINE Serving on https://172.21.15.40:7150 2024-02-16T19:17:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:14 smithi040 ceph-mon[29201]: [16/Feb/2024:19:17:12] ENGINE Bus STARTED 2024-02-16T19:17:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:17:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:16.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:16.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:16.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:16.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:17:16.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:16.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:17:16.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:17:16.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:15 smithi040 ceph-mon[29201]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:17:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.236+0000 7fd9d5573700 1 -- 172.21.15.163:0/3507774412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 msgr2=0x7fd9d00fdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.236+0000 7fd9d5573700 1 --2- 172.21.15.163:0/3507774412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 0x7fd9d00fdfa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40048c0 tx=0x7fd9c4009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.236+0000 7fd9d5573700 1 -- 172.21.15.163:0/3507774412 shutdown_connections 2024-02-16T19:17:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.236+0000 7fd9d5573700 1 --2- 172.21.15.163:0/3507774412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 0x7fd9d00fdfa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.236+0000 7fd9d5573700 1 -- 172.21.15.163:0/3507774412 >> 172.21.15.163:0/3507774412 conn(0x7fd9d00f7640 msgr2=0x7fd9d00f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.236+0000 7fd9d5573700 1 -- 172.21.15.163:0/3507774412 shutdown_connections 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.236+0000 7fd9d5573700 1 -- 172.21.15.163:0/3507774412 wait complete. 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.237+0000 7fd9d5573700 1 Processor -- start 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.237+0000 7fd9d5573700 1 -- start start 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.237+0000 7fd9d5573700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 0x7fd9d0112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.237+0000 7fd9d5573700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d0113530 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.237+0000 7fd9ceffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 0x7fd9d0112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.237+0000 7fd9ceffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 0x7fd9d0112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:58302/0 (socket says 172.21.15.163:58302) 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.237+0000 7fd9ceffd700 1 -- 172.21.15.163:0/1598440681 learned_addr learned my addr 172.21.15.163:0/1598440681 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.237+0000 7fd9ceffd700 1 -- 172.21.15.163:0/1598440681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4005040 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.238+0000 7fd9ceffd700 1 --2- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 0x7fd9d0112ff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009300 tx=0x7fd9c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.238+0000 7fd9b77fe700 1 -- 172.21.15.163:0/1598440681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9c4004e60 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.238+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d0113730 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.238+0000 7fd9b77fe700 1 -- 172.21.15.163:0/1598440681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9c4013420 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.238+0000 7fd9b77fe700 1 -- 172.21.15.163:0/1598440681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9c401c5b0 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.238+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d0113ba0 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.239+0000 7fd9b77fe700 1 -- 172.21.15.163:0/1598440681 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fd9c401c7f0 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.239+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d005f5c0 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.239+0000 7fd9b77fe700 1 --2- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd9b8036960 0x7fd9b8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.239+0000 7fd9b77fe700 1 -- 172.21.15.163:0/1598440681 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd9c404aca0 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.239+0000 7fd9ce7fc700 1 --2- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd9b8036960 0x7fd9b8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.239+0000 7fd9ce7fc700 1 --2- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd9b8036960 0x7fd9b8038e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009990 tx=0x7fd9c0006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:16.245 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.244+0000 7fd9b77fe700 1 -- 172.21.15.163:0/1598440681 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd9c4023300 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.522 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.521+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd9d0106f20 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.523 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.521+0000 7fd9b77fe700 1 -- 172.21.15.163:0/1598440681 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd9c404ae60 con 0x7fd9d00fdbd0 2024-02-16T19:17:16.523 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:16.523 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:17:16.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd9b8036960 msgr2=0x7fd9b8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:16.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 --2- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd9b8036960 0x7fd9b8038e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009990 tx=0x7fd9c0006e40 comp rx=0 tx=0).stop 2024-02-16T19:17:16.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 msgr2=0x7fd9d0112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:16.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 --2- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 0x7fd9d0112ff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009300 tx=0x7fd9c4009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:16.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 shutdown_connections 2024-02-16T19:17:16.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 --2- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd9b8036960 0x7fd9b8038e10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:16.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 --2- 172.21.15.163:0/1598440681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9d00fdbd0 0x7fd9d0112ff0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:16.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 >> 172.21.15.163:0/1598440681 conn(0x7fd9d00f7640 msgr2=0x7fd9d01013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:16.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 shutdown_connections 2024-02-16T19:17:16.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:16.524+0000 7fd9d5573700 1 -- 172.21.15.163:0/1598440681 wait complete. 2024-02-16T19:17:16.526 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:17:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: Deploying daemon crash.smithi163 on smithi163 2024-02-16T19:17:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:17 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/1598440681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:19.072 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:17:19.072 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:17:19.462 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:20.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:19 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:20.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:19 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:20.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:19 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:20.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:19 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:20.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:19 smithi040 ceph-mon[29201]: Deploying daemon node-exporter.smithi163 on smithi163 2024-02-16T19:17:21.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.020+0000 7eff566b7700 1 -- 172.21.15.163:0/548777966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 msgr2=0x7eff50102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:21.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.020+0000 7eff566b7700 1 --2- 172.21.15.163:0/548777966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 0x7eff50102fa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff400048c0 tx=0x7eff40009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:21.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.020+0000 7eff566b7700 1 -- 172.21.15.163:0/548777966 shutdown_connections 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.020+0000 7eff566b7700 1 --2- 172.21.15.163:0/548777966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 0x7eff50102fa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.020+0000 7eff566b7700 1 -- 172.21.15.163:0/548777966 >> 172.21.15.163:0/548777966 conn(0x7eff500f8760 msgr2=0x7eff500fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.020+0000 7eff566b7700 1 -- 172.21.15.163:0/548777966 shutdown_connections 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.020+0000 7eff566b7700 1 -- 172.21.15.163:0/548777966 wait complete. 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.021+0000 7eff566b7700 1 Processor -- start 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.021+0000 7eff566b7700 1 -- start start 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.021+0000 7eff566b7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 0x7eff50112f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.021+0000 7eff566b7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff50113470 con 0x7eff50102bd0 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.022+0000 7eff4ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 0x7eff50112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.022+0000 7eff4ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 0x7eff50112f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:58322/0 (socket says 172.21.15.163:58322) 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.022+0000 7eff4ffff700 1 -- 172.21.15.163:0/551713822 learned_addr learned my addr 172.21.15.163:0/551713822 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.022+0000 7eff4ffff700 1 -- 172.21.15.163:0/551713822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff40005040 con 0x7eff50102bd0 2024-02-16T19:17:21.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.022+0000 7eff4ffff700 1 --2- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 0x7eff50112f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff40009300 tx=0x7eff40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.022+0000 7eff4cff9700 1 -- 172.21.15.163:0/551713822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff40004e60 con 0x7eff50102bd0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.023+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff50113670 con 0x7eff50102bd0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.023+0000 7eff4cff9700 1 -- 172.21.15.163:0/551713822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff40013420 con 0x7eff50102bd0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.023+0000 7eff4cff9700 1 -- 172.21.15.163:0/551713822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff4001c5b0 con 0x7eff50102bd0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.023+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff50113b60 con 0x7eff50102bd0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.024+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5005f5c0 con 0x7eff50102bd0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.024+0000 7eff4cff9700 1 -- 172.21.15.163:0/551713822 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7eff4001c7f0 con 0x7eff50102bd0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.024+0000 7eff4cff9700 1 --2- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7eff38036960 0x7eff38038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.024+0000 7eff4cff9700 1 -- 172.21.15.163:0/551713822 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7eff40049c90 con 0x7eff50102bd0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.024+0000 7eff4f7fe700 1 --2- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7eff38036960 0x7eff38038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.024+0000 7eff4f7fe700 1 --2- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7eff38036960 0x7eff38038e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff44009990 tx=0x7eff44006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.029+0000 7eff4cff9700 1 -- 172.21.15.163:0/551713822 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7eff40023300 con 0x7eff50102bd0 2024-02-16T19:17:21.311 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.310+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7eff5010e6e0 con 0x7eff50102bd0 2024-02-16T19:17:21.311 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.310+0000 7eff4cff9700 1 -- 172.21.15.163:0/551713822 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7eff40011b60 con 0x7eff50102bd0 2024-02-16T19:17:21.312 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:21.312 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:17:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7eff38036960 msgr2=0x7eff38038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 --2- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7eff38036960 0x7eff38038e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff44009990 tx=0x7eff44006e40 comp rx=0 tx=0).stop 2024-02-16T19:17:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 msgr2=0x7eff50112f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 --2- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 0x7eff50112f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff40009300 tx=0x7eff40009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:21.314 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 shutdown_connections 2024-02-16T19:17:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 --2- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7eff38036960 0x7eff38038e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 --2- 172.21.15.163:0/551713822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff50102bd0 0x7eff50112f30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 >> 172.21.15.163:0/551713822 conn(0x7eff500f8760 msgr2=0x7eff500f93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 shutdown_connections 2024-02-16T19:17:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:21.313+0000 7eff566b7700 1 -- 172.21.15.163:0/551713822 wait complete. 2024-02-16T19:17:21.315 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:17:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:21 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:22 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/551713822' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:23.489 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:17:23.490 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:17:23.867 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:25.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:17:26.097 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.094+0000 7fa4b8e91700 1 -- 172.21.15.163:0/1647464995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 msgr2=0x7fa4b40fdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.094+0000 7fa4b8e91700 1 --2- 172.21.15.163:0/1647464995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 0x7fa4b40fdf80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40048c0 tx=0x7fa4a4009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.094+0000 7fa4b8e91700 1 -- 172.21.15.163:0/1647464995 shutdown_connections 2024-02-16T19:17:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.094+0000 7fa4b8e91700 1 --2- 172.21.15.163:0/1647464995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 0x7fa4b40fdf80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.094+0000 7fa4b8e91700 1 -- 172.21.15.163:0/1647464995 >> 172.21.15.163:0/1647464995 conn(0x7fa4b40f7660 msgr2=0x7fa4b40f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.094+0000 7fa4b8e91700 1 -- 172.21.15.163:0/1647464995 shutdown_connections 2024-02-16T19:17:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.094+0000 7fa4b8e91700 1 -- 172.21.15.163:0/1647464995 wait complete. 2024-02-16T19:17:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.095+0000 7fa4b8e91700 1 Processor -- start 2024-02-16T19:17:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.095+0000 7fa4b8e91700 1 -- start start 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.095+0000 7fa4b8e91700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 0x7fa4b4113020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.095+0000 7fa4b8e91700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b4113560 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.095+0000 7fa4b259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 0x7fa4b4113020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.095+0000 7fa4b259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 0x7fa4b4113020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:47692/0 (socket says 172.21.15.163:47692) 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.095+0000 7fa4b259c700 1 -- 172.21.15.163:0/182960975 learned_addr learned my addr 172.21.15.163:0/182960975 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.096+0000 7fa4b259c700 1 -- 172.21.15.163:0/182960975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4005040 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.096+0000 7fa4b259c700 1 --2- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 0x7fa4b4113020 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009300 tx=0x7fa4a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.096+0000 7fa4ab7fe700 1 -- 172.21.15.163:0/182960975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4a4004e60 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.096+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b4113760 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.096+0000 7fa4ab7fe700 1 -- 172.21.15.163:0/182960975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4a4013420 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.096+0000 7fa4ab7fe700 1 -- 172.21.15.163:0/182960975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4a401c5b0 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.096+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b4113bd0 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.097+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b405f5c0 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.097+0000 7fa4ab7fe700 1 -- 172.21.15.163:0/182960975 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fa4a4009600 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.097+0000 7fa4ab7fe700 1 --2- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa4a0036960 0x7fa4a0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:26.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.097+0000 7fa4ab7fe700 1 -- 172.21.15.163:0/182960975 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa4a4049c40 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.097+0000 7fa4a9d9b700 1 --2- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa4a0036960 0x7fa4a0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:26.101 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.098+0000 7fa4a9d9b700 1 --2- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa4a0036960 0x7fa4a0038e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009990 tx=0x7fa49c006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:26.103 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.102+0000 7fa4ab7fe700 1 -- 172.21.15.163:0/182960975 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa4a4025400 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.371 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.370+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa4b4106f00 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.372 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.370+0000 7fa4ab7fe700 1 -- 172.21.15.163:0/182960975 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa4a4011370 con 0x7fa4b40fdbb0 2024-02-16T19:17:26.372 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:26.372 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:17:26.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa4a0036960 msgr2=0x7fa4a0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:26.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 --2- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa4a0036960 0x7fa4a0038e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009990 tx=0x7fa49c006e40 comp rx=0 tx=0).stop 2024-02-16T19:17:26.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 msgr2=0x7fa4b4113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:26.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 --2- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 0x7fa4b4113020 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009300 tx=0x7fa4a4009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:26.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 shutdown_connections 2024-02-16T19:17:26.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 --2- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa4a0036960 0x7fa4a0038e10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:26.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 --2- 172.21.15.163:0/182960975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b40fdbb0 0x7fa4b4113020 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:26.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 >> 172.21.15.163:0/182960975 conn(0x7fa4b40f7660 msgr2=0x7fa4b41013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:26.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.372+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 shutdown_connections 2024-02-16T19:17:26.375 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:26.373+0000 7fa4b8e91700 1 -- 172.21.15.163:0/182960975 wait complete. 2024-02-16T19:17:26.375 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:17:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:26.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:17:26.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-16T19:17:26.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:17:26.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:26.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:26 smithi040 ceph-mon[29201]: Deploying daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:17:27.424 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:27 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/182960975' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:28.749 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:17:28.749 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:17:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:17:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:29 smithi040 ceph-mon[29201]: Deploying daemon mon.smithi163 on smithi163 2024-02-16T19:17:30.577 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:31.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:31 smithi040 ceph-mon[29201]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:31.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:31 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:32.944 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.733+0000 7f0a028c2700 1 -- 172.21.15.163:0/287080467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 msgr2=0x7f09fc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:32.944 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.733+0000 7f0a028c2700 1 --2- 172.21.15.163:0/287080467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 0x7f09fc101f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f09f00048c0 tx=0x7f09f0009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:32.944 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.733+0000 7f0a028c2700 1 -- 172.21.15.163:0/287080467 shutdown_connections 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.733+0000 7f0a028c2700 1 --2- 172.21.15.163:0/287080467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 0x7f09fc101f30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.733+0000 7f0a028c2700 1 -- 172.21.15.163:0/287080467 >> 172.21.15.163:0/287080467 conn(0x7f09fc072850 msgr2=0x7f09fc072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.733+0000 7f0a028c2700 1 -- 172.21.15.163:0/287080467 shutdown_connections 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.734+0000 7f0a028c2700 1 -- 172.21.15.163:0/287080467 wait complete. 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.734+0000 7f0a028c2700 1 Processor -- start 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.734+0000 7f0a028c2700 1 -- start start 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.734+0000 7f0a028c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 0x7f09fc10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.734+0000 7f0a028c2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc10e4b0 con 0x7f09fc101b60 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 0x7f09fc10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 0x7f09fc10df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.163:45448/0 (socket says 172.21.15.163:45448) 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f09fbfff700 1 -- 172.21.15.163:0/2544684584 learned_addr learned my addr 172.21.15.163:0/2544684584 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f09fbfff700 1 -- 172.21.15.163:0/2544684584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f0005040 con 0x7f09fc101b60 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f09fbfff700 1 --2- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 0x7f09fc10df70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009300 tx=0x7f09f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f09f8ff9700 1 -- 172.21.15.163:0/2544684584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09f0004e60 con 0x7f09fc101b60 2024-02-16T19:17:32.945 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f09f8ff9700 1 -- 172.21.15.163:0/2544684584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09f0013420 con 0x7f09fc101b60 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc10e6b0 con 0x7f09fc101b60 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f09f8ff9700 1 -- 172.21.15.163:0/2544684584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09f001c590 con 0x7f09fc101b60 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.735+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc10eb20 con 0x7f09fc101b60 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.736+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc0ffd70 con 0x7f09fc101b60 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.736+0000 7f09f8ff9700 1 -- 172.21.15.163:0/2544684584 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f09f001c850 con 0x7f09fc101b60 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.737+0000 7f09f8ff9700 1 --2- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09e4036910 0x7f09e4038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.737+0000 7f09f8ff9700 1 -- 172.21.15.163:0/2544684584 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f09f0049dd0 con 0x7f09fc101b60 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.737+0000 7f09fb7fe700 1 --2- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09e4036910 0x7f09e4038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.737+0000 7f09fb7fe700 1 --2- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09e4036910 0x7f09e4038dc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009990 tx=0x7f09ec006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:32.946 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:32.741+0000 7f09f8ff9700 1 -- 172.21.15.163:0/2544684584 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f09f004d030 con 0x7f09fc101b60 2024-02-16T19:17:33.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.012+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f09fc05f5c0 con 0x7f09fc101b60 2024-02-16T19:17:33.013 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.012+0000 7f09f8ff9700 1 -- 172.21.15.163:0/2544684584 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f09f0011dd0 con 0x7f09fc101b60 2024-02-16T19:17:33.014 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:33.014 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:14:52.798966Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.014+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09e4036910 msgr2=0x7f09e4038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.014+0000 7f0a028c2700 1 --2- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09e4036910 0x7f09e4038dc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009990 tx=0x7f09ec006e40 comp rx=0 tx=0).stop 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.014+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 msgr2=0x7f09fc10df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.014+0000 7f0a028c2700 1 --2- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 0x7f09fc10df70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009300 tx=0x7f09f0009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.014+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 shutdown_connections 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.014+0000 7f0a028c2700 1 --2- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09e4036910 0x7f09e4038dc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.014+0000 7f0a028c2700 1 --2- 172.21.15.163:0/2544684584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09fc101b60 0x7f09fc10df70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.014+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 >> 172.21.15.163:0/2544684584 conn(0x7f09fc072850 msgr2=0x7f09fc103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.015+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 shutdown_connections 2024-02-16T19:17:33.016 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:33.015+0000 7f0a028c2700 1 -- 172.21.15.163:0/2544684584 wait complete. 2024-02-16T19:17:33.017 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-02-16T19:17:33.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:33 smithi163 ceph-mon[36452]: mon.smithi163@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-02-16T19:17:33.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:33 smithi040 ceph-mon[29201]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:33.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:33 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/2544684584' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:33.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:33 smithi163 bash[36393]: 7679bf43e6a52cb951533cedfd5d7483d87dbee1cabd9b43ee7b28aca46cddf6 2024-02-16T19:17:33.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:33 smithi163 systemd[1]: Started Ceph mon.smithi163 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:17:34.791 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-16T19:17:34.791 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-16T19:17:35.046 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi163/config 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: mon.smithi040 calling monitor election 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/1984248138' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: mon.smithi163 calling monitor election 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: mon.smithi040 is new leader, mons smithi040,smithi163 in quorum (ranks 0,1) 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: monmap e2: 2 mons at {smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],smithi163=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: fsmap 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: mgrmap e17: smithi040.jzygzo(active, since 28s) 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/1984248138' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: overall HEALTH_OK 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/1984248138' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/1984248138' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:38.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: mon.smithi040 calling monitor election 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/1984248138' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: mon.smithi163 calling monitor election 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: mon.smithi040 is new leader, mons smithi040,smithi163 in quorum (ranks 0,1) 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: monmap e2: 2 mons at {smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],smithi163=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: fsmap 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: osdmap e5: 0 total, 0 up, 0 in 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: mgrmap e17: smithi040.jzygzo(active, since 28s) 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/1984248138' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: overall HEALTH_OK 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/1984248138' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/1984248138' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:17:38.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:38.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:39 smithi163 ceph-mon[36452]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:39 smithi163 ceph-mon[36452]: mgrmap e18: smithi040.jzygzo(active, since 28s), standbys: smithi163.wjfclz 2024-02-16T19:17:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi163.wjfclz", "id": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:17:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:17:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:39.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.524+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3032838720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b4003580 msgr2=0x7f05b4005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:39.525 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.524+0000 7f05ccb2f700 1 --2- 172.21.15.163:0/3032838720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b4003580 0x7f05b4005a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f05c80fcc10 tx=0x7f05b800b040 comp rx=0 tx=0).stop 2024-02-16T19:17:39.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.524+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3032838720 shutdown_connections 2024-02-16T19:17:39.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.524+0000 7f05ccb2f700 1 --2- 172.21.15.163:0/3032838720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b4003580 0x7f05b4005a00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:39.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.524+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3032838720 >> 172.21.15.163:0/3032838720 conn(0x7f05c80f8860 msgr2=0x7f05c80fac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:39.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.524+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3032838720 shutdown_connections 2024-02-16T19:17:39.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.524+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3032838720 wait complete. 2024-02-16T19:17:39.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05ccb2f700 1 Processor -- start 2024-02-16T19:17:39.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05ccb2f700 1 -- start start 2024-02-16T19:17:39.527 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05ccb2f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 0x7f05c8113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05ccb2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c8113670 0x7f05c810d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05ccb2f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8113c80 con 0x7f05c8113670 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05ccb2f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8113df0 con 0x7f05b4003580 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05c659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 0x7f05c8113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05c659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 0x7f05c8113130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36476/0 (socket says 172.21.15.163:36476) 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05c659c700 1 -- 172.21.15.163:0/3988075425 learned_addr learned my addr 172.21.15.163:0/3988075425 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05bdd9b700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c8113670 0x7f05c810d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05c659c700 1 -- 172.21.15.163:0/3988075425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 msgr2=0x7f05c8113130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05c659c700 1 -- 172.21.15.163:0/3988075425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 msgr2=0x7f05c8113130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05c659c700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 0x7f05c8113130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.525+0000 7f05c659c700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 0x7f05c8113130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05bdd9b700 1 -- 172.21.15.163:0/3988075425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 msgr2=0x7f05c8113130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05bdd9b700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 0x7f05c8113130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:39.528 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05bdd9b700 1 -- 172.21.15.163:0/3988075425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8005040 con 0x7f05c8113670 2024-02-16T19:17:39.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05bdd9b700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c8113670 0x7f05c810d010 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b790 tx=0x7f05b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:39.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05bf7fe700 1 -- 172.21.15.163:0/3988075425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b0011810 con 0x7f05c8113670 2024-02-16T19:17:39.529 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c810d670 con 0x7f05c8113670 2024-02-16T19:17:39.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c810dbb0 con 0x7f05c8113670 2024-02-16T19:17:39.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05bf7fe700 1 -- 172.21.15.163:0/3988075425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05b0011e40 con 0x7f05c8113670 2024-02-16T19:17:39.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.526+0000 7f05bf7fe700 1 -- 172.21.15.163:0/3988075425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b000f2e0 con 0x7f05c8113670 2024-02-16T19:17:39.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.528+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c805f5c0 con 0x7f05c8113670 2024-02-16T19:17:39.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.528+0000 7f05bf7fe700 1 -- 172.21.15.163:0/3988075425 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f05b0011970 con 0x7f05c8113670 2024-02-16T19:17:39.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.528+0000 7f05bf7fe700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f05a8069370 0x7f05a806b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:39.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.529+0000 7f05bf7fe700 1 -- 172.21.15.163:0/3988075425 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f05b0086300 con 0x7f05c8113670 2024-02-16T19:17:39.530 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.529+0000 7f05c659c700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f05a8069370 0x7f05a806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:39.531 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.529+0000 7f05c659c700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f05a8069370 0x7f05a806b820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f05b80045a0 tx=0x7f05b8018040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:39.534 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.533+0000 7f05bf7fe700 1 -- 172.21.15.163:0/3988075425 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f05b0053690 con 0x7f05c8113670 2024-02-16T19:17:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:39 smithi040 ceph-mon[29201]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:39 smithi040 ceph-mon[29201]: mgrmap e18: smithi040.jzygzo(active, since 28s), standbys: smithi163.wjfclz 2024-02-16T19:17:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi163.wjfclz", "id": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:17:39.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:39.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:39.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:17:39.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:17:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.803+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f05c810e750 con 0x7f05c8113670 2024-02-16T19:17:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.804+0000 7f05bf7fe700 1 -- 172.21.15.163:0/3988075425 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7f05b0015090 con 0x7f05c8113670 2024-02-16T19:17:39.806 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-02-16T19:17:39.806 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":2,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","modified":"2024-02-16T19:17:33.103073Z","created":"2024-02-16T19:14:52.798966Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi163","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-16T19:17:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f05a8069370 msgr2=0x7f05a806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f05a8069370 0x7f05a806b820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f05b80045a0 tx=0x7f05b8018040 comp rx=0 tx=0).stop 2024-02-16T19:17:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c8113670 msgr2=0x7f05c810d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c8113670 0x7f05c810d010 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b790 tx=0x7f05b00091a0 comp rx=0 tx=0).stop 2024-02-16T19:17:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 shutdown_connections 2024-02-16T19:17:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f05a8069370 0x7f05a806b820 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05b4003580 0x7f05c8113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 --2- 172.21.15.163:0/3988075425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c8113670 0x7f05c810d010 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 >> 172.21.15.163:0/3988075425 conn(0x7f05c80f8860 msgr2=0x7f05c80fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 shutdown_connections 2024-02-16T19:17:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:17:39.807+0000 7f05ccb2f700 1 -- 172.21.15.163:0/3988075425 wait complete. 2024-02-16T19:17:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 2 2024-02-16T19:17:40.460 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:40 smithi163 ceph-mon[36452]: from='client.? 172.21.15.163:0/3988075425' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:40.460 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:17:40.460 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:40.484 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-16T19:17:40.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-16T19:17:40.514 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:40 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/3988075425' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-16T19:17:40.514 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:17:40.514 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:40.764 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:17:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:17:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:41 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:17:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:41 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:17:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:17:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: Reconfiguring mon.smithi040 (unknown last config time)... 2024-02-16T19:17:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:17:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:17:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:42 smithi163 ceph-mon[36452]: Reconfiguring daemon mon.smithi040 on smithi040 2024-02-16T19:17:42.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:17:42.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:17:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:17:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: Reconfiguring mon.smithi040 (unknown last config time)... 2024-02-16T19:17:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:17:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:17:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:42 smithi040 ceph-mon[29201]: Reconfiguring daemon mon.smithi040 on smithi040 2024-02-16T19:17:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.964+0000 7f7c45ec7700 1 -- 172.21.15.40:0/96139461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 msgr2=0x7f7c400fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.964+0000 7f7c45ec7700 1 --2- 172.21.15.40:0/96139461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 0x7f7c400fdfb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c300048c0 tx=0x7f7c30009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.964+0000 7f7c45ec7700 1 -- 172.21.15.40:0/96139461 shutdown_connections 2024-02-16T19:17:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.964+0000 7f7c45ec7700 1 --2- 172.21.15.40:0/96139461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 0x7f7c400fdfb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.964+0000 7f7c45ec7700 1 -- 172.21.15.40:0/96139461 >> 172.21.15.40:0/96139461 conn(0x7f7c40072850 msgr2=0x7f7c40072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.964+0000 7f7c45ec7700 1 -- 172.21.15.40:0/96139461 shutdown_connections 2024-02-16T19:17:42.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.964+0000 7f7c45ec7700 1 -- 172.21.15.40:0/96139461 wait complete. 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.964+0000 7f7c45ec7700 1 Processor -- start 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c45ec7700 1 -- start start 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c45ec7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 0x7f7c401130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c45ec7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c40113630 0x7f7c4010cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c45ec7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c40113c40 con 0x7f7c400fdbe0 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c45ec7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4010d4d0 con 0x7f7c40113630 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c3f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 0x7f7c401130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c3f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 0x7f7c401130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44438/0 (socket says 172.21.15.40:44438) 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c3f7fe700 1 -- 172.21.15.40:0/3444808443 learned_addr learned my addr 172.21.15.40:0/3444808443 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c3f7fe700 1 -- 172.21.15.40:0/3444808443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c40113630 msgr2=0x7f7c4010cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c3f7fe700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c40113630 0x7f7c4010cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:42.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c3f7fe700 1 -- 172.21.15.40:0/3444808443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30005040 con 0x7f7c400fdbe0 2024-02-16T19:17:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c3f7fe700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 0x7f7c401130f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b870 tx=0x7f7c30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.965+0000 7f7c27fff700 1 -- 172.21.15.40:0/3444808443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3000e040 con 0x7f7c400fdbe0 2024-02-16T19:17:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.966+0000 7f7c27fff700 1 -- 172.21.15.40:0/3444808443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c30013420 con 0x7f7c400fdbe0 2024-02-16T19:17:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.966+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4010d750 con 0x7f7c400fdbe0 2024-02-16T19:17:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.966+0000 7f7c27fff700 1 -- 172.21.15.40:0/3444808443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3001c630 con 0x7f7c400fdbe0 2024-02-16T19:17:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.966+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4010dc90 con 0x7f7c400fdbe0 2024-02-16T19:17:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.966+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c401043b0 con 0x7f7c400fdbe0 2024-02-16T19:17:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.967+0000 7f7c27fff700 1 -- 172.21.15.40:0/3444808443 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f7c30013a20 con 0x7f7c400fdbe0 2024-02-16T19:17:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.967+0000 7f7c27fff700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c28069370 0x7f7c2806b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.968+0000 7f7c27fff700 1 -- 172.21.15.40:0/3444808443 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f7c30087330 con 0x7f7c400fdbe0 2024-02-16T19:17:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.968+0000 7f7c3effd700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c28069370 0x7f7c2806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:42.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.968+0000 7f7c3effd700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c28069370 0x7f7c2806b820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009930 tx=0x7f7c34006de0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:42.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:42.972+0000 7f7c27fff700 1 -- 172.21.15.40:0/3444808443 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7c30058270 con 0x7f7c400fdbe0 2024-02-16T19:17:43.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.166+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7c4005f5c0 con 0x7f7c400fdbe0 2024-02-16T19:17:43.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.166+0000 7f7c27fff700 1 -- 172.21.15.40:0/3444808443 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f7c4005f5c0 con 0x7f7c400fdbe0 2024-02-16T19:17:43.168 INFO:teuthology.orchestra.run.smithi040.stdout:# minimal ceph.conf for 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:17:43.168 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-02-16T19:17:43.168 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:17:43.168 INFO:teuthology.orchestra.run.smithi040.stdout: mon_host = [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] 2024-02-16T19:17:43.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c28069370 msgr2=0x7f7c2806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c28069370 0x7f7c2806b820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009930 tx=0x7f7c34006de0 comp rx=0 tx=0).stop 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 msgr2=0x7f7c401130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 0x7f7c401130f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b870 tx=0x7f7c30009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 shutdown_connections 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c28069370 0x7f7c2806b820 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c400fdbe0 0x7f7c401130f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 --2- 172.21.15.40:0/3444808443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c40113630 0x7f7c4010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 >> 172.21.15.40:0/3444808443 conn(0x7f7c40072850 msgr2=0x7f7c40101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 shutdown_connections 2024-02-16T19:17:43.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:43.168+0000 7f7c45ec7700 1 -- 172.21.15.40:0/3444808443 wait complete. 2024-02-16T19:17:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:43 smithi163 ceph-mon[36452]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:43 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3444808443' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:43.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:43 smithi040 ceph-mon[29201]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:43.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:43 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3444808443' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:43.989 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-16T19:17:43.990 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:17:43.990 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.conf 2024-02-16T19:17:44.059 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:17:44.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:17:44.127 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:17:44.127 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.conf 2024-02-16T19:17:44.162 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:17:44.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:17:44.234 INFO:tasks.cephadm:Deploying OSDs... 2024-02-16T19:17:44.234 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:17:44.234 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-02-16T19:17:44.249 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-16T19:17:44.249 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-02-16 19:17:01.635904528 +0000 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-02-16 19:11:03.023355630 +0000 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-02-16 19:11:03.023355630 +0000 2024-02-16T19:17:44.304 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-02-16 19:11:03.023355630 +0000 2024-02-16T19:17:44.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-16T19:17:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-02-16T19:17:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-02-16T19:17:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000244648 s, 2.1 MB/s 2024-02-16T19:17:44.370 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-16T19:17:44.425 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-02-16 19:17:01.636904515 +0000 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-02-16 19:11:03.375350852 +0000 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-02-16 19:11:03.375350852 +0000 2024-02-16T19:17:44.480 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-02-16 19:11:03.375350852 +0000 2024-02-16T19:17:44.481 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-16T19:17:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:44 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:44 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:44 smithi163 ceph-mon[36452]: Reconfiguring mgr.smithi040.jzygzo (unknown last config time)... 2024-02-16T19:17:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:44 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.jzygzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:17:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:44 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:17:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:44 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:44 smithi163 ceph-mon[36452]: Reconfiguring daemon mgr.smithi040.jzygzo on smithi040 2024-02-16T19:17:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:44 smithi163 ceph-mon[36452]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:44.543 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:44 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:44.543 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:44 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:44.543 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:44 smithi040 ceph-mon[29201]: Reconfiguring mgr.smithi040.jzygzo (unknown last config time)... 2024-02-16T19:17:44.543 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:44 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.jzygzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:17:44.543 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:44 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:17:44.543 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:44 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:44.543 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:44 smithi040 ceph-mon[29201]: Reconfiguring daemon mgr.smithi040.jzygzo on smithi040 2024-02-16T19:17:44.543 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:44 smithi040 ceph-mon[29201]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-02-16T19:17:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-02-16T19:17:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000186714 s, 2.7 MB/s 2024-02-16T19:17:44.547 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-16T19:17:44.602 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-02-16T19:17:44.657 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-16T19:17:44.657 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:17:44.657 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-16T19:17:44.657 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:17:44.657 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:17:44.657 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-02-16 19:17:01.636904515 +0000 2024-02-16T19:17:44.657 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-02-16 19:11:03.728346059 +0000 2024-02-16T19:17:44.658 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-02-16 19:11:03.728346059 +0000 2024-02-16T19:17:44.658 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-02-16 19:11:03.728346059 +0000 2024-02-16T19:17:44.658 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-16T19:17:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-02-16T19:17:44.722 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-02-16T19:17:44.722 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000236824 s, 2.2 MB/s 2024-02-16T19:17:44.723 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-16T19:17:44.778 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-02-16 19:17:01.636904515 +0000 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-02-16 19:11:04.061341539 +0000 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-02-16 19:11:04.061341539 +0000 2024-02-16T19:17:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-02-16 19:11:04.061341539 +0000 2024-02-16T19:17:44.835 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-16T19:17:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-02-16T19:17:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-02-16T19:17:44.900 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000204554 s, 2.5 MB/s 2024-02-16T19:17:44.901 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-16T19:17:44.957 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:17:44.957 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-02-16T19:17:44.976 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-16T19:17:44.976 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-02-16T19:17:45.035 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-16T19:17:45.035 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:17:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-16T19:17:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:17:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:17:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-02-16 19:17:31.897185834 +0000 2024-02-16T19:17:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-02-16 19:11:00.240828676 +0000 2024-02-16T19:17:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-02-16 19:11:00.240828676 +0000 2024-02-16T19:17:45.036 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-02-16 19:11:00.240828676 +0000 2024-02-16T19:17:45.036 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-16T19:17:45.105 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-02-16T19:17:45.105 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-02-16T19:17:45.105 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000258189 s, 2.0 MB/s 2024-02-16T19:17:45.106 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-16T19:17:45.166 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-02-16T19:17:45.223 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-16T19:17:45.224 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:17:45.224 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-16T19:17:45.224 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:17:45.224 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:17:45.224 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-02-16 19:17:31.897185834 +0000 2024-02-16T19:17:45.224 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-02-16 19:11:00.594820783 +0000 2024-02-16T19:17:45.224 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-02-16 19:11:00.595820760 +0000 2024-02-16T19:17:45.224 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-02-16 19:11:00.594820783 +0000 2024-02-16T19:17:45.224 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-16T19:17:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-02-16T19:17:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-02-16T19:17:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000257089 s, 2.0 MB/s 2024-02-16T19:17:45.294 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-16T19:17:45.352 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-02-16T19:17:45.409 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-16T19:17:45.410 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:17:45.410 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-16T19:17:45.410 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:17:45.410 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:17:45.410 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-02-16 19:17:31.897185834 +0000 2024-02-16T19:17:45.410 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-02-16 19:11:00.918813558 +0000 2024-02-16T19:17:45.410 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-02-16 19:11:00.918813558 +0000 2024-02-16T19:17:45.410 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-02-16 19:11:00.918813558 +0000 2024-02-16T19:17:45.410 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-16T19:17:45.478 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-02-16T19:17:45.478 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-02-16T19:17:45.478 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.00022303 s, 2.3 MB/s 2024-02-16T19:17:45.479 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-16T19:17:45.537 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-02-16T19:17:45.593 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-16T19:17:45.593 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-16T19:17:45.593 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-16T19:17:45.593 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:17:45.594 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-02-16T19:17:45.594 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-02-16 19:17:31.898185812 +0000 2024-02-16T19:17:45.594 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-02-16 19:11:01.266805798 +0000 2024-02-16T19:17:45.594 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-02-16 19:11:01.266805798 +0000 2024-02-16T19:17:45.594 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-02-16 19:11:01.266805798 +0000 2024-02-16T19:17:45.594 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-16T19:17:45.661 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-02-16T19:17:45.661 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-02-16T19:17:45.661 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000218424 s, 2.3 MB/s 2024-02-16T19:17:45.663 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-16T19:17:45.719 INFO:tasks.cephadm:Deploying osd.0 on smithi040 with /dev/vg_nvme/lv_4... 2024-02-16T19:17:45.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-16T19:17:46.548 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:46 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:46.548 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:46 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:46.549 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:46 smithi040 ceph-mon[29201]: Reconfiguring crash.smithi040 (monmap changed)... 2024-02-16T19:17:46.549 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:46 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:17:46.549 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:46 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:46.549 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:46 smithi040 ceph-mon[29201]: Reconfiguring daemon crash.smithi040 on smithi040 2024-02-16T19:17:46.580 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:46 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:46.580 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:46 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:46.580 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:46 smithi163 ceph-mon[36452]: Reconfiguring crash.smithi040 (monmap changed)... 2024-02-16T19:17:46.580 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:46 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:17:46.580 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:46 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:46.580 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:46 smithi163 ceph-mon[36452]: Reconfiguring daemon crash.smithi040 on smithi040 2024-02-16T19:17:46.591 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:17:47.513 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:47 smithi040 ceph-mon[29201]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:47.513 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:47 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:47.513 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:47 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:47 smithi163 ceph-mon[36452]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:47.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:47 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:47.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:47 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:48 smithi040 ceph-mon[29201]: Reconfiguring alertmanager.smithi040 (dependencies changed)... 2024-02-16T19:17:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:48 smithi040 ceph-mon[29201]: Reconfiguring daemon alertmanager.smithi040 on smithi040 2024-02-16T19:17:48.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:48 smithi163 ceph-mon[36452]: Reconfiguring alertmanager.smithi040 (dependencies changed)... 2024-02-16T19:17:48.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:48 smithi163 ceph-mon[36452]: Reconfiguring daemon alertmanager.smithi040 on smithi040 2024-02-16T19:17:49.342 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:49 smithi040 ceph-mon[29201]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:49 smithi163 ceph-mon[36452]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-16T19:17:50.925 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-16T19:17:50.926 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-02-16T19:17:50.926 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-02-16T19:17:50.926 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125954 s, 833 MB/s 2024-02-16T19:17:50.926 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-02-16T19:17:51.595 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:51 smithi040 ceph-mon[29201]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:51 smithi163 ceph-mon[36452]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:52.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi040:vg_nvme/lv_4 2024-02-16T19:17:52.762 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:17:53.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:53 smithi040 ceph-mon[29201]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:53.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:53 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:53.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:53 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:53 smithi163 ceph-mon[36452]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:53 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:53 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:54.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:54 smithi040 ceph-mon[29201]: Reconfiguring grafana.smithi040 (dependencies changed)... 2024-02-16T19:17:54.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:54 smithi040 ceph-mon[29201]: Reconfiguring daemon grafana.smithi040 on smithi040 2024-02-16T19:17:54.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:54 smithi163 ceph-mon[36452]: Reconfiguring grafana.smithi040 (dependencies changed)... 2024-02-16T19:17:54.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:54 smithi163 ceph-mon[36452]: Reconfiguring daemon grafana.smithi040 on smithi040 2024-02-16T19:17:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.075+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/4222944906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fcef0 msgr2=0x7fc8d80fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.075+0000 7fc8dcdf7700 1 --2- 172.21.15.40:0/4222944906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fcef0 0x7fc8d80fd320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004940 tx=0x7fc8c8009e30 comp rx=0 tx=0).stop 2024-02-16T19:17:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/4222944906 shutdown_connections 2024-02-16T19:17:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 --2- 172.21.15.40:0/4222944906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d80fe0e0 0x7fc8d80fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 --2- 172.21.15.40:0/4222944906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fcef0 0x7fc8d80fd320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/4222944906 >> 172.21.15.40:0/4222944906 conn(0x7fc8d80f8620 msgr2=0x7fc8d80faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/4222944906 shutdown_connections 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/4222944906 wait complete. 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 Processor -- start 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 -- start start 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d80fcef0 0x7fc8d810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fe0e0 0x7fc8d810def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d810e580 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.076+0000 7fc8dcdf7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d810edc0 con 0x7fc8d80fcef0 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8d5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fe0e0 0x7fc8d810def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8d5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fe0e0 0x7fc8d810def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56020/0 (socket says 172.21.15.40:56020) 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8d5d9b700 1 -- 172.21.15.40:0/2765063474 learned_addr learned my addr 172.21.15.40:0/2765063474 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8d659c700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d80fcef0 0x7fc8d810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8d5d9b700 1 -- 172.21.15.40:0/2765063474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d80fcef0 msgr2=0x7fc8d810d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:17:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8d5d9b700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d80fcef0 0x7fc8d810d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:17:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8d5d9b700 1 -- 172.21.15.40:0/2765063474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c8005040 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8d5d9b700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fe0e0 0x7fc8d810def0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00b790 tx=0x7fc8cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8c6ffd700 1 -- 172.21.15.40:0/2765063474 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8cc00f6f0 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d810f0a0 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d810f5e0 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8c6ffd700 1 -- 172.21.15.40:0/2765063474 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8cc00fd20 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.077+0000 7fc8c6ffd700 1 -- 172.21.15.40:0/2765063474 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8cc0103a0 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.078+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b80052e0 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.079+0000 7fc8c6ffd700 1 -- 172.21.15.40:0/2765063474 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fc8cc011400 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.079+0000 7fc8c6ffd700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc8c006d620 0x7fc8c006fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:17:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.079+0000 7fc8c6ffd700 1 -- 172.21.15.40:0/2765063474 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc8cc085d20 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.079+0000 7fc8d659c700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc8c006d620 0x7fc8c006fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:17:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.080+0000 7fc8d659c700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc8c006d620 0x7fc8c006fad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004890 tx=0x7fc8c8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:17:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.084+0000 7fc8c6ffd700 1 -- 172.21.15.40:0/2765063474 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc8cc056c60 con 0x7fc8d80fe0e0 2024-02-16T19:17:55.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:17:55.290+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8b8000bf0 con 0x7fc8c006d620 2024-02-16T19:17:55.545 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:55 smithi040 ceph-mon[29201]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:55.545 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:55 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:17:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:55 smithi163 ceph-mon[36452]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:55 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:17:56.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:56 smithi040 ceph-mon[29201]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:17:56.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:56 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:17:56.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:56 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:17:56.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:56 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:56 smithi163 ceph-mon[36452]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:17:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:56 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:17:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:56 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:17:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:56 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:17:57.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:57 smithi040 ceph-mon[29201]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:57 smithi163 ceph-mon[36452]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:59.519 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:59 smithi040 ceph-mon[29201]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:59.519 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:59 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:59.519 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:17:59 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:59 smithi163 ceph-mon[36452]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:17:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:59 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:17:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:17:59 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:00 smithi040 ceph-mon[29201]: Reconfiguring prometheus.smithi040 (dependencies changed)... 2024-02-16T19:18:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:00 smithi040 ceph-mon[29201]: Reconfiguring daemon prometheus.smithi040 on smithi040 2024-02-16T19:18:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:00 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/592991595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db82e60f-540c-4f54-8f54-758be43e2cf2"}]: dispatch 2024-02-16T19:18:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:00 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/592991595' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db82e60f-540c-4f54-8f54-758be43e2cf2"}]': finished 2024-02-16T19:18:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:00 smithi040 ceph-mon[29201]: osdmap e6: 1 total, 0 up, 1 in 2024-02-16T19:18:00.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:00 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:00.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:00 smithi163 ceph-mon[36452]: Reconfiguring prometheus.smithi040 (dependencies changed)... 2024-02-16T19:18:00.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:00 smithi163 ceph-mon[36452]: Reconfiguring daemon prometheus.smithi040 on smithi040 2024-02-16T19:18:00.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:00 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/592991595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db82e60f-540c-4f54-8f54-758be43e2cf2"}]: dispatch 2024-02-16T19:18:00.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:00 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/592991595' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db82e60f-540c-4f54-8f54-758be43e2cf2"}]': finished 2024-02-16T19:18:00.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:00 smithi163 ceph-mon[36452]: osdmap e6: 1 total, 0 up, 1 in 2024-02-16T19:18:00.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:00 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:01 smithi040 ceph-mon[29201]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:01 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/4267179121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:18:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:01 smithi163 ceph-mon[36452]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:01 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/4267179121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:18:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:03 smithi040 ceph-mon[29201]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:03 smithi163 ceph-mon[36452]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:05 smithi040 ceph-mon[29201]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:05.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:05 smithi163 ceph-mon[36452]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:07.536 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:07 smithi040 ceph-mon[29201]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:07 smithi163 ceph-mon[36452]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:09.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:09 smithi040 ceph-mon[29201]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:09 smithi163 ceph-mon[36452]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:10.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:18:10.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:10.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:10.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:18:10.385 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:10.650 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:18:10.651 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:10.651 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:10.651 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:18:10.651 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:11.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:11 smithi163 ceph-mon[36452]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:11.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:11 smithi163 ceph-mon[36452]: Reconfiguring crash.smithi163 (monmap changed)... 2024-02-16T19:18:11.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:11 smithi163 ceph-mon[36452]: Reconfiguring daemon crash.smithi163 on smithi163 2024-02-16T19:18:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:11 smithi040 ceph-mon[29201]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:11 smithi040 ceph-mon[29201]: Reconfiguring crash.smithi163 (monmap changed)... 2024-02-16T19:18:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:11 smithi040 ceph-mon[29201]: Reconfiguring daemon crash.smithi163 on smithi163 2024-02-16T19:18:12.433 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:12 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:18:12.434 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:12 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:12.434 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:12 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:12.434 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:12 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:12.434 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:12 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:18:12.434 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:12 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:18:12.434 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:12 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:18:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:18:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:18:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:13.452 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: Deploying daemon osd.0 on smithi040 2024-02-16T19:18:13.452 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: Reconfiguring mgr.smithi163.wjfclz (monmap changed)... 2024-02-16T19:18:13.452 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: Reconfiguring daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:18:13.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:13.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:13.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:13.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:18:13.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:18:13.453 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:13 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:13.622 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: Deploying daemon osd.0 on smithi040 2024-02-16T19:18:13.622 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: Reconfiguring mgr.smithi163.wjfclz (monmap changed)... 2024-02-16T19:18:13.623 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: Reconfiguring daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:18:13.623 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:13.623 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:13.623 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:13.623 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:18:13.623 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:18:13.623 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:13 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:14 smithi040 ceph-mon[29201]: Reconfiguring mon.smithi163 (monmap changed)... 2024-02-16T19:18:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:14 smithi040 ceph-mon[29201]: Reconfiguring daemon mon.smithi163 on smithi163 2024-02-16T19:18:14.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:14 smithi163 ceph-mon[36452]: Reconfiguring mon.smithi163 (monmap changed)... 2024-02-16T19:18:14.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:14 smithi163 ceph-mon[36452]: Reconfiguring daemon mon.smithi163 on smithi163 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi040.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi040.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:18:15.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi040.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T19:18:15.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:18:15.705 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:15.705 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.705 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.705 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:18:15.705 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi040.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T19:18:15.705 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.705 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:18:15.705 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi040.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T19:18:15.706 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.706 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:18:15.706 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi040.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T19:18:15.706 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:15.706 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:18:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:16 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:18:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:16 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi040.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T19:18:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:16 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:18:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:16 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi040.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T19:18:16.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:16 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:18:16.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:16 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi040.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T19:18:16.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:16 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:16.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:16 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:16.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:16 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:18:16.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:16 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi040.front.sepia.ceph.com:9093"}]: dispatch 2024-02-16T19:18:16.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:16 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:18:16.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:16 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi040.front.sepia.ceph.com:3000"}]: dispatch 2024-02-16T19:18:16.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:16 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:18:16.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:16 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi040.front.sepia.ceph.com:9095"}]: dispatch 2024-02-16T19:18:16.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:16 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:16.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:16 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:17.470 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:17 smithi040 ceph-mon[29201]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:17 smithi163 ceph-mon[36452]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:18 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:18 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:18 smithi040 ceph-mon[29201]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:18.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:18 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:18.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:18 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:18.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:18 smithi163 ceph-mon[36452]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:21.456 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:21 smithi040 ceph-mon[29201]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:21 smithi163 ceph-mon[36452]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.624+0000 7fc8c6ffd700 1 -- 172.21.15.40:0/2765063474 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc8b8000bf0 con 0x7fc8c006d620 2024-02-16T19:18:21.626 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 0 on host 'smithi040' 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc8c006d620 msgr2=0x7fc8c006fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc8c006d620 0x7fc8c006fad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004890 tx=0x7fc8c8009e30 comp rx=0 tx=0).stop 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fe0e0 msgr2=0x7fc8d810def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fe0e0 0x7fc8d810def0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00b790 tx=0x7fc8cc009250 comp rx=0 tx=0).stop 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 shutdown_connections 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc8c006d620 0x7fc8c006fad0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d80fcef0 0x7fc8d810d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 --2- 172.21.15.40:0/2765063474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d80fe0e0 0x7fc8d810def0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 >> 172.21.15.40:0/2765063474 conn(0x7fc8d80f8620 msgr2=0x7fc8d81013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 shutdown_connections 2024-02-16T19:18:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:21.626+0000 7fc8dcdf7700 1 -- 172.21.15.40:0/2765063474 wait complete. 2024-02-16T19:18:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:22 smithi040 ceph-mon[29201]: from='osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T19:18:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:22 smithi163 ceph-mon[36452]: from='osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T19:18:22.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:22.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:22.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:22.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:22.802 DEBUG:teuthology.orchestra.run.smithi040:osd.0> sudo journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.0.service 2024-02-16T19:18:22.804 INFO:tasks.cephadm:Deploying osd.1 on smithi040 with /dev/vg_nvme/lv_3... 2024-02-16T19:18:22.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-16T19:18:23.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:23 smithi040 ceph-mon[29201]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:23.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:23 smithi040 ceph-mon[29201]: from='osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T19:18:23.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:23 smithi040 ceph-mon[29201]: osdmap e7: 1 total, 0 up, 1 in 2024-02-16T19:18:23.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:23 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:23.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:23 smithi040 ceph-mon[29201]: from='osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:18:23.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:18:23 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[48162]: 2024-02-16T19:18:23.199+0000 7fe9dbe13700 -1 osd.0 0 waiting for initial osdmap 2024-02-16T19:18:23.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:18:23 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[48162]: 2024-02-16T19:18:23.203+0000 7fe9d7c10700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:18:23.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:23 smithi163 ceph-mon[36452]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:23 smithi163 ceph-mon[36452]: from='osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T19:18:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:23 smithi163 ceph-mon[36452]: osdmap e7: 1 total, 0 up, 1 in 2024-02-16T19:18:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:23 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:23 smithi163 ceph-mon[36452]: from='osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:18:23.910 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:18:24.297 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:24 smithi040 ceph-mon[29201]: from='osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-02-16T19:18:24.297 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:24 smithi040 ceph-mon[29201]: osdmap e8: 1 total, 0 up, 1 in 2024-02-16T19:18:24.297 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:24 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:24.297 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:24 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:24.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:24 smithi163 ceph-mon[36452]: from='osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-02-16T19:18:24.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:24 smithi163 ceph-mon[36452]: osdmap e8: 1 total, 0 up, 1 in 2024-02-16T19:18:24.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:24 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:24.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:24 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:25 smithi163 ceph-mon[36452]: purged_snaps scrub starts 2024-02-16T19:18:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:25 smithi163 ceph-mon[36452]: purged_snaps scrub ok 2024-02-16T19:18:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:25 smithi163 ceph-mon[36452]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:25 smithi163 ceph-mon[36452]: osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] boot 2024-02-16T19:18:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:25 smithi163 ceph-mon[36452]: osdmap e9: 1 total, 1 up, 1 in 2024-02-16T19:18:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:18:25.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:25 smithi040 ceph-mon[29201]: purged_snaps scrub starts 2024-02-16T19:18:25.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:25 smithi040 ceph-mon[29201]: purged_snaps scrub ok 2024-02-16T19:18:25.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:25 smithi040 ceph-mon[29201]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-16T19:18:25.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:25.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:25 smithi040 ceph-mon[29201]: osd.0 [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] boot 2024-02-16T19:18:25.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:25 smithi040 ceph-mon[29201]: osdmap e9: 1 total, 1 up, 1 in 2024-02-16T19:18:25.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:18:25.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:18:26.387 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:26 smithi040 ceph-mon[29201]: osdmap e10: 1 total, 1 up, 1 in 2024-02-16T19:18:26.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:26 smithi163 ceph-mon[36452]: osdmap e10: 1 total, 1 up, 1 in 2024-02-16T19:18:27.041 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-16T19:18:27.041 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-16T19:18:27.041 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-02-16T19:18:27.041 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-02-16T19:18:27.041 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011261 s, 931 MB/s 2024-02-16T19:18:27.041 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-02-16T19:18:27.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:27.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:27.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:18:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:18:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:18:27.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:27.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:27.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:27.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:18:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:18:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:18:27.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi040:vg_nvme/lv_3 2024-02-16T19:18:28.314 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:18:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:28 smithi040 ceph-mon[29201]: Detected new or changed devices on smithi040 2024-02-16T19:18:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:28 smithi040 ceph-mon[29201]: Adjusting osd_memory_target on smithi040 to 12869M 2024-02-16T19:18:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:28 smithi163 ceph-mon[36452]: Detected new or changed devices on smithi040 2024-02-16T19:18:28.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:28 smithi163 ceph-mon[36452]: Adjusting osd_memory_target on smithi040 to 12869M 2024-02-16T19:18:29.388 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:29 smithi040 ceph-mon[29201]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:29.388 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:29.388 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:29 smithi163 ceph-mon[36452]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:29 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:29 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:31.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.118+0000 7f12bf59e700 1 -- 172.21.15.40:0/2167232108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 msgr2=0x7f12c00fc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:18:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.118+0000 7f12bf59e700 1 --2- 172.21.15.40:0/2167232108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 0x7f12c00fc070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f12b0004940 tx=0x7f12b0009e30 comp rx=0 tx=0).stop 2024-02-16T19:18:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 -- 172.21.15.40:0/2167232108 shutdown_connections 2024-02-16T19:18:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 --2- 172.21.15.40:0/2167232108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12c00fce70 0x7f12c00fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 --2- 172.21.15.40:0/2167232108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 0x7f12c00fc070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 -- 172.21.15.40:0/2167232108 >> 172.21.15.40:0/2167232108 conn(0x7f12c00f7300 msgr2=0x7f12c00f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:18:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 -- 172.21.15.40:0/2167232108 shutdown_connections 2024-02-16T19:18:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 -- 172.21.15.40:0/2167232108 wait complete. 2024-02-16T19:18:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 Processor -- start 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 -- start start 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 0x7f12c010d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12c00fce70 0x7f12c010dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c010e310 con 0x7f12c00fbc60 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12bf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c010eb50 con 0x7f12c00fce70 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 0x7f12c010d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 0x7f12c010d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34044/0 (socket says 172.21.15.40:34044) 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.119+0000 7f12be59c700 1 -- 172.21.15.40:0/3609292586 learned_addr learned my addr 172.21.15.40:0/3609292586 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12bdd9b700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12c00fce70 0x7f12c010dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12be59c700 1 -- 172.21.15.40:0/3609292586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12c00fce70 msgr2=0x7f12c010dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12be59c700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12c00fce70 0x7f12c010dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12be59c700 1 -- 172.21.15.40:0/3609292586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0005040 con 0x7f12c00fbc60 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12be59c700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 0x7f12c010d740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f12b0000c00 tx=0x7f12b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:18:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12aeffd700 1 -- 172.21.15.40:0/3609292586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12b000e040 con 0x7f12c00fbc60 2024-02-16T19:18:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12aeffd700 1 -- 172.21.15.40:0/3609292586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12b000bcc0 con 0x7f12c00fbc60 2024-02-16T19:18:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12aeffd700 1 -- 172.21.15.40:0/3609292586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12b001c630 con 0x7f12c00fbc60 2024-02-16T19:18:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c010edd0 con 0x7f12c00fbc60 2024-02-16T19:18:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.120+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c010f390 con 0x7f12c00fbc60 2024-02-16T19:18:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.121+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c004bed0 con 0x7f12c00fbc60 2024-02-16T19:18:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.121+0000 7f12aeffd700 1 -- 172.21.15.40:0/3609292586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f12b0013980 con 0x7f12c00fbc60 2024-02-16T19:18:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.122+0000 7f12aeffd700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f12a8069210 0x7f12a806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:18:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.122+0000 7f12aeffd700 1 -- 172.21.15.40:0/3609292586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f12b0087710 con 0x7f12c00fbc60 2024-02-16T19:18:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.122+0000 7f12bdd9b700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f12a8069210 0x7f12a806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:18:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.122+0000 7f12bdd9b700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f12a8069210 0x7f12a806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f12b4006fd0 tx=0x7f12b4008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:18:31.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.126+0000 7f12aeffd700 1 -- 172.21.15.40:0/3609292586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f12b00582d0 con 0x7f12c00fbc60 2024-02-16T19:18:31.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:31.330+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f12c010f7b0 con 0x7f12a8069210 2024-02-16T19:18:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:31 smithi040 ceph-mon[29201]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:31 smithi163 ceph-mon[36452]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:32 smithi040 ceph-mon[29201]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:18:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:32 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:18:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:32 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:18:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:32 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:32 smithi163 ceph-mon[36452]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:18:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:32 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:18:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:32 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:18:32.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:32 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:33.474 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:33 smithi040 ceph-mon[29201]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:33.474 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:33 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:33.474 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:33 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:33.474 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:33 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:33.474 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:33 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:18:33.474 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:33 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:33 smithi163 ceph-mon[36452]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:33 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:33 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:33.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:33 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:33.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:33 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:18:33.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:33 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:35 smithi040 ceph-mon[29201]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:35 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3520665936' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22561508-cdb8-4c76-ab07-bf3efd6ee63d"}]: dispatch 2024-02-16T19:18:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:35 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3520665936' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22561508-cdb8-4c76-ab07-bf3efd6ee63d"}]': finished 2024-02-16T19:18:35.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:35 smithi040 ceph-mon[29201]: osdmap e11: 2 total, 1 up, 2 in 2024-02-16T19:18:35.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:35 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:35 smithi163 ceph-mon[36452]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:35 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3520665936' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22561508-cdb8-4c76-ab07-bf3efd6ee63d"}]: dispatch 2024-02-16T19:18:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:35 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3520665936' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22561508-cdb8-4c76-ab07-bf3efd6ee63d"}]': finished 2024-02-16T19:18:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:35 smithi163 ceph-mon[36452]: osdmap e11: 2 total, 1 up, 2 in 2024-02-16T19:18:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:35 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:36 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/855580473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:18:36.728 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:36 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/855580473' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:18:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:37 smithi040 ceph-mon[29201]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:37.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:37 smithi163 ceph-mon[36452]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:39.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:39 smithi040 ceph-mon[29201]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:39 smithi163 ceph-mon[36452]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:40.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:18:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:18:41.568 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:41 smithi040 ceph-mon[29201]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:41 smithi163 ceph-mon[36452]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:42.608 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:18:42.608 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:42.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:18:42.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:43.618 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:43 smithi040 ceph-mon[29201]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:43.618 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:43 smithi040 ceph-mon[29201]: Deploying daemon osd.1 on smithi040 2024-02-16T19:18:43.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:43 smithi163 ceph-mon[36452]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:43.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:43 smithi163 ceph-mon[36452]: Deploying daemon osd.1 on smithi040 2024-02-16T19:18:45.626 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:45 smithi040 ceph-mon[29201]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:45.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:45 smithi163 ceph-mon[36452]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:47 smithi040 ceph-mon[29201]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:47 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:47 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:47 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:18:47.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:47 smithi163 ceph-mon[36452]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:47.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:47 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:47.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:47 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:47.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:47 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:18:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:49 smithi040 ceph-mon[29201]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:49 smithi163 ceph-mon[36452]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:49.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.093+0000 7f12aeffd700 1 -- 172.21.15.40:0/3609292586 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f12c010f7b0 con 0x7f12a8069210 2024-02-16T19:18:50.095 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 1 on host 'smithi040' 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f12a8069210 msgr2=0x7f12a806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f12a8069210 0x7f12a806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f12b4006fd0 tx=0x7f12b4008040 comp rx=0 tx=0).stop 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 msgr2=0x7f12c010d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 0x7f12c010d740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f12b0000c00 tx=0x7f12b0009e30 comp rx=0 tx=0).stop 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 shutdown_connections 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f12a8069210 0x7f12a806b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12c00fbc60 0x7f12c010d740 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 --2- 172.21.15.40:0/3609292586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12c00fce70 0x7f12c010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:50.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.095+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 >> 172.21.15.40:0/3609292586 conn(0x7f12c00f7300 msgr2=0x7f12c0065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:18:50.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.096+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 shutdown_connections 2024-02-16T19:18:50.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:50.096+0000 7f12bf59e700 1 -- 172.21.15.40:0/3609292586 wait complete. 2024-02-16T19:18:50.970 DEBUG:teuthology.orchestra.run.smithi040:osd.1> sudo journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.1.service 2024-02-16T19:18:50.972 INFO:tasks.cephadm:Deploying osd.2 on smithi040 with /dev/vg_nvme/lv_2... 2024-02-16T19:18:50.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-16T19:18:51.300 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:51 smithi040 ceph-mon[29201]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:51.300 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:51 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:51.300 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:51 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:51.300 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:18:51 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[55736]: 2024-02-16T19:18:51.202+0000 7f3839e8f540 -1 osd.1 0 log_to_monitors true 2024-02-16T19:18:51.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:51 smithi163 ceph-mon[36452]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:51.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:51 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:51.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:51 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:51.803 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:18:52.253 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:52 smithi040 ceph-mon[29201]: from='osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T19:18:52.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:52 smithi163 ceph-mon[36452]: from='osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T19:18:53.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:53 smithi040 ceph-mon[29201]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:53.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:53 smithi040 ceph-mon[29201]: from='osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T19:18:53.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:53 smithi040 ceph-mon[29201]: osdmap e12: 2 total, 1 up, 2 in 2024-02-16T19:18:53.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:53 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:53.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:53 smithi040 ceph-mon[29201]: from='osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:18:53.145 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:18:53 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[55736]: 2024-02-16T19:18:53.090+0000 7f3830661700 -1 osd.1 0 waiting for initial osdmap 2024-02-16T19:18:53.145 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:18:53 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[55736]: 2024-02-16T19:18:53.095+0000 7f382cc5f700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:18:53.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:53 smithi163 ceph-mon[36452]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:53 smithi163 ceph-mon[36452]: from='osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T19:18:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:53 smithi163 ceph-mon[36452]: osdmap e12: 2 total, 1 up, 2 in 2024-02-16T19:18:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:53 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:53 smithi163 ceph-mon[36452]: from='osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:18:54.312 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:54 smithi040 ceph-mon[29201]: from='osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-02-16T19:18:54.312 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:54 smithi040 ceph-mon[29201]: osdmap e13: 2 total, 1 up, 2 in 2024-02-16T19:18:54.312 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:54 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:54.312 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:54 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:54.312 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:54 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:54 smithi163 ceph-mon[36452]: from='osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-02-16T19:18:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:54 smithi163 ceph-mon[36452]: osdmap e13: 2 total, 1 up, 2 in 2024-02-16T19:18:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:54 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:54 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:54 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:54.790 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-16T19:18:54.790 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-16T19:18:54.790 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-02-16T19:18:54.790 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-02-16T19:18:54.790 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121589 s, 862 MB/s 2024-02-16T19:18:54.790 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-02-16T19:18:55.352 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:55 smithi040 ceph-mon[29201]: purged_snaps scrub starts 2024-02-16T19:18:55.352 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:55 smithi040 ceph-mon[29201]: purged_snaps scrub ok 2024-02-16T19:18:55.352 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:55 smithi040 ceph-mon[29201]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:55.352 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:55 smithi040 ceph-mon[29201]: osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] boot 2024-02-16T19:18:55.352 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:55 smithi040 ceph-mon[29201]: osdmap e14: 2 total, 2 up, 2 in 2024-02-16T19:18:55.352 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:55 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:55.352 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:55 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:18:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:55 smithi163 ceph-mon[36452]: purged_snaps scrub starts 2024-02-16T19:18:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:55 smithi163 ceph-mon[36452]: purged_snaps scrub ok 2024-02-16T19:18:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:55 smithi163 ceph-mon[36452]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-16T19:18:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:55 smithi163 ceph-mon[36452]: osd.1 [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] boot 2024-02-16T19:18:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:55 smithi163 ceph-mon[36452]: osdmap e14: 2 total, 2 up, 2 in 2024-02-16T19:18:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:55 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:18:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:55 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:18:55.647 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi040:vg_nvme/lv_2 2024-02-16T19:18:55.922 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:18:56.356 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:56 smithi040 ceph-mon[29201]: osdmap e15: 2 total, 2 up, 2 in 2024-02-16T19:18:56.356 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:56 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:56.356 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:56 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:56.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:56 smithi163 ceph-mon[36452]: osdmap e15: 2 total, 2 up, 2 in 2024-02-16T19:18:56.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:56 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:56.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:56 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:18:57.267 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:57 smithi040 ceph-mon[29201]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:18:57.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:57 smithi163 ceph-mon[36452]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:18:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.489+0000 7fd883881700 1 -- 172.21.15.40:0/874519469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 msgr2=0x7fd87c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:18:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.489+0000 7fd883881700 1 --2- 172.21.15.40:0/874519469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 0x7fd87c0fd420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd86c004940 tx=0x7fd86c009e30 comp rx=0 tx=0).stop 2024-02-16T19:18:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.489+0000 7fd883881700 1 -- 172.21.15.40:0/874519469 shutdown_connections 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.489+0000 7fd883881700 1 --2- 172.21.15.40:0/874519469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 0x7fd87c0fd420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.489+0000 7fd883881700 1 --2- 172.21.15.40:0/874519469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd87c0fbe40 0x7fd87c0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.489+0000 7fd883881700 1 -- 172.21.15.40:0/874519469 >> 172.21.15.40:0/874519469 conn(0x7fd87c0f74e0 msgr2=0x7fd87c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.489+0000 7fd883881700 1 -- 172.21.15.40:0/874519469 shutdown_connections 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.489+0000 7fd883881700 1 -- 172.21.15.40:0/874519469 wait complete. 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd883881700 1 Processor -- start 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd883881700 1 -- start start 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd883881700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd87c0fbe40 0x7fd87c10fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd883881700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 0x7fd87c1100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd883881700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87c1106c0 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd883881700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87c110f60 con 0x7fd87c0fbe40 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd880e1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 0x7fd87c1100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd880e1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 0x7fd87c1100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32848/0 (socket says 172.21.15.40:32848) 2024-02-16T19:18:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd880e1c700 1 -- 172.21.15.40:0/2331162121 learned_addr learned my addr 172.21.15.40:0/2331162121 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.490+0000 7fd88161d700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd87c0fbe40 0x7fd87c10fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd880e1c700 1 -- 172.21.15.40:0/2331162121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd87c0fbe40 msgr2=0x7fd87c10fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd880e1c700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd87c0fbe40 0x7fd87c10fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd880e1c700 1 -- 172.21.15.40:0/2331162121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c005040 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd88161d700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd87c0fbe40 0x7fd87c10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd880e1c700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 0x7fd87c1100c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009fd0 tx=0x7fd86c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd871ffb700 1 -- 172.21.15.40:0/2331162121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd86c00e040 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd871ffb700 1 -- 172.21.15.40:0/2331162121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd86c00bcc0 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87c1111e0 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd871ffb700 1 -- 172.21.15.40:0/2331162121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd86c01c630 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.491+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87c111720 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.492+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87c04bed0 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.493+0000 7fd871ffb700 1 -- 172.21.15.40:0/2331162121 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fd86c013980 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.493+0000 7fd871ffb700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd868069210 0x7fd86806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:18:57.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.493+0000 7fd871ffb700 1 -- 172.21.15.40:0/2331162121 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7fd86c087950 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.493+0000 7fd88161d700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd868069210 0x7fd86806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:18:57.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.494+0000 7fd88161d700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd868069210 0x7fd86806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8780041d0 tx=0x7fd878009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:18:57.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.498+0000 7fd871ffb700 1 -- 172.21.15.40:0/2331162121 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd86c0583e0 con 0x7fd87c0fcfb0 2024-02-16T19:18:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:18:57.704+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd87c111b40 con 0x7fd868069210 2024-02-16T19:18:58.286 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:18:58.286 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:18:58.286 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:58.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:18:58.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:18:58.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:18:59.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:59 smithi040 ceph-mon[29201]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:18:59.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:18:59 smithi040 ceph-mon[29201]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:18:59.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:59 smithi163 ceph-mon[36452]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:18:59.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:18:59 smithi163 ceph-mon[36452]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: Detected new or changed devices on smithi040 2024-02-16T19:19:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: Adjusting osd_memory_target on smithi040 to 6434M 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:01.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:01 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: Detected new or changed devices on smithi040 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: Adjusting osd_memory_target on smithi040 to 6434M 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:01.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:01 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:02.596 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:02 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1471028218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58ff6654-72f1-4329-8f6c-4c3180353755"}]: dispatch 2024-02-16T19:19:02.596 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:02 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1471028218' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58ff6654-72f1-4329-8f6c-4c3180353755"}]': finished 2024-02-16T19:19:02.596 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:02 smithi040 ceph-mon[29201]: osdmap e16: 3 total, 2 up, 3 in 2024-02-16T19:19:02.596 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:02 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:02.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:02 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1471028218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58ff6654-72f1-4329-8f6c-4c3180353755"}]: dispatch 2024-02-16T19:19:02.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:02 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1471028218' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58ff6654-72f1-4329-8f6c-4c3180353755"}]': finished 2024-02-16T19:19:02.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:02 smithi163 ceph-mon[36452]: osdmap e16: 3 total, 2 up, 3 in 2024-02-16T19:19:02.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:02 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:03 smithi040 ceph-mon[29201]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:03 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/338349752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:19:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:03 smithi163 ceph-mon[36452]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:03 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/338349752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:19:05.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:05 smithi163 ceph-mon[36452]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:05.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:05 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:05.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:05 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:05.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:05 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:05.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:05 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:05.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:05 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:05 smithi040 ceph-mon[29201]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:05 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:05 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:05 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:05 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:05 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:07.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:07 smithi040 ceph-mon[29201]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:07 smithi163 ceph-mon[36452]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:09 smithi040 ceph-mon[29201]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:09 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:19:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:09 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:09 smithi163 ceph-mon[36452]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:09 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:19:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:09 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:10.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:10 smithi040 ceph-mon[29201]: Deploying daemon osd.2 on smithi040 2024-02-16T19:19:10.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:19:10.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:10 smithi163 ceph-mon[36452]: Deploying daemon osd.2 on smithi040 2024-02-16T19:19:10.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:19:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:11 smithi040 ceph-mon[29201]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:11.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:11 smithi163 ceph-mon[36452]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:13.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:13 smithi040 ceph-mon[29201]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:13.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:13 smithi163 ceph-mon[36452]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:15.561 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:15 smithi040 ceph-mon[29201]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:15.562 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:15.562 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:15.562 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:15 smithi163 ceph-mon[36452]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:15.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:16.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:16 smithi040 ceph-mon[29201]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:16.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:16 smithi163 ceph-mon[36452]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:18.116 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 2 on host 'smithi040' 2024-02-16T19:19:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.114+0000 7fd871ffb700 1 -- 172.21.15.40:0/2331162121 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd87c111b40 con 0x7fd868069210 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd868069210 msgr2=0x7fd86806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd868069210 0x7fd86806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8780041d0 tx=0x7fd878009040 comp rx=0 tx=0).stop 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 msgr2=0x7fd87c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 0x7fd87c1100c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009fd0 tx=0x7fd86c009e30 comp rx=0 tx=0).stop 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 shutdown_connections 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd868069210 0x7fd86806b6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd87c0fbe40 0x7fd87c10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 --2- 172.21.15.40:0/2331162121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd87c0fcfb0 0x7fd87c1100c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.116+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 >> 172.21.15.40:0/2331162121 conn(0x7fd87c0f74e0 msgr2=0x7fd87c0f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:18.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.117+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 shutdown_connections 2024-02-16T19:19:18.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:19:18.117+0000 7fd883881700 1 -- 172.21.15.40:0/2331162121 wait complete. 2024-02-16T19:19:18.806 DEBUG:teuthology.orchestra.run.smithi040:osd.2> sudo journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.2.service 2024-02-16T19:19:18.809 INFO:tasks.cephadm:Deploying osd.3 on smithi163 with /dev/vg_nvme/lv_4... 2024-02-16T19:19:18.809 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-16T19:19:19.071 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi163/config 2024-02-16T19:19:19.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:19 smithi163 ceph-mon[36452]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:19.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:19 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:19.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:19 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:19.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:19 smithi163 ceph-mon[36452]: from='osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T19:19:19.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:19 smithi040 ceph-mon[29201]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:19.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:19 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:19.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:19 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:19.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:19 smithi040 ceph-mon[29201]: from='osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T19:19:20.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:20 smithi163 ceph-mon[36452]: from='osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T19:19:20.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:20 smithi163 ceph-mon[36452]: osdmap e17: 3 total, 2 up, 3 in 2024-02-16T19:19:20.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:20 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:20.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:20 smithi163 ceph-mon[36452]: from='osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:19:20.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:20 smithi040 ceph-mon[29201]: from='osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T19:19:20.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:20 smithi040 ceph-mon[29201]: osdmap e17: 3 total, 2 up, 3 in 2024-02-16T19:19:20.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:20 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:20.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:20 smithi040 ceph-mon[29201]: from='osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:19:20.395 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:20 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[63510]: 2024-02-16T19:19:20.100+0000 7fb3cc4c6700 -1 osd.2 0 waiting for initial osdmap 2024-02-16T19:19:20.395 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:20 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[63510]: 2024-02-16T19:19:20.106+0000 7fb3c62bf700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:19:21.017 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-16T19:19:21.018 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-16T19:19:21.018 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-02-16T19:19:21.018 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-02-16T19:19:21.018 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111477 s, 941 MB/s 2024-02-16T19:19:21.018 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-02-16T19:19:21.314 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:21 smithi163 ceph-mon[36452]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:21.314 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:21 smithi163 ceph-mon[36452]: from='osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-02-16T19:19:21.315 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:21 smithi163 ceph-mon[36452]: osdmap e18: 3 total, 2 up, 3 in 2024-02-16T19:19:21.315 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:21 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:21.315 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:21 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:21 smithi040 ceph-mon[29201]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:21 smithi040 ceph-mon[29201]: from='osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-02-16T19:19:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:21 smithi040 ceph-mon[29201]: osdmap e18: 3 total, 2 up, 3 in 2024-02-16T19:19:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:21 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:21 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:21.665 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi163:vg_nvme/lv_4 2024-02-16T19:19:21.923 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi163/config 2024-02-16T19:19:22.640 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:22 smithi040 ceph-mon[29201]: purged_snaps scrub starts 2024-02-16T19:19:22.641 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:22 smithi040 ceph-mon[29201]: purged_snaps scrub ok 2024-02-16T19:19:22.641 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:22 smithi040 ceph-mon[29201]: osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] boot 2024-02-16T19:19:22.641 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:22 smithi040 ceph-mon[29201]: osdmap e19: 3 total, 3 up, 3 in 2024-02-16T19:19:22.641 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:22.641 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:22.641 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:22.688 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:22 smithi163 ceph-mon[36452]: purged_snaps scrub starts 2024-02-16T19:19:22.688 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:22 smithi163 ceph-mon[36452]: purged_snaps scrub ok 2024-02-16T19:19:22.688 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:22 smithi163 ceph-mon[36452]: osd.2 [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] boot 2024-02-16T19:19:22.689 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:22 smithi163 ceph-mon[36452]: osdmap e19: 3 total, 3 up, 3 in 2024-02-16T19:19:22.689 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:19:22.689 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:22.689 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:23.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.558+0000 7f39a48c6700 1 -- 172.21.15.163:0/2205853748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 msgr2=0x7f39a00fb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:23.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 --2- 172.21.15.163:0/2205853748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 0x7f39a00fb090 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3994004940 tx=0x7f3994009e30 comp rx=0 tx=0).stop 2024-02-16T19:19:23.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 -- 172.21.15.163:0/2205853748 shutdown_connections 2024-02-16T19:19:23.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 --2- 172.21.15.163:0/2205853748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 0x7f39a00fb090 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:23.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 --2- 172.21.15.163:0/2205853748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a00fe600 0x7f39a00fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:23.560 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 -- 172.21.15.163:0/2205853748 >> 172.21.15.163:0/2205853748 conn(0x7f39a00f6d20 msgr2=0x7f39a00f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:23.561 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:23 smithi163 ceph-mon[36452]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:23.561 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:23 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-16T19:19:23.561 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:23 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-16T19:19:23.561 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:23 smithi163 ceph-mon[36452]: osdmap e20: 3 total, 3 up, 3 in 2024-02-16T19:19:23.561 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:23 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 -- 172.21.15.163:0/2205853748 shutdown_connections 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 -- 172.21.15.163:0/2205853748 wait complete. 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 Processor -- start 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 -- start start 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a00fe600 0x7f39a010dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 0x7f39a010e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a010e750 con 0x7f39a00fe600 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.559+0000 7f39a48c6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a010eff0 con 0x7f39a00fef50 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f399dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 0x7f39a010e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f399dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 0x7f39a010e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46202/0 (socket says 172.21.15.163:46202) 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f399dd9b700 1 -- 172.21.15.163:0/3138442106 learned_addr learned my addr 172.21.15.163:0/3138442106 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f399e59c700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a00fe600 0x7f39a010dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f399dd9b700 1 -- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a00fe600 msgr2=0x7f39a010dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:23.563 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f399dd9b700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a00fe600 0x7f39a010dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f399dd9b700 1 -- 172.21.15.163:0/3138442106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3994005040 con 0x7f39a00fef50 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f399dd9b700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 0x7f39a010e150 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3994000c00 tx=0x7f3994009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f398effd700 1 -- 172.21.15.163:0/3138442106 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3994021070 con 0x7f39a00fef50 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f398effd700 1 -- 172.21.15.163:0/3138442106 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f399401c400 con 0x7f39a00fef50 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a010f270 con 0x7f39a00fef50 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f398effd700 1 -- 172.21.15.163:0/3138442106 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399400e070 con 0x7f39a00fef50 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.560+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a010f750 con 0x7f39a00fef50 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.561+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a0107a90 con 0x7f39a00fef50 2024-02-16T19:19:23.564 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.562+0000 7f398effd700 1 -- 172.21.15.163:0/3138442106 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f399401ca50 con 0x7f39a00fef50 2024-02-16T19:19:23.565 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.562+0000 7f398effd700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3988069210 0x7f398806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:23.565 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.562+0000 7f398effd700 1 -- 172.21.15.163:0/3138442106 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7f39940878e0 con 0x7f39a00fef50 2024-02-16T19:19:23.565 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.562+0000 7f399e59c700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3988069210 0x7f398806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:23.565 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.563+0000 7f399e59c700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3988069210 0x7f398806b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3990009b80 tx=0x7f399000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:23.568 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.567+0000 7f398effd700 1 -- 172.21.15.163:0/3138442106 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f39940580e0 con 0x7f39a00fef50 2024-02-16T19:19:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:23 smithi040 ceph-mon[29201]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-16T19:19:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:23 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-16T19:19:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:23 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-16T19:19:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:23 smithi040 ceph-mon[29201]: osdmap e20: 3 total, 3 up, 3 in 2024-02-16T19:19:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:23 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-16T19:19:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:23.762+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f39a0105ac0 con 0x7f3988069210 2024-02-16T19:19:24.089 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:19:23 smithi040 sudo[67643]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:19:24.089 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:19:23 smithi040 sudo[67643]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:24.089 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:19:23 smithi040 sudo[67643]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:24.090 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:19:23 smithi040 sudo[67643]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:24.359 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67648]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:19:24.359 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67648]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:24.359 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67648]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:24.359 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67648]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:24.452 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:24 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-16T19:19:24.453 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:24 smithi163 ceph-mon[36452]: osdmap e21: 3 total, 3 up, 3 in 2024-02-16T19:19:24.453 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:24 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:19:24.453 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:24 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:19:24.453 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:24 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:24.453 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:24 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:24.644 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67689]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:19:24.644 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67689]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:24.644 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67689]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:24.644 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67689]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:24.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:24 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-16T19:19:24.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:24 smithi040 ceph-mon[29201]: osdmap e21: 3 total, 3 up, 3 in 2024-02-16T19:19:24.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:24 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:19:24.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:24 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:19:24.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:24 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:24.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:24 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.091 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67787]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:19:25.092 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:25.092 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67787]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:25.092 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:19:24 smithi040 sudo[67787]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:25.362 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67842]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:19:25.362 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:25.362 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67842]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:25.362 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67842]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: osdmap e22: 3 total, 3 up, 3 in 2024-02-16T19:19:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:19:25.621 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67864]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:19:25.621 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67864]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:25.621 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67864]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:25.621 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67864]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: osdmap e22: 3 total, 3 up, 3 in 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:25.622 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:25.623 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:19:25.623 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67871]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:19:25.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67871]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:25.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67871]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:25.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67871]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:25.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:25.905+0000 7f398effd700 1 -- 172.21.15.163:0/3138442106 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3994054a20 con 0x7f39a00fef50 2024-02-16T19:19:26.166 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 sudo[39951]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-16T19:19:26.167 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 sudo[39951]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:26.167 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 sudo[39951]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:26.167 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 sudo[39951]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:26.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67949]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:19:26.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:26.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67949]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:26.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:25 smithi040 sudo[67949]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:26.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 sudo[39954]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-16T19:19:26.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 sudo[39954]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 sudo[39954]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 sudo[39954]: pam_unix(sudo:session): session closed for user root 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: Detected new or changed devices on smithi040 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: Adjusting osd_memory_target on smithi040 to 4289M 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: mgrmap e19: smithi040.jzygzo(active, since 2m), standbys: smithi163.wjfclz 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:19:26.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:26 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:19:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: Detected new or changed devices on smithi040 2024-02-16T19:19:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: Adjusting osd_memory_target on smithi040 to 4289M 2024-02-16T19:19:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T19:19:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T19:19:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:19:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:19:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: mgrmap e19: smithi040.jzygzo(active, since 2m), standbys: smithi163.wjfclz 2024-02-16T19:19:26.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:19:26.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:19:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: from='client.? 172.21.15.163:0/3169266350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f2b76f-f1fa-4459-8150-c36e624fb4aa"}]: dispatch 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f2b76f-f1fa-4459-8150-c36e624fb4aa"}]: dispatch 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6f2b76f-f1fa-4459-8150-c36e624fb4aa"}]': finished 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: osdmap e23: 4 total, 3 up, 4 in 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:27 smithi163 ceph-mon[36452]: from='client.? 172.21.15.163:0/3136687846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:19:27.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-16T19:19:27.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:27.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-16T19:19:27.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/3169266350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f2b76f-f1fa-4459-8150-c36e624fb4aa"}]: dispatch 2024-02-16T19:19:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6f2b76f-f1fa-4459-8150-c36e624fb4aa"}]: dispatch 2024-02-16T19:19:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6f2b76f-f1fa-4459-8150-c36e624fb4aa"}]': finished 2024-02-16T19:19:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: osdmap e23: 4 total, 3 up, 4 in 2024-02-16T19:19:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:27 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/3136687846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:19:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:28 smithi163 ceph-mon[36452]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:28.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:28 smithi040 ceph-mon[29201]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:30.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:30.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:30.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:30.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:30.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:29 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:30.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:29 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:30.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:29 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:30.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:29 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:30.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:29 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:30.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:29 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:31 smithi163 ceph-mon[36452]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:31 smithi040 ceph-mon[29201]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:33.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:33 smithi163 ceph-mon[36452]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-16T19:19:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:33 smithi040 ceph-mon[29201]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-16T19:19:34.470 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:34 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:19:34.470 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:34 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:34 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:19:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:34 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:35.411 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:35 smithi163 ceph-mon[36452]: Deploying daemon osd.3 on smithi163 2024-02-16T19:19:35.411 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:35 smithi163 ceph-mon[36452]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-16T19:19:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:35 smithi040 ceph-mon[29201]: Deploying daemon osd.3 on smithi163 2024-02-16T19:19:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:35 smithi040 ceph-mon[29201]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-16T19:19:37.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:37 smithi163 ceph-mon[36452]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-16T19:19:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:37 smithi040 ceph-mon[29201]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-16T19:19:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:39 smithi163 ceph-mon[36452]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-02-16T19:19:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:39 smithi040 ceph-mon[29201]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-02-16T19:19:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:19:40.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:19:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:41 smithi163 ceph-mon[36452]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-16T19:19:41.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:41 smithi040 ceph-mon[29201]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-16T19:19:41.977 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.976+0000 7f398effd700 1 -- 172.21.15.163:0/3138442106 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f39a0105ac0 con 0x7f3988069210 2024-02-16T19:19:41.977 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 3 on host 'smithi163' 2024-02-16T19:19:41.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3988069210 msgr2=0x7f398806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3988069210 0x7f398806b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3990009b80 tx=0x7f399000b040 comp rx=0 tx=0).stop 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 msgr2=0x7f39a010e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 0x7f39a010e150 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3994000c00 tx=0x7f3994009e30 comp rx=0 tx=0).stop 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 shutdown_connections 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3988069210 0x7f398806b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a00fe600 0x7f39a010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 --2- 172.21.15.163:0/3138442106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a00fef50 0x7f39a010e150 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 >> 172.21.15.163:0/3138442106 conn(0x7f39a00f6d20 msgr2=0x7f39a0101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 shutdown_connections 2024-02-16T19:19:41.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:41.978+0000 7f39a48c6700 1 -- 172.21.15.163:0/3138442106 wait complete. 2024-02-16T19:19:43.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:43.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:43.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:42 smithi163 ceph-mon[36452]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-16T19:19:43.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:42 smithi163 ceph-mon[36452]: from='osd.3 [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:19:43.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:42 smithi163 ceph-mon[36452]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:19:43.286 DEBUG:teuthology.orchestra.run.smithi163:osd.3> sudo journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.3.service 2024-02-16T19:19:43.288 INFO:tasks.cephadm:Deploying osd.4 on smithi163 with /dev/vg_nvme/lv_3... 2024-02-16T19:19:43.288 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-16T19:19:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:42 smithi040 ceph-mon[29201]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-16T19:19:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:42 smithi040 ceph-mon[29201]: from='osd.3 [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:19:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:42 smithi040 ceph-mon[29201]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:19:43.589 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi163/config 2024-02-16T19:19:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-mon[36452]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T19:19:43.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-mon[36452]: osdmap e24: 4 total, 3 up, 4 in 2024-02-16T19:19:43.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:43.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-mon[36452]: from='osd.3 [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:19:43.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-mon[36452]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:19:43.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:43.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:43.988 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[42055]: 2024-02-16T19:19:43.945+0000 7fbd6f8fd700 -1 osd.3 0 waiting for initial osdmap 2024-02-16T19:19:43.988 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:19:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[42055]: 2024-02-16T19:19:43.952+0000 7fbd68ef5700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:19:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:43 smithi040 ceph-mon[29201]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T19:19:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:43 smithi040 ceph-mon[29201]: osdmap e24: 4 total, 3 up, 4 in 2024-02-16T19:19:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:43 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:43 smithi040 ceph-mon[29201]: from='osd.3 [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:19:44.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:43 smithi040 ceph-mon[29201]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:19:44.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:43 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:44.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:43 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:45.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:44 smithi163 ceph-mon[36452]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-02-16T19:19:45.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:44 smithi163 ceph-mon[36452]: osdmap e25: 4 total, 3 up, 4 in 2024-02-16T19:19:45.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:44 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:45.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:44 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:45.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:44 smithi163 ceph-mon[36452]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:45.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:44 smithi163 ceph-mon[36452]: osd.3 [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] boot 2024-02-16T19:19:45.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:44 smithi163 ceph-mon[36452]: osdmap e26: 4 total, 4 up, 4 in 2024-02-16T19:19:45.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:44 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:44 smithi040 ceph-mon[29201]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-02-16T19:19:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:44 smithi040 ceph-mon[29201]: osdmap e25: 4 total, 3 up, 4 in 2024-02-16T19:19:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:44 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:44 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:45.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:44 smithi040 ceph-mon[29201]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-16T19:19:45.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:44 smithi040 ceph-mon[29201]: osd.3 [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] boot 2024-02-16T19:19:45.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:44 smithi040 ceph-mon[29201]: osdmap e26: 4 total, 4 up, 4 in 2024-02-16T19:19:45.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:44 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:19:46.068 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-16T19:19:46.068 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-16T19:19:46.068 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-02-16T19:19:46.068 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-02-16T19:19:46.069 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0146711 s, 715 MB/s 2024-02-16T19:19:46.069 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-02-16T19:19:46.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:46 smithi163 ceph-mon[36452]: purged_snaps scrub starts 2024-02-16T19:19:46.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:46 smithi163 ceph-mon[36452]: purged_snaps scrub ok 2024-02-16T19:19:46.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:46 smithi163 ceph-mon[36452]: osdmap e27: 4 total, 4 up, 4 in 2024-02-16T19:19:46.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:46 smithi040 ceph-mon[29201]: purged_snaps scrub starts 2024-02-16T19:19:46.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:46 smithi040 ceph-mon[29201]: purged_snaps scrub ok 2024-02-16T19:19:46.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:46 smithi040 ceph-mon[29201]: osdmap e27: 4 total, 4 up, 4 in 2024-02-16T19:19:46.797 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi163:vg_nvme/lv_3 2024-02-16T19:19:47.211 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi163/config 2024-02-16T19:19:47.308 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:47 smithi163 ceph-mon[36452]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:19:47.309 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:47 smithi163 ceph-mon[36452]: osdmap e28: 4 total, 4 up, 4 in 2024-02-16T19:19:47.579 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:47 smithi040 ceph-mon[29201]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:19:47.580 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:47 smithi040 ceph-mon[29201]: osdmap e28: 4 total, 4 up, 4 in 2024-02-16T19:19:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:48 smithi163 ceph-mon[36452]: osdmap e29: 4 total, 4 up, 4 in 2024-02-16T19:19:48.551 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:48 smithi040 ceph-mon[29201]: osdmap e29: 4 total, 4 up, 4 in 2024-02-16T19:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 -- 172.21.15.163:0/661978128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 msgr2=0x7f7ec40fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 --2- 172.21.15.163:0/661978128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 0x7f7ec40fc250 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4004940 tx=0x7f7eb4009e30 comp rx=0 tx=0).stop 2024-02-16T19:19:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 -- 172.21.15.163:0/661978128 shutdown_connections 2024-02-16T19:19:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 --2- 172.21.15.163:0/661978128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec40fcfb0 0x7f7ec40fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 --2- 172.21.15.163:0/661978128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 0x7f7ec40fc250 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 -- 172.21.15.163:0/661978128 >> 172.21.15.163:0/661978128 conn(0x7f7ec40f74e0 msgr2=0x7f7ec40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:19:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 -- 172.21.15.163:0/661978128 shutdown_connections 2024-02-16T19:19:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 -- 172.21.15.163:0/661978128 wait complete. 2024-02-16T19:19:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 Processor -- start 2024-02-16T19:19:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 -- start start 2024-02-16T19:19:49.341 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.338+0000 7f7eca42d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 0x7f7ec4110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 0x7f7ec4110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 0x7f7ec4110690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:45164/0 (socket says 172.21.15.163:45164) 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7eca42d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec40fcfb0 0x7f7ec4112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40664f0 con 0x7f7ec40fcfb0 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec3fff700 1 -- 172.21.15.163:0/2538957688 learned_addr learned my addr 172.21.15.163:0/2538957688 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec4066630 con 0x7f7ec40fbe40 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec3fff700 1 -- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec40fcfb0 msgr2=0x7f7ec4112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec3fff700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec40fcfb0 0x7f7ec4112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec3fff700 1 -- 172.21.15.163:0/2538957688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4005040 con 0x7f7ec40fbe40 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec3fff700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 0x7f7ec4110690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009fd0 tx=0x7f7eb4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec0ff9700 1 -- 172.21.15.163:0/2538957688 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb400e040 con 0x7f7ec40fbe40 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec4113210 con 0x7f7ec40fbe40 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec0ff9700 1 -- 172.21.15.163:0/2538957688 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7eb400bcc0 con 0x7f7ec40fbe40 2024-02-16T19:19:49.342 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7ec0ff9700 1 -- 172.21.15.163:0/2538957688 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb401c630 con 0x7f7ec40fbe40 2024-02-16T19:19:49.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.339+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec41136f0 con 0x7f7ec40fbe40 2024-02-16T19:19:49.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.340+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec404bed0 con 0x7f7ec40fbe40 2024-02-16T19:19:49.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.341+0000 7f7ec0ff9700 1 -- 172.21.15.163:0/2538957688 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7eb4013980 con 0x7f7ec40fbe40 2024-02-16T19:19:49.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.341+0000 7f7ec0ff9700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7eac0692b0 0x7f7eac06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:19:49.343 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.341+0000 7f7ec0ff9700 1 -- 172.21.15.163:0/2538957688 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f7eb4088070 con 0x7f7ec40fbe40 2024-02-16T19:19:49.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.341+0000 7f7ec37fe700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7eac0692b0 0x7f7eac06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:19:49.344 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.342+0000 7f7ec37fe700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7eac0692b0 0x7f7eac06b760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8006fd0 tx=0x7f7eb8006db0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:19:49.347 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.346+0000 7f7ec0ff9700 1 -- 172.21.15.163:0/2538957688 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7eb4054a40 con 0x7f7ec40fbe40 2024-02-16T19:19:49.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:19:49.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: osdmap e30: 4 total, 4 up, 4 in 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:49.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:49 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.537 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:19:49.536+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec4113910 con 0x7f7eac0692b0 2024-02-16T19:19:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:19:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: osdmap e30: 4 total, 4 up, 4 in 2024-02-16T19:19:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:19:49.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:49 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:19:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:50 smithi163 ceph-mon[36452]: Detected new or changed devices on smithi163 2024-02-16T19:19:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:50 smithi163 ceph-mon[36452]: Adjusting osd_memory_target on smithi163 to 15936M 2024-02-16T19:19:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:50 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:19:50.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:50 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:19:50.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:50 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:50 smithi040 ceph-mon[29201]: Detected new or changed devices on smithi163 2024-02-16T19:19:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:50 smithi040 ceph-mon[29201]: Adjusting osd_memory_target on smithi163 to 15936M 2024-02-16T19:19:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:50 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:19:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:50 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:19:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:50 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:51.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:51 smithi163 ceph-mon[36452]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:51.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:51 smithi163 ceph-mon[36452]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:19:51.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:51 smithi040 ceph-mon[29201]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:19:51.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:51 smithi040 ceph-mon[29201]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:19:52.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:52 smithi163 ceph-mon[36452]: from='client.? 172.21.15.163:0/1993334239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49afe3d8-2685-4c11-9830-77bd72a0b272"}]: dispatch 2024-02-16T19:19:52.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:52 smithi163 ceph-mon[36452]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49afe3d8-2685-4c11-9830-77bd72a0b272"}]: dispatch 2024-02-16T19:19:52.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:52 smithi163 ceph-mon[36452]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49afe3d8-2685-4c11-9830-77bd72a0b272"}]': finished 2024-02-16T19:19:52.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:52 smithi163 ceph-mon[36452]: osdmap e31: 5 total, 4 up, 5 in 2024-02-16T19:19:52.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:52 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:19:52.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:52 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/1993334239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49afe3d8-2685-4c11-9830-77bd72a0b272"}]: dispatch 2024-02-16T19:19:52.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:52 smithi040 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49afe3d8-2685-4c11-9830-77bd72a0b272"}]: dispatch 2024-02-16T19:19:52.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:52 smithi040 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49afe3d8-2685-4c11-9830-77bd72a0b272"}]': finished 2024-02-16T19:19:52.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:52 smithi040 ceph-mon[29201]: osdmap e31: 5 total, 4 up, 5 in 2024-02-16T19:19:52.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:52 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:19:53.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:53 smithi163 ceph-mon[36452]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-02-16T19:19:53.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:53 smithi163 ceph-mon[36452]: from='client.? 172.21.15.163:0/1428705303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:19:53.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:53 smithi040 ceph-mon[29201]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-02-16T19:19:53.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:53 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/1428705303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:19:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:55 smithi040 ceph-mon[29201]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-16T19:19:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:55 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:19:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:55 smithi163 ceph-mon[36452]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-16T19:19:55.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:55 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:19:57.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:57 smithi163 ceph-mon[36452]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-16T19:19:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:57 smithi040 ceph-mon[29201]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-16T19:19:59.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:59 smithi163 ceph-mon[36452]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-16T19:19:59.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:59 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:19:59.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:19:59 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:19:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:59 smithi040 ceph-mon[29201]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-16T19:19:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:59 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:19:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:19:59 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:00.429 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:00 smithi163 ceph-mon[36452]: Deploying daemon osd.4 on smithi163 2024-02-16T19:20:00.429 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:00 smithi163 ceph-mon[36452]: overall HEALTH_OK 2024-02-16T19:20:00.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:00 smithi040 ceph-mon[29201]: Deploying daemon osd.4 on smithi163 2024-02-16T19:20:00.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:00 smithi040 ceph-mon[29201]: overall HEALTH_OK 2024-02-16T19:20:01.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:01 smithi163 ceph-mon[36452]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-16T19:20:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:01 smithi040 ceph-mon[29201]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-16T19:20:03.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:03 smithi163 ceph-mon[36452]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:03 smithi040 ceph-mon[29201]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:05 smithi040 ceph-mon[29201]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:05 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:05 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:05.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:05 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:05.718 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:05 smithi163 ceph-mon[36452]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:05.718 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:05 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:05.718 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:05 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:05.718 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:05 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:07.303 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:07 smithi163 ceph-mon[36452]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:07 smithi040 ceph-mon[29201]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:08.203 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 4 on host 'smithi163' 2024-02-16T19:20:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.201+0000 7f7ec0ff9700 1 -- 172.21.15.163:0/2538957688 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7ec4113910 con 0x7f7eac0692b0 2024-02-16T19:20:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7eac0692b0 msgr2=0x7f7eac06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7eac0692b0 0x7f7eac06b760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8006fd0 tx=0x7f7eb8006db0 comp rx=0 tx=0).stop 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 msgr2=0x7f7ec4110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 0x7f7ec4110690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009fd0 tx=0x7f7eb4009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 shutdown_connections 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7eac0692b0 0x7f7eac06b760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec40fbe40 0x7f7ec4110690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 --2- 172.21.15.163:0/2538957688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ec40fcfb0 0x7f7ec4112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 >> 172.21.15.163:0/2538957688 conn(0x7f7ec40f74e0 msgr2=0x7f7ec4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 shutdown_connections 2024-02-16T19:20:08.207 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:08.205+0000 7f7eca42d700 1 -- 172.21.15.163:0/2538957688 wait complete. 2024-02-16T19:20:08.797 DEBUG:teuthology.orchestra.run.smithi163:osd.4> sudo journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.4.service 2024-02-16T19:20:08.799 INFO:tasks.cephadm:Deploying osd.5 on smithi163 with /dev/vg_nvme/lv_2... 2024-02-16T19:20:08.799 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-16T19:20:09.099 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi163/config 2024-02-16T19:20:09.447 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:09 smithi163 ceph-mon[36452]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:09.448 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:09 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:09.448 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:09 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:09.448 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:09 smithi163 ceph-mon[36452]: from='osd.4 [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:20:09.448 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:09 smithi163 ceph-mon[36452]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:20:09.448 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:09 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:09.448 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:09 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:09 smithi040 ceph-mon[29201]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:09 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:09 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:09 smithi040 ceph-mon[29201]: from='osd.4 [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:20:09.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:09 smithi040 ceph-mon[29201]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:20:09.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:09 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:09.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:09 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:10 smithi040 ceph-mon[29201]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T19:20:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:10 smithi040 ceph-mon[29201]: osdmap e32: 5 total, 4 up, 5 in 2024-02-16T19:20:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:20:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:10 smithi040 ceph-mon[29201]: from='osd.4 [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:20:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:10 smithi040 ceph-mon[29201]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:20:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:20:10.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:10 smithi163 ceph-mon[36452]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T19:20:10.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:10 smithi163 ceph-mon[36452]: osdmap e32: 5 total, 4 up, 5 in 2024-02-16T19:20:10.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:20:10.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:10 smithi163 ceph-mon[36452]: from='osd.4 [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:20:10.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:10 smithi163 ceph-mon[36452]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:20:10.412 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:20:10.412 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:20:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[48494]: 2024-02-16T19:20:10.176+0000 7fd75dcac700 -1 osd.4 0 waiting for initial osdmap 2024-02-16T19:20:10.412 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:20:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[48494]: 2024-02-16T19:20:10.183+0000 7fd758aa7700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:20:11.375 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:11 smithi163 ceph-mon[36452]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:11.375 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:11 smithi163 ceph-mon[36452]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-02-16T19:20:11.375 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:11 smithi163 ceph-mon[36452]: osdmap e33: 5 total, 4 up, 5 in 2024-02-16T19:20:11.375 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:20:11.376 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:20:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:11 smithi040 ceph-mon[29201]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:11 smithi040 ceph-mon[29201]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-02-16T19:20:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:11 smithi040 ceph-mon[29201]: osdmap e33: 5 total, 4 up, 5 in 2024-02-16T19:20:11.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:20:11.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:20:12.021 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-16T19:20:12.021 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-16T19:20:12.021 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-02-16T19:20:12.021 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-02-16T19:20:12.021 INFO:teuthology.orchestra.run.smithi163.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122667 s, 855 MB/s 2024-02-16T19:20:12.022 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 2024-02-16T19:20:12.022 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-02-16T19:20:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:12 smithi040 ceph-mon[29201]: purged_snaps scrub starts 2024-02-16T19:20:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:12 smithi040 ceph-mon[29201]: purged_snaps scrub ok 2024-02-16T19:20:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:12 smithi040 ceph-mon[29201]: osd.4 [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] boot 2024-02-16T19:20:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:12 smithi040 ceph-mon[29201]: osdmap e34: 5 total, 5 up, 5 in 2024-02-16T19:20:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:12 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:20:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:12 smithi163 ceph-mon[36452]: purged_snaps scrub starts 2024-02-16T19:20:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:12 smithi163 ceph-mon[36452]: purged_snaps scrub ok 2024-02-16T19:20:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:12 smithi163 ceph-mon[36452]: osd.4 [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] boot 2024-02-16T19:20:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:12 smithi163 ceph-mon[36452]: osdmap e34: 5 total, 5 up, 5 in 2024-02-16T19:20:12.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:12 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:20:12.824 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi163:vg_nvme/lv_2 2024-02-16T19:20:13.378 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:13 smithi163 ceph-mon[36452]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:13.379 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:13 smithi163 ceph-mon[36452]: osdmap e35: 5 total, 5 up, 5 in 2024-02-16T19:20:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi163/config 2024-02-16T19:20:13.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:13 smithi040 ceph-mon[29201]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-16T19:20:13.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:13 smithi040 ceph-mon[29201]: osdmap e35: 5 total, 5 up, 5 in 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: Detected new or changed devices on smithi163 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: Adjusting osd_memory_target on smithi163 to 7968M 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:14.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:20:14.659 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.659 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:14.659 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:14.659 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:20:14.659 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.659 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:14 smithi163 ceph-mon[36452]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:14.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: Detected new or changed devices on smithi163 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: Adjusting osd_memory_target on smithi163 to 7968M 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:20:14.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:14.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:14 smithi040 ceph-mon[29201]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 -- 172.21.15.163:0/3026022235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fcef0 msgr2=0x7f1f0c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 --2- 172.21.15.163:0/3026022235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fcef0 0x7f1f0c0fd320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1efc004970 tx=0x7f1efc009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 -- 172.21.15.163:0/3026022235 shutdown_connections 2024-02-16T19:20:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 --2- 172.21.15.163:0/3026022235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c0fe0e0 0x7f1f0c0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 --2- 172.21.15.163:0/3026022235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fcef0 0x7f1f0c0fd320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 -- 172.21.15.163:0/3026022235 >> 172.21.15.163:0/3026022235 conn(0x7f1f0c0f8620 msgr2=0x7f1f0c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 -- 172.21.15.163:0/3026022235 shutdown_connections 2024-02-16T19:20:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 -- 172.21.15.163:0/3026022235 wait complete. 2024-02-16T19:20:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.903+0000 7f1f11144700 1 Processor -- start 2024-02-16T19:20:14.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f11144700 1 -- start start 2024-02-16T19:20:14.906 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f11144700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c0fcef0 0x7f1f0c10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f11144700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fe0e0 0x7f1f0c10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f11144700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c10e560 con 0x7f1f0c0fcef0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f11144700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c10eda0 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f0259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fe0e0 0x7f1f0c10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f0259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fe0e0 0x7f1f0c10ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34668/0 (socket says 172.21.15.163:34668) 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f0259c700 1 -- 172.21.15.163:0/2774859187 learned_addr learned my addr 172.21.15.163:0/2774859187 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f0ad9d700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c0fcef0 0x7f1f0c10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f0259c700 1 -- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c0fcef0 msgr2=0x7f1f0c10d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f0259c700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c0fcef0 0x7f1f0c10d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f0259c700 1 -- 172.21.15.163:0/2774859187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc005040 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f0259c700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fe0e0 0x7f1f0c10ded0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400b790 tx=0x7f1ef4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f03fff700 1 -- 172.21.15.163:0/2774859187 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ef400f6f0 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f03fff700 1 -- 172.21.15.163:0/2774859187 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ef400fd20 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.904+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c10f080 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.907 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.905+0000 7f1f03fff700 1 -- 172.21.15.163:0/2774859187 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ef40103a0 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.905+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c10f5c0 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.905+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c04bed0 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.908 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.906+0000 7f1f03fff700 1 -- 172.21.15.163:0/2774859187 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1ef4011400 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.906+0000 7f1f03fff700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1ef80692b0 0x7f1ef806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:14.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.906+0000 7f1f03fff700 1 -- 172.21.15.163:0/2774859187 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f1ef4086a80 con 0x7f1f0c0fe0e0 2024-02-16T19:20:14.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.907+0000 7f1f0ad9d700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1ef80692b0 0x7f1ef806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:14.909 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.907+0000 7f1f0ad9d700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1ef80692b0 0x7f1ef806b760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00b800 tx=0x7f1efc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:14.913 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:14.912+0000 7f1f03fff700 1 -- 172.21.15.163:0/2774859187 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1ef4056d30 con 0x7f1f0c0fe0e0 2024-02-16T19:20:15.105 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:15.103+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f0c105c40 con 0x7f1ef80692b0 2024-02-16T19:20:15.798 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:15 smithi163 ceph-mon[36452]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:15.798 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:20:15.798 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:20:15.798 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:15.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:15 smithi040 ceph-mon[29201]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:20:15.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-16T19:20:15.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-16T19:20:15.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:16.873 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:16 smithi163 ceph-mon[36452]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:16.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:16 smithi040 ceph-mon[29201]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:17 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/2119624938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7"}]: dispatch 2024-02-16T19:20:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:17 smithi040 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7"}]: dispatch 2024-02-16T19:20:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:17 smithi040 ceph-mon[29201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7"}]': finished 2024-02-16T19:20:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:17 smithi040 ceph-mon[29201]: osdmap e36: 6 total, 5 up, 6 in 2024-02-16T19:20:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:17 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:17.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:17 smithi163 ceph-mon[36452]: from='client.? 172.21.15.163:0/2119624938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7"}]: dispatch 2024-02-16T19:20:17.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:17 smithi163 ceph-mon[36452]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7"}]: dispatch 2024-02-16T19:20:17.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:17 smithi163 ceph-mon[36452]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7"}]': finished 2024-02-16T19:20:17.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:17 smithi163 ceph-mon[36452]: osdmap e36: 6 total, 5 up, 6 in 2024-02-16T19:20:17.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:17 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:18 smithi040 ceph-mon[29201]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:18 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/1117664869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:20:18.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:18 smithi163 ceph-mon[36452]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:18.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:18 smithi163 ceph-mon[36452]: from='client.? 172.21.15.163:0/1117664869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-16T19:20:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:21 smithi163 ceph-mon[36452]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:21.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:21 smithi040 ceph-mon[29201]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:23.281 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:23 smithi163 ceph-mon[36452]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:23 smithi040 ceph-mon[29201]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:25 smithi163 ceph-mon[36452]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:20:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:20:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:25 smithi040 ceph-mon[29201]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:20:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:25.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:20:26.464 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:26 smithi163 ceph-mon[36452]: Deploying daemon osd.5 on smithi163 2024-02-16T19:20:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:26 smithi040 ceph-mon[29201]: Deploying daemon osd.5 on smithi163 2024-02-16T19:20:27.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:27 smithi163 ceph-mon[36452]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:27.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:27 smithi040 ceph-mon[29201]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:29.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:29 smithi163 ceph-mon[36452]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:29 smithi040 ceph-mon[29201]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:30.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:30 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:30.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:30 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:30.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:30 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:30.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:30 smithi163 ceph-mon[36452]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:31.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:30 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:31.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:30 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:31.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:30 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:31.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:30 smithi040 ceph-mon[29201]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:33.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:33 smithi040 ceph-mon[29201]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:33.404 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:33 smithi163 ceph-mon[36452]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:33.479 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.478+0000 7f1f03fff700 1 -- 172.21.15.163:0/2774859187 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f0c105c40 con 0x7f1ef80692b0 2024-02-16T19:20:33.479 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 5 on host 'smithi163' 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1ef80692b0 msgr2=0x7f1ef806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1ef80692b0 0x7f1ef806b760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00b800 tx=0x7f1efc009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fe0e0 msgr2=0x7f1f0c10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fe0e0 0x7f1f0c10ded0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400b790 tx=0x7f1ef4009250 comp rx=0 tx=0).stop 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 shutdown_connections 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1ef80692b0 0x7f1ef806b760 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c0fcef0 0x7f1f0c10d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 --2- 172.21.15.163:0/2774859187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f0c0fe0e0 0x7f1f0c10ded0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 >> 172.21.15.163:0/2774859187 conn(0x7f1f0c0f8620 msgr2=0x7f1f0c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:33.482 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 shutdown_connections 2024-02-16T19:20:33.483 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:33.481+0000 7f1f11144700 1 -- 172.21.15.163:0/2774859187 wait complete. 2024-02-16T19:20:34.085 DEBUG:teuthology.orchestra.run.smithi163:osd.5> sudo journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.5.service 2024-02-16T19:20:34.087 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-16T19:20:34.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-16T19:20:34.363 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:20:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:34 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:34 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:34 smithi163 ceph-mon[36452]: from='osd.5 [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:20:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:34 smithi163 ceph-mon[36452]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:20:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:34 smithi163 ceph-mon[36452]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:34 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:34 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:34.739 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:34 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:34.740 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:34 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:34.740 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:34 smithi040 ceph-mon[29201]: from='osd.5 [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:20:34.740 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:34 smithi040 ceph-mon[29201]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:20:34.740 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:34 smithi040 ceph-mon[29201]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:34.740 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:34 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:34.740 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:34 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:35 smithi040 ceph-mon[29201]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T19:20:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:35 smithi040 ceph-mon[29201]: osdmap e37: 6 total, 5 up, 6 in 2024-02-16T19:20:35.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:35 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:35.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:35 smithi040 ceph-mon[29201]: from='osd.5 [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:20:35.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:35 smithi040 ceph-mon[29201]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:20:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:35 smithi163 ceph-mon[36452]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T19:20:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:35 smithi163 ceph-mon[36452]: osdmap e37: 6 total, 5 up, 6 in 2024-02-16T19:20:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:35 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:35 smithi163 ceph-mon[36452]: from='osd.5 [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:20:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:35 smithi163 ceph-mon[36452]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:20:35.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:20:35 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[55137]: 2024-02-16T19:20:35.453+0000 7f45ae931700 -1 osd.5 0 waiting for initial osdmap 2024-02-16T19:20:35.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:20:35 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[55137]: 2024-02-16T19:20:35.463+0000 7f45a872a700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 -- 172.21.15.40:0/2328543244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fe0e0 msgr2=0x7f8a280fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 --2- 172.21.15.40:0/2328543244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fe0e0 0x7f8a280fe550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c004940 tx=0x7f8a1c009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 -- 172.21.15.40:0/2328543244 shutdown_connections 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 --2- 172.21.15.40:0/2328543244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fe0e0 0x7f8a280fe550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 --2- 172.21.15.40:0/2328543244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a280fcef0 0x7f8a280fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 -- 172.21.15.40:0/2328543244 >> 172.21.15.40:0/2328543244 conn(0x7f8a280f8620 msgr2=0x7f8a280faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 -- 172.21.15.40:0/2328543244 shutdown_connections 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 -- 172.21.15.40:0/2328543244 wait complete. 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 Processor -- start 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 -- start start 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fcef0 0x7f8a28071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a280fe0e0 0x7f8a280702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280708e0 con 0x7f8a280fcef0 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2ca50700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a28070a50 con 0x7f8a280fe0e0 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.195+0000 7f8a2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fcef0 0x7f8a28071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a25d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a280fe0e0 0x7f8a280702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fcef0 0x7f8a28071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36252/0 (socket says 172.21.15.40:36252) 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a2659c700 1 -- 172.21.15.40:0/223439798 learned_addr learned my addr 172.21.15.40:0/223439798 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a2659c700 1 -- 172.21.15.40:0/223439798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a280fe0e0 msgr2=0x7f8a280702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a2659c700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a280fe0e0 0x7f8a280702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a2659c700 1 -- 172.21.15.40:0/223439798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c005040 con 0x7f8a280fcef0 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a25d9b700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a280fe0e0 0x7f8a280702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a2659c700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fcef0 0x7f8a28071e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800eb70 tx=0x7f8a1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a16ffd700 1 -- 172.21.15.40:0/223439798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a18007cc0 con 0x7f8a280fcef0 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a16ffd700 1 -- 172.21.15.40:0/223439798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a1800f040 con 0x7f8a280fcef0 2024-02-16T19:20:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a28070cd0 con 0x7f8a280fcef0 2024-02-16T19:20:36.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a16ffd700 1 -- 172.21.15.40:0/223439798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a1800cb20 con 0x7f8a280fcef0 2024-02-16T19:20:36.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.196+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a28115920 con 0x7f8a280fcef0 2024-02-16T19:20:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.197+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2804bed0 con 0x7f8a280fcef0 2024-02-16T19:20:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.198+0000 7f8a16ffd700 1 -- 172.21.15.40:0/223439798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8a1800cc80 con 0x7f8a280fcef0 2024-02-16T19:20:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.198+0000 7f8a16ffd700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8a100692a0 0x7f8a1006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.198+0000 7f8a16ffd700 1 -- 172.21.15.40:0/223439798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f8a18086720 con 0x7f8a280fcef0 2024-02-16T19:20:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.198+0000 7f8a25d9b700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8a100692a0 0x7f8a1006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.198+0000 7f8a25d9b700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8a100692a0 0x7f8a1006b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009b30 tx=0x7f8a1c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.203+0000 7f8a16ffd700 1 -- 172.21.15.40:0/223439798 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8a1804dbd0 con 0x7f8a280fcef0 2024-02-16T19:20:36.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.392+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8a28115d50 con 0x7f8a280fcef0 2024-02-16T19:20:36.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.393+0000 7f8a16ffd700 1 -- 172.21.15.40:0/223439798 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7f8a18056940 con 0x7f8a280fcef0 2024-02-16T19:20:36.395 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:20:36.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8a100692a0 msgr2=0x7f8a1006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:36.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8a100692a0 0x7f8a1006b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009b30 tx=0x7f8a1c009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:36.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fcef0 msgr2=0x7f8a28071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:36.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fcef0 0x7f8a28071e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800eb70 tx=0x7f8a1800b040 comp rx=0 tx=0).stop 2024-02-16T19:20:36.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 shutdown_connections 2024-02-16T19:20:36.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8a100692a0 0x7f8a1006b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:36.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a280fcef0 0x7f8a28071e60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:36.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 --2- 172.21.15.40:0/223439798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a280fe0e0 0x7f8a280702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:36.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 >> 172.21.15.40:0/223439798 conn(0x7f8a280f8620 msgr2=0x7f8a281013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:36.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 shutdown_connections 2024-02-16T19:20:36.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:36.395+0000 7f8a2ca50700 1 -- 172.21.15.40:0/223439798 wait complete. 2024-02-16T19:20:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:36 smithi040 ceph-mon[29201]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-02-16T19:20:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:36 smithi040 ceph-mon[29201]: osdmap e38: 6 total, 5 up, 6 in 2024-02-16T19:20:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:36 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:36 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:36 smithi040 ceph-mon[29201]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:36 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/223439798' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T19:20:36.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:36 smithi163 ceph-mon[36452]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-02-16T19:20:36.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:36 smithi163 ceph-mon[36452]: osdmap e38: 6 total, 5 up, 6 in 2024-02-16T19:20:36.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:36 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:36.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:36 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:36.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:36 smithi163 ceph-mon[36452]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-16T19:20:36.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:36 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/223439798' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T19:20:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1708111211,"num_in_osds":6,"osd_in_since":1708111217,"num_remapped_pgs":0} 2024-02-16T19:20:37.579 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:37 smithi163 ceph-mon[36452]: purged_snaps scrub starts 2024-02-16T19:20:37.579 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:37 smithi163 ceph-mon[36452]: purged_snaps scrub ok 2024-02-16T19:20:37.579 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:37 smithi163 ceph-mon[36452]: osd.5 [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] boot 2024-02-16T19:20:37.579 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:37 smithi163 ceph-mon[36452]: osdmap e39: 6 total, 6 up, 6 in 2024-02-16T19:20:37.580 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:37 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:37.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:37 smithi040 ceph-mon[29201]: purged_snaps scrub starts 2024-02-16T19:20:37.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:37 smithi040 ceph-mon[29201]: purged_snaps scrub ok 2024-02-16T19:20:37.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:37 smithi040 ceph-mon[29201]: osd.5 [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] boot 2024-02-16T19:20:37.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:37 smithi040 ceph-mon[29201]: osdmap e39: 6 total, 6 up, 6 in 2024-02-16T19:20:37.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:37 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:20:38.085 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-16T19:20:38.355 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:20:38.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: osdmap e40: 6 total, 6 up, 6 in 2024-02-16T19:20:38.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: Detected new or changed devices on smithi163 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: Adjusting osd_memory_target on smithi163 to 5312M 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:38 smithi163 ceph-mon[36452]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:38.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: osdmap e40: 6 total, 6 up, 6 in 2024-02-16T19:20:38.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: Detected new or changed devices on smithi163 2024-02-16T19:20:38.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: Adjusting osd_memory_target on smithi163 to 5312M 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:20:38.795 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:38 smithi040 ceph-mon[29201]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 -- 172.21.15.40:0/2276594035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fbe40 msgr2=0x7f7e480fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 --2- 172.21.15.40:0/2276594035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fbe40 0x7f7e480fc250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c004940 tx=0x7f7e3c009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 -- 172.21.15.40:0/2276594035 shutdown_connections 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 --2- 172.21.15.40:0/2276594035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e480fcfb0 0x7f7e480fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 --2- 172.21.15.40:0/2276594035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fbe40 0x7f7e480fc250 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 -- 172.21.15.40:0/2276594035 >> 172.21.15.40:0/2276594035 conn(0x7f7e480f7520 msgr2=0x7f7e480f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 -- 172.21.15.40:0/2276594035 shutdown_connections 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 -- 172.21.15.40:0/2276594035 wait complete. 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 Processor -- start 2024-02-16T19:20:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.030+0000 7f7e4eafa700 1 -- start start 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e4eafa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e480fbe40 0x7f7e48112a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e4eafa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fcfb0 0x7f7e48114fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e4eafa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48066be0 con 0x7f7e480fcfb0 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e4eafa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48066d20 con 0x7f7e480fbe40 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fcfb0 0x7f7e48114fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e47fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fcfb0 0x7f7e48114fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51220/0 (socket says 172.21.15.40:51220) 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e47fff700 1 -- 172.21.15.40:0/3843638864 learned_addr learned my addr 172.21.15.40:0/3843638864 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e4c896700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e480fbe40 0x7f7e48112a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e47fff700 1 -- 172.21.15.40:0/3843638864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e480fbe40 msgr2=0x7f7e48112a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e47fff700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e480fbe40 0x7f7e48112a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e47fff700 1 -- 172.21.15.40:0/3843638864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c005040 con 0x7f7e480fcfb0 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e47fff700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fcfb0 0x7f7e48114fc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400b790 tx=0x7f7e34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e457fa700 1 -- 172.21.15.40:0/3843638864 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e3400f6f0 con 0x7f7e480fcfb0 2024-02-16T19:20:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e457fa700 1 -- 172.21.15.40:0/3843638864 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e3400fd20 con 0x7f7e480fcfb0 2024-02-16T19:20:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e48115500 con 0x7f7e480fcfb0 2024-02-16T19:20:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e457fa700 1 -- 172.21.15.40:0/3843638864 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e340103a0 con 0x7f7e480fcfb0 2024-02-16T19:20:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.031+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e48115990 con 0x7f7e480fcfb0 2024-02-16T19:20:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.033+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4804bed0 con 0x7f7e480fcfb0 2024-02-16T19:20:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.033+0000 7f7e457fa700 1 -- 172.21.15.40:0/3843638864 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7e3400f850 con 0x7f7e480fcfb0 2024-02-16T19:20:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.033+0000 7f7e457fa700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7e380692a0 0x7f7e3806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.033+0000 7f7e457fa700 1 -- 172.21.15.40:0/3843638864 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7e34086c30 con 0x7f7e480fcfb0 2024-02-16T19:20:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.033+0000 7f7e4c896700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7e380692a0 0x7f7e3806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.034+0000 7f7e4c896700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7e380692a0 0x7f7e3806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c004890 tx=0x7f7e3c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:40.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.038+0000 7f7e457fa700 1 -- 172.21.15.40:0/3843638864 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7e340532a0 con 0x7f7e480fcfb0 2024-02-16T19:20:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.237+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7e48115e40 con 0x7f7e480fcfb0 2024-02-16T19:20:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.237+0000 7f7e457fa700 1 -- 172.21.15.40:0/3843638864 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f7e340568c0 con 0x7f7e480fcfb0 2024-02-16T19:20:40.238 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:20:40.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7e380692a0 msgr2=0x7f7e3806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7e380692a0 0x7f7e3806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c004890 tx=0x7f7e3c009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fcfb0 msgr2=0x7f7e48114fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fcfb0 0x7f7e48114fc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400b790 tx=0x7f7e34009250 comp rx=0 tx=0).stop 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 shutdown_connections 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7e380692a0 0x7f7e3806b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e480fbe40 0x7f7e48112a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 --2- 172.21.15.40:0/3843638864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e480fcfb0 0x7f7e48114fc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 >> 172.21.15.40:0/3843638864 conn(0x7f7e480f7520 msgr2=0x7f7e48100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 shutdown_connections 2024-02-16T19:20:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:40.239+0000 7f7e4eafa700 1 -- 172.21.15.40:0/3843638864 wait complete. 2024-02-16T19:20:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:20:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:20:40.992 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1708111236,"num_in_osds":6,"osd_in_since":1708111217,"num_remapped_pgs":0} 2024-02-16T19:20:40.993 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T19:20:41.258 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:20:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:41 smithi163 ceph-mon[36452]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:41 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3843638864' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T19:20:41.540 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:41 smithi040 ceph-mon[29201]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:41.540 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:41 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3843638864' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-16T19:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.083+0000 7f4d399a0700 1 -- 172.21.15.40:0/4235742095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 msgr2=0x7f4d340fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.083+0000 7f4d399a0700 1 --2- 172.21.15.40:0/4235742095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 0x7f4d340fe390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d24004940 tx=0x7f4d24009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.083+0000 7f4d399a0700 1 -- 172.21.15.40:0/4235742095 shutdown_connections 2024-02-16T19:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.083+0000 7f4d399a0700 1 --2- 172.21.15.40:0/4235742095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d340fe960 0x7f4d34106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.083+0000 7f4d399a0700 1 --2- 172.21.15.40:0/4235742095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 0x7f4d340fe390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.083+0000 7f4d399a0700 1 -- 172.21.15.40:0/4235742095 >> 172.21.15.40:0/4235742095 conn(0x7f4d340f7a10 msgr2=0x7f4d340f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:43.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.083+0000 7f4d399a0700 1 -- 172.21.15.40:0/4235742095 shutdown_connections 2024-02-16T19:20:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.084+0000 7f4d399a0700 1 -- 172.21.15.40:0/4235742095 wait complete. 2024-02-16T19:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.084+0000 7f4d399a0700 1 Processor -- start 2024-02-16T19:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.084+0000 7f4d399a0700 1 -- start start 2024-02-16T19:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d399a0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 0x7f4d340fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d399a0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d340fe960 0x7f4d340fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d399a0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340fc930 con 0x7f4d340fdfc0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d399a0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340fcaa0 con 0x7f4d340fe960 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 0x7f4d340fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 0x7f4d340fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51232/0 (socket says 172.21.15.40:51232) 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d32ffd700 1 -- 172.21.15.40:0/756076328 learned_addr learned my addr 172.21.15.40:0/756076328 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d327fc700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d340fe960 0x7f4d340fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d32ffd700 1 -- 172.21.15.40:0/756076328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d340fe960 msgr2=0x7f4d340fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d32ffd700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d340fe960 0x7f4d340fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d32ffd700 1 -- 172.21.15.40:0/756076328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d24005040 con 0x7f4d340fdfc0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d32ffd700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 0x7f4d340fbdf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009180 tx=0x7f4d24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.085+0000 7f4d1b7fe700 1 -- 172.21.15.40:0/756076328 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2400e040 con 0x7f4d340fdfc0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.086+0000 7f4d1b7fe700 1 -- 172.21.15.40:0/756076328 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d2400bbf0 con 0x7f4d340fdfc0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.086+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d34119c50 con 0x7f4d340fdfc0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.086+0000 7f4d1b7fe700 1 -- 172.21.15.40:0/756076328 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2401c630 con 0x7f4d340fdfc0 2024-02-16T19:20:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.086+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3411a130 con 0x7f4d340fdfc0 2024-02-16T19:20:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.087+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d341045b0 con 0x7f4d340fdfc0 2024-02-16T19:20:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.087+0000 7f4d1b7fe700 1 -- 172.21.15.40:0/756076328 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4d24013420 con 0x7f4d340fdfc0 2024-02-16T19:20:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.087+0000 7f4d1b7fe700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4d1c0692b0 0x7f4d1c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.087+0000 7f4d1b7fe700 1 -- 172.21.15.40:0/756076328 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4d24088260 con 0x7f4d340fdfc0 2024-02-16T19:20:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.087+0000 7f4d327fc700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4d1c0692b0 0x7f4d1c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.088+0000 7f4d327fc700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4d1c0692b0 0x7f4d1c06b760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d28007fd0 tx=0x7f4d28009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:43.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.092+0000 7f4d1b7fe700 1 -- 172.21.15.40:0/756076328 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4d240548d0 con 0x7f4d340fdfc0 2024-02-16T19:20:43.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.284+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d3404bed0 con 0x7f4d340fdfc0 2024-02-16T19:20:43.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.285+0000 7f4d1b7fe700 1 -- 172.21.15.40:0/756076328 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11071 (secure 0 0 0) 0x7f4d24057ef0 con 0x7f4d340fdfc0 2024-02-16T19:20:43.286 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:20:43.287 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","created":"2024-02-16T19:14:55.436141+0000","modified":"2024-02-16T19:20:37.457815+0000","last_up_change":"2024-02-16T19:20:36.450204+0000","last_in_change":"2024-02-16T19:20:17.443690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:19:22.349243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"db82e60f-540c-4f54-8f54-758be43e2cf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6803","nonce":800063463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6805","nonce":800063463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6809","nonce":800063463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6807","nonce":800063463}]},"public_addr":"172.21.15.40:6803/800063463","cluster_addr":"172.21.15.40:6805/800063463","heartbeat_back_addr":"172.21.15.40:6809/800063463","heartbeat_front_addr":"172.21.15.40:6807/800063463","state":["exists","up"]},{"osd":1,"uuid":"22561508-cdb8-4c76-ab07-bf3efd6ee63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6811","nonce":3466922800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6813","nonce":3466922800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6817","nonce":3466922800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6815","nonce":3466922800}]},"public_addr":"172.21.15.40:6811/3466922800","cluster_addr":"172.21.15.40:6813/3466922800","heartbeat_back_addr":"172.21.15.40:6817/3466922800","heartbeat_front_addr":"172.21.15.40:6815/3466922800","state":["exists","up"]},{"osd":2,"uuid":"58ff6654-72f1-4329-8f6c-4c3180353755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6819","nonce":3340187472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6821","nonce":3340187472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6825","nonce":3340187472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6823","nonce":3340187472}]},"public_addr":"172.21.15.40:6819/3340187472","cluster_addr":"172.21.15.40:6821/3340187472","heartbeat_back_addr":"172.21.15.40:6825/3340187472","heartbeat_front_addr":"172.21.15.40:6823/3340187472","state":["exists","up"]},{"osd":3,"uuid":"b6f2b76f-f1fa-4459-8150-c36e624fb4aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6801","nonce":2273487961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6803","nonce":2273487961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6807","nonce":2273487961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6805","nonce":2273487961}]},"public_addr":"172.21.15.163:6801/2273487961","cluster_addr":"172.21.15.163:6803/2273487961","heartbeat_back_addr":"172.21.15.163:6807/2273487961","heartbeat_front_addr":"172.21.15.163:6805/2273487961","state":["exists","up"]},{"osd":4,"uuid":"49afe3d8-2685-4c11-9830-77bd72a0b272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6809","nonce":1296776670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6811","nonce":1296776670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6815","nonce":1296776670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6813","nonce":1296776670}]},"public_addr":"172.21.15.163:6809/1296776670","cluster_addr":"172.21.15.163:6811/1296776670","heartbeat_back_addr":"172.21.15.163:6815/1296776670","heartbeat_front_addr":"172.21.15.163:6813/1296776670","state":["exists","up"]},{"osd":5,"uuid":"c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6817","nonce":737810550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6819","nonce":737810550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6823","nonce":737810550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6821","nonce":737810550}]},"public_addr":"172.21.15.163:6817/737810550","cluster_addr":"172.21.15.163:6819/737810550","heartbeat_back_addr":"172.21.15.163:6823/737810550","heartbeat_front_addr":"172.21.15.163:6821/737810550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:22.484106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:52.190602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:19.421062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:43.759349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:09.243860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:34.636467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6801/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6800/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2189950867":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/1306942330":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2673381923":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/1754668438":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/4011363422":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1423773760":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1058311844":"2024-02-17T19:15:23.873919+0000","172.21.15.40:6800/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2009141374":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6801/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6801/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2144454600":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6800/2140771334":"2024-02-17T19:15:54.760982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4d1c0692b0 msgr2=0x7f4d1c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4d1c0692b0 0x7f4d1c06b760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d28007fd0 tx=0x7f4d28009040 comp rx=0 tx=0).stop 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 msgr2=0x7f4d340fbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 0x7f4d340fbdf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009180 tx=0x7f4d24009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 shutdown_connections 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4d1c0692b0 0x7f4d1c06b760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d340fdfc0 0x7f4d340fbdf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 --2- 172.21.15.40:0/756076328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d340fe960 0x7f4d340fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 >> 172.21.15.40:0/756076328 conn(0x7f4d340f7a10 msgr2=0x7f4d34101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 shutdown_connections 2024-02-16T19:20:43.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:43.287+0000 7f4d399a0700 1 -- 172.21.15.40:0/756076328 wait complete. 2024-02-16T19:20:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:43 smithi040 ceph-mon[29201]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:43 smithi163 ceph-mon[36452]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:44.001 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:19:22.349243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-02-16T19:20:44.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-16T19:20:44.267 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:20:44.390 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:44 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/756076328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:20:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:44 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/756076328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:20:45.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:45 smithi163 ceph-mon[36452]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:45.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:45 smithi040 ceph-mon[29201]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:46.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 -- 172.21.15.40:0/4127592081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314102800 msgr2=0x7f03140fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:46.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 --2- 172.21.15.40:0/4127592081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314102800 0x7f03140fbe70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0300004940 tx=0x7f0300009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:46.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 -- 172.21.15.40:0/4127592081 shutdown_connections 2024-02-16T19:20:46.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 --2- 172.21.15.40:0/4127592081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314102800 0x7f03140fbe70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 --2- 172.21.15.40:0/4127592081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0314101e60 0x7f0314102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 -- 172.21.15.40:0/4127592081 >> 172.21.15.40:0/4127592081 conn(0x7f03140f7a10 msgr2=0x7f03140f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:46.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 -- 172.21.15.40:0/4127592081 shutdown_connections 2024-02-16T19:20:46.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 -- 172.21.15.40:0/4127592081 wait complete. 2024-02-16T19:20:46.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 Processor -- start 2024-02-16T19:20:46.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.133+0000 7f031ab63700 1 -- start start 2024-02-16T19:20:46.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f031ab63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314101e60 0x7f0314113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f031ab63700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0314102800 0x7f03141140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f031ab63700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03141146f0 con 0x7f0314101e60 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f031ab63700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031410da20 con 0x7f0314102800 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03188ff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314101e60 0x7f0314113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03188ff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314101e60 0x7f0314113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51246/0 (socket says 172.21.15.40:51246) 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03188ff700 1 -- 172.21.15.40:0/4170443615 learned_addr learned my addr 172.21.15.40:0/4170443615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f0313fff700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0314102800 0x7f03141140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03188ff700 1 -- 172.21.15.40:0/4170443615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0314102800 msgr2=0x7f03141140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03188ff700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0314102800 0x7f03141140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03188ff700 1 -- 172.21.15.40:0/4170443615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300005040 con 0x7f0314101e60 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03188ff700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314101e60 0x7f0314113b60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0308002a50 tx=0x7f03080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03117fa700 1 -- 172.21.15.40:0/4170443615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0308007cc0 con 0x7f0314101e60 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031410dd00 con 0x7f0314101e60 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03117fa700 1 -- 172.21.15.40:0/4170443615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f030800f040 con 0x7f0314101e60 2024-02-16T19:20:46.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f03117fa700 1 -- 172.21.15.40:0/4170443615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f030800da30 con 0x7f0314101e60 2024-02-16T19:20:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.134+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031410e210 con 0x7f0314101e60 2024-02-16T19:20:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.135+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03140fb610 con 0x7f0314101e60 2024-02-16T19:20:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.136+0000 7f03117fa700 1 -- 172.21.15.40:0/4170443615 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f030800dbb0 con 0x7f0314101e60 2024-02-16T19:20:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.136+0000 7f03117fa700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03040692a0 0x7f030406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.136+0000 7f03117fa700 1 -- 172.21.15.40:0/4170443615 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0308086c00 con 0x7f0314101e60 2024-02-16T19:20:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.136+0000 7f0313fff700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03040692a0 0x7f030406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:46.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.137+0000 7f0313fff700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03040692a0 0x7f030406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f030000b3b0 tx=0x7f0300009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:46.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.141+0000 7f03117fa700 1 -- 172.21.15.40:0/4170443615 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f030808c020 con 0x7f0314101e60 2024-02-16T19:20:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.331+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0314108f70 con 0x7f0314101e60 2024-02-16T19:20:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.331+0000 7f03117fa700 1 -- 172.21.15.40:0/4170443615 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0308056c80 con 0x7f0314101e60 2024-02-16T19:20:46.333 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.333+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03040692a0 msgr2=0x7f030406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.333+0000 7f031ab63700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03040692a0 0x7f030406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f030000b3b0 tx=0x7f0300009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.333+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314101e60 msgr2=0x7f0314113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.333+0000 7f031ab63700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314101e60 0x7f0314113b60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0308002a50 tx=0x7f03080046b0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.334+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 shutdown_connections 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.334+0000 7f031ab63700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03040692a0 0x7f030406b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.334+0000 7f031ab63700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0314101e60 0x7f0314113b60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.334+0000 7f031ab63700 1 --2- 172.21.15.40:0/4170443615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0314102800 0x7f03141140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.334+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 >> 172.21.15.40:0/4170443615 conn(0x7f03140f7a10 msgr2=0x7f0314066070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.334+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 shutdown_connections 2024-02-16T19:20:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:46.334+0000 7f031ab63700 1 -- 172.21.15.40:0/4170443615 wait complete. 2024-02-16T19:20:47.002 INFO:tasks.cephadm:Setting up client nodes... 2024-02-16T19:20:47.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-16T19:20:47.270 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:20:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:47 smithi163 ceph-mon[36452]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:47 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/4170443615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:20:47.556 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:47 smithi040 ceph-mon[29201]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:47.556 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:47 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/4170443615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:20:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.131+0000 7f6ab031a700 1 -- 172.21.15.40:0/2493023856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fcef0 msgr2=0x7f6aa80fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.131+0000 7f6ab031a700 1 --2- 172.21.15.40:0/2493023856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fcef0 0x7f6aa80fd320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4004940 tx=0x7f6aa4009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 -- 172.21.15.40:0/2493023856 shutdown_connections 2024-02-16T19:20:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 --2- 172.21.15.40:0/2493023856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa80fe0e0 0x7f6aa80fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 --2- 172.21.15.40:0/2493023856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fcef0 0x7f6aa80fd320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 -- 172.21.15.40:0/2493023856 >> 172.21.15.40:0/2493023856 conn(0x7f6aa80f8620 msgr2=0x7f6aa80faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 -- 172.21.15.40:0/2493023856 shutdown_connections 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 -- 172.21.15.40:0/2493023856 wait complete. 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 Processor -- start 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 -- start start 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa80fcef0 0x7f6aa810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fe0e0 0x7f6aa810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa810e560 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.132+0000 7f6ab031a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa810eda0 con 0x7f6aa80fcef0 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6aad8b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fe0e0 0x7f6aa810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6aad8b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fe0e0 0x7f6aa810ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32920/0 (socket says 172.21.15.40:32920) 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6aad8b5700 1 -- 172.21.15.40:0/3572439700 learned_addr learned my addr 172.21.15.40:0/3572439700 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6aae0b6700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa80fcef0 0x7f6aa810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6aad8b5700 1 -- 172.21.15.40:0/3572439700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa80fcef0 msgr2=0x7f6aa810d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6aad8b5700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa80fcef0 0x7f6aa810d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:49.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6aad8b5700 1 -- 172.21.15.40:0/3572439700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4005040 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6aad8b5700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fe0e0 0x7f6aa810ded0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b790 tx=0x7f6a98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6a9effd700 1 -- 172.21.15.40:0/3572439700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9800f6f0 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa810f080 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6a9effd700 1 -- 172.21.15.40:0/3572439700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a9800fd20 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6a9effd700 1 -- 172.21.15.40:0/3572439700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a980103a0 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.133+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa810f5c0 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.134+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa804bed0 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.135+0000 7f6a9effd700 1 -- 172.21.15.40:0/3572439700 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6a98011400 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.135+0000 7f6a9effd700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6a940692a0 0x7f6a9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.135+0000 7f6a9effd700 1 -- 172.21.15.40:0/3572439700 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6a98086b50 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.135+0000 7f6aae0b6700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6a940692a0 0x7f6a9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:49.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.136+0000 7f6aae0b6700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6a940692a0 0x7f6a9406b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4004890 tx=0x7f6aa4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:49.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.139+0000 7f6a9effd700 1 -- 172.21.15.40:0/3572439700 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6a98056cf0 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.409+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f6aa810fa10 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.414+0000 7f6a9effd700 1 -- 172.21.15.40:0/3572439700 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f6a98056880 con 0x7f6aa80fe0e0 2024-02-16T19:20:49.416 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-02-16T19:20:49.416 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQCRtc9lvPWDGBAAOtsUMWQu+mXHbLMH/UnfpQ== 2024-02-16T19:20:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6a940692a0 msgr2=0x7f6a9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6a940692a0 0x7f6a9406b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4004890 tx=0x7f6aa4009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fe0e0 msgr2=0x7f6aa810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fe0e0 0x7f6aa810ded0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b790 tx=0x7f6a98009250 comp rx=0 tx=0).stop 2024-02-16T19:20:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 shutdown_connections 2024-02-16T19:20:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6a940692a0 0x7f6a9406b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa80fcef0 0x7f6aa810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 --2- 172.21.15.40:0/3572439700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa80fe0e0 0x7f6aa810ded0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 >> 172.21.15.40:0/3572439700 conn(0x7f6aa80f8620 msgr2=0x7f6aa81013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 shutdown_connections 2024-02-16T19:20:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:49.417+0000 7f6ab031a700 1 -- 172.21.15.40:0/3572439700 wait complete. 2024-02-16T19:20:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:49 smithi040 ceph-mon[29201]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:49 smithi163 ceph-mon[36452]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:50.153 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:20:50.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-16T19:20:50.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-16T19:20:50.197 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-16T19:20:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:50 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3572439700' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:20:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:50 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3572439700' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T19:20:50.464 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi163/config 2024-02-16T19:20:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:50 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3572439700' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:20:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:50 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3572439700' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T19:20:51.574 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:51 smithi163 ceph-mon[36452]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:51.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:51 smithi040 ceph-mon[29201]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:51.997 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.996+0000 7fb3a87f1700 1 -- 172.21.15.163:0/4061895509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 msgr2=0x7fb3a0102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.996+0000 7fb3a87f1700 1 --2- 172.21.15.163:0/4061895509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 0x7fb3a0102520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb390004940 tx=0x7fb390009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.996+0000 7fb3a87f1700 1 -- 172.21.15.163:0/4061895509 shutdown_connections 2024-02-16T19:20:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.996+0000 7fb3a87f1700 1 --2- 172.21.15.163:0/4061895509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 0x7fb3a0102520 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.996+0000 7fb3a87f1700 1 --2- 172.21.15.163:0/4061895509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0066c10 0x7fb3a00ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.996+0000 7fb3a87f1700 1 -- 172.21.15.163:0/4061895509 >> 172.21.15.163:0/4061895509 conn(0x7fb3a00f7520 msgr2=0x7fb3a00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.996+0000 7fb3a87f1700 1 -- 172.21.15.163:0/4061895509 shutdown_connections 2024-02-16T19:20:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.996+0000 7fb3a87f1700 1 -- 172.21.15.163:0/4061895509 wait complete. 2024-02-16T19:20:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a87f1700 1 Processor -- start 2024-02-16T19:20:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a87f1700 1 -- start start 2024-02-16T19:20:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a87f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0066c10 0x7fb3a010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a87f1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 0x7fb3a010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a87f1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a010e550 con 0x7fb3a0066c10 2024-02-16T19:20:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a87f1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a010ed90 con 0x7fb3a0100120 2024-02-16T19:20:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a5d8c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 0x7fb3a010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:52.000 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a5d8c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 0x7fb3a010dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50034/0 (socket says 172.21.15.163:50034) 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.997+0000 7fb3a5d8c700 1 -- 172.21.15.163:0/1622769461 learned_addr learned my addr 172.21.15.163:0/1622769461 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb3a5d8c700 1 -- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0066c10 msgr2=0x7fb3a010d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb3a5d8c700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0066c10 0x7fb3a010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb3a5d8c700 1 -- 172.21.15.163:0/1622769461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390005040 con 0x7fb3a0100120 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb3a5d8c700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 0x7fb3a010dec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb39000b3b0 tx=0x7fb390009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb396ffd700 1 -- 172.21.15.163:0/1622769461 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb39000e040 con 0x7fb3a0100120 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb396ffd700 1 -- 172.21.15.163:0/1622769461 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb39000bcc0 con 0x7fb3a0100120 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a010f010 con 0x7fb3a0100120 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb396ffd700 1 -- 172.21.15.163:0/1622769461 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb39001c630 con 0x7fb3a0100120 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.998+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a010f550 con 0x7fb3a0100120 2024-02-16T19:20:52.001 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.999+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a0107ad0 con 0x7fb3a0100120 2024-02-16T19:20:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:51.999+0000 7fb396ffd700 1 -- 172.21.15.163:0/1622769461 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb3900139f0 con 0x7fb3a0100120 2024-02-16T19:20:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.000+0000 7fb396ffd700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb38c0692a0 0x7fb38c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.000+0000 7fb396ffd700 1 -- 172.21.15.163:0/1622769461 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb390088300 con 0x7fb3a0100120 2024-02-16T19:20:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.000+0000 7fb3a658d700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb38c0692a0 0x7fb38c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.000+0000 7fb3a658d700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb38c0692a0 0x7fb38c06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006fd0 tx=0x7fb39c006db0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:52.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.004+0000 7fb396ffd700 1 -- 172.21.15.163:0/1622769461 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb390058400 con 0x7fb3a0100120 2024-02-16T19:20:52.262 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.261+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb3a0105a90 con 0x7fb3a0100120 2024-02-16T19:20:52.268 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.267+0000 7fb396ffd700 1 -- 172.21.15.163:0/1622769461 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb390057f90 con 0x7fb3a0100120 2024-02-16T19:20:52.268 INFO:teuthology.orchestra.run.smithi163.stdout:[client.1] 2024-02-16T19:20:52.268 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQCUtc9lPi6uDxAAetzzK07iHw/aCYQ4mKGawg== 2024-02-16T19:20:52.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb38c0692a0 msgr2=0x7fb38c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:52.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb38c0692a0 0x7fb38c06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006fd0 tx=0x7fb39c006db0 comp rx=0 tx=0).stop 2024-02-16T19:20:52.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 msgr2=0x7fb3a010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:52.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 0x7fb3a010dec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb39000b3b0 tx=0x7fb390009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:52.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 shutdown_connections 2024-02-16T19:20:52.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb38c0692a0 0x7fb38c06b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:52.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a0066c10 0x7fb3a010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:52.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 --2- 172.21.15.163:0/1622769461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3a0100120 0x7fb3a010dec0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:52.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 >> 172.21.15.163:0/1622769461 conn(0x7fb3a00f7520 msgr2=0x7fb3a00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:52.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 shutdown_connections 2024-02-16T19:20:52.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-02-16T19:20:52.269+0000 7fb3a87f1700 1 -- 172.21.15.163:0/1622769461 wait complete. 2024-02-16T19:20:52.890 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:20:52.890 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-16T19:20:52.890 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-16T19:20:52.931 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-16T19:20:52.932 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-16T19:20:52.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-16T19:20:53.202 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:20:53.545 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:53 smithi040 ceph-mon[29201]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:53.546 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:53 smithi040 ceph-mon[29201]: from='client.? 172.21.15.163:0/1622769461' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:20:53.546 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:53 smithi040 ceph-mon[29201]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:20:53.546 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:53 smithi040 ceph-mon[29201]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T19:20:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:53 smithi163 ceph-mon[36452]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:53.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:53 smithi163 ceph-mon[36452]: from='client.? 172.21.15.163:0/1622769461' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:20:53.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:53 smithi163 ceph-mon[36452]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-16T19:20:53.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:53 smithi163 ceph-mon[36452]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.701+0000 7ff9123f8700 1 -- 172.21.15.40:0/1476799841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 msgr2=0x7ff90c0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.701+0000 7ff9123f8700 1 --2- 172.21.15.40:0/1476799841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 0x7ff90c0fcf60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff900004940 tx=0x7ff900009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.701+0000 7ff9123f8700 1 -- 172.21.15.40:0/1476799841 shutdown_connections 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.701+0000 7ff9123f8700 1 --2- 172.21.15.40:0/1476799841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 0x7ff90c0fcf60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.701+0000 7ff9123f8700 1 --2- 172.21.15.40:0/1476799841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff90c102f50 0x7ff90c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.701+0000 7ff9123f8700 1 -- 172.21.15.40:0/1476799841 >> 172.21.15.40:0/1476799841 conn(0x7ff90c0f8ae0 msgr2=0x7ff90c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.701+0000 7ff9123f8700 1 -- 172.21.15.40:0/1476799841 shutdown_connections 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.701+0000 7ff9123f8700 1 -- 172.21.15.40:0/1476799841 wait complete. 2024-02-16T19:20:54.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff9123f8700 1 Processor -- start 2024-02-16T19:20:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff9123f8700 1 -- start start 2024-02-16T19:20:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff9123f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff90c102f50 0x7ff90c113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff9123f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 0x7ff90c114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff9123f8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c1146e0 con 0x7ff90c1038f0 2024-02-16T19:20:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff9123f8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c10d980 con 0x7ff90c102f50 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff90b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 0x7ff90c114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff90b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 0x7ff90c114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32940/0 (socket says 172.21.15.40:32940) 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff90b7fe700 1 -- 172.21.15.40:0/2236731859 learned_addr learned my addr 172.21.15.40:0/2236731859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff90bfff700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff90c102f50 0x7ff90c113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff90b7fe700 1 -- 172.21.15.40:0/2236731859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff90c102f50 msgr2=0x7ff90c113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff90b7fe700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff90c102f50 0x7ff90c113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.702+0000 7ff90b7fe700 1 -- 172.21.15.40:0/2236731859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff900005040 con 0x7ff90c1038f0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.703+0000 7ff90b7fe700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 0x7ff90c114000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff900009100 tx=0x7ff900009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.703+0000 7ff908ff9700 1 -- 172.21.15.40:0/2236731859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff90000e040 con 0x7ff90c1038f0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.703+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c10dc00 con 0x7ff90c1038f0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.703+0000 7ff908ff9700 1 -- 172.21.15.40:0/2236731859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff900004290 con 0x7ff90c1038f0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.703+0000 7ff908ff9700 1 -- 172.21.15.40:0/2236731859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff90001c3f0 con 0x7ff90c1038f0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.703+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90c10e140 con 0x7ff90c1038f0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.704+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90c0fc6e0 con 0x7ff90c1038f0 2024-02-16T19:20:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.704+0000 7ff908ff9700 1 -- 172.21.15.40:0/2236731859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff90000bbf0 con 0x7ff90c1038f0 2024-02-16T19:20:54.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.705+0000 7ff908ff9700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff8f40692b0 0x7ff8f406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:54.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.705+0000 7ff90bfff700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff8f40692b0 0x7ff8f406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:54.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.705+0000 7ff908ff9700 1 -- 172.21.15.40:0/2236731859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff900088060 con 0x7ff90c1038f0 2024-02-16T19:20:54.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.705+0000 7ff90bfff700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff8f40692b0 0x7ff8f406b760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009bf0 tx=0x7ff8fc0091a0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:54.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.709+0000 7ff908ff9700 1 -- 172.21.15.40:0/2236731859 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff900058160 con 0x7ff90c1038f0 2024-02-16T19:20:54.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.955+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff90c04bed0 con 0x7ff90c1038f0 2024-02-16T19:20:54.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.958+0000 7ff908ff9700 1 -- 172.21.15.40:0/2236731859 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167169 (secure 0 0 0) 0x7ff900057cf0 con 0x7ff90c1038f0 2024-02-16T19:20:54.962 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff8f40692b0 msgr2=0x7ff8f406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff8f40692b0 0x7ff8f406b760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009bf0 tx=0x7ff8fc0091a0 comp rx=0 tx=0).stop 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 msgr2=0x7ff90c114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 0x7ff90c114000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff900009100 tx=0x7ff900009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 shutdown_connections 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff8f40692b0 0x7ff8f406b760 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff90c102f50 0x7ff90c113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 --2- 172.21.15.40:0/2236731859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff90c1038f0 0x7ff90c114000 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.960+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 >> 172.21.15.40:0/2236731859 conn(0x7ff90c0f8ae0 msgr2=0x7ff90c0fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.961+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 shutdown_connections 2024-02-16T19:20:54.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:54.961+0000 7ff9123f8700 1 -- 172.21.15.40:0/2236731859 wait complete. 2024-02-16T19:20:55.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:55 smithi040 ceph-mon[29201]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:55.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:55 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2236731859' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-16T19:20:55.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:55 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:20:55.639 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"active_gid":14212,"active_name":"smithi040.jzygzo","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1321006798},{"type":"v1","addr":"172.21.15.40:6801","nonce":1321006798}]},"active_addr":"172.21.15.40:6801/1321006798","active_change":"2024-02-16T19:17:10.019116+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14232,"name":"smithi163.wjfclz","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.40:8443/","prometheus":"http://172.21.15.40:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":1412951630}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":250713212}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":2148905528}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":726664255}]}]} 2024-02-16T19:20:55.643 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-16T19:20:55.643 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-16T19:20:55.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T19:20:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:55 smithi163 ceph-mon[36452]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:55.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:55 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2236731859' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-16T19:20:55.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:55 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:20:55.905 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:20:57.526 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:57 smithi040 ceph-mon[29201]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:57.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/538066715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc1027e0 msgr2=0x7f5cbc0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 --2- 172.21.15.40:0/538066715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc1027e0 0x7f5cbc0fbea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4004940 tx=0x7f5ca4009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/538066715 shutdown_connections 2024-02-16T19:20:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 --2- 172.21.15.40:0/538066715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc1027e0 0x7f5cbc0fbea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 --2- 172.21.15.40:0/538066715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cbc101e40 0x7f5cbc102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/538066715 >> 172.21.15.40:0/538066715 conn(0x7f5cbc071d90 msgr2=0x7f5cbc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/538066715 shutdown_connections 2024-02-16T19:20:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/538066715 wait complete. 2024-02-16T19:20:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.527+0000 7f5cc1eb9700 1 Processor -- start 2024-02-16T19:20:57.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cc1eb9700 1 -- start start 2024-02-16T19:20:57.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cc1eb9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc101e40 0x7f5cbc10eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:57.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cc1eb9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cbc1027e0 0x7f5cbc10f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:57.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cc1eb9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc10f600 con 0x7f5cbc101e40 2024-02-16T19:20:57.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cc1eb9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc10f770 con 0x7f5cbc1027e0 2024-02-16T19:20:57.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cbb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc101e40 0x7f5cbc10eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:57.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cbb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc101e40 0x7f5cbc10eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48648/0 (socket says 172.21.15.40:48648) 2024-02-16T19:20:57.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cbb7fe700 1 -- 172.21.15.40:0/1609646851 learned_addr learned my addr 172.21.15.40:0/1609646851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:20:57.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cbb7fe700 1 -- 172.21.15.40:0/1609646851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cbc1027e0 msgr2=0x7f5cbc10f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:57.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cbb7fe700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cbc1027e0 0x7f5cbc10f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:57.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cbb7fe700 1 -- 172.21.15.40:0/1609646851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca4005040 con 0x7f5cbc101e40 2024-02-16T19:20:57.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cbb7fe700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc101e40 0x7f5cbc10eb10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00cb10 tx=0x7f5cac007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:57.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.528+0000 7f5cb3fff700 1 -- 172.21.15.40:0/1609646851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cac004200 con 0x7f5cbc101e40 2024-02-16T19:20:57.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.529+0000 7f5cb3fff700 1 -- 172.21.15.40:0/1609646851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cac00acc0 con 0x7f5cbc101e40 2024-02-16T19:20:57.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.529+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc117ae0 con 0x7f5cbc101e40 2024-02-16T19:20:57.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.529+0000 7f5cb3fff700 1 -- 172.21.15.40:0/1609646851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cac003d90 con 0x7f5cbc101e40 2024-02-16T19:20:57.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.529+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc117ff0 con 0x7f5cbc101e40 2024-02-16T19:20:57.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.530+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc104310 con 0x7f5cbc101e40 2024-02-16T19:20:57.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.530+0000 7f5cb3fff700 1 -- 172.21.15.40:0/1609646851 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5cac00a7f0 con 0x7f5cbc101e40 2024-02-16T19:20:57.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.530+0000 7f5cb3fff700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5ca80692a0 0x7f5ca806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:20:57.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.530+0000 7f5cb3fff700 1 -- 172.21.15.40:0/1609646851 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5cac0874c0 con 0x7f5cbc101e40 2024-02-16T19:20:57.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.530+0000 7f5cbaffd700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5ca80692a0 0x7f5ca806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:20:57.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.531+0000 7f5cbaffd700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5ca80692a0 0x7f5ca806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4004f90 tx=0x7f5ca4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:20:57.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.535+0000 7f5cb3fff700 1 -- 172.21.15.40:0/1609646851 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5cac053ab0 con 0x7f5cbc101e40 2024-02-16T19:20:57.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.727+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cbc04bed0 con 0x7f5cbc101e40 2024-02-16T19:20:57.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.728+0000 7f5cb3fff700 1 -- 172.21.15.40:0/1609646851 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11071 (secure 0 0 0) 0x7f5cac0570d0 con 0x7f5cbc101e40 2024-02-16T19:20:57.730 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:20:57.730 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","created":"2024-02-16T19:14:55.436141+0000","modified":"2024-02-16T19:20:37.457815+0000","last_up_change":"2024-02-16T19:20:36.450204+0000","last_in_change":"2024-02-16T19:20:17.443690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:19:22.349243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"db82e60f-540c-4f54-8f54-758be43e2cf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6803","nonce":800063463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6805","nonce":800063463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6809","nonce":800063463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6807","nonce":800063463}]},"public_addr":"172.21.15.40:6803/800063463","cluster_addr":"172.21.15.40:6805/800063463","heartbeat_back_addr":"172.21.15.40:6809/800063463","heartbeat_front_addr":"172.21.15.40:6807/800063463","state":["exists","up"]},{"osd":1,"uuid":"22561508-cdb8-4c76-ab07-bf3efd6ee63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6811","nonce":3466922800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6813","nonce":3466922800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6817","nonce":3466922800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6815","nonce":3466922800}]},"public_addr":"172.21.15.40:6811/3466922800","cluster_addr":"172.21.15.40:6813/3466922800","heartbeat_back_addr":"172.21.15.40:6817/3466922800","heartbeat_front_addr":"172.21.15.40:6815/3466922800","state":["exists","up"]},{"osd":2,"uuid":"58ff6654-72f1-4329-8f6c-4c3180353755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6819","nonce":3340187472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6821","nonce":3340187472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6825","nonce":3340187472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6823","nonce":3340187472}]},"public_addr":"172.21.15.40:6819/3340187472","cluster_addr":"172.21.15.40:6821/3340187472","heartbeat_back_addr":"172.21.15.40:6825/3340187472","heartbeat_front_addr":"172.21.15.40:6823/3340187472","state":["exists","up"]},{"osd":3,"uuid":"b6f2b76f-f1fa-4459-8150-c36e624fb4aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6801","nonce":2273487961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6803","nonce":2273487961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6807","nonce":2273487961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6805","nonce":2273487961}]},"public_addr":"172.21.15.163:6801/2273487961","cluster_addr":"172.21.15.163:6803/2273487961","heartbeat_back_addr":"172.21.15.163:6807/2273487961","heartbeat_front_addr":"172.21.15.163:6805/2273487961","state":["exists","up"]},{"osd":4,"uuid":"49afe3d8-2685-4c11-9830-77bd72a0b272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6809","nonce":1296776670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6811","nonce":1296776670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6815","nonce":1296776670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6813","nonce":1296776670}]},"public_addr":"172.21.15.163:6809/1296776670","cluster_addr":"172.21.15.163:6811/1296776670","heartbeat_back_addr":"172.21.15.163:6815/1296776670","heartbeat_front_addr":"172.21.15.163:6813/1296776670","state":["exists","up"]},{"osd":5,"uuid":"c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6817","nonce":737810550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6819","nonce":737810550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6823","nonce":737810550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6821","nonce":737810550}]},"public_addr":"172.21.15.163:6817/737810550","cluster_addr":"172.21.15.163:6819/737810550","heartbeat_back_addr":"172.21.15.163:6823/737810550","heartbeat_front_addr":"172.21.15.163:6821/737810550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:22.484106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:52.190602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:19.421062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:43.759349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:09.243860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:34.636467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6801/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6800/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2189950867":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/1306942330":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2673381923":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/1754668438":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/4011363422":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1423773760":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1058311844":"2024-02-17T19:15:23.873919+0000","172.21.15.40:6800/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2009141374":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6801/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6801/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2144454600":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6800/2140771334":"2024-02-17T19:15:54.760982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:20:57.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5ca80692a0 msgr2=0x7f5ca806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5ca80692a0 0x7f5ca806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4004f90 tx=0x7f5ca4009e30 comp rx=0 tx=0).stop 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc101e40 msgr2=0x7f5cbc10eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc101e40 0x7f5cbc10eb10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00cb10 tx=0x7f5cac007f90 comp rx=0 tx=0).stop 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 shutdown_connections 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5ca80692a0 0x7f5ca806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5cbc101e40 0x7f5cbc10eb10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 --2- 172.21.15.40:0/1609646851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cbc1027e0 0x7f5cbc10f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 >> 172.21.15.40:0/1609646851 conn(0x7f5cbc071d90 msgr2=0x7f5cbc0fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 shutdown_connections 2024-02-16T19:20:57.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:20:57.730+0000 7f5cc1eb9700 1 -- 172.21.15.40:0/1609646851 wait complete. 2024-02-16T19:20:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:57 smithi163 ceph-mon[36452]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:58.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:58 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1609646851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:20:58.525 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-16T19:20:58.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-16T19:20:58.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:58 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1609646851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:20:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:20:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:20:59 smithi040 ceph-mon[29201]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:20:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:20:59 smithi163 ceph-mon[36452]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:00.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 -- 172.21.15.40:0/1732168895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 msgr2=0x7fc32c0fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:00.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 --2- 172.21.15.40:0/1732168895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 0x7fc32c0fe320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009ce0 tx=0x7fc31c004d10 comp rx=0 tx=0).stop 2024-02-16T19:21:00.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 -- 172.21.15.40:0/1732168895 shutdown_connections 2024-02-16T19:21:00.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 --2- 172.21.15.40:0/1732168895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fe8f0 0x7fc32c106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:00.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 --2- 172.21.15.40:0/1732168895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 0x7fc32c0fe320 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:00.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 -- 172.21.15.40:0/1732168895 >> 172.21.15.40:0/1732168895 conn(0x7fc32c0f7a10 msgr2=0x7fc32c0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:00.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 -- 172.21.15.40:0/1732168895 shutdown_connections 2024-02-16T19:21:00.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 -- 172.21.15.40:0/1732168895 wait complete. 2024-02-16T19:21:00.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.480+0000 7fc333ec0700 1 Processor -- start 2024-02-16T19:21:00.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc333ec0700 1 -- start start 2024-02-16T19:21:00.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc333ec0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 0x7fc32c115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:00.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc333ec0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fe8f0 0x7fc32c1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:00.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc333ec0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c116800 con 0x7fc32c0fdf50 2024-02-16T19:21:00.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc333ec0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c10fb30 con 0x7fc32c0fe8f0 2024-02-16T19:21:00.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc331c5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 0x7fc32c115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:00.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc331c5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 0x7fc32c115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48658/0 (socket says 172.21.15.40:48658) 2024-02-16T19:21:00.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc331c5c700 1 -- 172.21.15.40:0/959144500 learned_addr learned my addr 172.21.15.40:0/959144500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:00.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc33145b700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fe8f0 0x7fc32c1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:00.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc331c5c700 1 -- 172.21.15.40:0/959144500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fe8f0 msgr2=0x7fc32c1161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:00.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc331c5c700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fe8f0 0x7fc32c1161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:00.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc331c5c700 1 -- 172.21.15.40:0/959144500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c009940 con 0x7fc32c0fdf50 2024-02-16T19:21:00.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.481+0000 7fc331c5c700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 0x7fc32c115c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc31c007c00 tx=0x7fc31c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:00.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.482+0000 7fc32a7fc700 1 -- 172.21.15.40:0/959144500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc31c00e070 con 0x7fc32c0fdf50 2024-02-16T19:21:00.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.482+0000 7fc32a7fc700 1 -- 172.21.15.40:0/959144500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc31c004610 con 0x7fc32c0fdf50 2024-02-16T19:21:00.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.482+0000 7fc32a7fc700 1 -- 172.21.15.40:0/959144500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc31c01b960 con 0x7fc32c0fdf50 2024-02-16T19:21:00.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.482+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c10fdb0 con 0x7fc32c0fdf50 2024-02-16T19:21:00.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.482+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c1102f0 con 0x7fc32c0fdf50 2024-02-16T19:21:00.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.483+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32c106a80 con 0x7fc32c0fdf50 2024-02-16T19:21:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.483+0000 7fc32a7fc700 1 -- 172.21.15.40:0/959144500 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc31c019040 con 0x7fc32c0fdf50 2024-02-16T19:21:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.483+0000 7fc32a7fc700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc3140692a0 0x7fc31406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.484+0000 7fc32a7fc700 1 -- 172.21.15.40:0/959144500 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc31c088550 con 0x7fc32c0fdf50 2024-02-16T19:21:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.484+0000 7fc33145b700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc3140692a0 0x7fc31406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.484+0000 7fc33145b700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc3140692a0 0x7fc31406b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc32c1114a0 tx=0x7fc320009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:00.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.488+0000 7fc32a7fc700 1 -- 172.21.15.40:0/959144500 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc31c0585d0 con 0x7fc32c0fdf50 2024-02-16T19:21:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.684+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc32c04bed0 con 0x7fc32c0fdf50 2024-02-16T19:21:00.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.684+0000 7fc32a7fc700 1 -- 172.21.15.40:0/959144500 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11071 (secure 0 0 0) 0x7fc31c058160 con 0x7fc32c0fdf50 2024-02-16T19:21:00.686 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:21:00.686 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","created":"2024-02-16T19:14:55.436141+0000","modified":"2024-02-16T19:20:37.457815+0000","last_up_change":"2024-02-16T19:20:36.450204+0000","last_in_change":"2024-02-16T19:20:17.443690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:19:22.349243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"db82e60f-540c-4f54-8f54-758be43e2cf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6803","nonce":800063463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6805","nonce":800063463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6809","nonce":800063463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6807","nonce":800063463}]},"public_addr":"172.21.15.40:6803/800063463","cluster_addr":"172.21.15.40:6805/800063463","heartbeat_back_addr":"172.21.15.40:6809/800063463","heartbeat_front_addr":"172.21.15.40:6807/800063463","state":["exists","up"]},{"osd":1,"uuid":"22561508-cdb8-4c76-ab07-bf3efd6ee63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6811","nonce":3466922800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6813","nonce":3466922800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6817","nonce":3466922800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6815","nonce":3466922800}]},"public_addr":"172.21.15.40:6811/3466922800","cluster_addr":"172.21.15.40:6813/3466922800","heartbeat_back_addr":"172.21.15.40:6817/3466922800","heartbeat_front_addr":"172.21.15.40:6815/3466922800","state":["exists","up"]},{"osd":2,"uuid":"58ff6654-72f1-4329-8f6c-4c3180353755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6819","nonce":3340187472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6821","nonce":3340187472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6825","nonce":3340187472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6823","nonce":3340187472}]},"public_addr":"172.21.15.40:6819/3340187472","cluster_addr":"172.21.15.40:6821/3340187472","heartbeat_back_addr":"172.21.15.40:6825/3340187472","heartbeat_front_addr":"172.21.15.40:6823/3340187472","state":["exists","up"]},{"osd":3,"uuid":"b6f2b76f-f1fa-4459-8150-c36e624fb4aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6801","nonce":2273487961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6803","nonce":2273487961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6807","nonce":2273487961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6805","nonce":2273487961}]},"public_addr":"172.21.15.163:6801/2273487961","cluster_addr":"172.21.15.163:6803/2273487961","heartbeat_back_addr":"172.21.15.163:6807/2273487961","heartbeat_front_addr":"172.21.15.163:6805/2273487961","state":["exists","up"]},{"osd":4,"uuid":"49afe3d8-2685-4c11-9830-77bd72a0b272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6809","nonce":1296776670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6811","nonce":1296776670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6815","nonce":1296776670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6813","nonce":1296776670}]},"public_addr":"172.21.15.163:6809/1296776670","cluster_addr":"172.21.15.163:6811/1296776670","heartbeat_back_addr":"172.21.15.163:6815/1296776670","heartbeat_front_addr":"172.21.15.163:6813/1296776670","state":["exists","up"]},{"osd":5,"uuid":"c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6817","nonce":737810550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6819","nonce":737810550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6823","nonce":737810550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6821","nonce":737810550}]},"public_addr":"172.21.15.163:6817/737810550","cluster_addr":"172.21.15.163:6819/737810550","heartbeat_back_addr":"172.21.15.163:6823/737810550","heartbeat_front_addr":"172.21.15.163:6821/737810550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:22.484106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:52.190602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:19.421062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:43.759349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:09.243860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:34.636467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6801/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6800/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2189950867":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/1306942330":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2673381923":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/1754668438":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/4011363422":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1423773760":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1058311844":"2024-02-17T19:15:23.873919+0000","172.21.15.40:6800/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2009141374":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6801/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6801/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2144454600":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6800/2140771334":"2024-02-17T19:15:54.760982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc3140692a0 msgr2=0x7fc31406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc3140692a0 0x7fc31406b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc32c1114a0 tx=0x7fc320009040 comp rx=0 tx=0).stop 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 msgr2=0x7fc32c115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 0x7fc32c115c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc31c007c00 tx=0x7fc31c004d10 comp rx=0 tx=0).stop 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 shutdown_connections 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc3140692a0 0x7fc31406b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fdf50 0x7fc32c115c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 --2- 172.21.15.40:0/959144500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fe8f0 0x7fc32c1161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 >> 172.21.15.40:0/959144500 conn(0x7fc32c0f7a10 msgr2=0x7fc32c0fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 shutdown_connections 2024-02-16T19:21:00.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:00.686+0000 7fc333ec0700 1 -- 172.21.15.40:0/959144500 wait complete. 2024-02-16T19:21:01.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-16T19:21:01.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-16T19:21:01.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-16T19:21:01.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-16T19:21:01.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-16T19:21:01.444 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-16T19:21:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:01 smithi040 ceph-mon[29201]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:01 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/959144500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:21:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:01 smithi163 ceph-mon[36452]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:01 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/959144500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:21:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:01.785 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:03 smithi040 ceph-mon[29201]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:03 smithi163 ceph-mon[36452]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:04.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.510+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1238077869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fe900 msgr2=0x7ff7d4106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.510+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1238077869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fe900 0x7ff7d4106f40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0004940 tx=0x7ff7d0009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.510+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1238077869 shutdown_connections 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.510+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1238077869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fe900 0x7ff7d4106f40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.510+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1238077869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d40fdf60 0x7ff7d40fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.510+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1238077869 >> 172.21.15.40:0/1238077869 conn(0x7ff7d40f7a10 msgr2=0x7ff7d40f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.510+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1238077869 shutdown_connections 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.510+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1238077869 wait complete. 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7db7bd700 1 Processor -- start 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7db7bd700 1 -- start start 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7db7bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fdf60 0x7ff7d4113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7db7bd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d40fe900 0x7ff7d4114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7db7bd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4114730 con 0x7ff7d40fdf60 2024-02-16T19:21:04.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7db7bd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d410d9d0 con 0x7ff7d40fe900 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7d9559700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fdf60 0x7ff7d4113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7d9559700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fdf60 0x7ff7d4113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48664/0 (socket says 172.21.15.40:48664) 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7d9559700 1 -- 172.21.15.40:0/1185909020 learned_addr learned my addr 172.21.15.40:0/1185909020 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7d8d58700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d40fe900 0x7ff7d4114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7d9559700 1 -- 172.21.15.40:0/1185909020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d40fe900 msgr2=0x7ff7d4114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7d9559700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d40fe900 0x7ff7d4114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.511+0000 7ff7d9559700 1 -- 172.21.15.40:0/1185909020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d0005040 con 0x7ff7d40fdf60 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.512+0000 7ff7d9559700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fdf60 0x7ff7d4113b10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b790 tx=0x7ff7c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.512+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c400f770 con 0x7ff7d40fdf60 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.512+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7c400fda0 con 0x7ff7d40fdf60 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.512+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d410dcb0 con 0x7ff7d40fdf60 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.512+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c400d310 con 0x7ff7d40fdf60 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.512+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d410e1f0 con 0x7ff7d40fdf60 2024-02-16T19:21:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.513+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7d410e740 con 0x7ff7d40fdf60 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.513+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff7c4010420 con 0x7ff7d40fdf60 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.513+0000 7ff7c9ffb700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7c00692a0 0x7ff7c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.514+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff7c4086d60 con 0x7ff7d40fdf60 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.514+0000 7ff7d8d58700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7c00692a0 0x7ff7c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.514+0000 7ff7c9ffb700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] conn(0x7ff7c006ef70 0x7ff7c0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.514+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 --> [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7c0071a30 con 0x7ff7c006ef70 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.514+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7c40893d0 con 0x7ff7d40fdf60 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.514+0000 7ff7d9d5a700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] conn(0x7ff7c006ef70 0x7ff7c0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.514+0000 7ff7d8d58700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7c00692a0 0x7ff7c006b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000b3b0 tx=0x7ff7d0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.514+0000 7ff7d9d5a700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] conn(0x7ff7c006ef70 0x7ff7c0071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.515+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 <== osd.1 v2:172.21.15.40:6810/3466922800 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff7c0071a30 con 0x7ff7c006ef70 2024-02-16T19:21:04.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 --> [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff7d40fb610 con 0x7ff7c006ef70 2024-02-16T19:21:04.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7c9ffb700 1 -- 172.21.15.40:0/1185909020 <== osd.1 v2:172.21.15.40:6810/3466922800 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff7d40fb610 con 0x7ff7c006ef70 2024-02-16T19:21:04.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] conn(0x7ff7c006ef70 msgr2=0x7ff7c0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:04.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] conn(0x7ff7c006ef70 0x7ff7c0071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:04.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7c00692a0 msgr2=0x7ff7c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:04.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7c00692a0 0x7ff7c006b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000b3b0 tx=0x7ff7d0009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fdf60 msgr2=0x7ff7d4113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fdf60 0x7ff7d4113b10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b790 tx=0x7ff7c4009250 comp rx=0 tx=0).stop 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 shutdown_connections 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6810/3466922800,v1:172.21.15.40:6811/3466922800] conn(0x7ff7c006ef70 0x7ff7c0071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7c00692a0 0x7ff7c006b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7d40fdf60 0x7ff7d4113b10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 --2- 172.21.15.40:0/1185909020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d40fe900 0x7ff7d4114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.530+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 >> 172.21.15.40:0/1185909020 conn(0x7ff7d40f7a10 msgr2=0x7ff7d41016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.531+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 shutdown_connections 2024-02-16T19:21:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:04.531+0000 7ff7db7bd700 1 -- 172.21.15.40:0/1185909020 wait complete. 2024-02-16T19:21:04.763 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.236+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/316232811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c981027e0 msgr2=0x7f7c980fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.236+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/316232811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c981027e0 0x7f7c980fbea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7c88004940 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.236+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/316232811 shutdown_connections 2024-02-16T19:21:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.236+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/316232811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c981027e0 0x7f7c980fbea0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.236+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/316232811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c98101e40 0x7f7c98102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.236+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/316232811 >> 172.21.15.40:0/316232811 conn(0x7f7c98071d90 msgr2=0x7f7c98072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.236+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/316232811 shutdown_connections 2024-02-16T19:21:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.236+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/316232811 wait complete. 2024-02-16T19:21:05.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.237+0000 7f7c9f3a1700 1 Processor -- start 2024-02-16T19:21:05.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.237+0000 7f7c9f3a1700 1 -- start start 2024-02-16T19:21:05.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9f3a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c98101e40 0x7f7c98112970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:05.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9f3a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981027e0 0x7f7c98112eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:05.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9f3a1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98113500 con 0x7f7c98101e40 2024-02-16T19:21:05.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9f3a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9810c830 con 0x7f7c981027e0 2024-02-16T19:21:05.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9c93c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981027e0 0x7f7c98112eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:05.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9d13d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c98101e40 0x7f7c98112970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:05.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9d13d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c98101e40 0x7f7c98112970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48676/0 (socket says 172.21.15.40:48676) 2024-02-16T19:21:05.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9d13d700 1 -- 172.21.15.40:0/927108323 learned_addr learned my addr 172.21.15.40:0/927108323 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9d13d700 1 -- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981027e0 msgr2=0x7f7c98112eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9d13d700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981027e0 0x7f7c98112eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.238+0000 7f7c9d13d700 1 -- 172.21.15.40:0/927108323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c88005040 con 0x7f7c98101e40 2024-02-16T19:21:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.239+0000 7f7c9d13d700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c98101e40 0x7f7c98112970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7c94002a50 tx=0x7f7c940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.239+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c94007cc0 con 0x7f7c98101e40 2024-02-16T19:21:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.239+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9810ca30 con 0x7f7c98101e40 2024-02-16T19:21:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.239+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c9400f040 con 0x7f7c98101e40 2024-02-16T19:21:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.239+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c9400da30 con 0x7f7c98101e40 2024-02-16T19:21:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.239+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9810cf40 con 0x7f7c98101e40 2024-02-16T19:21:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.240+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c9810d440 con 0x7f7c98101e40 2024-02-16T19:21:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.241+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7c9400db90 con 0x7f7c98101e40 2024-02-16T19:21:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.241+0000 7f7c8dffb700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c840692a0 0x7f7c8406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.242+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7c94086b50 con 0x7f7c98101e40 2024-02-16T19:21:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.242+0000 7f7c8dffb700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] conn(0x7f7c8406ef70 0x7f7c84071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.242+0000 7f7c9c93c700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c840692a0 0x7f7c8406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.242+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 --> [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c84071a30 con 0x7f7c8406ef70 2024-02-16T19:21:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.242+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c9408b220 con 0x7f7c98101e40 2024-02-16T19:21:05.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.242+0000 7f7c9d93e700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] conn(0x7f7c8406ef70 0x7f7c84071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:05.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.242+0000 7f7c9c93c700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c840692a0 0x7f7c8406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b3b0 tx=0x7f7c88009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:05.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.243+0000 7f7c9d93e700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] conn(0x7f7c8406ef70 0x7f7c84071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:05.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.244+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 <== osd.4 v2:172.21.15.163:6808/1296776670 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f7c84071a30 con 0x7f7c8406ef70 2024-02-16T19:21:05.245 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:05.248 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:05.259 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:05.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.263+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 --> [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c980738b0 con 0x7f7c8406ef70 2024-02-16T19:21:05.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.263+0000 7f7c8dffb700 1 -- 172.21.15.40:0/927108323 <== osd.4 v2:172.21.15.163:6808/1296776670 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7c980738b0 con 0x7f7c8406ef70 2024-02-16T19:21:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.264+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] conn(0x7f7c8406ef70 msgr2=0x7f7c84071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.264+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] conn(0x7f7c8406ef70 0x7f7c84071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.264+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c840692a0 msgr2=0x7f7c8406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c840692a0 0x7f7c8406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b3b0 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c98101e40 msgr2=0x7f7c98112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c98101e40 0x7f7c98112970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7c94002a50 tx=0x7f7c940046b0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 shutdown_connections 2024-02-16T19:21:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:6808/1296776670,v1:172.21.15.163:6809/1296776670] conn(0x7f7c8406ef70 0x7f7c84071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c840692a0 0x7f7c8406b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c98101e40 0x7f7c98112970 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 --2- 172.21.15.40:0/927108323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981027e0 0x7f7c98112eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 >> 172.21.15.40:0/927108323 conn(0x7f7c98071d90 msgr2=0x7f7c980fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 shutdown_connections 2024-02-16T19:21:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:05.265+0000 7f7c9f3a1700 1 -- 172.21.15.40:0/927108323 wait complete. 2024-02-16T19:21:05.385 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:05 smithi040 ceph-mon[29201]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:05.540 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:05 smithi163 ceph-mon[36452]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:06.074 INFO:teuthology.orchestra.run.smithi040.stdout:60129542173 2024-02-16T19:21:06.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-16T19:21:06.962 INFO:teuthology.orchestra.run.smithi040.stdout:146028888078 2024-02-16T19:21:06.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-16T19:21:07.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:07 smithi040 ceph-mon[29201]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:07.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:07 smithi163 ceph-mon[36452]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:09.354 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:09 smithi040 ceph-mon[29201]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.457+0000 7f1f86286700 1 -- 172.21.15.40:0/2607790052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 msgr2=0x7f1f800fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.457+0000 7f1f86286700 1 --2- 172.21.15.40:0/2607790052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 0x7f1f800fcf60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f74004940 tx=0x7f1f74009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.457+0000 7f1f86286700 1 -- 172.21.15.40:0/2607790052 shutdown_connections 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.457+0000 7f1f86286700 1 --2- 172.21.15.40:0/2607790052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 0x7f1f800fcf60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.457+0000 7f1f86286700 1 --2- 172.21.15.40:0/2607790052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80102f50 0x7f1f80103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.457+0000 7f1f86286700 1 -- 172.21.15.40:0/2607790052 >> 172.21.15.40:0/2607790052 conn(0x7f1f800f8ae0 msgr2=0x7f1f800faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.457+0000 7f1f86286700 1 -- 172.21.15.40:0/2607790052 shutdown_connections 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.457+0000 7f1f86286700 1 -- 172.21.15.40:0/2607790052 wait complete. 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f86286700 1 Processor -- start 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f86286700 1 -- start start 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f86286700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80102f50 0x7f1f80070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f86286700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 0x7f1f800707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f86286700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f80070e60 con 0x7f1f801038f0 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f86286700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f80070fd0 con 0x7f1f80102f50 2024-02-16T19:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f7f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 0x7f1f800707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f7f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 0x7f1f800707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56460/0 (socket says 172.21.15.40:56460) 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.458+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/3346582025 learned_addr learned my addr 172.21.15.40:0/3346582025 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7ffff700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80102f50 0x7f1f80070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80102f50 msgr2=0x7f1f80070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7f7fe700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80102f50 0x7f1f80070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/3346582025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74005040 con 0x7f1f801038f0 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7ffff700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80102f50 0x7f1f80070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7f7fe700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 0x7f1f800707d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b3b0 tx=0x7f1f74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f7400e040 con 0x7f1f801038f0 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f80119d00 con 0x7f1f801038f0 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f7400bbf0 con 0x7f1f801038f0 2024-02-16T19:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f7401c430 con 0x7f1f801038f0 2024-02-16T19:21:09.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.459+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8011a1f0 con 0x7f1f801038f0 2024-02-16T19:21:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.460+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f800faee0 con 0x7f1f801038f0 2024-02-16T19:21:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.461+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1f7400bd60 con 0x7f1f801038f0 2024-02-16T19:21:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.461+0000 7f1f7cff9700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f680692a0 0x7f1f6806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.461+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1f740882a0 con 0x7f1f801038f0 2024-02-16T19:21:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.461+0000 7f1f7ffff700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f680692a0 0x7f1f6806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.462+0000 7f1f7cff9700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] conn(0x7f1f6806ef70 0x7f1f68071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.462+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 --> [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f68071a30 con 0x7f1f6806ef70 2024-02-16T19:21:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.462+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f7408a7f0 con 0x7f1f801038f0 2024-02-16T19:21:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.462+0000 7f1f84823700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] conn(0x7f1f6806ef70 0x7f1f68071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.462+0000 7f1f7ffff700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f680692a0 0x7f1f6806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f70006fd0 tx=0x7f1f70008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.462+0000 7f1f84823700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] conn(0x7f1f6806ef70 0x7f1f68071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.463+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 <== osd.5 v2:172.21.15.163:6816/737810550 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1f68071a30 con 0x7f1f6806ef70 2024-02-16T19:21:09.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.480+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 --> [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f800fc6e0 con 0x7f1f6806ef70 2024-02-16T19:21:09.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f7cff9700 1 -- 172.21.15.40:0/3346582025 <== osd.5 v2:172.21.15.163:6816/737810550 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1f800fc6e0 con 0x7f1f6806ef70 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] conn(0x7f1f6806ef70 msgr2=0x7f1f68071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] conn(0x7f1f6806ef70 0x7f1f68071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f680692a0 msgr2=0x7f1f6806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f680692a0 0x7f1f6806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f70006fd0 tx=0x7f1f70008040 comp rx=0 tx=0).stop 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 msgr2=0x7f1f800707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 0x7f1f800707d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b3b0 tx=0x7f1f74009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 shutdown_connections 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:6816/737810550,v1:172.21.15.163:6817/737810550] conn(0x7f1f6806ef70 0x7f1f68071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f680692a0 0x7f1f6806b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f80102f50 0x7f1f80070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 --2- 172.21.15.40:0/3346582025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f801038f0 0x7f1f800707d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.481+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 >> 172.21.15.40:0/3346582025 conn(0x7f1f800f8ae0 msgr2=0x7f1f800fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.482+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 shutdown_connections 2024-02-16T19:21:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:09.482+0000 7f1f86286700 1 -- 172.21.15.40:0/3346582025 wait complete. 2024-02-16T19:21:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:09 smithi163 ceph-mon[36452]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:10.596 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:10.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:11.367 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:11 smithi040 ceph-mon[29201]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:11.675 INFO:teuthology.orchestra.run.smithi040.stdout:167503724554 2024-02-16T19:21:11.675 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-16T19:21:11.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:11 smithi163 ceph-mon[36452]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 -- 172.21.15.40:0/3817818705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fe900 msgr2=0x7ff758106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/3817818705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fe900 0x7ff758106fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff748004970 tx=0x7ff748009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 -- 172.21.15.40:0/3817818705 shutdown_connections 2024-02-16T19:21:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/3817818705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fe900 0x7ff758106fd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/3817818705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7580fdf60 0x7ff7580fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 -- 172.21.15.40:0/3817818705 >> 172.21.15.40:0/3817818705 conn(0x7ff7580f79e0 msgr2=0x7ff7580f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 -- 172.21.15.40:0/3817818705 shutdown_connections 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 -- 172.21.15.40:0/3817818705 wait complete. 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 Processor -- start 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.867+0000 7ff75e7e0700 1 -- start start 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff75e7e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fdf60 0x7ff7580fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff75e7e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7580fe900 0x7ff7580fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff75e7e0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580fc960 con 0x7ff7580fdf60 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff75e7e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580fcad0 con 0x7ff7580fe900 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff757fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fdf60 0x7ff7580fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff757fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fdf60 0x7ff7580fbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56482/0 (socket says 172.21.15.40:56482) 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff757fff700 1 -- 172.21.15.40:0/1302418015 learned_addr learned my addr 172.21.15.40:0/1302418015 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff7577fe700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7580fe900 0x7ff7580fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff757fff700 1 -- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7580fe900 msgr2=0x7ff7580fc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff757fff700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7580fe900 0x7ff7580fc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.868+0000 7ff757fff700 1 -- 172.21.15.40:0/1302418015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748005040 con 0x7ff7580fdf60 2024-02-16T19:21:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.869+0000 7ff757fff700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fdf60 0x7ff7580fbe20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff74c002a50 tx=0x7ff74c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.869+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c007cc0 con 0x7ff7580fdf60 2024-02-16T19:21:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.869+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff758119c60 con 0x7ff7580fdf60 2024-02-16T19:21:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.869+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff74c00f040 con 0x7ff7580fdf60 2024-02-16T19:21:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.869+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c00da30 con 0x7ff7580fdf60 2024-02-16T19:21:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.869+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75811a110 con 0x7ff7580fdf60 2024-02-16T19:21:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.870+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff75805e650 con 0x7ff7580fdf60 2024-02-16T19:21:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.870+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff74c00db90 con 0x7ff7580fdf60 2024-02-16T19:21:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.871+0000 7ff754ff9700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7400692a0 0x7ff74006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.871+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff74c086be0 con 0x7ff7580fdf60 2024-02-16T19:21:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.871+0000 7ff7577fe700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7400692a0 0x7ff74006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.871+0000 7ff754ff9700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] conn(0x7ff74006ef70 0x7ff740071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.871+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 --> [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff740071a30 con 0x7ff74006ef70 2024-02-16T19:21:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.871+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff74c08c1e0 con 0x7ff7580fdf60 2024-02-16T19:21:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.871+0000 7ff7577fe700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7400692a0 0x7ff74006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff748000c00 tx=0x7ff748009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.871+0000 7ff75cd7d700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] conn(0x7ff74006ef70 0x7ff740071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.872+0000 7ff75cd7d700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] conn(0x7ff74006ef70 0x7ff740071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.873+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 <== osd.3 v2:172.21.15.163:6800/2273487961 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff740071a30 con 0x7ff74006ef70 2024-02-16T19:21:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.889+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 --> [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff75805ead0 con 0x7ff74006ef70 2024-02-16T19:21:11.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff754ff9700 1 -- 172.21.15.40:0/1302418015 <== osd.3 v2:172.21.15.163:6800/2273487961 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff75805ead0 con 0x7ff74006ef70 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] conn(0x7ff74006ef70 msgr2=0x7ff740071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] conn(0x7ff74006ef70 0x7ff740071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7400692a0 msgr2=0x7ff74006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7400692a0 0x7ff74006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff748000c00 tx=0x7ff748009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fdf60 msgr2=0x7ff7580fbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fdf60 0x7ff7580fbe20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff74c002a50 tx=0x7ff74c0046b0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 shutdown_connections 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:6800/2273487961,v1:172.21.15.163:6801/2273487961] conn(0x7ff74006ef70 0x7ff740071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff7400692a0 0x7ff74006b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7580fdf60 0x7ff7580fbe20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 --2- 172.21.15.40:0/1302418015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7580fe900 0x7ff7580fc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:11.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 >> 172.21.15.40:0/1302418015 conn(0x7ff7580f79e0 msgr2=0x7ff758101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:11.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 shutdown_connections 2024-02-16T19:21:11.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:11.890+0000 7ff75e7e0700 1 -- 172.21.15.40:0/1302418015 wait complete. 2024-02-16T19:21:12.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.422+0000 7f3aa64da700 1 -- 172.21.15.40:0/1316594015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0101e60 msgr2=0x7f3aa0102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:12.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.422+0000 7f3aa64da700 1 --2- 172.21.15.40:0/1316594015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0101e60 0x7f3aa0102230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3a90004940 tx=0x7f3a90009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:12.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.422+0000 7f3aa64da700 1 -- 172.21.15.40:0/1316594015 shutdown_connections 2024-02-16T19:21:12.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.422+0000 7f3aa64da700 1 --2- 172.21.15.40:0/1316594015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aa0102800 0x7f3aa00fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:12.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.422+0000 7f3aa64da700 1 --2- 172.21.15.40:0/1316594015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0101e60 0x7f3aa0102230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:12.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.422+0000 7f3aa64da700 1 -- 172.21.15.40:0/1316594015 >> 172.21.15.40:0/1316594015 conn(0x7f3aa00f7a10 msgr2=0x7f3aa00f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:12.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.422+0000 7f3aa64da700 1 -- 172.21.15.40:0/1316594015 shutdown_connections 2024-02-16T19:21:12.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.422+0000 7f3aa64da700 1 -- 172.21.15.40:0/1316594015 wait complete. 2024-02-16T19:21:12.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3aa64da700 1 Processor -- start 2024-02-16T19:21:12.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3aa64da700 1 -- start start 2024-02-16T19:21:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3aa64da700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aa0101e60 0x7f3aa0113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3aa64da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0102800 0x7f3aa0113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3aa64da700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa01146d0 con 0x7f3aa0102800 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3aa64da700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa010d970 con 0x7f3aa0101e60 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3a9f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0102800 0x7f3aa0113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3a9f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0102800 0x7f3aa0113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56512/0 (socket says 172.21.15.40:56512) 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.423+0000 7f3a9f7fe700 1 -- 172.21.15.40:0/3516099560 learned_addr learned my addr 172.21.15.40:0/3516099560 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3a9ffff700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aa0101e60 0x7f3aa0113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3a9f7fe700 1 -- 172.21.15.40:0/3516099560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aa0101e60 msgr2=0x7f3aa0113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3a9f7fe700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aa0101e60 0x7f3aa0113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3a9f7fe700 1 -- 172.21.15.40:0/3516099560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a90005040 con 0x7f3aa0102800 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3a9f7fe700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0102800 0x7f3aa0113ff0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800b790 tx=0x7f3a88009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8800f6f0 con 0x7f3aa0102800 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa010dc50 con 0x7f3aa0102800 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a8800fd20 con 0x7f3aa0102800 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a880103a0 con 0x7f3aa0102800 2024-02-16T19:21:12.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.424+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa010e190 con 0x7f3aa0102800 2024-02-16T19:21:12.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.425+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3aa010e710 con 0x7f3aa0102800 2024-02-16T19:21:12.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.425+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3a88011400 con 0x7f3aa0102800 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3a9cff9700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3a8c0692a0 0x7f3a8c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3a88086d10 con 0x7f3aa0102800 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3a9ffff700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3a8c0692a0 0x7f3a8c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3a9cff9700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] conn(0x7f3a8c06ef70 0x7f3a8c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 --> [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a8c071a30 con 0x7f3a8c06ef70 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a8808c310 con 0x7f3aa0102800 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3aa4a77700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] conn(0x7f3a8c06ef70 0x7f3a8c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3a9ffff700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3a8c0692a0 0x7f3a8c06b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a90004f90 tx=0x7f3a90009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.426+0000 7f3aa4a77700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] conn(0x7f3a8c06ef70 0x7f3a8c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:12.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.427+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 <== osd.0 v2:172.21.15.40:6802/800063463 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3a8c071a30 con 0x7f3a8c06ef70 2024-02-16T19:21:12.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.442+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 --> [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3aa0066920 con 0x7f3a8c06ef70 2024-02-16T19:21:12.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.442+0000 7f3a9cff9700 1 -- 172.21.15.40:0/3516099560 <== osd.0 v2:172.21.15.40:6802/800063463 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3aa0066920 con 0x7f3a8c06ef70 2024-02-16T19:21:12.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.442+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] conn(0x7f3a8c06ef70 msgr2=0x7f3a8c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:12.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.442+0000 7f3aa64da700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] conn(0x7f3a8c06ef70 0x7f3a8c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:12.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.442+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3a8c0692a0 msgr2=0x7f3a8c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:12.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.442+0000 7f3aa64da700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3a8c0692a0 0x7f3a8c06b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a90004f90 tx=0x7f3a90009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:12.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.442+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0102800 msgr2=0x7f3aa0113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.442+0000 7f3aa64da700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0102800 0x7f3aa0113ff0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800b790 tx=0x7f3a88009220 comp rx=0 tx=0).stop 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.443+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 shutdown_connections 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.443+0000 7f3aa64da700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6802/800063463,v1:172.21.15.40:6803/800063463] conn(0x7f3a8c06ef70 0x7f3a8c071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.443+0000 7f3aa64da700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3a8c0692a0 0x7f3a8c06b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.443+0000 7f3aa64da700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aa0101e60 0x7f3aa0113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.443+0000 7f3aa64da700 1 --2- 172.21.15.40:0/3516099560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3aa0102800 0x7f3aa0113ff0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.443+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 >> 172.21.15.40:0/3516099560 conn(0x7f3aa00f7a10 msgr2=0x7f3aa0065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.443+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 shutdown_connections 2024-02-16T19:21:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:12.443+0000 7f3aa64da700 1 -- 172.21.15.40:0/3516099560 wait complete. 2024-02-16T19:21:13.463 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:13 smithi040 ceph-mon[29201]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:13.489 INFO:teuthology.orchestra.run.smithi040.stdout:111669149716 2024-02-16T19:21:13.489 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-16T19:21:13.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:13 smithi163 ceph-mon[36452]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:14.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 -- 172.21.15.40:0/501863436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 msgr2=0x7fc4940fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:14.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 --2- 172.21.15.40:0/501863436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 0x7fc4940fb9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc484004940 tx=0x7fc484009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:14.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 -- 172.21.15.40:0/501863436 shutdown_connections 2024-02-16T19:21:14.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 --2- 172.21.15.40:0/501863436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 0x7fc4940fb9c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:14.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 --2- 172.21.15.40:0/501863436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc494103140 0x7fc494103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:14.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 -- 172.21.15.40:0/501863436 >> 172.21.15.40:0/501863436 conn(0x7fc494071560 msgr2=0x7fc494071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:14.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 -- 172.21.15.40:0/501863436 shutdown_connections 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 -- 172.21.15.40:0/501863436 wait complete. 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 Processor -- start 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 -- start start 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.260+0000 7fc49ba29700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 0x7fc494113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc49ba29700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc494103140 0x7fc4941140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc49ba29700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc494114720 con 0x7fc4940fb570 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc49ba29700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49410da50 con 0x7fc494103140 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc4997c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 0x7fc494113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc4997c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 0x7fc494113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56530/0 (socket says 172.21.15.40:56530) 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc4997c5700 1 -- 172.21.15.40:0/1076063515 learned_addr learned my addr 172.21.15.40:0/1076063515 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc498fc4700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc494103140 0x7fc4941140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc4997c5700 1 -- 172.21.15.40:0/1076063515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc494103140 msgr2=0x7fc4941140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc4997c5700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc494103140 0x7fc4941140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:14.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc4997c5700 1 -- 172.21.15.40:0/1076063515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc484005040 con 0x7fc4940fb570 2024-02-16T19:21:14.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc4997c5700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 0x7fc494113b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc49000c830 tx=0x7fc490004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:14.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4900106f0 con 0x7fc4940fb570 2024-02-16T19:21:14.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc490010d20 con 0x7fc4940fb570 2024-02-16T19:21:14.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4900113a0 con 0x7fc4940fb570 2024-02-16T19:21:14.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.261+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49410dd30 con 0x7fc4940fb570 2024-02-16T19:21:14.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.262+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49410e1f0 con 0x7fc4940fb570 2024-02-16T19:21:14.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.263+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc49410e740 con 0x7fc4940fb570 2024-02-16T19:21:14.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.263+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc490012400 con 0x7fc4940fb570 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.263+0000 7fc489ffb700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc4800692a0 0x7fc48006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.263+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc490086e10 con 0x7fc4940fb570 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.264+0000 7fc498fc4700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc4800692a0 0x7fc48006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.264+0000 7fc489ffb700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] conn(0x7fc48006ef70 0x7fc480071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.264+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 --> [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc480071a30 con 0x7fc48006ef70 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.264+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc49008c470 con 0x7fc4940fb570 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.264+0000 7fc499fc6700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] conn(0x7fc48006ef70 0x7fc480071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.264+0000 7fc498fc4700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc4800692a0 0x7fc48006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc49410ed80 tx=0x7fc484009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.264+0000 7fc499fc6700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] conn(0x7fc48006ef70 0x7fc480071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:14.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.265+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 <== osd.2 v2:172.21.15.40:6818/3340187472 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc480071a30 con 0x7fc48006ef70 2024-02-16T19:21:14.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.282+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 --> [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc49410e8b0 con 0x7fc48006ef70 2024-02-16T19:21:14.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.282+0000 7fc489ffb700 1 -- 172.21.15.40:0/1076063515 <== osd.2 v2:172.21.15.40:6818/3340187472 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc49410e8b0 con 0x7fc48006ef70 2024-02-16T19:21:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.282+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] conn(0x7fc48006ef70 msgr2=0x7fc480071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.282+0000 7fc49ba29700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] conn(0x7fc48006ef70 0x7fc480071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc4800692a0 msgr2=0x7fc48006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc4800692a0 0x7fc48006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc49410ed80 tx=0x7fc484009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 msgr2=0x7fc494113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 0x7fc494113b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc49000c830 tx=0x7fc490004470 comp rx=0 tx=0).stop 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 shutdown_connections 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6818/3340187472,v1:172.21.15.40:6819/3340187472] conn(0x7fc48006ef70 0x7fc480071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc4800692a0 0x7fc48006b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4940fb570 0x7fc494113b90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 --2- 172.21.15.40:0/1076063515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc494103140 0x7fc4941140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 >> 172.21.15.40:0/1076063515 conn(0x7fc494071560 msgr2=0x7fc4940f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 shutdown_connections 2024-02-16T19:21:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:14.283+0000 7fc49ba29700 1 -- 172.21.15.40:0/1076063515 wait complete. 2024-02-16T19:21:14.721 INFO:teuthology.orchestra.run.smithi040.stdout:38654705700 2024-02-16T19:21:14.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-16T19:21:14.903 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:14.903 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:15.359 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:15 smithi040 ceph-mon[29201]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:15 smithi163 ceph-mon[36452]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:17.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:17 smithi163 ceph-mon[36452]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:17.589 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:17 smithi040 ceph-mon[29201]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:18.390 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:18.783 INFO:teuthology.orchestra.run.smithi040.stdout:81604378650 2024-02-16T19:21:18.783 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-16T19:21:19.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:19 smithi040 ceph-mon[29201]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:19 smithi163 ceph-mon[36452]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 -- 172.21.15.40:0/1544031203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 msgr2=0x7f96880fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 --2- 172.21.15.40:0/1544031203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 0x7f96880fbea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f967c004940 tx=0x7f967c009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 -- 172.21.15.40:0/1544031203 shutdown_connections 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 --2- 172.21.15.40:0/1544031203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 0x7f96880fbea0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 --2- 172.21.15.40:0/1544031203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9688101e40 0x7f9688102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 -- 172.21.15.40:0/1544031203 >> 172.21.15.40:0/1544031203 conn(0x7f9688071d90 msgr2=0x7f9688072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 -- 172.21.15.40:0/1544031203 shutdown_connections 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 -- 172.21.15.40:0/1544031203 wait complete. 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.272+0000 7f968e130700 1 Processor -- start 2024-02-16T19:21:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f968e130700 1 -- start start 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f968e130700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9688101e40 0x7f96880fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f968e130700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 0x7f96880fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f968e130700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880fe600 con 0x7f96881027e0 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f968e130700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880fe770 con 0x7f9688101e40 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f9686ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 0x7f96880fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f9686ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 0x7f96880fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33588/0 (socket says 172.21.15.40:33588) 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f9686ffd700 1 -- 172.21.15.40:0/371140744 learned_addr learned my addr 172.21.15.40:0/371140744 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f96877fe700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9688101e40 0x7f96880fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f9686ffd700 1 -- 172.21.15.40:0/371140744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9688101e40 msgr2=0x7f96880fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f9686ffd700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9688101e40 0x7f96880fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f9686ffd700 1 -- 172.21.15.40:0/371140744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c005040 con 0x7f96881027e0 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f96877fe700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9688101e40 0x7f96880fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f9686ffd700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 0x7f96880fe000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f967c009150 tx=0x7f967c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f966ffff700 1 -- 172.21.15.40:0/371140744 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f967c00e040 con 0x7f96881027e0 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f966ffff700 1 -- 172.21.15.40:0/371140744 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f967c00bbf0 con 0x7f96881027e0 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.273+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9688119cb0 con 0x7f96881027e0 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.274+0000 7f966ffff700 1 -- 172.21.15.40:0/371140744 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f967c01c630 con 0x7f96881027e0 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.274+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968811a110 con 0x7f96881027e0 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.274+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96880738b0 con 0x7f96881027e0 2024-02-16T19:21:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.275+0000 7f966ffff700 1 -- 172.21.15.40:0/371140744 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f967c00bd60 con 0x7f96881027e0 2024-02-16T19:21:20.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.275+0000 7f966ffff700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f96700692a0 0x7f967006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:20.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.275+0000 7f966ffff700 1 -- 172.21.15.40:0/371140744 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f967c0881c0 con 0x7f96881027e0 2024-02-16T19:21:20.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.275+0000 7f96877fe700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f96700692a0 0x7f967006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:20.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.276+0000 7f96877fe700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f96700692a0 0x7f967006b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9678007fd0 tx=0x7f9678009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:20.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.280+0000 7f966ffff700 1 -- 172.21.15.40:0/371140744 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f967c054830 con 0x7f96881027e0 2024-02-16T19:21:20.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.474+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f96880ff200 con 0x7f96881027e0 2024-02-16T19:21:20.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:20 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/66865429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-16T19:21:20.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.475+0000 7f966ffff700 1 -- 172.21.15.40:0/371140744 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f967c057e50 con 0x7f96881027e0 2024-02-16T19:21:20.477 INFO:teuthology.orchestra.run.smithi040.stdout:167503724556 2024-02-16T19:21:20.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f96700692a0 msgr2=0x7f967006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f96700692a0 0x7f967006b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9678007fd0 tx=0x7f9678009040 comp rx=0 tx=0).stop 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 msgr2=0x7f96880fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 0x7f96880fe000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f967c009150 tx=0x7f967c009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 shutdown_connections 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f96700692a0 0x7f967006b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9688101e40 0x7f96880fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 --2- 172.21.15.40:0/371140744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96881027e0 0x7f96880fe000 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.477+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 >> 172.21.15.40:0/371140744 conn(0x7f9688071d90 msgr2=0x7f9688065760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.478+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 shutdown_connections 2024-02-16T19:21:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.478+0000 7f968e130700 1 -- 172.21.15.40:0/371140744 wait complete. 2024-02-16T19:21:20.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:20 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/66865429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-16T19:21:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 -- 172.21.15.40:0/4112718511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe8102f40 msgr2=0x7f2fe8103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 --2- 172.21.15.40:0/4112718511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe8102f40 0x7f2fe8103310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8004940 tx=0x7f2fd8009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 -- 172.21.15.40:0/4112718511 shutdown_connections 2024-02-16T19:21:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 --2- 172.21.15.40:0/4112718511 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe81038e0 0x7f2fe80fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 --2- 172.21.15.40:0/4112718511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe8102f40 0x7f2fe8103310 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 -- 172.21.15.40:0/4112718511 >> 172.21.15.40:0/4112718511 conn(0x7f2fe80f8af0 msgr2=0x7f2fe80faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 -- 172.21.15.40:0/4112718511 shutdown_connections 2024-02-16T19:21:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 -- 172.21.15.40:0/4112718511 wait complete. 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.902+0000 7f2fee9e8700 1 Processor -- start 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fee9e8700 1 -- start start 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fee9e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe8102f40 0x7f2fe8113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fee9e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe81038e0 0x7f2fe8113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fee9e8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8114680 con 0x7f2fe81038e0 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fee9e8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe810d920 con 0x7f2fe8102f40 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe81038e0 0x7f2fe8113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe81038e0 0x7f2fe8113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33606/0 (socket says 172.21.15.40:33606) 2024-02-16T19:21:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe77fe700 1 -- 172.21.15.40:0/3246881788 learned_addr learned my addr 172.21.15.40:0/3246881788 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe7fff700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe8102f40 0x7f2fe8113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe77fe700 1 -- 172.21.15.40:0/3246881788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe8102f40 msgr2=0x7f2fe8113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe77fe700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe8102f40 0x7f2fe8113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe77fe700 1 -- 172.21.15.40:0/3246881788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd8005040 con 0x7f2fe81038e0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe77fe700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe81038e0 0x7f2fe8113fa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00b790 tx=0x7f2fdc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.903+0000 7f2fe4ff9700 1 -- 172.21.15.40:0/3246881788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fdc00f6f0 con 0x7f2fe81038e0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.904+0000 7f2fe4ff9700 1 -- 172.21.15.40:0/3246881788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fdc00fd20 con 0x7f2fe81038e0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.904+0000 7f2fe4ff9700 1 -- 172.21.15.40:0/3246881788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fdc0103a0 con 0x7f2fe81038e0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.904+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe810dc00 con 0x7f2fe81038e0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.904+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe810e0f0 con 0x7f2fe81038e0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.905+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe80fc6f0 con 0x7f2fe81038e0 2024-02-16T19:21:20.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.905+0000 7f2fe4ff9700 1 -- 172.21.15.40:0/3246881788 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2fdc011400 con 0x7f2fe81038e0 2024-02-16T19:21:20.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.905+0000 7f2fe4ff9700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fd00692a0 0x7f2fd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:20.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.905+0000 7f2fe4ff9700 1 -- 172.21.15.40:0/3246881788 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2fdc086d10 con 0x7f2fe81038e0 2024-02-16T19:21:20.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.905+0000 7f2fe7fff700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fd00692a0 0x7f2fd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:20.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.906+0000 7f2fe7fff700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fd00692a0 0x7f2fd006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8004f90 tx=0x7f2fd8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:20.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:20.910+0000 7f2fe4ff9700 1 -- 172.21.15.40:0/3246881788 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2fdc056d90 con 0x7f2fe81038e0 2024-02-16T19:21:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.274+0000 7f82bd7e6700 1 -- 172.21.15.40:0/2191136458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b8102f50 msgr2=0x7f82b8103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.274+0000 7f82bd7e6700 1 --2- 172.21.15.40:0/2191136458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b8102f50 0x7f82b8103320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f82a8004940 tx=0x7f82a8009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.274+0000 7f82bd7e6700 1 -- 172.21.15.40:0/2191136458 shutdown_connections 2024-02-16T19:21:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.274+0000 7f82bd7e6700 1 --2- 172.21.15.40:0/2191136458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b81038f0 0x7f82b80fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.274+0000 7f82bd7e6700 1 --2- 172.21.15.40:0/2191136458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b8102f50 0x7f82b8103320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.274+0000 7f82bd7e6700 1 -- 172.21.15.40:0/2191136458 >> 172.21.15.40:0/2191136458 conn(0x7f82b80f8ae0 msgr2=0x7f82b80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.274+0000 7f82bd7e6700 1 -- 172.21.15.40:0/2191136458 shutdown_connections 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.274+0000 7f82bd7e6700 1 -- 172.21.15.40:0/2191136458 wait complete. 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82bd7e6700 1 Processor -- start 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82bd7e6700 1 -- start start 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82bd7e6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b8102f50 0x7f82b8113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82bd7e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b81038f0 0x7f82b8113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82bd7e6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b81145f0 con 0x7f82b81038f0 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82bd7e6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b810d920 con 0x7f82b8102f50 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82b67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b81038f0 0x7f82b8113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82b67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b81038f0 0x7f82b8113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33572/0 (socket says 172.21.15.40:33572) 2024-02-16T19:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82b67fc700 1 -- 172.21.15.40:0/66865429 learned_addr learned my addr 172.21.15.40:0/66865429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:21.013 INFO:teuthology.orchestra.run.smithi040.stdout:146028888081 2024-02-16T19:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82b67fc700 1 -- 172.21.15.40:0/66865429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b8102f50 msgr2=0x7f82b8113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82b6ffd700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b8102f50 0x7f82b8113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.275+0000 7f82b67fc700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b8102f50 0x7f82b8113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.276+0000 7f82b67fc700 1 -- 172.21.15.40:0/66865429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8005040 con 0x7f82b81038f0 2024-02-16T19:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.276+0000 7f82b67fc700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b81038f0 0x7f82b8113fa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c800 tx=0x7f82ac004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.276+0000 7f829f7fe700 1 -- 172.21.15.40:0/66865429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82ac00a760 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.276+0000 7f829f7fe700 1 -- 172.21.15.40:0/66865429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82ac00ad90 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.276+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b810dc00 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.276+0000 7f829f7fe700 1 -- 172.21.15.40:0/66865429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82ac0113a0 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.276+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b810e140 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.277+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b8100260 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.277+0000 7f829f7fe700 1 -- 172.21.15.40:0/66865429 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f82ac012400 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.278+0000 7f829f7fe700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f82a00692b0 0x7f82a006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.278+0000 7f82b6ffd700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f82a00692b0 0x7f82a006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.278+0000 7f829f7fe700 1 -- 172.21.15.40:0/66865429 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f82ac086f30 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.278+0000 7f82b6ffd700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f82a00692b0 0x7f82a006b760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82a8000c00 tx=0x7f82a8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.283+0000 7f829f7fe700 1 -- 172.21.15.40:0/66865429 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f82ac056f90 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.487+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f82b810a050 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.487+0000 7f829f7fe700 1 -- 172.21.15.40:0/66865429 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82ac056b20 con 0x7f82b81038f0 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f82a00692b0 msgr2=0x7f82a006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f82a00692b0 0x7f82a006b760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82a8000c00 tx=0x7f82a8009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b81038f0 msgr2=0x7f82b8113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b81038f0 0x7f82b8113fa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c800 tx=0x7f82ac004470 comp rx=0 tx=0).stop 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 shutdown_connections 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f82a00692b0 0x7f82a006b760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b8102f50 0x7f82b8113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 --2- 172.21.15.40:0/66865429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b81038f0 0x7f82b8113fa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.489+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 >> 172.21.15.40:0/66865429 conn(0x7f82b80f8ae0 msgr2=0x7f82b80fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.490+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 shutdown_connections 2024-02-16T19:21:21.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:19.490+0000 7f82bd7e6700 1 -- 172.21.15.40:0/66865429 wait complete. 2024-02-16T19:21:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.102+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2fe810f040 con 0x7f2fe81038e0 2024-02-16T19:21:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.102+0000 7f2fe4ff9700 1 -- 172.21.15.40:0/3246881788 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2fdc056920 con 0x7f2fe81038e0 2024-02-16T19:21:21.104 INFO:teuthology.orchestra.run.smithi040.stdout:111669149718 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fd00692a0 msgr2=0x7f2fd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fd00692a0 0x7f2fd006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8004f90 tx=0x7f2fd8009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe81038e0 msgr2=0x7f2fe8113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe81038e0 0x7f2fe8113fa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00b790 tx=0x7f2fdc009220 comp rx=0 tx=0).stop 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 shutdown_connections 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fd00692a0 0x7f2fd006b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe8102f40 0x7f2fe8113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 --2- 172.21.15.40:0/3246881788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe81038e0 0x7f2fe8113fa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 >> 172.21.15.40:0/3246881788 conn(0x7f2fe80f8af0 msgr2=0x7f2fe80fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 shutdown_connections 2024-02-16T19:21:21.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:21.105+0000 7f2fee9e8700 1 -- 172.21.15.40:0/3246881788 wait complete. 2024-02-16T19:21:21.222 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:21 smithi040 ceph-mon[29201]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:21 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/371140744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-16T19:21:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:21 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3246881788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-16T19:21:21.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888081 for osd.4 2024-02-16T19:21:21.726 DEBUG:teuthology.parallel:result is None 2024-02-16T19:21:21.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:21 smithi163 ceph-mon[36452]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:21.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:21 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/371140744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-16T19:21:21.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:21 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3246881788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-16T19:21:22.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149716 got 111669149718 for osd.3 2024-02-16T19:21:22.440 DEBUG:teuthology.parallel:result is None 2024-02-16T19:21:22.860 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724556 for osd.5 2024-02-16T19:21:22.860 DEBUG:teuthology.parallel:result is None 2024-02-16T19:21:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:23 smithi040 ceph-mon[29201]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:23 smithi163 ceph-mon[36452]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:25.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.158+0000 7f41c3458700 1 -- 172.21.15.40:0/1089437646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc1038f0 msgr2=0x7f41bc0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:25.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.158+0000 7f41c3458700 1 --2- 172.21.15.40:0/1089437646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc1038f0 0x7f41bc0fcf60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f41b0004940 tx=0x7f41b0009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:25.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.158+0000 7f41c3458700 1 -- 172.21.15.40:0/1089437646 shutdown_connections 2024-02-16T19:21:25.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.158+0000 7f41c3458700 1 --2- 172.21.15.40:0/1089437646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc1038f0 0x7f41bc0fcf60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:25.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.158+0000 7f41c3458700 1 --2- 172.21.15.40:0/1089437646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41bc102f50 0x7f41bc103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:25.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.158+0000 7f41c3458700 1 -- 172.21.15.40:0/1089437646 >> 172.21.15.40:0/1089437646 conn(0x7f41bc0f8ae0 msgr2=0x7f41bc0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:25.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.159+0000 7f41c3458700 1 -- 172.21.15.40:0/1089437646 shutdown_connections 2024-02-16T19:21:25.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.159+0000 7f41c3458700 1 -- 172.21.15.40:0/1089437646 wait complete. 2024-02-16T19:21:25.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.159+0000 7f41c3458700 1 Processor -- start 2024-02-16T19:21:25.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.159+0000 7f41c3458700 1 -- start start 2024-02-16T19:21:25.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c3458700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc102f50 0x7f41bc113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:25.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c3458700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41bc1038f0 0x7f41bc113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:25.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c3458700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc114600 con 0x7f41bc102f50 2024-02-16T19:21:25.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c3458700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc10d930 con 0x7f41bc1038f0 2024-02-16T19:21:25.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c11f4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc102f50 0x7f41bc113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c11f4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc102f50 0x7f41bc113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33622/0 (socket says 172.21.15.40:33622) 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c11f4700 1 -- 172.21.15.40:0/4163442606 learned_addr learned my addr 172.21.15.40:0/4163442606 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c09f3700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41bc1038f0 0x7f41bc113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c11f4700 1 -- 172.21.15.40:0/4163442606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41bc1038f0 msgr2=0x7f41bc113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c11f4700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41bc1038f0 0x7f41bc113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c11f4700 1 -- 172.21.15.40:0/4163442606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b0005040 con 0x7f41bc102f50 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c11f4700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc102f50 0x7f41bc113a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41ac001d80 tx=0x7f41ac009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41b9ffb700 1 -- 172.21.15.40:0/4163442606 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ac007d10 con 0x7f41bc102f50 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc10dc10 con 0x7f41bc102f50 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41b9ffb700 1 -- 172.21.15.40:0/4163442606 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41ac001e80 con 0x7f41bc102f50 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.160+0000 7f41b9ffb700 1 -- 172.21.15.40:0/4163442606 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ac014640 con 0x7f41bc102f50 2024-02-16T19:21:25.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.161+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc10e120 con 0x7f41bc102f50 2024-02-16T19:21:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.162+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc105420 con 0x7f41bc102f50 2024-02-16T19:21:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.162+0000 7f41b9ffb700 1 -- 172.21.15.40:0/4163442606 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f41ac00e960 con 0x7f41bc102f50 2024-02-16T19:21:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.162+0000 7f41b9ffb700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f41a40692a0 0x7f41a406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.163+0000 7f41b9ffb700 1 -- 172.21.15.40:0/4163442606 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f41ac086bc0 con 0x7f41bc102f50 2024-02-16T19:21:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.163+0000 7f41c09f3700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f41a40692a0 0x7f41a406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:25.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.163+0000 7f41c09f3700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f41a40692a0 0x7f41a406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b3b0 tx=0x7f41b0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:25.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.167+0000 7f41b9ffb700 1 -- 172.21.15.40:0/4163442606 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f41ac04dbd0 con 0x7f41bc102f50 2024-02-16T19:21:25.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.363+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f41bc04bed0 con 0x7f41bc102f50 2024-02-16T19:21:25.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.364+0000 7f41b9ffb700 1 -- 172.21.15.40:0/4163442606 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f41ac056cf0 con 0x7f41bc102f50 2024-02-16T19:21:25.365 INFO:teuthology.orchestra.run.smithi040.stdout:60129542177 2024-02-16T19:21:25.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f41a40692a0 msgr2=0x7f41a406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:25.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f41a40692a0 0x7f41a406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b3b0 tx=0x7f41b0009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:25.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc102f50 msgr2=0x7f41bc113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:25.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc102f50 0x7f41bc113a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41ac001d80 tx=0x7f41ac009ec0 comp rx=0 tx=0).stop 2024-02-16T19:21:25.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 shutdown_connections 2024-02-16T19:21:25.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f41a40692a0 0x7f41a406b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:25.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41bc102f50 0x7f41bc113a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:25.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 --2- 172.21.15.40:0/4163442606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41bc1038f0 0x7f41bc113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:25.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 >> 172.21.15.40:0/4163442606 conn(0x7f41bc0f8ae0 msgr2=0x7f41bc0fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:25.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 shutdown_connections 2024-02-16T19:21:25.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:25.366+0000 7f41c3458700 1 -- 172.21.15.40:0/4163442606 wait complete. 2024-02-16T19:21:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:25 smithi040 ceph-mon[29201]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:25 smithi163 ceph-mon[36452]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:26.607 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:26 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/4163442606' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-16T19:21:26.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:26 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/4163442606' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.792+0000 7f97ff40d700 1 -- 172.21.15.40:0/3366321831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f8101e40 msgr2=0x7f97f8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.792+0000 7f97ff40d700 1 --2- 172.21.15.40:0/3366321831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f8101e40 0x7f97f8102210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009ce0 tx=0x7f97e8004d10 comp rx=0 tx=0).stop 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.792+0000 7f97ff40d700 1 -- 172.21.15.40:0/3366321831 shutdown_connections 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.792+0000 7f97ff40d700 1 --2- 172.21.15.40:0/3366321831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f81027e0 0x7f97f80fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.792+0000 7f97ff40d700 1 --2- 172.21.15.40:0/3366321831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f8101e40 0x7f97f8102210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.792+0000 7f97ff40d700 1 -- 172.21.15.40:0/3366321831 >> 172.21.15.40:0/3366321831 conn(0x7f97f8071d90 msgr2=0x7f97f8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.792+0000 7f97ff40d700 1 -- 172.21.15.40:0/3366321831 shutdown_connections 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.792+0000 7f97ff40d700 1 -- 172.21.15.40:0/3366321831 wait complete. 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97ff40d700 1 Processor -- start 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97ff40d700 1 -- start start 2024-02-16T19:21:26.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97ff40d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f8101e40 0x7f97f8115e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97ff40d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f81027e0 0x7f97f8116360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97ff40d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81169b0 con 0x7f97f81027e0 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97ff40d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f810fcc0 con 0x7f97f8101e40 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97fc9a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f81027e0 0x7f97f8116360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97fc9a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f81027e0 0x7f97f8116360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33638/0 (socket says 172.21.15.40:33638) 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97fc9a8700 1 -- 172.21.15.40:0/2715555160 learned_addr learned my addr 172.21.15.40:0/2715555160 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97fd1a9700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f8101e40 0x7f97f8115e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97fc9a8700 1 -- 172.21.15.40:0/2715555160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f8101e40 msgr2=0x7f97f8115e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97fc9a8700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f8101e40 0x7f97f8115e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97fc9a8700 1 -- 172.21.15.40:0/2715555160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8009940 con 0x7f97f81027e0 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.793+0000 7f97fc9a8700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f81027e0 0x7f97f8116360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00c800 tx=0x7f97ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.794+0000 7f97f5ffb700 1 -- 172.21.15.40:0/2715555160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97ec0106f0 con 0x7f97f81027e0 2024-02-16T19:21:26.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.794+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f810ffa0 con 0x7f97f81027e0 2024-02-16T19:21:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.794+0000 7f97f5ffb700 1 -- 172.21.15.40:0/2715555160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97ec010d20 con 0x7f97f81027e0 2024-02-16T19:21:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.794+0000 7f97f5ffb700 1 -- 172.21.15.40:0/2715555160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97ec0113a0 con 0x7f97f81027e0 2024-02-16T19:21:26.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.794+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f81104e0 con 0x7f97f81027e0 2024-02-16T19:21:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.795+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f80738b0 con 0x7f97f81027e0 2024-02-16T19:21:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.795+0000 7f97f5ffb700 1 -- 172.21.15.40:0/2715555160 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f97ec012400 con 0x7f97f81027e0 2024-02-16T19:21:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.796+0000 7f97f5ffb700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f97e0069370 0x7f97e006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.796+0000 7f97f5ffb700 1 -- 172.21.15.40:0/2715555160 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f97ec086d90 con 0x7f97f81027e0 2024-02-16T19:21:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.796+0000 7f97fd1a9700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f97e0069370 0x7f97e006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:26.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.796+0000 7f97fd1a9700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f97e0069370 0x7f97e006b820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97e8015040 tx=0x7f97e8004d10 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:26.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.800+0000 7f97f5ffb700 1 -- 172.21.15.40:0/2715555160 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f97ec056e70 con 0x7f97f81027e0 2024-02-16T19:21:26.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.995+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f97f8108f40 con 0x7f97f81027e0 2024-02-16T19:21:26.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.995+0000 7f97f5ffb700 1 -- 172.21.15.40:0/2715555160 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97ec056a00 con 0x7f97f81027e0 2024-02-16T19:21:26.997 INFO:teuthology.orchestra.run.smithi040.stdout:38654705703 2024-02-16T19:21:26.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f97e0069370 msgr2=0x7f97e006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f97e0069370 0x7f97e006b820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97e8015040 tx=0x7f97e8004d10 comp rx=0 tx=0).stop 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f81027e0 msgr2=0x7f97f8116360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f81027e0 0x7f97f8116360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00c800 tx=0x7f97ec004470 comp rx=0 tx=0).stop 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 shutdown_connections 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f97e0069370 0x7f97e006b820 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f8101e40 0x7f97f8115e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 --2- 172.21.15.40:0/2715555160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97f81027e0 0x7f97f8116360 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 >> 172.21.15.40:0/2715555160 conn(0x7f97f8071d90 msgr2=0x7f97f80656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 shutdown_connections 2024-02-16T19:21:26.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:26.997+0000 7f97ff40d700 1 -- 172.21.15.40:0/2715555160 wait complete. 2024-02-16T19:21:27.623 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:27 smithi040 ceph-mon[29201]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:27.623 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:27 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2715555160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-16T19:21:27.649 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705703 for osd.0 2024-02-16T19:21:27.649 DEBUG:teuthology.parallel:result is None 2024-02-16T19:21:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:27 smithi163 ceph-mon[36452]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:27 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2715555160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-16T19:21:28.026 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542177 for osd.1 2024-02-16T19:21:28.026 DEBUG:teuthology.parallel:result is None 2024-02-16T19:21:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.290+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/1126439367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c781038f0 msgr2=0x7f9c780fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.290+0000 7f9c7d1cd700 1 --2- 172.21.15.40:0/1126439367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c781038f0 0x7f9c780fcf60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c60004940 tx=0x7f9c60009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.290+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/1126439367 shutdown_connections 2024-02-16T19:21:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.290+0000 7f9c7d1cd700 1 --2- 172.21.15.40:0/1126439367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c781038f0 0x7f9c780fcf60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.290+0000 7f9c7d1cd700 1 --2- 172.21.15.40:0/1126439367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c78102f50 0x7f9c78103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.290+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/1126439367 >> 172.21.15.40:0/1126439367 conn(0x7f9c780f8ae0 msgr2=0x7f9c780faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.290+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/1126439367 shutdown_connections 2024-02-16T19:21:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.290+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/1126439367 wait complete. 2024-02-16T19:21:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c7d1cd700 1 Processor -- start 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c7d1cd700 1 -- start start 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c7d1cd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78102f50 0x7f9c78113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c7d1cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c781038f0 0x7f9c78114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c7d1cd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781146e0 con 0x7f9c78102f50 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c7d1cd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7810d980 con 0x7f9c781038f0 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c76d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78102f50 0x7f9c78113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c76d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78102f50 0x7f9c78113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33164/0 (socket says 172.21.15.40:33164) 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c76d9d700 1 -- 172.21.15.40:0/170937473 learned_addr learned my addr 172.21.15.40:0/170937473 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c7659c700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c781038f0 0x7f9c78114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c76d9d700 1 -- 172.21.15.40:0/170937473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c781038f0 msgr2=0x7f9c78114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c76d9d700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c781038f0 0x7f9c78114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.291+0000 7f9c76d9d700 1 -- 172.21.15.40:0/170937473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c60005040 con 0x7f9c78102f50 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.292+0000 7f9c76d9d700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78102f50 0x7f9c78113ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800bbb0 tx=0x7f9c68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.292+0000 7f9c6f7fe700 1 -- 172.21.15.40:0/170937473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c680092e0 con 0x7f9c78102f50 2024-02-16T19:21:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.292+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7810dc60 con 0x7f9c78102f50 2024-02-16T19:21:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.292+0000 7f9c6f7fe700 1 -- 172.21.15.40:0/170937473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c6800cb20 con 0x7f9c78102f50 2024-02-16T19:21:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.292+0000 7f9c6f7fe700 1 -- 172.21.15.40:0/170937473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c68015d70 con 0x7f9c78102f50 2024-02-16T19:21:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.292+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7810e170 con 0x7f9c78102f50 2024-02-16T19:21:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.293+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c780fc6e0 con 0x7f9c78102f50 2024-02-16T19:21:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.293+0000 7f9c6f7fe700 1 -- 172.21.15.40:0/170937473 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9c68012070 con 0x7f9c78102f50 2024-02-16T19:21:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.294+0000 7f9c6f7fe700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9c640692a0 0x7f9c6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.294+0000 7f9c6f7fe700 1 -- 172.21.15.40:0/170937473 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9c68086b90 con 0x7f9c78102f50 2024-02-16T19:21:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.294+0000 7f9c7659c700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9c640692a0 0x7f9c6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.294+0000 7f9c7659c700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9c640692a0 0x7f9c6406b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c7810ecb0 tx=0x7f9c60009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.298+0000 7f9c6f7fe700 1 -- 172.21.15.40:0/170937473 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9c68056c10 con 0x7f9c78102f50 2024-02-16T19:21:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.491+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9c7810f0a0 con 0x7f9c78102f50 2024-02-16T19:21:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.491+0000 7f9c6f7fe700 1 -- 172.21.15.40:0/170937473 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9c680567a0 con 0x7f9c78102f50 2024-02-16T19:21:28.493 INFO:teuthology.orchestra.run.smithi040.stdout:81604378653 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9c640692a0 msgr2=0x7f9c6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9c640692a0 0x7f9c6406b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c7810ecb0 tx=0x7f9c60009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78102f50 msgr2=0x7f9c78113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78102f50 0x7f9c78113ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800bbb0 tx=0x7f9c68009840 comp rx=0 tx=0).stop 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 shutdown_connections 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9c640692a0 0x7f9c6406b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c78102f50 0x7f9c78113ac0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 --2- 172.21.15.40:0/170937473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c781038f0 0x7f9c78114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 >> 172.21.15.40:0/170937473 conn(0x7f9c780f8ae0 msgr2=0x7f9c780fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 shutdown_connections 2024-02-16T19:21:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:28.493+0000 7f9c7d1cd700 1 -- 172.21.15.40:0/170937473 wait complete. 2024-02-16T19:21:29.132 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378650 got 81604378653 for osd.2 2024-02-16T19:21:29.133 DEBUG:teuthology.parallel:result is None 2024-02-16T19:21:29.133 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-16T19:21:29.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-16T19:21:29.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:29 smithi040 ceph-mon[29201]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:29.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:29 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/170937473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-16T19:21:29.398 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:29 smithi163 ceph-mon[36452]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:29 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/170937473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-16T19:21:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 -- 172.21.15.40:0/1961043772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fdf70 msgr2=0x7fb9d40fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 --2- 172.21.15.40:0/1961043772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fdf70 0x7fb9d40fe340 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc004970 tx=0x7fb9bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 -- 172.21.15.40:0/1961043772 shutdown_connections 2024-02-16T19:21:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 --2- 172.21.15.40:0/1961043772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9d40fe910 0x7fb9d4106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:31.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 --2- 172.21.15.40:0/1961043772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fdf70 0x7fb9d40fe340 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 -- 172.21.15.40:0/1961043772 >> 172.21.15.40:0/1961043772 conn(0x7fb9d40f7a10 msgr2=0x7fb9d40f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 -- 172.21.15.40:0/1961043772 shutdown_connections 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 -- 172.21.15.40:0/1961043772 wait complete. 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.050+0000 7fb9da741700 1 Processor -- start 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9da741700 1 -- start start 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9da741700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9d40fdf70 0x7fb9d4113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9da741700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fe910 0x7fb9d4114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9da741700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d41146f0 con 0x7fb9d40fe910 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9da741700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d410d990 con 0x7fb9d40fdf70 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9d37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fe910 0x7fb9d4114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9d37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fe910 0x7fb9d4114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33176/0 (socket says 172.21.15.40:33176) 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9d37fe700 1 -- 172.21.15.40:0/3965421871 learned_addr learned my addr 172.21.15.40:0/3965421871 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9d3fff700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9d40fdf70 0x7fb9d4113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9d37fe700 1 -- 172.21.15.40:0/3965421871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9d40fdf70 msgr2=0x7fb9d4113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:31.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9d37fe700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9d40fdf70 0x7fb9d4113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.051+0000 7fb9d37fe700 1 -- 172.21.15.40:0/3965421871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc005040 con 0x7fb9d40fe910 2024-02-16T19:21:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.052+0000 7fb9d37fe700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fe910 0x7fb9d4114010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400b790 tx=0x7fb9c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.052+0000 7fb9d0ff9700 1 -- 172.21.15.40:0/3965421871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9c400f6f0 con 0x7fb9d40fe910 2024-02-16T19:21:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.052+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d410dc70 con 0x7fb9d40fe910 2024-02-16T19:21:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.052+0000 7fb9d0ff9700 1 -- 172.21.15.40:0/3965421871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9c400fd20 con 0x7fb9d40fe910 2024-02-16T19:21:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.052+0000 7fb9d0ff9700 1 -- 172.21.15.40:0/3965421871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9c40103a0 con 0x7fb9d40fe910 2024-02-16T19:21:31.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.052+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d410e1b0 con 0x7fb9d40fe910 2024-02-16T19:21:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.053+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d4104680 con 0x7fb9d40fe910 2024-02-16T19:21:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.053+0000 7fb9d0ff9700 1 -- 172.21.15.40:0/3965421871 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb9c4011400 con 0x7fb9d40fe910 2024-02-16T19:21:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.054+0000 7fb9d0ff9700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb9c00692a0 0x7fb9c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.054+0000 7fb9d0ff9700 1 -- 172.21.15.40:0/3965421871 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb9c4086c20 con 0x7fb9d40fe910 2024-02-16T19:21:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.054+0000 7fb9d3fff700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb9c00692a0 0x7fb9c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:31.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.054+0000 7fb9d3fff700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb9c00692a0 0x7fb9c006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc000c00 tx=0x7fb9bc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:31.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.058+0000 7fb9d0ff9700 1 -- 172.21.15.40:0/3965421871 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb9c4056d20 con 0x7fb9d40fe910 2024-02-16T19:21:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.246+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9d410a0d0 con 0x7fb9c00692a0 2024-02-16T19:21:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.248+0000 7fb9d0ff9700 1 -- 172.21.15.40:0/3965421871 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28208 (secure 0 0 0) 0x7fb9d410a0d0 con 0x7fb9c00692a0 2024-02-16T19:21:31.249 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb9c00692a0 msgr2=0x7fb9c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb9c00692a0 0x7fb9c006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc000c00 tx=0x7fb9bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fe910 msgr2=0x7fb9d4114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fe910 0x7fb9d4114010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400b790 tx=0x7fb9c4009250 comp rx=0 tx=0).stop 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 shutdown_connections 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb9c00692a0 0x7fb9c006b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9d40fdf70 0x7fb9d4113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 --2- 172.21.15.40:0/3965421871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d40fe910 0x7fb9d4114010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 >> 172.21.15.40:0/3965421871 conn(0x7fb9d40f7a10 msgr2=0x7fb9d4101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:31.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 shutdown_connections 2024-02-16T19:21:31.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:31.250+0000 7fb9da741700 1 -- 172.21.15.40:0/3965421871 wait complete. 2024-02-16T19:21:31.252 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-02-16T19:21:31.544 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:31 smithi040 ceph-mon[29201]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:31 smithi163 ceph-mon[36452]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:31.854 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-02-16T19:21:30.066835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787600,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683312,"statfs":{"total":575970213888,"available":574139711488,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002426"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-02-16T19:20:36.455691+0000","last_change":"2024-02-16T19:20:11.179111+0000","last_active":"2024-02-16T19:20:36.455691+0000","last_peered":"2024-02-16T19:20:36.455691+0000","last_clean":"2024-02-16T19:20:36.455691+0000","last_became_active":"2024-02-16T19:20:11.178803+0000","last_became_peered":"2024-02-16T19:20:11.178803+0000","last_unstale":"2024-02-16T19:20:36.455691+0000","last_undegraded":"2024-02-16T19:20:36.455691+0000","last_fullsized":"2024-02-16T19:20:36.455691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-16T19:19:22.349668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-16T19:19:22.349668+0000","last_clean_scrub_stamp":"2024-02-16T19:19:22.349668+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T03:39:21.626756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.438},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.495}]},{"osd":2,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.315}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.49199999999999999}]},{"osd":1,"last update":"Fri Feb 16 19:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Fri Feb 16 19:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447512,"statfs":{"total":95995035648,"available":95690252288,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.29399999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Feb 16 19:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.25700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.46500000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.36499999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.47299999999999998}]},{"osd":3,"last update":"Fri Feb 16 19:20:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 16 19:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.23599999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.48699999999999999,"5min":0.495,"15min":0.495},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.27500000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Feb 16 19:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38800000000000001}]},{"osd":3,"last update":"Fri Feb 16 19:20:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.25900000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33000000000000002},"min":{"1min":0.24299999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.312}]},{"osd":2,"last update":"Fri Feb 16 19:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Feb 16 19:20:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Feb 16 19:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.83599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-16T19:21:31.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-16T19:21:32.118 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:32.514 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:32 smithi040 ceph-mon[29201]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:21:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:32 smithi163 ceph-mon[36452]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:21:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:33 smithi040 ceph-mon[29201]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:33 smithi163 ceph-mon[36452]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.745+0000 7efc4862c700 1 -- 172.21.15.40:0/446238363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc401038f0 msgr2=0x7efc400fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.745+0000 7efc4862c700 1 --2- 172.21.15.40:0/446238363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc401038f0 0x7efc400fcf60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efc30004940 tx=0x7efc30009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.745+0000 7efc4862c700 1 -- 172.21.15.40:0/446238363 shutdown_connections 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.745+0000 7efc4862c700 1 --2- 172.21.15.40:0/446238363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc401038f0 0x7efc400fcf60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.745+0000 7efc4862c700 1 --2- 172.21.15.40:0/446238363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc40102f50 0x7efc40103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.745+0000 7efc4862c700 1 -- 172.21.15.40:0/446238363 >> 172.21.15.40:0/446238363 conn(0x7efc400f8ae0 msgr2=0x7efc400faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.745+0000 7efc4862c700 1 -- 172.21.15.40:0/446238363 shutdown_connections 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.745+0000 7efc4862c700 1 -- 172.21.15.40:0/446238363 wait complete. 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc4862c700 1 Processor -- start 2024-02-16T19:21:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc4862c700 1 -- start start 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc4862c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc40102f50 0x7efc40113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc4862c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc401038f0 0x7efc40113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc4862c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc40114620 con 0x7efc40102f50 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc4862c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4010d950 con 0x7efc401038f0 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc463c8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc40102f50 0x7efc40113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc463c8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc40102f50 0x7efc40113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33196/0 (socket says 172.21.15.40:33196) 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc463c8700 1 -- 172.21.15.40:0/488616979 learned_addr learned my addr 172.21.15.40:0/488616979 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc45bc7700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc401038f0 0x7efc40113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc463c8700 1 -- 172.21.15.40:0/488616979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc401038f0 msgr2=0x7efc40113fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc463c8700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc401038f0 0x7efc40113fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.746+0000 7efc463c8700 1 -- 172.21.15.40:0/488616979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc30005040 con 0x7efc40102f50 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.747+0000 7efc463c8700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc40102f50 0x7efc40113a90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efc3c002a50 tx=0x7efc3c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.747+0000 7efc36ffd700 1 -- 172.21.15.40:0/488616979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3c007cc0 con 0x7efc40102f50 2024-02-16T19:21:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.747+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4010dc30 con 0x7efc40102f50 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.747+0000 7efc36ffd700 1 -- 172.21.15.40:0/488616979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc3c00f040 con 0x7efc40102f50 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.747+0000 7efc36ffd700 1 -- 172.21.15.40:0/488616979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3c009640 con 0x7efc40102f50 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.747+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4010e140 con 0x7efc40102f50 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.748+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc40105420 con 0x7efc40102f50 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.748+0000 7efc36ffd700 1 -- 172.21.15.40:0/488616979 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efc3c0097a0 con 0x7efc40102f50 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.749+0000 7efc36ffd700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7efc2c0692b0 0x7efc2c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.749+0000 7efc45bc7700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7efc2c0692b0 0x7efc2c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.749+0000 7efc36ffd700 1 -- 172.21.15.40:0/488616979 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efc3c086c80 con 0x7efc40102f50 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.749+0000 7efc45bc7700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7efc2c0692b0 0x7efc2c06b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc4010ec80 tx=0x7efc30009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.753+0000 7efc36ffd700 1 -- 172.21.15.40:0/488616979 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7efc3c056d00 con 0x7efc40102f50 2024-02-16T19:21:33.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.940+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc4004bf40 con 0x7efc2c0692b0 2024-02-16T19:21:33.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.942+0000 7efc36ffd700 1 -- 172.21.15.40:0/488616979 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28192 (secure 0 0 0) 0x7efc4004bf40 con 0x7efc2c0692b0 2024-02-16T19:21:33.943 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7efc2c0692b0 msgr2=0x7efc2c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7efc2c0692b0 0x7efc2c06b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc4010ec80 tx=0x7efc30009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc40102f50 msgr2=0x7efc40113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc40102f50 0x7efc40113a90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efc3c002a50 tx=0x7efc3c0046b0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 shutdown_connections 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7efc2c0692b0 0x7efc2c06b760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc40102f50 0x7efc40113a90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 --2- 172.21.15.40:0/488616979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc401038f0 0x7efc40113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 >> 172.21.15.40:0/488616979 conn(0x7efc400f8ae0 msgr2=0x7efc400fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 shutdown_connections 2024-02-16T19:21:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:33.944+0000 7efc4862c700 1 -- 172.21.15.40:0/488616979 wait complete. 2024-02-16T19:21:33.947 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-02-16T19:21:34.595 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-02-16T19:21:32.067220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787600,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683312,"statfs":{"total":575970213888,"available":574139711488,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002459"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-02-16T19:20:36.455691+0000","last_change":"2024-02-16T19:20:11.179111+0000","last_active":"2024-02-16T19:20:36.455691+0000","last_peered":"2024-02-16T19:20:36.455691+0000","last_clean":"2024-02-16T19:20:36.455691+0000","last_became_active":"2024-02-16T19:20:11.178803+0000","last_became_peered":"2024-02-16T19:20:11.178803+0000","last_unstale":"2024-02-16T19:20:36.455691+0000","last_undegraded":"2024-02-16T19:20:36.455691+0000","last_fullsized":"2024-02-16T19:20:36.455691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-16T19:19:22.349668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-16T19:19:22.349668+0000","last_clean_scrub_stamp":"2024-02-16T19:19:22.349668+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T03:39:21.626756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.438},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.495}]},{"osd":2,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.315}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.49199999999999999}]},{"osd":1,"last update":"Fri Feb 16 19:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Fri Feb 16 19:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447512,"statfs":{"total":95995035648,"available":95690252288,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22800000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.29399999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.48999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Feb 16 19:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.25700000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.46500000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.36499999999999999,"15min":0.36299999999999999},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.47299999999999998}]},{"osd":3,"last update":"Fri Feb 16 19:20:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 16 19:20:58 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.23599999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.48699999999999999,"5min":0.495,"15min":0.495},"last":0.434},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.27500000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Feb 16 19:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.51100000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.56100000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Feb 16 19:20:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Feb 16 19:21:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.68500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 16 19:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.32100000000000001,"15min":0.312},"min":{"1min":0.25900000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33000000000000002},"min":{"1min":0.24299999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Feb 16 19:21:26 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41199999999999998,"15min":0.41599999999999998},"min":{"1min":0.318,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.31900000000000001}]},{"osd":3,"last update":"Fri Feb 16 19:20:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Feb 16 19:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-16T19:21:34.596 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-16T19:21:34.596 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-16T19:21:34.596 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-16T19:21:34.596 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-16T19:21:34.862 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:35 smithi040 ceph-mon[29201]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:21:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:35 smithi040 ceph-mon[29201]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:35 smithi163 ceph-mon[36452]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-16T19:21:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:35 smithi163 ceph-mon[36452]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.450+0000 7f310ecc1700 1 -- 172.21.15.40:0/588550675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3108103140 msgr2=0x7f3108103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.450+0000 7f310ecc1700 1 --2- 172.21.15.40:0/588550675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3108103140 0x7f3108103510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30fc004940 tx=0x7f30fc009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.450+0000 7f310ecc1700 1 -- 172.21.15.40:0/588550675 shutdown_connections 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.450+0000 7f310ecc1700 1 --2- 172.21.15.40:0/588550675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31080fb570 0x7f31080fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.450+0000 7f310ecc1700 1 --2- 172.21.15.40:0/588550675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3108103140 0x7f3108103510 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.450+0000 7f310ecc1700 1 -- 172.21.15.40:0/588550675 >> 172.21.15.40:0/588550675 conn(0x7f3108071560 msgr2=0x7f3108071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.450+0000 7f310ecc1700 1 -- 172.21.15.40:0/588550675 shutdown_connections 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.450+0000 7f310ecc1700 1 -- 172.21.15.40:0/588550675 wait complete. 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ecc1700 1 Processor -- start 2024-02-16T19:21:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ecc1700 1 -- start start 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ecc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31080fb570 0x7f3108113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ecc1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3108103140 0x7f3108114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ecc1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31081146a0 con 0x7f31080fb570 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ecc1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310810d9d0 con 0x7f3108103140 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ca5d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31080fb570 0x7f3108113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ca5d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31080fb570 0x7f3108113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33218/0 (socket says 172.21.15.40:33218) 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ca5d700 1 -- 172.21.15.40:0/1312179914 learned_addr learned my addr 172.21.15.40:0/1312179914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f3107fff700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3108103140 0x7f3108114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ca5d700 1 -- 172.21.15.40:0/1312179914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3108103140 msgr2=0x7f3108114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ca5d700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3108103140 0x7f3108114050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ca5d700 1 -- 172.21.15.40:0/1312179914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc005040 con 0x7f31080fb570 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.451+0000 7f310ca5d700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31080fb570 0x7f3108113b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00b3b0 tx=0x7f30fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.452+0000 7f31057fa700 1 -- 172.21.15.40:0/1312179914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30fc00e040 con 0x7f31080fb570 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.452+0000 7f31057fa700 1 -- 172.21.15.40:0/1312179914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30fc00bbf0 con 0x7f31080fb570 2024-02-16T19:21:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.452+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310810dc50 con 0x7f31080fb570 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.452+0000 7f31057fa700 1 -- 172.21.15.40:0/1312179914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30fc01c630 con 0x7f31080fb570 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.452+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310810e190 con 0x7f31080fb570 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.453+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3108100310 con 0x7f31080fb570 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.453+0000 7f31057fa700 1 -- 172.21.15.40:0/1312179914 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f30fc00bd60 con 0x7f31080fb570 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.453+0000 7f31057fa700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f30f80692a0 0x7f30f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.454+0000 7f31057fa700 1 -- 172.21.15.40:0/1312179914 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f30fc0882f0 con 0x7f31080fb570 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.454+0000 7f3107fff700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f30f80692a0 0x7f30f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.454+0000 7f3107fff700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f30f80692a0 0x7f30f806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f310810f340 tx=0x7f30f4009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:36.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.458+0000 7f31057fa700 1 -- 172.21.15.40:0/1312179914 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f30fc058350 con 0x7f31080fb570 2024-02-16T19:21:36.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.689+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f310804bed0 con 0x7f31080fb570 2024-02-16T19:21:36.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.690+0000 7f31057fa700 1 -- 172.21.15.40:0/1312179914 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f30fc057ee0 con 0x7f31080fb570 2024-02-16T19:21:36.691 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:21:36.691 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-16T19:21:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f30f80692a0 msgr2=0x7f30f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f30f80692a0 0x7f30f806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f310810f340 tx=0x7f30f4009ee0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31080fb570 msgr2=0x7f3108113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31080fb570 0x7f3108113b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00b3b0 tx=0x7f30fc009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 shutdown_connections 2024-02-16T19:21:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f30f80692a0 0x7f30f806b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31080fb570 0x7f3108113b10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 --2- 172.21.15.40:0/1312179914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3108103140 0x7f3108114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:36.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 >> 172.21.15.40:0/1312179914 conn(0x7f3108071560 msgr2=0x7f31080fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:36.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 shutdown_connections 2024-02-16T19:21:36.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:36.692+0000 7f310ecc1700 1 -- 172.21.15.40:0/1312179914 wait complete. 2024-02-16T19:21:37.313 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-16T19:21:37.313 INFO:tasks.cephadm:Setup complete, yielding 2024-02-16T19:21:37.314 INFO:teuthology.run_tasks:Running task print... 2024-02-16T19:21:37.323 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-02-16T19:21:37.323 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:21:37.372 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:21:37.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-02-16T19:21:37.644 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:37 smithi040 ceph-mon[29201]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:37 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1312179914' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-16T19:21:37.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:37 smithi163 ceph-mon[36452]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:37.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:37 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1312179914' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-16T19:21:38.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:38 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:21:38.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:38 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.260+0000 7f25f56d9700 1 -- 172.21.15.40:0/1908702608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 msgr2=0x7f25f00fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.260+0000 7f25f56d9700 1 --2- 172.21.15.40:0/1908702608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 0x7f25f00fd440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25e4004940 tx=0x7f25e4009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.260+0000 7f25f56d9700 1 -- 172.21.15.40:0/1908702608 shutdown_connections 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.260+0000 7f25f56d9700 1 --2- 172.21.15.40:0/1908702608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 0x7f25f00fd440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.260+0000 7f25f56d9700 1 --2- 172.21.15.40:0/1908702608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f00fbde0 0x7f25f00fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.260+0000 7f25f56d9700 1 -- 172.21.15.40:0/1908702608 >> 172.21.15.40:0/1908702608 conn(0x7f25f00f7480 msgr2=0x7f25f00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.260+0000 7f25f56d9700 1 -- 172.21.15.40:0/1908702608 shutdown_connections 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.260+0000 7f25f56d9700 1 -- 172.21.15.40:0/1908702608 wait complete. 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25f56d9700 1 Processor -- start 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25f56d9700 1 -- start start 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25f56d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f00fbde0 0x7f25f01105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25f56d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 0x7f25f0112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25f56d9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0066d80 con 0x7f25f00fcff0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25f56d9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0066ec0 con 0x7f25f00fbde0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25ee7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 0x7f25f0112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25ee7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 0x7f25f0112b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32936/0 (socket says 172.21.15.40:32936) 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.261+0000 7f25ee7fc700 1 -- 172.21.15.40:0/1199816893 learned_addr learned my addr 172.21.15.40:0/1199816893 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25eeffd700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f00fbde0 0x7f25f01105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25ee7fc700 1 -- 172.21.15.40:0/1199816893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f00fbde0 msgr2=0x7f25f01105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25ee7fc700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f00fbde0 0x7f25f01105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25ee7fc700 1 -- 172.21.15.40:0/1199816893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e4005040 con 0x7f25f00fcff0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25eeffd700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f00fbde0 0x7f25f01105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25ee7fc700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 0x7f25f0112b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f25e40093b0 tx=0x7f25e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25d77fe700 1 -- 172.21.15.40:0/1199816893 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25e400e040 con 0x7f25f00fcff0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25d77fe700 1 -- 172.21.15.40:0/1199816893 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25e400bcc0 con 0x7f25f00fcff0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25d77fe700 1 -- 172.21.15.40:0/1199816893 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25e401c630 con 0x7f25f00fcff0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f0113150 con 0x7f25f00fcff0 2024-02-16T19:21:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.262+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f01136b0 con 0x7f25f00fcff0 2024-02-16T19:21:39.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.263+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f004bed0 con 0x7f25f00fcff0 2024-02-16T19:21:39.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.264+0000 7f25d77fe700 1 -- 172.21.15.40:0/1199816893 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f25e40139f0 con 0x7f25f00fcff0 2024-02-16T19:21:39.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.264+0000 7f25d77fe700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f25d80692a0 0x7f25d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:39.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.264+0000 7f25d77fe700 1 -- 172.21.15.40:0/1199816893 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f25e4088250 con 0x7f25f00fcff0 2024-02-16T19:21:39.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.264+0000 7f25eeffd700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f25d80692a0 0x7f25d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:39.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.264+0000 7f25eeffd700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f25d80692a0 0x7f25d806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f25e0006fd0 tx=0x7f25e0008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:39.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.268+0000 7f25d77fe700 1 -- 172.21.15.40:0/1199816893 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f25e4054960 con 0x7f25f00fcff0 2024-02-16T19:21:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.461+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f25f01139e0 con 0x7f25f00fcff0 2024-02-16T19:21:39.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.465+0000 7f25d77fe700 1 -- 172.21.15.40:0/1199816893 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f25e4057f80 con 0x7f25f00fcff0 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f25d80692a0 msgr2=0x7f25d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f25d80692a0 0x7f25d806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f25e0006fd0 tx=0x7f25e0008040 comp rx=0 tx=0).stop 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 msgr2=0x7f25f0112b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 0x7f25f0112b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f25e40093b0 tx=0x7f25e4009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 shutdown_connections 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f25d80692a0 0x7f25d806b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f00fbde0 0x7f25f01105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 --2- 172.21.15.40:0/1199816893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25f00fcff0 0x7f25f0112b20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 >> 172.21.15.40:0/1199816893 conn(0x7f25f00f7480 msgr2=0x7f25f0100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 shutdown_connections 2024-02-16T19:21:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:39.467+0000 7f25f56d9700 1 -- 172.21.15.40:0/1199816893 wait complete. 2024-02-16T19:21:39.587 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:39 smithi040 ceph-mon[29201]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:39.587 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:39.587 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:21:39.588 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:39 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:21:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:39 smithi163 ceph-mon[36452]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:21:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:39 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:21:40.089 INFO:teuthology.run_tasks:Running task print... 2024-02-16T19:21:40.101 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-02-16T19:21:40.101 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:21:40.137 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:21:40.138 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-16T19:21:40.402 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:40.721 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:40 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1199816893' entity='client.admin' 2024-02-16T19:21:40.721 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:21:40.721 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:40.721 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:21:40.721 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:21:40.721 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:40 smithi040 ceph-mon[29201]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:40.721 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:40 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1199816893' entity='client.admin' 2024-02-16T19:21:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:21:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:21:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:21:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:40 smithi163 ceph-mon[36452]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:42.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.101+0000 7f99a2563700 1 -- 172.21.15.40:0/4078168367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 msgr2=0x7f999c070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:42.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.101+0000 7f99a2563700 1 --2- 172.21.15.40:0/4078168367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 0x7f999c070cd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9984004940 tx=0x7f9984009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:42.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.101+0000 7f99a2563700 1 -- 172.21.15.40:0/4078168367 shutdown_connections 2024-02-16T19:21:42.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.101+0000 7f99a2563700 1 --2- 172.21.15.40:0/4078168367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 0x7f999c070cd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:42.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.101+0000 7f99a2563700 1 --2- 172.21.15.40:0/4078168367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f999c071eb0 0x7f999c070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:42.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.101+0000 7f99a2563700 1 -- 172.21.15.40:0/4078168367 >> 172.21.15.40:0/4078168367 conn(0x7f999c0f8620 msgr2=0x7f999c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.101+0000 7f99a2563700 1 -- 172.21.15.40:0/4078168367 shutdown_connections 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.101+0000 7f99a2563700 1 -- 172.21.15.40:0/4078168367 wait complete. 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f99a2563700 1 Processor -- start 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f99a2563700 1 -- start start 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f99a2563700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 0x7f999c10fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f99a2563700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f999c071eb0 0x7f999c110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f99a2563700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c110710 con 0x7f999c070860 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f99a2563700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c110f50 con 0x7f999c071eb0 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f999bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 0x7f999c10fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f999bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 0x7f999c10fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32944/0 (socket says 172.21.15.40:32944) 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f999bfff700 1 -- 172.21.15.40:0/3129352916 learned_addr learned my addr 172.21.15.40:0/3129352916 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f999bfff700 1 -- 172.21.15.40:0/3129352916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f999c071eb0 msgr2=0x7f999c110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f999bfff700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f999c071eb0 0x7f999c110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.102+0000 7f999bfff700 1 -- 172.21.15.40:0/3129352916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9984005040 con 0x7f999c070860 2024-02-16T19:21:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.103+0000 7f999bfff700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 0x7f999c10fb40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b790 tx=0x7f998c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.103+0000 7f9998ff9700 1 -- 172.21.15.40:0/3129352916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998c00f770 con 0x7f999c070860 2024-02-16T19:21:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.103+0000 7f9998ff9700 1 -- 172.21.15.40:0/3129352916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f998c00fda0 con 0x7f999c070860 2024-02-16T19:21:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.103+0000 7f9998ff9700 1 -- 172.21.15.40:0/3129352916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998c00d310 con 0x7f999c070860 2024-02-16T19:21:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.103+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c111230 con 0x7f999c070860 2024-02-16T19:21:42.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.103+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c111770 con 0x7f999c070860 2024-02-16T19:21:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.104+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999c04bed0 con 0x7f999c070860 2024-02-16T19:21:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.104+0000 7f9998ff9700 1 -- 172.21.15.40:0/3129352916 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f998c010420 con 0x7f999c070860 2024-02-16T19:21:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.105+0000 7f9998ff9700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f99880692a0 0x7f998806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.105+0000 7f9998ff9700 1 -- 172.21.15.40:0/3129352916 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f998c086d60 con 0x7f999c070860 2024-02-16T19:21:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.105+0000 7f999b7fe700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f99880692a0 0x7f998806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:42.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.105+0000 7f999b7fe700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f99880692a0 0x7f998806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9984004f90 tx=0x7f9984009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:42.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.109+0000 7f9998ff9700 1 -- 172.21.15.40:0/3129352916 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f998c056f00 con 0x7f999c070860 2024-02-16T19:21:42.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.315+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f999c104550 con 0x7f99880692a0 2024-02-16T19:21:42.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.316+0000 7f9998ff9700 1 -- 172.21.15.40:0/3129352916 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f999c104550 con 0x7f99880692a0 2024-02-16T19:21:42.317 INFO:teuthology.orchestra.run.smithi040.stdout:Backend: cephadm 2024-02-16T19:21:42.317 INFO:teuthology.orchestra.run.smithi040.stdout:Available: Yes 2024-02-16T19:21:42.317 INFO:teuthology.orchestra.run.smithi040.stdout:Paused: No 2024-02-16T19:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.317+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f99880692a0 msgr2=0x7f998806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.317+0000 7f99a2563700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f99880692a0 0x7f998806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9984004f90 tx=0x7f9984009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.317+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 msgr2=0x7f999c10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.317+0000 7f99a2563700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 0x7f999c10fb40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b790 tx=0x7f998c009250 comp rx=0 tx=0).stop 2024-02-16T19:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.318+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 shutdown_connections 2024-02-16T19:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.318+0000 7f99a2563700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f99880692a0 0x7f998806b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.318+0000 7f99a2563700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f999c070860 0x7f999c10fb40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:42.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.318+0000 7f99a2563700 1 --2- 172.21.15.40:0/3129352916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f999c071eb0 0x7f999c110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.318+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 >> 172.21.15.40:0/3129352916 conn(0x7f999c0f8620 msgr2=0x7f999c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.318+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 shutdown_connections 2024-02-16T19:21:42.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:42.318+0000 7f99a2563700 1 -- 172.21.15.40:0/3129352916 wait complete. 2024-02-16T19:21:42.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-16T19:21:43.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:43 smithi040 ceph-mon[29201]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:43.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:43 smithi040 ceph-mon[29201]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:43.173 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:43 smithi163 ceph-mon[36452]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:43 smithi163 ceph-mon[36452]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 -- 172.21.15.40:0/3792463937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fbe40 msgr2=0x7f53340fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 --2- 172.21.15.40:0/3792463937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fbe40 0x7f53340fc250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5328004940 tx=0x7f5328009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 -- 172.21.15.40:0/3792463937 shutdown_connections 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 --2- 172.21.15.40:0/3792463937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53340fcfb0 0x7f53340fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 --2- 172.21.15.40:0/3792463937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fbe40 0x7f53340fc250 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 -- 172.21.15.40:0/3792463937 >> 172.21.15.40:0/3792463937 conn(0x7f53340f74e0 msgr2=0x7f53340f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 -- 172.21.15.40:0/3792463937 shutdown_connections 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 -- 172.21.15.40:0/3792463937 wait complete. 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 Processor -- start 2024-02-16T19:21:44.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.719+0000 7f533ab62700 1 -- start start 2024-02-16T19:21:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f533ab62700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53340fbe40 0x7f533410fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f533ab62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fcfb0 0x7f5334110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f533ab62700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5334110700 con 0x7f53340fcfb0 2024-02-16T19:21:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f533ab62700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5334110f40 con 0x7f53340fbe40 2024-02-16T19:21:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f5333fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fcfb0 0x7f5334110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f5333fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fcfb0 0x7f5334110070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32960/0 (socket says 172.21.15.40:32960) 2024-02-16T19:21:44.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f5333fff700 1 -- 172.21.15.40:0/4262941060 learned_addr learned my addr 172.21.15.40:0/4262941060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f53388fe700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53340fbe40 0x7f533410fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f5333fff700 1 -- 172.21.15.40:0/4262941060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53340fbe40 msgr2=0x7f533410fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f5333fff700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53340fbe40 0x7f533410fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f5333fff700 1 -- 172.21.15.40:0/4262941060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5328005040 con 0x7f53340fcfb0 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f5333fff700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fcfb0 0x7f5334110070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f532000b790 tx=0x7f5320009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.720+0000 7f53317fa700 1 -- 172.21.15.40:0/4262941060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f532000f770 con 0x7f53340fcfb0 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.721+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5334111220 con 0x7f53340fcfb0 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.721+0000 7f53317fa700 1 -- 172.21.15.40:0/4262941060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f532000fda0 con 0x7f53340fcfb0 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.721+0000 7f53317fa700 1 -- 172.21.15.40:0/4262941060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f532000d310 con 0x7f53340fcfb0 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.721+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5334111760 con 0x7f53340fcfb0 2024-02-16T19:21:44.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.722+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533404bed0 con 0x7f53340fcfb0 2024-02-16T19:21:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.722+0000 7f53317fa700 1 -- 172.21.15.40:0/4262941060 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5320010420 con 0x7f53340fcfb0 2024-02-16T19:21:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.722+0000 7f53317fa700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f53240692a0 0x7f532406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.723+0000 7f53317fa700 1 -- 172.21.15.40:0/4262941060 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5320086f10 con 0x7f53340fcfb0 2024-02-16T19:21:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.723+0000 7f53388fe700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f53240692a0 0x7f532406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.723+0000 7f53388fe700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f53240692a0 0x7f532406b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5328004f90 tx=0x7f5328009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:44.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.727+0000 7f53317fa700 1 -- 172.21.15.40:0/4262941060 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5320056f70 con 0x7f53340fcfb0 2024-02-16T19:21:44.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.928+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5334111c30 con 0x7f53240692a0 2024-02-16T19:21:44.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.936+0000 7f53317fa700 1 -- 172.21.15.40:0/4262941060 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7f5334111c30 con 0x7f53240692a0 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (3m) 2m ago 5m 26.3M - 0.25.0 c8568f914cd2 aadccae1ccec 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (5m) 2m ago 5m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (4m) 70s ago 4m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (3m) 2m ago 5m 78.6M - 9.4.7 954c08fa6188 8447b5580554 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:9283 running (6m) 2m ago 6m 472M - 17.2.7-513-gb88fae66 870a0944ccb8 4e9e9bb4389a 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283 running (4m) 70s ago 4m 435M - 17.2.7-513-gb88fae66 870a0944ccb8 06e26641f085 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (6m) 2m ago 6m 40.2M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (4m) 70s ago 4m 38.0M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (5m) 2m ago 5m 14.2M - 1.5.0 0da6a335fe13 61d16f3f00d4 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (4m) 70s ago 4m 16.5M - 1.5.0 0da6a335fe13 c45ebf41f1b8 2024-02-16T19:21:44.938 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (3m) 2m ago 3m 40.6M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:21:44.939 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (2m) 2m ago 2m 39.7M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:21:44.939 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (2m) 2m ago 2m 38.5M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:21:44.939 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (2m) 70s ago 2m 39.2M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:21:44.939 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (101s) 70s ago 100s 36.6M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:21:44.939 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (75s) 70s ago 75s 13.1M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:21:44.939 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (3m) 2m ago 4m 38.0M - 2.43.0 a07b618ecd1d 557bd885dbc5 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f53240692a0 msgr2=0x7f532406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f53240692a0 0x7f532406b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5328004f90 tx=0x7f5328009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fcfb0 msgr2=0x7f5334110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fcfb0 0x7f5334110070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f532000b790 tx=0x7f5320009250 comp rx=0 tx=0).stop 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 shutdown_connections 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f53240692a0 0x7f532406b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53340fbe40 0x7f533410fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 --2- 172.21.15.40:0/4262941060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53340fcfb0 0x7f5334110070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 >> 172.21.15.40:0/4262941060 conn(0x7f53340f74e0 msgr2=0x7f53340656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 shutdown_connections 2024-02-16T19:21:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:44.938+0000 7f533ab62700 1 -- 172.21.15.40:0/4262941060 wait complete. 2024-02-16T19:21:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:45 smithi040 ceph-mon[29201]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:45.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:45 smithi163 ceph-mon[36452]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:45.540 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-16T19:21:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:46.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:46 smithi040 ceph-mon[29201]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:46.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:46 smithi163 ceph-mon[36452]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 -- 172.21.15.40:0/2552486760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fe0e0 msgr2=0x7f6f8c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 --2- 172.21.15.40:0/2552486760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fe0e0 0x7f6f8c0fe550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6f80004940 tx=0x7f6f80009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 -- 172.21.15.40:0/2552486760 shutdown_connections 2024-02-16T19:21:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 --2- 172.21.15.40:0/2552486760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fe0e0 0x7f6f8c0fe550 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 --2- 172.21.15.40:0/2552486760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c0fcef0 0x7f6f8c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 -- 172.21.15.40:0/2552486760 >> 172.21.15.40:0/2552486760 conn(0x7f6f8c0f8620 msgr2=0x7f6f8c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 -- 172.21.15.40:0/2552486760 shutdown_connections 2024-02-16T19:21:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 -- 172.21.15.40:0/2552486760 wait complete. 2024-02-16T19:21:47.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.498+0000 7f6f9130f700 1 Processor -- start 2024-02-16T19:21:47.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:47 smithi040 ceph-mon[29201]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f9130f700 1 -- start start 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f9130f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fcef0 0x7f6f8c10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f9130f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c0fe0e0 0x7f6f8c10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f9130f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c10e560 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f9130f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c10eda0 con 0x7f6f8c0fe0e0 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f8affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fcef0 0x7f6f8c10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f8affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fcef0 0x7f6f8c10d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33600/0 (socket says 172.21.15.40:33600) 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f8affd700 1 -- 172.21.15.40:0/2956436251 learned_addr learned my addr 172.21.15.40:0/2956436251 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f8a7fc700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c0fe0e0 0x7f6f8c10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f8affd700 1 -- 172.21.15.40:0/2956436251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c0fe0e0 msgr2=0x7f6f8c10ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f8affd700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c0fe0e0 0x7f6f8c10ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f8affd700 1 -- 172.21.15.40:0/2956436251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f80005040 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.499+0000 7f6f8affd700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fcef0 0x7f6f8c10d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00b790 tx=0x7f6f7c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.500+0000 7f6f737fe700 1 -- 172.21.15.40:0/2956436251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f7c00f6f0 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.500+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c10f080 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.500+0000 7f6f737fe700 1 -- 172.21.15.40:0/2956436251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f7c00fd20 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.500+0000 7f6f737fe700 1 -- 172.21.15.40:0/2956436251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f7c0103a0 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.500+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c10f5c0 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.501+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c04bed0 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.501+0000 7f6f737fe700 1 -- 172.21.15.40:0/2956436251 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6f7c010500 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.501+0000 7f6f737fe700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6f740692a0 0x7f6f7406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.502+0000 7f6f737fe700 1 -- 172.21.15.40:0/2956436251 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6f7c086c30 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.502+0000 7f6f8a7fc700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6f740692a0 0x7f6f7406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.502+0000 7f6f8a7fc700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6f740692a0 0x7f6f7406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000b3b0 tx=0x7f6f80009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.506+0000 7f6f737fe700 1 -- 172.21.15.40:0/2956436251 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6f7c056d30 con 0x7f6f8c0fcef0 2024-02-16T19:21:47.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.707+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f8c10f210 con 0x7f6f740692a0 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.712+0000 7f6f737fe700 1 -- 172.21.15.40:0/2956436251 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7f6f8c10f210 con 0x7f6f740692a0 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stdout:crash 2/2 2m ago 6m * 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stdout:mgr 2/2 2m ago 6m count:2 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stdout:mon 2/2 2m ago 5m smithi040:172.21.15.40=smithi040;smithi163:172.21.15.163=smithi163;count:2 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-02-16T19:21:47.713 INFO:teuthology.orchestra.run.smithi040.stdout:osd 6 2m ago - 2024-02-16T19:21:47.714 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-02-16T19:21:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6f740692a0 msgr2=0x7f6f7406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6f740692a0 0x7f6f7406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000b3b0 tx=0x7f6f80009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fcef0 msgr2=0x7f6f8c10d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fcef0 0x7f6f8c10d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00b790 tx=0x7f6f7c009220 comp rx=0 tx=0).stop 2024-02-16T19:21:47.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 shutdown_connections 2024-02-16T19:21:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6f740692a0 0x7f6f7406b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f8c0fcef0 0x7f6f8c10d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 --2- 172.21.15.40:0/2956436251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f8c0fe0e0 0x7f6f8c10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 >> 172.21.15.40:0/2956436251 conn(0x7f6f8c0f8620 msgr2=0x7f6f8c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 shutdown_connections 2024-02-16T19:21:47.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:47.714+0000 7f6f9130f700 1 -- 172.21.15.40:0/2956436251 wait complete. 2024-02-16T19:21:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:47 smithi163 ceph-mon[36452]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:48.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-16T19:21:48.678 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:49.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:49 smithi040 ceph-mon[29201]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:49.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:49 smithi040 ceph-mon[29201]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:49.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:49 smithi163 ceph-mon[36452]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:49.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:49 smithi163 ceph-mon[36452]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.270+0000 7f238bbfb700 1 -- 172.21.15.40:0/375004155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 msgr2=0x7f23840fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.270+0000 7f238bbfb700 1 --2- 172.21.15.40:0/375004155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 0x7f23840fc290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f237c009ce0 tx=0x7f237c004d10 comp rx=0 tx=0).stop 2024-02-16T19:21:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.270+0000 7f238bbfb700 1 -- 172.21.15.40:0/375004155 shutdown_connections 2024-02-16T19:21:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.270+0000 7f238bbfb700 1 --2- 172.21.15.40:0/375004155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23840fd080 0x7f23840fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.270+0000 7f238bbfb700 1 --2- 172.21.15.40:0/375004155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 0x7f23840fc290 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.270+0000 7f238bbfb700 1 -- 172.21.15.40:0/375004155 >> 172.21.15.40:0/375004155 conn(0x7f23840f7590 msgr2=0x7f23840f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.270+0000 7f238bbfb700 1 -- 172.21.15.40:0/375004155 shutdown_connections 2024-02-16T19:21:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.270+0000 7f238bbfb700 1 -- 172.21.15.40:0/375004155 wait complete. 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f238bbfb700 1 Processor -- start 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f238bbfb700 1 -- start start 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f238bbfb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 0x7f23841107c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f238bbfb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23840fd080 0x7f2384112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f238bbfb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2384066e00 con 0x7f23840fbe80 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f238bbfb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2384066f40 con 0x7f23840fd080 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f2389997700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 0x7f23841107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f2389997700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 0x7f23841107c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33620/0 (socket says 172.21.15.40:33620) 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f2389997700 1 -- 172.21.15.40:0/3880928070 learned_addr learned my addr 172.21.15.40:0/3880928070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f2389997700 1 -- 172.21.15.40:0/3880928070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23840fd080 msgr2=0x7f2384112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f2389196700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23840fd080 0x7f2384112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f2389997700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23840fd080 0x7f2384112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.271+0000 7f2389997700 1 -- 172.21.15.40:0/3880928070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c009940 con 0x7f23840fbe80 2024-02-16T19:21:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.272+0000 7f2389997700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 0x7f23841107c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f237c009910 tx=0x7f237c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.272+0000 7f23827fc700 1 -- 172.21.15.40:0/3880928070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f237c00e040 con 0x7f23840fbe80 2024-02-16T19:21:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.272+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2384113390 con 0x7f23840fbe80 2024-02-16T19:21:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.272+0000 7f23827fc700 1 -- 172.21.15.40:0/3880928070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f237c0128e0 con 0x7f23840fbe80 2024-02-16T19:21:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.272+0000 7f23827fc700 1 -- 172.21.15.40:0/3880928070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f237c01c950 con 0x7f23840fbe80 2024-02-16T19:21:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.272+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23841138d0 con 0x7f23840fbe80 2024-02-16T19:21:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.273+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238404bed0 con 0x7f23840fbe80 2024-02-16T19:21:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.273+0000 7f23827fc700 1 -- 172.21.15.40:0/3880928070 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f237c012a50 con 0x7f23840fbe80 2024-02-16T19:21:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.274+0000 7f23827fc700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f236c0692a0 0x7f236c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.274+0000 7f2389196700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f236c0692a0 0x7f236c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.274+0000 7f23827fc700 1 -- 172.21.15.40:0/3880928070 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f237c088cc0 con 0x7f23840fbe80 2024-02-16T19:21:50.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.274+0000 7f2389196700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f236c0692a0 0x7f236c06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2378006fd0 tx=0x7f2378008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:50.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.278+0000 7f23827fc700 1 -- 172.21.15.40:0/3880928070 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f237c0552b0 con 0x7f23840fbe80 2024-02-16T19:21:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.476+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f23841019d0 con 0x7f236c0692a0 2024-02-16T19:21:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.477+0000 7f23827fc700 1 -- 172.21.15.40:0/3880928070 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f23841019d0 con 0x7f236c0692a0 2024-02-16T19:21:50.479 INFO:teuthology.orchestra.run.smithi040.stdout:HOST ADDR LABELS STATUS 2024-02-16T19:21:50.479 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040 172.21.15.40 2024-02-16T19:21:50.479 INFO:teuthology.orchestra.run.smithi040.stdout:smithi163 172.21.15.163 2024-02-16T19:21:50.479 INFO:teuthology.orchestra.run.smithi040.stdout:2 hosts in cluster 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f236c0692a0 msgr2=0x7f236c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f236c0692a0 0x7f236c06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2378006fd0 tx=0x7f2378008040 comp rx=0 tx=0).stop 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 msgr2=0x7f23841107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 0x7f23841107c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f237c009910 tx=0x7f237c004d10 comp rx=0 tx=0).stop 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 shutdown_connections 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f236c0692a0 0x7f236c06b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23840fbe80 0x7f23841107c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 --2- 172.21.15.40:0/3880928070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23840fd080 0x7f2384112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 >> 172.21.15.40:0/3880928070 conn(0x7f23840f7590 msgr2=0x7f23841002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 shutdown_connections 2024-02-16T19:21:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:50.479+0000 7f238bbfb700 1 -- 172.21.15.40:0/3880928070 wait complete. 2024-02-16T19:21:51.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-16T19:21:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:51 smithi040 ceph-mon[29201]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:51 smithi163 ceph-mon[36452]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:52.376 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:52 smithi040 ceph-mon[29201]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:52.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:52 smithi163 ceph-mon[36452]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:52.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.830+0000 7fc61df12700 1 -- 172.21.15.40:0/61624711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 msgr2=0x7fc6180fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:52.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.830+0000 7fc61df12700 1 --2- 172.21.15.40:0/61624711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 0x7fc6180fd420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc60c004940 tx=0x7fc60c009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:52.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.830+0000 7fc61df12700 1 -- 172.21.15.40:0/61624711 shutdown_connections 2024-02-16T19:21:52.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.830+0000 7fc61df12700 1 --2- 172.21.15.40:0/61624711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 0x7fc6180fd420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.830+0000 7fc61df12700 1 --2- 172.21.15.40:0/61624711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6180fbe40 0x7fc6180fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.830+0000 7fc61df12700 1 -- 172.21.15.40:0/61624711 >> 172.21.15.40:0/61624711 conn(0x7fc6180f7520 msgr2=0x7fc6180f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.830+0000 7fc61df12700 1 -- 172.21.15.40:0/61624711 shutdown_connections 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.830+0000 7fc61df12700 1 -- 172.21.15.40:0/61624711 wait complete. 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc61df12700 1 Processor -- start 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc61df12700 1 -- start start 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc61df12700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6180fbe40 0x7fc6180fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc61df12700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 0x7fc6180fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc61df12700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180fea10 con 0x7fc6180fcfb0 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc61df12700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180feb80 con 0x7fc6180fbe40 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc616ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 0x7fc6180fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc616ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 0x7fc6180fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33648/0 (socket says 172.21.15.40:33648) 2024-02-16T19:21:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc616ffd700 1 -- 172.21.15.40:0/164709582 learned_addr learned my addr 172.21.15.40:0/164709582 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc6177fe700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6180fbe40 0x7fc6180fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc616ffd700 1 -- 172.21.15.40:0/164709582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6180fbe40 msgr2=0x7fc6180fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc616ffd700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6180fbe40 0x7fc6180fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.831+0000 7fc616ffd700 1 -- 172.21.15.40:0/164709582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c005040 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.832+0000 7fc616ffd700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 0x7fc6180fe380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0093b0 tx=0x7fc60c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.832+0000 7fc5fffff700 1 -- 172.21.15.40:0/164709582 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60c00e040 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.832+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6180fee00 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.832+0000 7fc5fffff700 1 -- 172.21.15.40:0/164709582 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc60c00bcc0 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.832+0000 7fc5fffff700 1 -- 172.21.15.40:0/164709582 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60c01c630 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.832+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc618115950 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.833+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61804bed0 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.833+0000 7fc5fffff700 1 -- 172.21.15.40:0/164709582 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc60c013980 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.834+0000 7fc5fffff700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc6000692a0 0x7fc60006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.834+0000 7fc5fffff700 1 -- 172.21.15.40:0/164709582 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc60c0881c0 con 0x7fc6180fcfb0 2024-02-16T19:21:52.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.834+0000 7fc6177fe700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc6000692a0 0x7fc60006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:52.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.834+0000 7fc6177fe700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc6000692a0 0x7fc60006b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6180ffc90 tx=0x7fc6080091a0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:52.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:52.839+0000 7fc5fffff700 1 -- 172.21.15.40:0/164709582 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc60c0582c0 con 0x7fc6180fcfb0 2024-02-16T19:21:53.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.039+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc618066660 con 0x7fc6000692a0 2024-02-16T19:21:53.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.040+0000 7fc5fffff700 1 -- 172.21.15.40:0/164709582 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fc618066660 con 0x7fc6000692a0 2024-02-16T19:21:53.041 INFO:teuthology.orchestra.run.smithi040.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-16T19:21:53.042 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009J400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-16T19:21:53.042 INFO:teuthology.orchestra.run.smithi040.stdout:smithi163 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GU400BGN 372G No 75s ago Has a FileSystem, LVM detected 2024-02-16T19:21:53.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc6000692a0 msgr2=0x7fc60006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:53.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc6000692a0 0x7fc60006b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6180ffc90 tx=0x7fc6080091a0 comp rx=0 tx=0).stop 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 msgr2=0x7fc6180fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 0x7fc6180fe380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0093b0 tx=0x7fc60c009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 shutdown_connections 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc6000692a0 0x7fc60006b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6180fbe40 0x7fc6180fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 --2- 172.21.15.40:0/164709582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6180fcfb0 0x7fc6180fe380 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 >> 172.21.15.40:0/164709582 conn(0x7fc6180f7520 msgr2=0x7fc618065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 shutdown_connections 2024-02-16T19:21:53.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:53.042+0000 7fc61df12700 1 -- 172.21.15.40:0/164709582 wait complete. 2024-02-16T19:21:53.600 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:53 smithi040 ceph-mon[29201]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:53.625 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-16T19:21:53.635 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:21:53.671 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:21:53.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-16T19:21:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:53 smithi163 ceph-mon[36452]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:53.933 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:54.378 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:54 smithi040 ceph-mon[29201]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:54.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:54 smithi163 ceph-mon[36452]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:55.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 -- 172.21.15.40:0/1827776545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d001003a0 msgr2=0x7f6d00102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:55.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 --2- 172.21.15.40:0/1827776545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d001003a0 0x7f6d00102780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4004940 tx=0x7f6cf4009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:55.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 -- 172.21.15.40:0/1827776545 shutdown_connections 2024-02-16T19:21:55.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 --2- 172.21.15.40:0/1827776545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d001003a0 0x7f6d00102780 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:55.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 --2- 172.21.15.40:0/1827776545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d000fda80 0x7f6d000ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:55.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 -- 172.21.15.40:0/1827776545 >> 172.21.15.40:0/1827776545 conn(0x7f6d000f74e0 msgr2=0x7f6d000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:55.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 -- 172.21.15.40:0/1827776545 shutdown_connections 2024-02-16T19:21:55.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 -- 172.21.15.40:0/1827776545 wait complete. 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 Processor -- start 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.487+0000 7f6d06caf700 1 -- start start 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d06caf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d000fda80 0x7f6d0010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d06caf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d001003a0 0x7f6d0010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d06caf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0010e480 con 0x7f6d000fda80 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d06caf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0010ed20 con 0x7f6d001003a0 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d04a4b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d000fda80 0x7f6d0010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d04a4b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d000fda80 0x7f6d0010d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33676/0 (socket says 172.21.15.40:33676) 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d04a4b700 1 -- 172.21.15.40:0/1972321318 learned_addr learned my addr 172.21.15.40:0/1972321318 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6cfffff700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d001003a0 0x7f6d0010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d04a4b700 1 -- 172.21.15.40:0/1972321318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d001003a0 msgr2=0x7f6d0010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d04a4b700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d001003a0 0x7f6d0010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d04a4b700 1 -- 172.21.15.40:0/1972321318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf4005040 con 0x7f6d000fda80 2024-02-16T19:21:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d04a4b700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d000fda80 0x7f6d0010d940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8001d80 tx=0x7f6cf8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:55.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6cfd7fa700 1 -- 172.21.15.40:0/1972321318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf8007d10 con 0x7f6d000fda80 2024-02-16T19:21:55.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0010f000 con 0x7f6d000fda80 2024-02-16T19:21:55.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6cfd7fa700 1 -- 172.21.15.40:0/1972321318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cf8007e70 con 0x7f6d000fda80 2024-02-16T19:21:55.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.488+0000 7f6cfd7fa700 1 -- 172.21.15.40:0/1972321318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf8014640 con 0x7f6d000fda80 2024-02-16T19:21:55.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.489+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0010f540 con 0x7f6d000fda80 2024-02-16T19:21:55.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.489+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0004bed0 con 0x7f6d000fda80 2024-02-16T19:21:55.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.490+0000 7f6cfd7fa700 1 -- 172.21.15.40:0/1972321318 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6cf800e960 con 0x7f6d000fda80 2024-02-16T19:21:55.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.490+0000 7f6cfd7fa700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6ce80692a0 0x7f6ce806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:21:55.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.490+0000 7f6cfd7fa700 1 -- 172.21.15.40:0/1972321318 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6cf8086c30 con 0x7f6d000fda80 2024-02-16T19:21:55.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.490+0000 7f6cfffff700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6ce80692a0 0x7f6ce806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:21:55.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.491+0000 7f6cfffff700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6ce80692a0 0x7f6ce806b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4004f90 tx=0x7f6cf4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:21:55.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.495+0000 7f6cfd7fa700 1 -- 172.21.15.40:0/1972321318 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6cf8056d60 con 0x7f6d000fda80 2024-02-16T19:21:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:55 smithi040 ceph-mon[29201]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:55 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:55.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:55.716+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d0010f9c0 con 0x7f6ce80692a0 2024-02-16T19:21:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:55 smithi163 ceph-mon[36452]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:55 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:21:56.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:56 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-16T19:21:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:56 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-16T19:21:57.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.297+0000 7f6cfd7fa700 1 -- 172.21.15.40:0/1972321318 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6d0010f9c0 con 0x7f6ce80692a0 2024-02-16T19:21:57.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6ce80692a0 msgr2=0x7f6ce806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6ce80692a0 0x7f6ce806b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4004f90 tx=0x7f6cf4009e30 comp rx=0 tx=0).stop 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d000fda80 msgr2=0x7f6d0010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d000fda80 0x7f6d0010d940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8001d80 tx=0x7f6cf8009ec0 comp rx=0 tx=0).stop 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 shutdown_connections 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6ce80692a0 0x7f6ce806b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d000fda80 0x7f6d0010d940 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 --2- 172.21.15.40:0/1972321318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d001003a0 0x7f6d0010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 >> 172.21.15.40:0/1972321318 conn(0x7f6d000f74e0 msgr2=0x7f6d000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 shutdown_connections 2024-02-16T19:21:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:21:57.300+0000 7f6d06caf700 1 -- 172.21.15.40:0/1972321318 wait complete. 2024-02-16T19:21:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:57 smithi040 ceph-mon[29201]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:57 smithi040 ceph-mon[29201]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:57 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-16T19:21:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:57 smithi040 ceph-mon[29201]: osdmap e41: 6 total, 6 up, 6 in 2024-02-16T19:21:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:57 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-16T19:21:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:57 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[29197]: 2024-02-16T19:21:57.271+0000 7f326691d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:21:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:57 smithi163 ceph-mon[36452]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:21:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:57 smithi163 ceph-mon[36452]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:57 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-16T19:21:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:57 smithi163 ceph-mon[36452]: osdmap e41: 6 total, 6 up, 6 in 2024-02-16T19:21:57.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:57 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-16T19:21:57.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T19:21:58.568 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-16T19:21:58.568 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: osdmap e42: 6 total, 6 up, 6 in 2024-02-16T19:21:58.568 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: osdmap e43: 6 total, 6 up, 6 in 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: fsmap cephfs:0 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: Saving service mds.cephfs spec with placement count:4 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.ezkppl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.ezkppl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:58.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:58 smithi040 ceph-mon[29201]: Deploying daemon mds.cephfs.smithi040.ezkppl on smithi040 2024-02-16T19:21:58.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: osdmap e42: 6 total, 6 up, 6 in 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: osdmap e43: 6 total, 6 up, 6 in 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: fsmap cephfs:0 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: Saving service mds.cephfs spec with placement count:4 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:21:58.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:21:58.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.ezkppl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:21:58.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.ezkppl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:21:58.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:21:58.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:58 smithi163 ceph-mon[36452]: Deploying daemon mds.cephfs.smithi040.ezkppl on smithi040 2024-02-16T19:21:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:21:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:59 smithi040 ceph-mon[29201]: pgmap v175: 65 pgs: 16 creating+peering, 6 active+clean, 43 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:21:59 smithi040 ceph-mon[29201]: osdmap e44: 6 total, 6 up, 6 in 2024-02-16T19:21:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:59 smithi163 ceph-mon[36452]: pgmap v175: 65 pgs: 16 creating+peering, 6 active+clean, 43 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:21:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:21:59 smithi163 ceph-mon[36452]: osdmap e44: 6 total, 6 up, 6 in 2024-02-16T19:22:00.580 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:00 smithi040 ceph-mon[29201]: osdmap e45: 6 total, 6 up, 6 in 2024-02-16T19:22:00.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:00 smithi163 ceph-mon[36452]: osdmap e45: 6 total, 6 up, 6 in 2024-02-16T19:22:00.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 -- 172.21.15.40:0/3562910193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 msgr2=0x7f32fc102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:00.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 --2- 172.21.15.40:0/3562910193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 0x7f32fc102530 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32f0004940 tx=0x7f32f0009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:00.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 -- 172.21.15.40:0/3562910193 shutdown_connections 2024-02-16T19:22:00.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 --2- 172.21.15.40:0/3562910193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 0x7f32fc102530 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 --2- 172.21.15.40:0/3562910193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32fc066c50 0x7f32fc0ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 -- 172.21.15.40:0/3562910193 >> 172.21.15.40:0/3562910193 conn(0x7f32fc0f7510 msgr2=0x7f32fc0f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 -- 172.21.15.40:0/3562910193 shutdown_connections 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 -- 172.21.15.40:0/3562910193 wait complete. 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.911+0000 7f33027f8700 1 Processor -- start 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f33027f8700 1 -- start start 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f33027f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32fc066c50 0x7f32fc10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f33027f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 0x7f32fc10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f33027f8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc10e4f0 con 0x7f32fc1000f0 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f33027f8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc10ed90 con 0x7f32fc066c50 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f32fb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 0x7f32fc10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f32fb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 0x7f32fc10def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34018/0 (socket says 172.21.15.40:34018) 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f32fb7fe700 1 -- 172.21.15.40:0/2767871148 learned_addr learned my addr 172.21.15.40:0/2767871148 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f32fbfff700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32fc066c50 0x7f32fc10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f32fb7fe700 1 -- 172.21.15.40:0/2767871148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32fc066c50 msgr2=0x7f32fc10d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f32fb7fe700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32fc066c50 0x7f32fc10d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f32fb7fe700 1 -- 172.21.15.40:0/2767871148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f0005040 con 0x7f32fc1000f0 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.912+0000 7f32fbfff700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32fc066c50 0x7f32fc10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.913+0000 7f32fb7fe700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 0x7f32fc10def0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b3b0 tx=0x7f32f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.913+0000 7f32f8ff9700 1 -- 172.21.15.40:0/2767871148 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f000e040 con 0x7f32fc1000f0 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.913+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc10f010 con 0x7f32fc1000f0 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.913+0000 7f32f8ff9700 1 -- 172.21.15.40:0/2767871148 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32f000bcc0 con 0x7f32fc1000f0 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.913+0000 7f32f8ff9700 1 -- 172.21.15.40:0/2767871148 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f001c630 con 0x7f32fc1000f0 2024-02-16T19:22:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.913+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc10f550 con 0x7f32fc1000f0 2024-02-16T19:22:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.914+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc107a60 con 0x7f32fc1000f0 2024-02-16T19:22:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.915+0000 7f32f8ff9700 1 -- 172.21.15.40:0/2767871148 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f32f0013420 con 0x7f32fc1000f0 2024-02-16T19:22:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.915+0000 7f32f8ff9700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f32e40692a0 0x7f32e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.915+0000 7f32f8ff9700 1 -- 172.21.15.40:0/2767871148 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f32f0088620 con 0x7f32fc1000f0 2024-02-16T19:22:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.915+0000 7f32fbfff700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f32e40692a0 0x7f32e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.915+0000 7f32fbfff700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f32e40692a0 0x7f32e406b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009fd0 tx=0x7f32ec009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:00.920+0000 7f32f8ff9700 1 -- 172.21.15.40:0/2767871148 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f32f00583e0 con 0x7f32fc1000f0 2024-02-16T19:22:01.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.159+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f32fc04bed0 con 0x7f32fc1000f0 2024-02-16T19:22:01.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.160+0000 7f32f8ff9700 1 -- 172.21.15.40:0/2767871148 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f32f0057f70 con 0x7f32fc1000f0 2024-02-16T19:22:01.161 INFO:teuthology.orchestra.run.smithi040.stdout:e2 2024-02-16T19:22:01.161 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:22:01.161 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:22:01.161 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 2 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:21:57.272607+0000 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:22:01.162 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 0 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 1 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:in 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:up {} 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 0 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:01.163 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:01.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f32e40692a0 msgr2=0x7f32e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:01.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f32e40692a0 0x7f32e406b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009fd0 tx=0x7f32ec009ee0 comp rx=0 tx=0).stop 2024-02-16T19:22:01.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 msgr2=0x7f32fc10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 0x7f32fc10def0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b3b0 tx=0x7f32f0009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 shutdown_connections 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f32e40692a0 0x7f32e406b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32fc066c50 0x7f32fc10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 --2- 172.21.15.40:0/2767871148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32fc1000f0 0x7f32fc10def0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 >> 172.21.15.40:0/2767871148 conn(0x7f32fc0f7510 msgr2=0x7f32fc0f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 shutdown_connections 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:01.162+0000 7f33027f8700 1 -- 172.21.15.40:0/2767871148 wait complete. 2024-02-16T19:22:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2024-02-16T19:22:01.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:01 smithi040 ceph-mon[29201]: pgmap v178: 65 pgs: 16 creating+peering, 21 active+clean, 28 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:01.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:01 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2767871148' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:22:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:01 smithi163 ceph-mon[36452]: pgmap v178: 65 pgs: 16 creating+peering, 21 active+clean, 28 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:01 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2767871148' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:22:02.024 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:22:02.034 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:22:02.034 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-02-16T19:22:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.rggdhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.rggdhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6826/659310888,v1:172.21.15.40:6827/659310888] up:boot 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: daemon mds.cephfs.smithi040.ezkppl assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: Cluster is now healthy 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: fsmap cephfs:0 1 up:standby 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:creating} 2024-02-16T19:22:02.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:02 smithi163 ceph-mon[36452]: daemon mds.cephfs.smithi040.ezkppl is now active in filesystem cephfs as rank 0 2024-02-16T19:22:02.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:02.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:02.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:02.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.rggdhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:22:02.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.rggdhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:22:02.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:02.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6826/659310888,v1:172.21.15.40:6827/659310888] up:boot 2024-02-16T19:22:02.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: daemon mds.cephfs.smithi040.ezkppl assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-16T19:22:02.762 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:22:02.762 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T19:22:02.762 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: Cluster is now healthy 2024-02-16T19:22:02.762 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: fsmap cephfs:0 1 up:standby 2024-02-16T19:22:02.762 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:22:02.762 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:creating} 2024-02-16T19:22:02.762 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:02 smithi040 ceph-mon[29201]: daemon mds.cephfs.smithi040.ezkppl is now active in filesystem cephfs as rank 0 2024-02-16T19:22:03.526 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:03 smithi163 ceph-mon[36452]: Deploying daemon mds.cephfs.smithi163.rggdhh on smithi163 2024-02-16T19:22:03.526 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:03 smithi163 ceph-mon[36452]: pgmap v179: 65 pgs: 16 creating+peering, 42 active+clean, 7 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:03.526 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:03 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6826/659310888,v1:172.21.15.40:6827/659310888] up:active 2024-02-16T19:22:03.527 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:03 smithi163 ceph-mon[36452]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:active} 2024-02-16T19:22:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:03 smithi040 ceph-mon[29201]: Deploying daemon mds.cephfs.smithi163.rggdhh on smithi163 2024-02-16T19:22:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:03 smithi040 ceph-mon[29201]: pgmap v179: 65 pgs: 16 creating+peering, 42 active+clean, 7 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:03 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6826/659310888,v1:172.21.15.40:6827/659310888] up:active 2024-02-16T19:22:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:03 smithi040 ceph-mon[29201]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:active} 2024-02-16T19:22:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.017+0000 7fcce0fb1700 1 -- 172.21.15.40:0/2935199968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 msgr2=0x7fccdc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.017+0000 7fcce0fb1700 1 --2- 172.21.15.40:0/2935199968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 0x7fccdc0fe550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fccc4004940 tx=0x7fccc4009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 -- 172.21.15.40:0/2935199968 shutdown_connections 2024-02-16T19:22:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 --2- 172.21.15.40:0/2935199968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 0x7fccdc0fe550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 --2- 172.21.15.40:0/2935199968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc0fcef0 0x7fccdc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 -- 172.21.15.40:0/2935199968 >> 172.21.15.40:0/2935199968 conn(0x7fccdc0f8620 msgr2=0x7fccdc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 -- 172.21.15.40:0/2935199968 shutdown_connections 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 -- 172.21.15.40:0/2935199968 wait complete. 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 Processor -- start 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 -- start start 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc0fcef0 0x7fccdc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 0x7fccdc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc10e4b0 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.018+0000 7fcce0fb1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc10ed50 con 0x7fccdc0fcef0 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 0x7fccdc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 0x7fccdc10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34052/0 (socket says 172.21.15.40:34052) 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd9d9b700 1 -- 172.21.15.40:0/34962280 learned_addr learned my addr 172.21.15.40:0/34962280 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccda59c700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc0fcef0 0x7fccdc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd9d9b700 1 -- 172.21.15.40:0/34962280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc0fcef0 msgr2=0x7fccdc10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd9d9b700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc0fcef0 0x7fccdc10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd9d9b700 1 -- 172.21.15.40:0/34962280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4005040 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd9d9b700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 0x7fccdc10deb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fccc40093b0 tx=0x7fccc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd2ffd700 1 -- 172.21.15.40:0/34962280 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccc400e040 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd2ffd700 1 -- 172.21.15.40:0/34962280 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccc400bcc0 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccdc10efd0 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fccd2ffd700 1 -- 172.21.15.40:0/34962280 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccc401c630 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.019+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc10f510 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.020+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccdc04bed0 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.021+0000 7fccd2ffd700 1 -- 172.21.15.40:0/34962280 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fccc4013980 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.022+0000 7fccd2ffd700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fccc80692a0 0x7fccc806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:04.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.022+0000 7fccd2ffd700 1 -- 172.21.15.40:0/34962280 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fccc4088520 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.022+0000 7fccda59c700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fccc80692a0 0x7fccc806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:04.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.022+0000 7fccda59c700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fccc80692a0 0x7fccc806b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0fcc30 tx=0x7fcccc00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:04.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.031+0000 7fccd2ffd700 1 -- 172.21.15.40:0/34962280 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fccc40582e0 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.282+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fccdc10fa40 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.431+0000 7fccd2ffd700 1 -- 172.21.15.40:0/34962280 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7fccc4057e70 con 0x7fccdc0fe0e0 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fccc80692a0 msgr2=0x7fccc806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fccc80692a0 0x7fccc806b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0fcc30 tx=0x7fcccc00b040 comp rx=0 tx=0).stop 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 msgr2=0x7fccdc10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 0x7fccdc10deb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fccc40093b0 tx=0x7fccc4009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 shutdown_connections 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fccc80692a0 0x7fccc806b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccdc0fcef0 0x7fccdc10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 --2- 172.21.15.40:0/34962280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0fe0e0 0x7fccdc10deb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.433+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 >> 172.21.15.40:0/34962280 conn(0x7fccdc0f8620 msgr2=0x7fccdc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.434+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 shutdown_connections 2024-02-16T19:22:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:04.434+0000 7fcce0fb1700 1 -- 172.21.15.40:0/34962280 wait complete. 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.bmpjpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.bmpjpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: Deploying daemon mds.cephfs.smithi040.bmpjpk on smithi040 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/34962280' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] up:boot 2024-02-16T19:22:05.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/34962280' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-16T19:22:05.057 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: daemon mds.cephfs.smithi163.rggdhh assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-16T19:22:05.057 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-16T19:22:05.057 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T19:22:05.057 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:active} 1 up:standby 2024-02-16T19:22:05.057 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:22:05.057 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:creating} 2024-02-16T19:22:05.057 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:04 smithi040 ceph-mon[29201]: daemon mds.cephfs.smithi163.rggdhh is now active in filesystem cephfs as rank 1 2024-02-16T19:22:05.058 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:22:05.071 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:22:05.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-02-16T19:22:05.139 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:05.139 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:05.139 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:05.139 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.bmpjpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:22:05.139 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.bmpjpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:22:05.139 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:05.139 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: Deploying daemon mds.cephfs.smithi040.bmpjpk on smithi040 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/34962280' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] up:boot 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/34962280' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: daemon mds.cephfs.smithi163.rggdhh assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:active} 1 up:standby 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:creating} 2024-02-16T19:22:05.140 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:04 smithi163 ceph-mon[36452]: daemon mds.cephfs.smithi163.rggdhh is now active in filesystem cephfs as rank 1 2024-02-16T19:22:05.953 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:06.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:06 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] up:active 2024-02-16T19:22:06.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:06 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 2024-02-16T19:22:06.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:06 smithi040 ceph-mon[29201]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-02-16T19:22:06.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:06 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] up:active 2024-02-16T19:22:06.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:06 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 2024-02-16T19:22:06.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:06 smithi163 ceph-mon[36452]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-02-16T19:22:07.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.643+0000 7ff37cf51700 1 -- 172.21.15.40:0/525487390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fb5e0 msgr2=0x7ff3780fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:07.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.643+0000 7ff37cf51700 1 --2- 172.21.15.40:0/525487390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fb5e0 0x7ff3780fb9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff360004a50 tx=0x7ff36000af40 comp rx=0 tx=0).stop 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.643+0000 7ff37cf51700 1 -- 172.21.15.40:0/525487390 shutdown_connections 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.643+0000 7ff37cf51700 1 --2- 172.21.15.40:0/525487390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3780fc7f0 0x7ff3780fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.643+0000 7ff37cf51700 1 --2- 172.21.15.40:0/525487390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fb5e0 0x7ff3780fb9f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.643+0000 7ff37cf51700 1 -- 172.21.15.40:0/525487390 >> 172.21.15.40:0/525487390 conn(0x7ff3780f6ca0 msgr2=0x7ff3780f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.643+0000 7ff37cf51700 1 -- 172.21.15.40:0/525487390 shutdown_connections 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.643+0000 7ff37cf51700 1 -- 172.21.15.40:0/525487390 wait complete. 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff37cf51700 1 Processor -- start 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff37cf51700 1 -- start start 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff37cf51700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3780fb5e0 0x7ff37810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff37cf51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fc7f0 0x7ff37810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff37cf51700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37810e600 con 0x7ff3780fc7f0 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff37cf51700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37810ee40 con 0x7ff3780fb5e0 2024-02-16T19:22:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff375d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fc7f0 0x7ff37810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff375d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fc7f0 0x7ff37810df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39334/0 (socket says 172.21.15.40:39334) 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff375d9b700 1 -- 172.21.15.40:0/2403391626 learned_addr learned my addr 172.21.15.40:0/2403391626 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff37659c700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3780fb5e0 0x7ff37810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff375d9b700 1 -- 172.21.15.40:0/2403391626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3780fb5e0 msgr2=0x7ff37810da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.644+0000 7ff375d9b700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3780fb5e0 0x7ff37810da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.645+0000 7ff375d9b700 1 -- 172.21.15.40:0/2403391626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360005040 con 0x7ff3780fc7f0 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.645+0000 7ff375d9b700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fc7f0 0x7ff37810df70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff36800b790 tx=0x7ff368009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.645+0000 7ff36effd700 1 -- 172.21.15.40:0/2403391626 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff36800f6f0 con 0x7ff3780fc7f0 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.645+0000 7ff36effd700 1 -- 172.21.15.40:0/2403391626 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff36800fd20 con 0x7ff3780fc7f0 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.645+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37810f120 con 0x7ff3780fc7f0 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.645+0000 7ff36effd700 1 -- 172.21.15.40:0/2403391626 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3680103a0 con 0x7ff3780fc7f0 2024-02-16T19:22:07.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.645+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37810f660 con 0x7ff3780fc7f0 2024-02-16T19:22:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.646+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37804bed0 con 0x7ff3780fc7f0 2024-02-16T19:22:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.646+0000 7ff36effd700 1 -- 172.21.15.40:0/2403391626 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff368011400 con 0x7ff3780fc7f0 2024-02-16T19:22:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.647+0000 7ff36effd700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff3640692a0 0x7ff36406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.647+0000 7ff37659c700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff3640692a0 0x7ff36406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.647+0000 7ff36effd700 1 -- 172.21.15.40:0/2403391626 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff368086f70 con 0x7ff3780fc7f0 2024-02-16T19:22:07.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.647+0000 7ff37659c700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff3640692a0 0x7ff36406b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff36000afd0 tx=0x7ff36000af40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:07.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.651+0000 7ff36effd700 1 -- 172.21.15.40:0/2403391626 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff368056d30 con 0x7ff3780fc7f0 2024-02-16T19:22:07.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.892+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff37810fae0 con 0x7ff3780fc7f0 2024-02-16T19:22:07.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.946+0000 7ff36effd700 1 -- 172.21.15.40:0/2403391626 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff3680568c0 con 0x7ff3780fc7f0 2024-02-16T19:22:07.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.948+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff3640692a0 msgr2=0x7ff36406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:07.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.948+0000 7ff37cf51700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff3640692a0 0x7ff36406b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff36000afd0 tx=0x7ff36000af40 comp rx=0 tx=0).stop 2024-02-16T19:22:07.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fc7f0 msgr2=0x7ff37810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:07.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fc7f0 0x7ff37810df70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff36800b790 tx=0x7ff368009250 comp rx=0 tx=0).stop 2024-02-16T19:22:07.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 shutdown_connections 2024-02-16T19:22:07.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7ff3640692a0 0x7ff36406b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:07.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3780fb5e0 0x7ff37810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:07.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 --2- 172.21.15.40:0/2403391626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3780fc7f0 0x7ff37810df70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:07.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 >> 172.21.15.40:0/2403391626 conn(0x7ff3780f6ca0 msgr2=0x7ff3780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:07.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 shutdown_connections 2024-02-16T19:22:07.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:07.949+0000 7ff37cf51700 1 -- 172.21.15.40:0/2403391626 wait complete. 2024-02-16T19:22:08.071 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:07 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2403391626' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T19:22:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:07 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2403391626' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-16T19:22:08.840 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:22:08.850 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:22:08.850 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-02-16T19:22:09.117 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2403391626' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-16T19:22:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 2024-02-16T19:22:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s wr, 8 op/s 2024-02-16T19:22:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.qovvde", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:22:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.qovvde", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:22:09.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:08 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:09.173 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2403391626' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-16T19:22:09.173 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 2024-02-16T19:22:09.173 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s wr, 8 op/s 2024-02-16T19:22:09.173 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:09.174 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:09.174 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:09.174 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.qovvde", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:22:09.174 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.qovvde", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-16T19:22:09.174 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:08 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:10.187 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:09 smithi040 ceph-mon[29201]: Deploying daemon mds.cephfs.smithi163.qovvde on smithi163 2024-02-16T19:22:10.187 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:09 smithi040 ceph-mon[29201]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-16T19:22:10.188 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:09 smithi040 ceph-mon[29201]: Cluster is now healthy 2024-02-16T19:22:10.188 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:09 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:boot 2024-02-16T19:22:10.188 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:09 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 1 up:standby 2024-02-16T19:22:10.188 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:09 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:22:10.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:09 smithi163 ceph-mon[36452]: Deploying daemon mds.cephfs.smithi163.qovvde on smithi163 2024-02-16T19:22:10.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:09 smithi163 ceph-mon[36452]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-16T19:22:10.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:09 smithi163 ceph-mon[36452]: Cluster is now healthy 2024-02-16T19:22:10.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:09 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:boot 2024-02-16T19:22:10.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:09 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 1 up:standby 2024-02-16T19:22:10.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:09 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:22:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.701+0000 7f560c7e4700 1 -- 172.21.15.40:0/2919303197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 msgr2=0x7f56040fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.701+0000 7f560c7e4700 1 --2- 172.21.15.40:0/2919303197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 0x7f56040fe550 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f55f4004940 tx=0x7f55f4009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 -- 172.21.15.40:0/2919303197 shutdown_connections 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 --2- 172.21.15.40:0/2919303197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 0x7f56040fe550 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 --2- 172.21.15.40:0/2919303197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56040fcef0 0x7f56040fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 -- 172.21.15.40:0/2919303197 >> 172.21.15.40:0/2919303197 conn(0x7f56040f8620 msgr2=0x7f56040faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 -- 172.21.15.40:0/2919303197 shutdown_connections 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 -- 172.21.15.40:0/2919303197 wait complete. 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 Processor -- start 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 -- start start 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56040fcef0 0x7f560410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 0x7f560410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560410e4a0 con 0x7f56040fe0e0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560c7e4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560410ed40 con 0x7f56040fcef0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f5609d7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 0x7f560410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f5609d7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 0x7f560410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39382/0 (socket says 172.21.15.40:39382) 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f5609d7f700 1 -- 172.21.15.40:0/3718913988 learned_addr learned my addr 172.21.15.40:0/3718913988 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.702+0000 7f560a580700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56040fcef0 0x7f560410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f5609d7f700 1 -- 172.21.15.40:0/3718913988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56040fcef0 msgr2=0x7f560410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f5609d7f700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56040fcef0 0x7f560410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f5609d7f700 1 -- 172.21.15.40:0/3718913988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f4005040 con 0x7f56040fe0e0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f5609d7f700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 0x7f560410dea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f55f40093b0 tx=0x7f55f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f55faffd700 1 -- 172.21.15.40:0/3718913988 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55f400e040 con 0x7f56040fe0e0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f55faffd700 1 -- 172.21.15.40:0/3718913988 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55f400bcc0 con 0x7f56040fe0e0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560410efc0 con 0x7f56040fe0e0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f55faffd700 1 -- 172.21.15.40:0/3718913988 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55f401c630 con 0x7f56040fe0e0 2024-02-16T19:22:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.703+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560410f500 con 0x7f56040fe0e0 2024-02-16T19:22:10.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.704+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560404bed0 con 0x7f56040fe0e0 2024-02-16T19:22:10.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.704+0000 7f55faffd700 1 -- 172.21.15.40:0/3718913988 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f55f4013420 con 0x7f56040fe0e0 2024-02-16T19:22:10.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.705+0000 7f55faffd700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f55f00692a0 0x7f55f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:10.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.705+0000 7f55faffd700 1 -- 172.21.15.40:0/3718913988 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f55f4088430 con 0x7f56040fe0e0 2024-02-16T19:22:10.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.705+0000 7f560a580700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f55f00692a0 0x7f55f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:10.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.705+0000 7f560a580700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f55f00692a0 0x7f55f006b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56040fcc30 tx=0x7f5600009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:10.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.709+0000 7f55faffd700 1 -- 172.21.15.40:0/3718913988 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f55f4058290 con 0x7f56040fe0e0 2024-02-16T19:22:10.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:10.947+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f5604105a30 con 0x7f56040fe0e0 2024-02-16T19:22:11.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:10 smithi163 ceph-mon[36452]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-16T19:22:11.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:11.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:10 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3718913988' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-16T19:22:11.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.238+0000 7f55faffd700 1 -- 172.21.15.40:0/3718913988 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7f55f4057e20 con 0x7f56040fe0e0 2024-02-16T19:22:11.240 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:10 smithi040 ceph-mon[29201]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-16T19:22:11.240 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:11.240 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:10 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3718913988' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-16T19:22:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.241+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f55f00692a0 msgr2=0x7f55f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.241+0000 7f560c7e4700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f55f00692a0 0x7f55f006b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56040fcc30 tx=0x7f5600009040 comp rx=0 tx=0).stop 2024-02-16T19:22:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.241+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 msgr2=0x7f560410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.241+0000 7f560c7e4700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 0x7f560410dea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f55f40093b0 tx=0x7f55f4009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.241+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 shutdown_connections 2024-02-16T19:22:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.242+0000 7f560c7e4700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f55f00692a0 0x7f55f006b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.242+0000 7f560c7e4700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56040fcef0 0x7f560410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.242+0000 7f560c7e4700 1 --2- 172.21.15.40:0/3718913988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56040fe0e0 0x7f560410dea0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.242+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 >> 172.21.15.40:0/3718913988 conn(0x7f56040f8620 msgr2=0x7f56041013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.242+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 shutdown_connections 2024-02-16T19:22:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:11.242+0000 7f560c7e4700 1 -- 172.21.15.40:0/3718913988 wait complete. 2024-02-16T19:22:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:inline data disabled 2024-02-16T19:22:11.910 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:22:11.921 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:22:11.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T19:22:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:12.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:boot 2024-02-16T19:22:12.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3718913988' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-16T19:22:12.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:12.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:11 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:22:12.360 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.360 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.361 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.361 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.361 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:12.361 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:12.361 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:boot 2024-02-16T19:22:12.361 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3718913988' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-16T19:22:12.361 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:12.361 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:11 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:22:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:13.237 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:12 smithi040 ceph-mon[29201]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-02-16T19:22:13.271 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:12 smithi163 ceph-mon[36452]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-02-16T19:22:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:standby 2024-02-16T19:22:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] up:active 2024-02-16T19:22:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: Dropping low affinity active daemon mds.cephfs.smithi040.ezkppl in favor of higher affinity standby. 2024-02-16T19:22:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: Replacing daemon mds.cephfs.smithi040.ezkppl as rank 0 with standby daemon mds.cephfs.smithi040.bmpjpk 2024-02-16T19:22:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:22:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: osdmap e46: 6 total, 6 up, 6 in 2024-02-16T19:22:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: fsmap cephfs:2/2 {0=cephfs.smithi040.bmpjpk=up:replay,1=cephfs.smithi163.rggdhh=up:active} 1 up:standby 2024-02-16T19:22:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:13 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:standby 2024-02-16T19:22:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] up:active 2024-02-16T19:22:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: Dropping low affinity active daemon mds.cephfs.smithi040.ezkppl in favor of higher affinity standby. 2024-02-16T19:22:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: Replacing daemon mds.cephfs.smithi040.ezkppl as rank 0 with standby daemon mds.cephfs.smithi040.bmpjpk 2024-02-16T19:22:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:22:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: osdmap e46: 6 total, 6 up, 6 in 2024-02-16T19:22:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: fsmap cephfs:2/2 {0=cephfs.smithi040.bmpjpk=up:replay,1=cephfs.smithi163.rggdhh=up:active} 1 up:standby 2024-02-16T19:22:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:13 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:15.236 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:15 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:resolve 2024-02-16T19:22:15.236 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:15 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] up:boot 2024-02-16T19:22:15.236 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:15 smithi040 ceph-mon[29201]: fsmap cephfs:2/2 {0=cephfs.smithi040.bmpjpk=up:resolve,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:15.237 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:15 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:22:15.237 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:15 smithi040 ceph-mon[29201]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 8 op/s 2024-02-16T19:22:15.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:15 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:resolve 2024-02-16T19:22:15.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:15 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] up:boot 2024-02-16T19:22:15.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:15 smithi163 ceph-mon[36452]: fsmap cephfs:2/2 {0=cephfs.smithi040.bmpjpk=up:resolve,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:15.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:15 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:22:15.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:15 smithi163 ceph-mon[36452]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 8 op/s 2024-02-16T19:22:15.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.734+0000 7fb8d203f700 1 -- 172.21.15.40:0/3543040384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc066c70 msgr2=0x7fb8cc0ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:15.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.734+0000 7fb8d203f700 1 --2- 172.21.15.40:0/3543040384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc066c70 0x7fb8cc0ffaf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc004940 tx=0x7fb8bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:15.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.734+0000 7fb8d203f700 1 -- 172.21.15.40:0/3543040384 shutdown_connections 2024-02-16T19:22:15.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.734+0000 7fb8d203f700 1 --2- 172.21.15.40:0/3543040384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8cc1000c0 0x7fb8cc102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:15.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.734+0000 7fb8d203f700 1 --2- 172.21.15.40:0/3543040384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc066c70 0x7fb8cc0ffaf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:15.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.734+0000 7fb8d203f700 1 -- 172.21.15.40:0/3543040384 >> 172.21.15.40:0/3543040384 conn(0x7fb8cc0f7520 msgr2=0x7fb8cc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:15.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.735+0000 7fb8d203f700 1 -- 172.21.15.40:0/3543040384 shutdown_connections 2024-02-16T19:22:15.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.735+0000 7fb8d203f700 1 -- 172.21.15.40:0/3543040384 wait complete. 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.735+0000 7fb8d203f700 1 Processor -- start 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.735+0000 7fb8d203f700 1 -- start start 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8d203f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8cc066c70 0x7fb8cc10d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8d203f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1000c0 0x7fb8cc10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8d203f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc10e4c0 con 0x7fb8cc1000c0 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8d203f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc10ed60 con 0x7fb8cc066c70 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8caffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1000c0 0x7fb8cc10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8caffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1000c0 0x7fb8cc10dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39420/0 (socket says 172.21.15.40:39420) 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8caffd700 1 -- 172.21.15.40:0/1745150648 learned_addr learned my addr 172.21.15.40:0/1745150648 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8cb7fe700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8cc066c70 0x7fb8cc10d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:15.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8caffd700 1 -- 172.21.15.40:0/1745150648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8cc066c70 msgr2=0x7fb8cc10d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8caffd700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8cc066c70 0x7fb8cc10d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8caffd700 1 -- 172.21.15.40:0/1745150648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc005040 con 0x7fb8cc1000c0 2024-02-16T19:22:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8caffd700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1000c0 0x7fb8cc10dec0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c800 tx=0x7fb8c0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8b3fff700 1 -- 172.21.15.40:0/1745150648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8c000a760 con 0x7fb8cc1000c0 2024-02-16T19:22:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc10f040 con 0x7fb8cc1000c0 2024-02-16T19:22:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc10f580 con 0x7fb8cc1000c0 2024-02-16T19:22:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.736+0000 7fb8b3fff700 1 -- 172.21.15.40:0/1745150648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8c000ad90 con 0x7fb8cc1000c0 2024-02-16T19:22:15.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.737+0000 7fb8b3fff700 1 -- 172.21.15.40:0/1745150648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8c00113a0 con 0x7fb8cc1000c0 2024-02-16T19:22:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.737+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc107a70 con 0x7fb8cc1000c0 2024-02-16T19:22:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.738+0000 7fb8b3fff700 1 -- 172.21.15.40:0/1745150648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb8c0012400 con 0x7fb8cc1000c0 2024-02-16T19:22:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.738+0000 7fb8b3fff700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb8b40692a0 0x7fb8b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.738+0000 7fb8b3fff700 1 -- 172.21.15.40:0/1745150648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb8c0087130 con 0x7fb8cc1000c0 2024-02-16T19:22:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.738+0000 7fb8cb7fe700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb8b40692a0 0x7fb8b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.739+0000 7fb8cb7fe700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb8b40692a0 0x7fb8b406b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc004f90 tx=0x7fb8bc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.743+0000 7fb8b3fff700 1 -- 172.21.15.40:0/1745150648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb8c0056f40 con 0x7fb8cc1000c0 2024-02-16T19:22:15.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.982+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb8cc105a30 con 0x7fb8cc1000c0 2024-02-16T19:22:15.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.982+0000 7fb8b3fff700 1 -- 172.21.15.40:0/1745150648 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1823 (secure 0 0 0) 0x7fb8c0056ad0 con 0x7fb8cc1000c0 2024-02-16T19:22:15.984 INFO:teuthology.orchestra.run.smithi040.stdout:e15 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 15 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:22:15.003873+0000 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:22:15.987 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:22:15.988 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:22:15.988 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:22:15.988 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:22:15.988 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:22:15.988 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:22:15.988 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:22:15.989 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:22:15.989 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:22:15.989 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:22:15.989 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:22:15.989 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:22:15.989 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:22:15.989 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:22:15.989 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:reconnect seq 4 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:15.990 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:22:15.991 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:22:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb8b40692a0 msgr2=0x7fb8b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb8b40692a0 0x7fb8b406b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc004f90 tx=0x7fb8bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1000c0 msgr2=0x7fb8cc10dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1000c0 0x7fb8cc10dec0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c800 tx=0x7fb8c0004470 comp rx=0 tx=0).stop 2024-02-16T19:22:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 shutdown_connections 2024-02-16T19:22:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb8b40692a0 0x7fb8b406b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8cc066c70 0x7fb8cc10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 --2- 172.21.15.40:0/1745150648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1000c0 0x7fb8cc10dec0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.984+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 >> 172.21.15.40:0/1745150648 conn(0x7fb8cc0f7520 msgr2=0x7fb8cc0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.985+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 shutdown_connections 2024-02-16T19:22:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:15.985+0000 7fb8d203f700 1 -- 172.21.15.40:0/1745150648 wait complete. 2024-02-16T19:22:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 15 2024-02-16T19:22:16.352 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:16 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:reconnect 2024-02-16T19:22:16.353 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:16 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:standby 2024-02-16T19:22:16.353 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:16 smithi040 ceph-mon[29201]: fsmap cephfs:2/2 {0=cephfs.smithi040.bmpjpk=up:reconnect,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:16.353 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:16 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:16.353 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:16 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1745150648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:22:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:16 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:reconnect 2024-02-16T19:22:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:16 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:standby 2024-02-16T19:22:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:16 smithi163 ceph-mon[36452]: fsmap cephfs:2/2 {0=cephfs.smithi040.bmpjpk=up:reconnect,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:16 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:16.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:16 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1745150648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:22:16.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-16T19:22:17.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:17 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:rejoin 2024-02-16T19:22:17.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:17 smithi040 ceph-mon[29201]: fsmap cephfs:2/2 {0=cephfs.smithi040.bmpjpk=up:rejoin,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:17.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:17 smithi040 ceph-mon[29201]: daemon mds.cephfs.smithi040.bmpjpk is now active in filesystem cephfs as rank 0 2024-02-16T19:22:17.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:17 smithi040 ceph-mon[29201]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 716 B/s wr, 4 op/s 2024-02-16T19:22:17.204 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:17.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:17 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:rejoin 2024-02-16T19:22:17.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:17 smithi163 ceph-mon[36452]: fsmap cephfs:2/2 {0=cephfs.smithi040.bmpjpk=up:rejoin,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:17.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:17 smithi163 ceph-mon[36452]: daemon mds.cephfs.smithi040.bmpjpk is now active in filesystem cephfs as rank 0 2024-02-16T19:22:17.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:17 smithi163 ceph-mon[36452]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 716 B/s wr, 4 op/s 2024-02-16T19:22:18.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:18 smithi040 ceph-mon[29201]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:22:18.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:18 smithi040 ceph-mon[29201]: Cluster is now healthy 2024-02-16T19:22:18.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:18 smithi040 ceph-mon[29201]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:active 2024-02-16T19:22:18.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:18 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:18 smithi163 ceph-mon[36452]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:22:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:18 smithi163 ceph-mon[36452]: Cluster is now healthy 2024-02-16T19:22:18.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:18 smithi163 ceph-mon[36452]: mds.? [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] up:active 2024-02-16T19:22:18.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:18 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:22:19.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:19 smithi040 ceph-mon[29201]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-16T19:22:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 -- 172.21.15.40:0/1633451960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fb5e0 msgr2=0x7f5ddc0fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 --2- 172.21.15.40:0/1633451960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fb5e0 0x7f5ddc0fb9f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc004a50 tx=0x7f5dcc00af40 comp rx=0 tx=0).stop 2024-02-16T19:22:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 -- 172.21.15.40:0/1633451960 shutdown_connections 2024-02-16T19:22:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 --2- 172.21.15.40:0/1633451960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ddc0fc7f0 0x7f5ddc0fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 --2- 172.21.15.40:0/1633451960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fb5e0 0x7f5ddc0fb9f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 -- 172.21.15.40:0/1633451960 >> 172.21.15.40:0/1633451960 conn(0x7f5ddc0f6ca0 msgr2=0x7f5ddc0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 -- 172.21.15.40:0/1633451960 shutdown_connections 2024-02-16T19:22:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 -- 172.21.15.40:0/1633451960 wait complete. 2024-02-16T19:22:19.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.394+0000 7f5de15c0700 1 Processor -- start 2024-02-16T19:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5de15c0700 1 -- start start 2024-02-16T19:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5de15c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ddc0fb5e0 0x7f5ddc10fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5de15c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fc7f0 0x7f5ddc110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5de15c0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc110750 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5de15c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc110ff0 con 0x7f5ddc0fb5e0 2024-02-16T19:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5dda7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fc7f0 0x7f5ddc110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5dda7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fc7f0 0x7f5ddc110150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48308/0 (socket says 172.21.15.40:48308) 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5dda7fc700 1 -- 172.21.15.40:0/556397039 learned_addr learned my addr 172.21.15.40:0/556397039 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5ddaffd700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ddc0fb5e0 0x7f5ddc10fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5dda7fc700 1 -- 172.21.15.40:0/556397039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ddc0fb5e0 msgr2=0x7f5ddc10fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5dda7fc700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ddc0fb5e0 0x7f5ddc10fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5dda7fc700 1 -- 172.21.15.40:0/556397039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc005040 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.395+0000 7f5dda7fc700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fc7f0 0x7f5ddc110150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0002a50 tx=0x7f5dd00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.396+0000 7f5dc37fe700 1 -- 172.21.15.40:0/556397039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dd0007cc0 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.396+0000 7f5dc37fe700 1 -- 172.21.15.40:0/556397039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dd000f040 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.396+0000 7f5dc37fe700 1 -- 172.21.15.40:0/556397039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dd0009640 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.396+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc1112d0 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.396+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc111750 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.397+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc04bed0 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.397+0000 7f5dc37fe700 1 -- 172.21.15.40:0/556397039 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5dd00097a0 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.398+0000 7f5dc37fe700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5dc40692a0 0x7f5dc406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.398+0000 7f5dc37fe700 1 -- 172.21.15.40:0/556397039 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5dd0086f90 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.398+0000 7f5ddaffd700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5dc40692a0 0x7f5dc406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.398+0000 7f5ddaffd700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5dc40692a0 0x7f5dc406b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00c3b0 tx=0x7f5dcc00af40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.403+0000 7f5dc37fe700 1 -- 172.21.15.40:0/556397039 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5dd0056ca0 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:19 smithi163 ceph-mon[36452]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-16T19:22:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.665+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ddc103840 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.666+0000 7f5dc37fe700 1 -- 172.21.15.40:0/556397039 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f5dd005ae60 con 0x7f5ddc0fc7f0 2024-02-16T19:22:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5dc40692a0 msgr2=0x7f5dc406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5dc40692a0 0x7f5dc406b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00c3b0 tx=0x7f5dcc00af40 comp rx=0 tx=0).stop 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fc7f0 msgr2=0x7f5ddc110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fc7f0 0x7f5ddc110150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0002a50 tx=0x7f5dd00046b0 comp rx=0 tx=0).stop 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 shutdown_connections 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5dc40692a0 0x7f5dc406b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ddc0fb5e0 0x7f5ddc10fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 --2- 172.21.15.40:0/556397039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ddc0fc7f0 0x7f5ddc110150 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 >> 172.21.15.40:0/556397039 conn(0x7f5ddc0f6ca0 msgr2=0x7f5ddc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 shutdown_connections 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:19.668+0000 7f5de15c0700 1 -- 172.21.15.40:0/556397039 wait complete. 2024-02-16T19:22:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-02-16T19:22:19.681 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:22:20.337 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-16T19:22:20.625 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:20 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:20.625 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:20 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:20.625 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:20 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:20.625 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:20 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:20.625 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:20 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:20.626 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:20 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:20.626 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:20 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/556397039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:22:20.633 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:20 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:20.634 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:20 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:20.634 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:20 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:20.634 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:20 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:20.634 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:20 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:20.634 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:20 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:20.634 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:20 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/556397039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:22:21.040 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:21.460 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:21 smithi040 ceph-mon[29201]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-16T19:22:21.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:21 smithi163 ceph-mon[36452]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-16T19:22:22.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:22.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:22 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:22.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:22.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:22 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.271+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/3029657520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fe0e0 msgr2=0x7fd1ec0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.271+0000 7fd1f1ce6700 1 --2- 172.21.15.40:0/3029657520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fe0e0 0x7fd1ec0fe550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0004940 tx=0x7fd1e0009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.271+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/3029657520 shutdown_connections 2024-02-16T19:22:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.271+0000 7fd1f1ce6700 1 --2- 172.21.15.40:0/3029657520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fe0e0 0x7fd1ec0fe550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.271+0000 7fd1f1ce6700 1 --2- 172.21.15.40:0/3029657520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1ec0fcef0 0x7fd1ec0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.271+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/3029657520 >> 172.21.15.40:0/3029657520 conn(0x7fd1ec0f8620 msgr2=0x7fd1ec0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.271+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/3029657520 shutdown_connections 2024-02-16T19:22:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.271+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/3029657520 wait complete. 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.272+0000 7fd1f1ce6700 1 Processor -- start 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.272+0000 7fd1f1ce6700 1 -- start start 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.272+0000 7fd1f1ce6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fcef0 0x7fd1ec10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.272+0000 7fd1f1ce6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1ec0fe0e0 0x7fd1ec10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.272+0000 7fd1f1ce6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec10e4b0 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.272+0000 7fd1f1ce6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec10ed50 con 0x7fd1ec0fe0e0 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fcef0 0x7fd1ec10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fcef0 0x7fd1ec10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48326/0 (socket says 172.21.15.40:48326) 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1eb7fe700 1 -- 172.21.15.40:0/14116254 learned_addr learned my addr 172.21.15.40:0/14116254 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1eaffd700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1ec0fe0e0 0x7fd1ec10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1eb7fe700 1 -- 172.21.15.40:0/14116254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1ec0fe0e0 msgr2=0x7fd1ec10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1eb7fe700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1ec0fe0e0 0x7fd1ec10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1eb7fe700 1 -- 172.21.15.40:0/14116254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0005040 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1eb7fe700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fcef0 0x7fd1ec10d970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc002a80 tx=0x7fd1dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1d3fff700 1 -- 172.21.15.40:0/14116254 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1dc007cc0 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1d3fff700 1 -- 172.21.15.40:0/14116254 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1dc00f040 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec10f030 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1d3fff700 1 -- 172.21.15.40:0/14116254 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1dc00da30 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.273+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec10f570 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.275+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec04bed0 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.275+0000 7fd1d3fff700 1 -- 172.21.15.40:0/14116254 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd1dc00db90 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.275+0000 7fd1d3fff700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd1d40692a0 0x7fd1d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:23.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.275+0000 7fd1d3fff700 1 -- 172.21.15.40:0/14116254 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd1dc086f60 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.275+0000 7fd1eaffd700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd1d40692a0 0x7fd1d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:23.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.276+0000 7fd1eaffd700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd1d40692a0 0x7fd1d406b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b3b0 tx=0x7fd1e0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.280+0000 7fd1d3fff700 1 -- 172.21.15.40:0/14116254 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd1dc056c50 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.465 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:23 smithi040 ceph-mon[29201]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-16T19:22:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.559+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fd1ec10f950 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.560+0000 7fd1d3fff700 1 -- 172.21.15.40:0/14116254 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+99 (secure 0 0 0) 0x7fd1dc0567e0 con 0x7fd1ec0fcef0 2024-02-16T19:22:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd1d40692a0 msgr2=0x7fd1d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd1d40692a0 0x7fd1d406b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b3b0 tx=0x7fd1e0009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fcef0 msgr2=0x7fd1ec10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fcef0 0x7fd1ec10d970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc002a80 tx=0x7fd1dc004600 comp rx=0 tx=0).stop 2024-02-16T19:22:23.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 shutdown_connections 2024-02-16T19:22:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fd1d40692a0 0x7fd1d406b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0fcef0 0x7fd1ec10d970 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 --2- 172.21.15.40:0/14116254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1ec0fe0e0 0x7fd1ec10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 >> 172.21.15.40:0/14116254 conn(0x7fd1ec0f8620 msgr2=0x7fd1ec1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 shutdown_connections 2024-02-16T19:22:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:23.562+0000 7fd1f1ce6700 1 -- 172.21.15.40:0/14116254 wait complete. 2024-02-16T19:22:23.573 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:22:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:23 smithi163 ceph-mon[36452]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-16T19:22:24.422 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-16T19:22:24.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.587+0000 7f1f4eb14640 1 -- 172.21.15.40:0/76707005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fc440 msgr2=0x7f1f480fc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.587+0000 7f1f4eb14640 1 --2- 172.21.15.40:0/76707005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fc440 0x7f1f480fc8b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1f340048d0 tx=0x7f1f34009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.587+0000 7f1f4eb14640 1 -- 172.21.15.40:0/76707005 shutdown_connections 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.587+0000 7f1f4eb14640 1 --2- 172.21.15.40:0/76707005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fc440 0x7f1f480fc8b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.587+0000 7f1f4eb14640 1 --2- 172.21.15.40:0/76707005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f480fb170 0x7f1f480fb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.587+0000 7f1f4eb14640 1 -- 172.21.15.40:0/76707005 >> 172.21.15.40:0/76707005 conn(0x7f1f480f6910 msgr2=0x7f1f480f8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.587+0000 7f1f4eb14640 1 -- 172.21.15.40:0/76707005 shutdown_connections 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.587+0000 7f1f4eb14640 1 -- 172.21.15.40:0/76707005 wait complete. 2024-02-16T19:22:24.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4eb14640 1 Processor -- start 2024-02-16T19:22:24.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4eb14640 1 -- start start 2024-02-16T19:22:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4eb14640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fb170 0x7f1f4810cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4eb14640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f480fc440 0x7f1f4810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4eb14640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4810d6e0 con 0x7f1f480fb170 2024-02-16T19:22:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4eb14640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48112130 con 0x7f1f480fc440 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4c8b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fb170 0x7f1f4810cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4c8b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fb170 0x7f1f4810cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48344/0 (socket says 172.21.15.40:48344) 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f4c8b0640 1 -- 172.21.15.40:0/3113023255 learned_addr learned my addr 172.21.15.40:0/3113023255 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.588+0000 7f1f47fff640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f480fc440 0x7f1f4810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f4c8b0640 1 -- 172.21.15.40:0/3113023255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f480fc440 msgr2=0x7f1f4810d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f4c8b0640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f480fc440 0x7f1f4810d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f4c8b0640 1 -- 172.21.15.40:0/3113023255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34005040 con 0x7f1f480fb170 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f4c8b0640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fb170 0x7f1f4810cb50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00eb70 tx=0x7f1f3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f457fa640 1 -- 172.21.15.40:0/3113023255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f3c007c30 con 0x7f1f480fb170 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f48112270 con 0x7f1f480fb170 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f457fa640 1 -- 172.21.15.40:0/3113023255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f3c00f040 con 0x7f1f480fb170 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f457fa640 1 -- 172.21.15.40:0/3113023255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f3c0095b0 con 0x7f1f480fb170 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.589+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f48112760 con 0x7f1f480fb170 2024-02-16T19:22:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.590+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f48103ce0 con 0x7f1f480fb170 2024-02-16T19:22:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.591+0000 7f1f457fa640 1 -- 172.21.15.40:0/3113023255 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1f3c009730 con 0x7f1f480fb170 2024-02-16T19:22:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.591+0000 7f1f457fa640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f38069210 0x7f1f3806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.591+0000 7f1f457fa640 1 -- 172.21.15.40:0/3113023255 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1f3c086f10 con 0x7f1f480fb170 2024-02-16T19:22:24.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.591+0000 7f1f47fff640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f38069210 0x7f1f3806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.592+0000 7f1f47fff640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f38069210 0x7f1f3806b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f4810e0a0 tx=0x7f1f34009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:24.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.595+0000 7f1f457fa640 1 -- 172.21.15.40:0/3113023255 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1f3c0531c0 con 0x7f1f480fb170 2024-02-16T19:22:24.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:24 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/14116254' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-16T19:22:24.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:24 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/14116254' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-16T19:22:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.768+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f48112980 con 0x7f1f480fb170 2024-02-16T19:22:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.769+0000 7f1f457fa640 1 -- 172.21.15.40:0/3113023255 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14384 (secure 0 0 0) 0x7f1f3c0567e0 con 0x7f1f480fb170 2024-02-16T19:22:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:24.770 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":46,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","created":"2024-02-16T19:14:55.436141+0000","modified":"2024-02-16T19:22:12.998867+0000","last_up_change":"2024-02-16T19:20:36.450204+0000","last_in_change":"2024-02-16T19:20:17.443690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:19:22.349243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T19:21:55.970810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T19:21:56.515184+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"db82e60f-540c-4f54-8f54-758be43e2cf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6803","nonce":800063463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6805","nonce":800063463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6809","nonce":800063463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":800063463},{"type":"v1","addr":"172.21.15.40:6807","nonce":800063463}]},"public_addr":"172.21.15.40:6803/800063463","cluster_addr":"172.21.15.40:6805/800063463","heartbeat_back_addr":"172.21.15.40:6809/800063463","heartbeat_front_addr":"172.21.15.40:6807/800063463","state":["exists","up"]},{"osd":1,"uuid":"22561508-cdb8-4c76-ab07-bf3efd6ee63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6811","nonce":3466922800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6813","nonce":3466922800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6817","nonce":3466922800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3466922800},{"type":"v1","addr":"172.21.15.40:6815","nonce":3466922800}]},"public_addr":"172.21.15.40:6811/3466922800","cluster_addr":"172.21.15.40:6813/3466922800","heartbeat_back_addr":"172.21.15.40:6817/3466922800","heartbeat_front_addr":"172.21.15.40:6815/3466922800","state":["exists","up"]},{"osd":2,"uuid":"58ff6654-72f1-4329-8f6c-4c3180353755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6819","nonce":3340187472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6821","nonce":3340187472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6825","nonce":3340187472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3340187472},{"type":"v1","addr":"172.21.15.40:6823","nonce":3340187472}]},"public_addr":"172.21.15.40:6819/3340187472","cluster_addr":"172.21.15.40:6821/3340187472","heartbeat_back_addr":"172.21.15.40:6825/3340187472","heartbeat_front_addr":"172.21.15.40:6823/3340187472","state":["exists","up"]},{"osd":3,"uuid":"b6f2b76f-f1fa-4459-8150-c36e624fb4aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6801","nonce":2273487961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6803","nonce":2273487961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6807","nonce":2273487961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2273487961},{"type":"v1","addr":"172.21.15.163:6805","nonce":2273487961}]},"public_addr":"172.21.15.163:6801/2273487961","cluster_addr":"172.21.15.163:6803/2273487961","heartbeat_back_addr":"172.21.15.163:6807/2273487961","heartbeat_front_addr":"172.21.15.163:6805/2273487961","state":["exists","up"]},{"osd":4,"uuid":"49afe3d8-2685-4c11-9830-77bd72a0b272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6809","nonce":1296776670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6811","nonce":1296776670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6815","nonce":1296776670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1296776670},{"type":"v1","addr":"172.21.15.163:6813","nonce":1296776670}]},"public_addr":"172.21.15.163:6809/1296776670","cluster_addr":"172.21.15.163:6811/1296776670","heartbeat_back_addr":"172.21.15.163:6815/1296776670","heartbeat_front_addr":"172.21.15.163:6813/1296776670","state":["exists","up"]},{"osd":5,"uuid":"c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6817","nonce":737810550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6819","nonce":737810550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6823","nonce":737810550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":737810550},{"type":"v1","addr":"172.21.15.163:6821","nonce":737810550}]},"public_addr":"172.21.15.163:6817/737810550","cluster_addr":"172.21.15.163:6819/737810550","heartbeat_back_addr":"172.21.15.163:6823/737810550","heartbeat_front_addr":"172.21.15.163:6821/737810550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:22.484106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:52.190602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:19.421062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:43.759349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:09.243860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:34.636467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6827/659310888":"2024-02-17T19:22:12.998828+0000","172.21.15.40:6800/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/2144454600":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6800/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1423773760":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1058311844":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/4011363422":"2024-02-17T19:15:23.873919+0000","172.21.15.40:6801/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/1754668438":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2673381923":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/1306942330":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2189950867":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6800/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6801/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:6801/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2009141374":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6826/659310888":"2024-02-17T19:22:12.998828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:22:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f38069210 msgr2=0x7f1f3806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f38069210 0x7f1f3806b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f4810e0a0 tx=0x7f1f34009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fb170 msgr2=0x7f1f4810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fb170 0x7f1f4810cb50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00eb70 tx=0x7f1f3c00b040 comp rx=0 tx=0).stop 2024-02-16T19:22:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 shutdown_connections 2024-02-16T19:22:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f1f38069210 0x7f1f3806b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f480fc440 0x7f1f4810d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 --2- 172.21.15.40:0/3113023255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f480fb170 0x7f1f4810cb50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 >> 172.21.15.40:0/3113023255 conn(0x7f1f480f6910 msgr2=0x7f1f48065710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 shutdown_connections 2024-02-16T19:22:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.771+0000 7f1f4eb14640 1 -- 172.21.15.40:0/3113023255 wait complete. 2024-02-16T19:22:24.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:19:22.349243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T19:21:55.970810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T19:21:56.515184+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-16T19:22:24.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T19:22:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 -- 172.21.15.40:0/4167593229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 msgr2=0x7f354c0fb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 --2- 172.21.15.40:0/4167593229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 0x7f354c0fb5e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35400048d0 tx=0x7f3540009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 -- 172.21.15.40:0/4167593229 shutdown_connections 2024-02-16T19:22:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 --2- 172.21.15.40:0/4167593229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c0fc400 0x7f354c0fc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 --2- 172.21.15.40:0/4167593229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 0x7f354c0fb5e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 -- 172.21.15.40:0/4167593229 >> 172.21.15.40:0/4167593229 conn(0x7f354c0f68d0 msgr2=0x7f354c0f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 -- 172.21.15.40:0/4167593229 shutdown_connections 2024-02-16T19:22:24.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 -- 172.21.15.40:0/4167593229 wait complete. 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 Processor -- start 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.895+0000 7f3553e2e640 1 -- start start 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3553e2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 0x7f354c064f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3553e2e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c0fc400 0x7f354c111940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3553e2e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c111e80 con 0x7f354c0fb1d0 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3553e2e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c111fc0 con 0x7f354c0fc400 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3551bca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 0x7f354c064f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3551bca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 0x7f354c064f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48364/0 (socket says 172.21.15.40:48364) 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3551bca640 1 -- 172.21.15.40:0/3312753948 learned_addr learned my addr 172.21.15.40:0/3312753948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f35513c9640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c0fc400 0x7f354c111940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3551bca640 1 -- 172.21.15.40:0/3312753948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c0fc400 msgr2=0x7f354c111940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3551bca640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c0fc400 0x7f354c111940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3551bca640 1 -- 172.21.15.40:0/3312753948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3540005040 con 0x7f354c0fb1d0 2024-02-16T19:22:24.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f35513c9640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c0fc400 0x7f354c111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:22:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3551bca640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 0x7f354c064f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3540034040 tx=0x7f3540009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f353e7fc640 1 -- 172.21.15.40:0/3312753948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3540009590 con 0x7f354c0fb1d0 2024-02-16T19:22:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c112240 con 0x7f354c0fb1d0 2024-02-16T19:22:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f353e7fc640 1 -- 172.21.15.40:0/3312753948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35400096f0 con 0x7f354c0fb1d0 2024-02-16T19:22:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f353e7fc640 1 -- 172.21.15.40:0/3312753948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354003b6b0 con 0x7f354c0fb1d0 2024-02-16T19:22:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.896+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c112760 con 0x7f354c0fb1d0 2024-02-16T19:22:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.897+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c103d20 con 0x7f354c0fb1d0 2024-02-16T19:22:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.898+0000 7f353e7fc640 1 -- 172.21.15.40:0/3312753948 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f354002f070 con 0x7f354c0fb1d0 2024-02-16T19:22:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.898+0000 7f353e7fc640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3538069210 0x7f353806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.898+0000 7f353e7fc640 1 -- 172.21.15.40:0/3312753948 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f35400a9990 con 0x7f354c0fb1d0 2024-02-16T19:22:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.898+0000 7f35513c9640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3538069210 0x7f353806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.899+0000 7f35513c9640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3538069210 0x7f353806b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f354c064d20 tx=0x7f3548009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:24.902+0000 7f353e7fc640 1 -- 172.21.15.40:0/3312753948 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3540075bf0 con 0x7f354c0fb1d0 2024-02-16T19:22:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.079+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f354c112dc0 con 0x7f354c0fb1d0 2024-02-16T19:22:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.080+0000 7f353e7fc640 1 -- 172.21.15.40:0/3312753948 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3540079210 con 0x7f354c0fb1d0 2024-02-16T19:22:25.081 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3538069210 msgr2=0x7f353806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3538069210 0x7f353806b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f354c064d20 tx=0x7f3548009040 comp rx=0 tx=0).stop 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 msgr2=0x7f354c064f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 0x7f354c064f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3540034040 tx=0x7f3540009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 shutdown_connections 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3538069210 0x7f353806b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c0fc400 0x7f354c111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 --2- 172.21.15.40:0/3312753948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c0fb1d0 0x7f354c064f70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 >> 172.21.15.40:0/3312753948 conn(0x7f354c0f68d0 msgr2=0x7f354c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 shutdown_connections 2024-02-16T19:22:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.082+0000 7f3553e2e640 1 -- 172.21.15.40:0/3312753948 wait complete. 2024-02-16T19:22:25.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T19:22:25.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.206+0000 7f16beb5d640 1 -- 172.21.15.40:0/3460429809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 msgr2=0x7f16b80fc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.206+0000 7f16beb5d640 1 --2- 172.21.15.40:0/3460429809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 0x7f16b80fc5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f16a40048d0 tx=0x7f16a4009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.206+0000 7f16beb5d640 1 -- 172.21.15.40:0/3460429809 shutdown_connections 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.206+0000 7f16beb5d640 1 --2- 172.21.15.40:0/3460429809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 0x7f16b80fc5d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.206+0000 7f16beb5d640 1 --2- 172.21.15.40:0/3460429809 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b80fb780 0x7f16b80fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.206+0000 7f16beb5d640 1 -- 172.21.15.40:0/3460429809 >> 172.21.15.40:0/3460429809 conn(0x7f16b80f6050 msgr2=0x7f16b80f8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.206+0000 7f16beb5d640 1 -- 172.21.15.40:0/3460429809 shutdown_connections 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.206+0000 7f16beb5d640 1 -- 172.21.15.40:0/3460429809 wait complete. 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16beb5d640 1 Processor -- start 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16beb5d640 1 -- start start 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16beb5d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b80fb780 0x7f16b810ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16beb5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 0x7f16b810bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16beb5d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b810c5f0 con 0x7f16b80fc160 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16beb5d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8111040 con 0x7f16b80fb780 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 0x7f16b810bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 0x7f16b810bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48376/0 (socket says 172.21.15.40:48376) 2024-02-16T19:22:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.207+0000 7f16b7fff640 1 -- 172.21.15.40:0/2872286213 learned_addr learned my addr 172.21.15.40:0/2872286213 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16bc8f9640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b80fb780 0x7f16b810ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16b7fff640 1 -- 172.21.15.40:0/2872286213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b80fb780 msgr2=0x7f16b810ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16b7fff640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b80fb780 0x7f16b810ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16b7fff640 1 -- 172.21.15.40:0/2872286213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4005040 con 0x7f16b80fc160 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16b7fff640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 0x7f16b810bfa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f16a402bda0 tx=0x7f16a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16b57fa640 1 -- 172.21.15.40:0/2872286213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a40094c0 con 0x7f16b80fc160 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b8111180 con 0x7f16b80fc160 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b8111670 con 0x7f16b80fc160 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16b57fa640 1 -- 172.21.15.40:0/2872286213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16a4009620 con 0x7f16b80fc160 2024-02-16T19:22:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.208+0000 7f16b57fa640 1 -- 172.21.15.40:0/2872286213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a403b400 con 0x7f16b80fc160 2024-02-16T19:22:25.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.209+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c005240 con 0x7f16b80fc160 2024-02-16T19:22:25.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.210+0000 7f16b57fa640 1 -- 172.21.15.40:0/2872286213 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f16a402f070 con 0x7f16b80fc160 2024-02-16T19:22:25.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.210+0000 7f16b57fa640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16a8069210 0x7f16a806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.210+0000 7f16b57fa640 1 -- 172.21.15.40:0/2872286213 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f16a40a9730 con 0x7f16b80fc160 2024-02-16T19:22:25.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.210+0000 7f16bc8f9640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16a8069210 0x7f16a806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.211+0000 7f16bc8f9640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16a8069210 0x7f16a806b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16b8064160 tx=0x7f16ac00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.215+0000 7f16b57fa640 1 -- 172.21.15.40:0/2872286213 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f16a4075ab0 con 0x7f16b80fc160 2024-02-16T19:22:25.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.393+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f169c005a70 con 0x7f16b80fc160 2024-02-16T19:22:25.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.393+0000 7f16b57fa640 1 -- 172.21.15.40:0/2872286213 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f16a40790d0 con 0x7f16b80fc160 2024-02-16T19:22:25.395 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-02-16T19:22:25.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16a8069210 msgr2=0x7f16a806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16a8069210 0x7f16a806b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16b8064160 tx=0x7f16ac00b040 comp rx=0 tx=0).stop 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 msgr2=0x7f16b810bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 0x7f16b810bfa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f16a402bda0 tx=0x7f16a4009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 shutdown_connections 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16b80fc160 0x7f16b810bfa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16a8069210 0x7f16a806b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 --2- 172.21.15.40:0/2872286213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b80fb780 0x7f16b810ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 >> 172.21.15.40:0/2872286213 conn(0x7f16b80f6050 msgr2=0x7f16b80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 shutdown_connections 2024-02-16T19:22:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.395+0000 7f16beb5d640 1 -- 172.21.15.40:0/2872286213 wait complete. 2024-02-16T19:22:25.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T19:22:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 -- 172.21.15.40:0/3963729641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 msgr2=0x7f7e000fd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 --2- 172.21.15.40:0/3963729641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 0x7f7e000fd9e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7df00048d0 tx=0x7f7df0009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 -- 172.21.15.40:0/3963729641 shutdown_connections 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 --2- 172.21.15.40:0/3963729641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 0x7f7e000fd9e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 --2- 172.21.15.40:0/3963729641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e000fc2a0 0x7f7e000fc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 -- 172.21.15.40:0/3963729641 >> 172.21.15.40:0/3963729641 conn(0x7f7e000f7a40 msgr2=0x7f7e000f9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 -- 172.21.15.40:0/3963729641 shutdown_connections 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 -- 172.21.15.40:0/3963729641 wait complete. 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 Processor -- start 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 -- start start 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e000fc2a0 0x7f7e0010cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 0x7f7e0010d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0010d6c0 con 0x7f7e000fd570 2024-02-16T19:22:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e07f5d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e00112110 con 0x7f7e000fc2a0 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e054f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 0x7f7e0010d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e054f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 0x7f7e0010d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48402/0 (socket says 172.21.15.40:48402) 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.508+0000 7f7e054f8640 1 -- 172.21.15.40:0/2772599872 learned_addr learned my addr 172.21.15.40:0/2772599872 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7e05cf9640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e000fc2a0 0x7f7e0010cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7e054f8640 1 -- 172.21.15.40:0/2772599872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e000fc2a0 msgr2=0x7f7e0010cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7e054f8640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e000fc2a0 0x7f7e0010cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7e054f8640 1 -- 172.21.15.40:0/2772599872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0005040 con 0x7f7e000fd570 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7e054f8640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 0x7f7e0010d070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7df0007da0 tx=0x7f7df0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7df67fc640 1 -- 172.21.15.40:0/2772599872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df00094c0 con 0x7f7e000fd570 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7df67fc640 1 -- 172.21.15.40:0/2772599872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7df0009620 con 0x7f7e000fd570 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e00112250 con 0x7f7e000fd570 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7df67fc640 1 -- 172.21.15.40:0/2772599872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df003b620 con 0x7f7e000fd570 2024-02-16T19:22:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.509+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e00112740 con 0x7f7e000fd570 2024-02-16T19:22:25.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.510+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e00104e10 con 0x7f7e000fd570 2024-02-16T19:22:25.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.510+0000 7f7df67fc640 1 -- 172.21.15.40:0/2772599872 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7df002f070 con 0x7f7e000fd570 2024-02-16T19:22:25.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.511+0000 7f7df67fc640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7dec069210 0x7f7dec06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.511+0000 7f7df67fc640 1 -- 172.21.15.40:0/2772599872 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7df00a9770 con 0x7f7e000fd570 2024-02-16T19:22:25.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.511+0000 7f7e05cf9640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7dec069210 0x7f7dec06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.511+0000 7f7e05cf9640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7dec069210 0x7f7dec06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc004ce0 tx=0x7f7dfc00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:25.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.515+0000 7f7df67fc640 1 -- 172.21.15.40:0/2772599872 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7df0075a50 con 0x7f7e000fd570 2024-02-16T19:22:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.709+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f7e00112920 con 0x7f7e000fd570 2024-02-16T19:22:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.710+0000 7f7df67fc640 1 -- 172.21.15.40:0/2772599872 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7df0079070 con 0x7f7e000fd570 2024-02-16T19:22:25.711 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-02-16T19:22:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7dec069210 msgr2=0x7f7dec06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7dec069210 0x7f7dec06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc004ce0 tx=0x7f7dfc00b040 comp rx=0 tx=0).stop 2024-02-16T19:22:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 msgr2=0x7f7e0010d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 0x7f7e0010d070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7df0007da0 tx=0x7f7df0009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 shutdown_connections 2024-02-16T19:22:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e000fd570 0x7f7e0010d070 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7dec069210 0x7f7dec06b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 --2- 172.21.15.40:0/2772599872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e000fc2a0 0x7f7e0010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 >> 172.21.15.40:0/2772599872 conn(0x7f7e000f7a40 msgr2=0x7f7e001007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 shutdown_connections 2024-02-16T19:22:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.711+0000 7f7e07f5d640 1 -- 172.21.15.40:0/2772599872 wait complete. 2024-02-16T19:22:25.722 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:25 smithi040 ceph-mon[29201]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-16T19:22:25.722 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:25 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3113023255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:22:25.722 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:25 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3312753948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:22:25.722 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:25.722 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:25 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2872286213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:22:25.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-16T19:22:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:25 smithi163 ceph-mon[36452]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-16T19:22:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:25 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3113023255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:22:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:25 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3312753948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:22:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:25 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2872286213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:22:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.820+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4086548859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 msgr2=0x7fa9880feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.820+0000 7fa98ec2e640 1 --2- 172.21.15.40:0/4086548859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 0x7fa9880feeb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0048d0 tx=0x7fa97c009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.820+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4086548859 shutdown_connections 2024-02-16T19:22:25.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.820+0000 7fa98ec2e640 1 --2- 172.21.15.40:0/4086548859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9880ff3f0 0x7fa9881017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.820+0000 7fa98ec2e640 1 --2- 172.21.15.40:0/4086548859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 0x7fa9880feeb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.820+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4086548859 >> 172.21.15.40:0/4086548859 conn(0x7fa9880f6960 msgr2=0x7fa9880f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.820+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4086548859 shutdown_connections 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.820+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4086548859 wait complete. 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98ec2e640 1 Processor -- start 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98ec2e640 1 -- start start 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98ec2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 0x7fa9880fc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98ec2e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9880ff3f0 0x7fa9880fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98ec2e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9880fb2f0 con 0x7fa9880663f0 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98ec2e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9880fb460 con 0x7fa9880ff3f0 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98c9ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 0x7fa9880fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98c9ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 0x7fa9880fc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48434/0 (socket says 172.21.15.40:48434) 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98c9ca640 1 -- 172.21.15.40:0/4254601379 learned_addr learned my addr 172.21.15.40:0/4254601379 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:25.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98c9ca640 1 -- 172.21.15.40:0/4254601379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9880ff3f0 msgr2=0x7fa9880fadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa987fff640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9880ff3f0 0x7fa9880fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98c9ca640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9880ff3f0 0x7fa9880fadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.821+0000 7fa98c9ca640 1 -- 172.21.15.40:0/4254601379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c005040 con 0x7fa9880663f0 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.822+0000 7fa98c9ca640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 0x7fa9880fc730 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa97c034040 tx=0x7fa97c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.822+0000 7fa9857fa640 1 -- 172.21.15.40:0/4254601379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa97c009590 con 0x7fa9880663f0 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.822+0000 7fa9857fa640 1 -- 172.21.15.40:0/4254601379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa97c0096f0 con 0x7fa9880663f0 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.822+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9880fb6e0 con 0x7fa9880663f0 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.822+0000 7fa9857fa640 1 -- 172.21.15.40:0/4254601379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa97c03b6b0 con 0x7fa9880663f0 2024-02-16T19:22:25.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.822+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9880fbc00 con 0x7fa9880663f0 2024-02-16T19:22:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.823+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa988064480 con 0x7fa9880663f0 2024-02-16T19:22:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.823+0000 7fa9857fa640 1 -- 172.21.15.40:0/4254601379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa97c02f070 con 0x7fa9880663f0 2024-02-16T19:22:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.823+0000 7fa9857fa640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa978069210 0x7fa97806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.824+0000 7fa9857fa640 1 -- 172.21.15.40:0/4254601379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa97c0a9990 con 0x7fa9880663f0 2024-02-16T19:22:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.824+0000 7fa987fff640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa978069210 0x7fa97806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:25.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.824+0000 7fa987fff640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa978069210 0x7fa97806b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa974007fd0 tx=0x7fa974009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:25.828+0000 7fa9857fa640 1 -- 172.21.15.40:0/4254601379 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa97c075bf0 con 0x7fa9880663f0 2024-02-16T19:22:26.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.041+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa988106da0 con 0x7fa9880663f0 2024-02-16T19:22:26.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.042+0000 7fa9857fa640 1 -- 172.21.15.40:0/4254601379 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fa97c079210 con 0x7fa9880663f0 2024-02-16T19:22:26.043 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:26.043 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/4253297290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4253297290},{"type":"v1","addr":"172.21.15.40:6827","nonce":4253297290}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:22:17.019628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi163.rggdhh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6825/1432724463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1432724463},{"type":"v1","addr":"172.21.15.163:6825","nonce":1432724463}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-16T19:22:26.043 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.043+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa978069210 msgr2=0x7fa97806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.043+0000 7fa98ec2e640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa978069210 0x7fa97806b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa974007fd0 tx=0x7fa974009040 comp rx=0 tx=0).stop 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.043+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 msgr2=0x7fa9880fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.043+0000 7fa98ec2e640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 0x7fa9880fc730 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa97c034040 tx=0x7fa97c009da0 comp rx=0 tx=0).stop 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.044+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 shutdown_connections 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.044+0000 7fa98ec2e640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fa978069210 0x7fa97806b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.044+0000 7fa98ec2e640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9880ff3f0 0x7fa9880fadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.044+0000 7fa98ec2e640 1 --2- 172.21.15.40:0/4254601379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9880663f0 0x7fa9880fc730 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:26.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.044+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 >> 172.21.15.40:0/4254601379 conn(0x7fa9880f6960 msgr2=0x7fa9880f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:26.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.044+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 shutdown_connections 2024-02-16T19:22:26.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:26.044+0000 7fa98ec2e640 1 -- 172.21.15.40:0/4254601379 wait complete. 2024-02-16T19:22:26.056 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-02-16T19:22:26.056 INFO:teuthology.run_tasks:Running task kclient... 2024-02-16T19:22:26.081 INFO:tasks.kclient:Mounting kernel clients... 2024-02-16T19:22:26.081 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-02-16T19:22:26.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:26.082 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-02-16T19:22:26.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:26.097 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-02-16T19:22:26.165 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-02-16T19:22:26.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:22:26.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:26.167 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-02-16T19:22:26.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:26.186 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-02-16T19:22:26.263 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-02-16T19:22:26.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:22:26.264 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-02-16T19:22:26.264 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-16T19:22:26.264 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-16T19:22:26.265 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-16T19:22:26.550 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:26.673 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:26 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2772599872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:22:26.674 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:26 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/4254601379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:22:26.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:26 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2772599872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:22:26.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:26 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/4254601379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:22:27.690 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:27 smithi040 ceph-mon[29201]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-16T19:22:27.690 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:27.690 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:27.690 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:27.691 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:27.691 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:27 smithi163 ceph-mon[36452]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-16T19:22:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.219+0000 7f09b18a4700 1 -- 172.21.15.40:0/844593984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fdf50 msgr2=0x7f09ac0fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.219+0000 7f09b18a4700 1 --2- 172.21.15.40:0/844593984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fdf50 0x7f09ac0fe320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f099c004940 tx=0x7f099c009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.219+0000 7f09b18a4700 1 -- 172.21.15.40:0/844593984 shutdown_connections 2024-02-16T19:22:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.219+0000 7f09b18a4700 1 --2- 172.21.15.40:0/844593984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09ac0fe8f0 0x7f09ac106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.219+0000 7f09b18a4700 1 --2- 172.21.15.40:0/844593984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fdf50 0x7f09ac0fe320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.219+0000 7f09b18a4700 1 -- 172.21.15.40:0/844593984 >> 172.21.15.40:0/844593984 conn(0x7f09ac0f79e0 msgr2=0x7f09ac0f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.219+0000 7f09b18a4700 1 -- 172.21.15.40:0/844593984 shutdown_connections 2024-02-16T19:22:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.219+0000 7f09b18a4700 1 -- 172.21.15.40:0/844593984 wait complete. 2024-02-16T19:22:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.220+0000 7f09b18a4700 1 Processor -- start 2024-02-16T19:22:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.220+0000 7f09b18a4700 1 -- start start 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09b18a4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09ac0fdf50 0x7f09ac113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09b18a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fe8f0 0x7f09ac113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09b18a4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac114610 con 0x7f09ac0fe8f0 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09b18a4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac10d940 con 0x7f09ac0fdf50 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fe8f0 0x7f09ac113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fe8f0 0x7f09ac113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37684/0 (socket says 172.21.15.40:37684) 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aa7fc700 1 -- 172.21.15.40:0/3920944855 learned_addr learned my addr 172.21.15.40:0/3920944855 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aaffd700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09ac0fdf50 0x7f09ac113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aaffd700 1 -- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fe8f0 msgr2=0x7f09ac113fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aaffd700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fe8f0 0x7f09ac113fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aaffd700 1 -- 172.21.15.40:0/3920944855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c005040 con 0x7f09ac0fdf50 2024-02-16T19:22:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aa7fc700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fe8f0 0x7f09ac113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:22:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.221+0000 7f09aaffd700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09ac0fdf50 0x7f09ac113a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b3b0 tx=0x7f099c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.222+0000 7f09937fe700 1 -- 172.21.15.40:0/3920944855 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099c00e040 con 0x7f09ac0fdf50 2024-02-16T19:22:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.222+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac10dbc0 con 0x7f09ac0fdf50 2024-02-16T19:22:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.222+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac10e100 con 0x7f09ac0fdf50 2024-02-16T19:22:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.222+0000 7f09937fe700 1 -- 172.21.15.40:0/3920944855 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f099c009b10 con 0x7f09ac0fdf50 2024-02-16T19:22:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.222+0000 7f09937fe700 1 -- 172.21.15.40:0/3920944855 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099c013660 con 0x7f09ac0fdf50 2024-02-16T19:22:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.223+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac104540 con 0x7f09ac0fdf50 2024-02-16T19:22:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.223+0000 7f09937fe700 1 -- 172.21.15.40:0/3920944855 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f099c0137c0 con 0x7f09ac0fdf50 2024-02-16T19:22:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.224+0000 7f09937fe700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09940692a0 0x7f099406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.224+0000 7f09937fe700 1 -- 172.21.15.40:0/3920944855 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f099c087fb0 con 0x7f09ac0fdf50 2024-02-16T19:22:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.224+0000 7f09aa7fc700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09940692a0 0x7f099406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.224+0000 7f09aa7fc700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09940692a0 0x7f099406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f09a0004fd0 tx=0x7f09a0009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.229+0000 7f09937fe700 1 -- 172.21.15.40:0/3920944855 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f099c0b9bd0 con 0x7f09ac0fdf50 2024-02-16T19:22:28.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.467+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f09ac064b00 con 0x7f09ac0fdf50 2024-02-16T19:22:28.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.468+0000 7f09937fe700 1 -- 172.21.15.40:0/3920944855 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f099c057d00 con 0x7f09ac0fdf50 2024-02-16T19:22:28.469 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-16T19:22:28.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09940692a0 msgr2=0x7f099406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:28.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09940692a0 0x7f099406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f09a0004fd0 tx=0x7f09a0009040 comp rx=0 tx=0).stop 2024-02-16T19:22:28.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09ac0fdf50 msgr2=0x7f09ac113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:28.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09ac0fdf50 0x7f09ac113a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b3b0 tx=0x7f099c009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:28.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 shutdown_connections 2024-02-16T19:22:28.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f09940692a0 0x7f099406b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:28.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09ac0fdf50 0x7f09ac113a80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 --2- 172.21.15.40:0/3920944855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09ac0fe8f0 0x7f09ac113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 >> 172.21.15.40:0/3920944855 conn(0x7f09ac0f79e0 msgr2=0x7f09ac1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 shutdown_connections 2024-02-16T19:22:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:28.470+0000 7f09b18a4700 1 -- 172.21.15.40:0/3920944855 wait complete. 2024-02-16T19:22:29.161 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-16T19:22:29.161 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-16T19:22:29.162 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-02-16T19:22:29.162 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-16T19:22:29.162 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-16T19:22:29.162 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-16T19:22:29.162 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-16T19:22:29.162 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-16T19:22:29.162 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-16T19:22:29.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.162 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-02-16T19:22:29.179 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-16T19:22:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-02-16T19:22:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-02-16T19:22:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-16T19:22:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42245sec preferred_lft 42245sec 2024-02-16T19:22:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-02-16T19:22:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:22:29.181 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:22:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-02-16T19:22:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-02-16T19:22:29.182 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-16T19:22:29.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.182 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-02-16T19:22:29.183 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-02-16T19:22:29.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-02-16T19:22:29.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-02-16T19:22:29.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-02-16T19:22:29.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-16T19:22:29.183 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-02-16T19:22:29.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.360 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-16T19:22:29.388 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-02-16T19:22:29.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.389 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-02-16T19:22:29.460 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-02-16T19:22:29.460 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-16T19:22:29.460 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-16T19:22:29.460 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-16T19:22:29.460 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-16T19:22:29.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.461 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-02-16T19:22:29.461 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-02-16T19:22:29.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-16T19:22:29.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-16T19:22:29.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-16T19:22:29.461 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-02-16T19:22:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:29 smithi040 ceph-mon[29201]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-16T19:22:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:29 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3920944855' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T19:22:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:29 smithi163 ceph-mon[36452]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-16T19:22:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:29 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3920944855' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T19:22:29.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.788 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-02-16T19:22:29.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.803 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-02-16T19:22:29.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.858 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-02-16T19:22:29.858 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-02-16T19:22:29.858 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T19:22:29.858 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-16T19:22:29.858 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-02-16T19:22:29.941 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-16T19:22:29.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:29.941 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-02-16T19:22:29.941 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-02-16T19:22:29.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-16T19:22:29.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-16T19:22:29.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-16T19:22:29.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-16T19:22:29.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-16T19:22:29.942 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-02-16T19:22:30.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:30.075 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-02-16T19:22:30.075 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-02-16T19:22:30.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-02-16T19:22:30.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-02-16T19:22:30.075 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-02-16T19:22:30.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T19:22:30.117 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-16T19:22:30.172 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-16T19:22:30.172 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T19:22:30.173 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-16T19:22:30.227 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-02-16T19:22:30.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:30.227 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-16T19:22:30.345 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-16T19:22:30.345 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-16T19:22:30.345 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-02-16T19:22:30.463 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-16T19:22:30.463 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-16T19:22:30.464 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-02-16T19:22:30.464 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.163:3300" 2024-02-16T19:22:30.464 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-16T19:22:30.468 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-16T19:22:30.468 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-16T19:22:30.469 INFO:teuthology.orchestra.run.smithi040.stdout:invalid new device string format 2024-02-16T19:22:30.469 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.163:3300" 2024-02-16T19:22:30.469 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-16T19:22:30.469 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.163:6789" 2024-02-16T19:22:30.469 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.163:6789:/ 2024-02-16T19:22:30.469 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8c23bd6c-ccff-11ee-95ba-87774f69a715" will pass to kernel 2024-02-16T19:22:30.470 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-16T19:22:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2024-02-16T19:22:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-16T19:22:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2024-02-16T19:22:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-16T19:22:30.471 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-16T19:22:30.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:30.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-16T19:22:30.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:30.505 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-16T19:22:30.569 INFO:teuthology.orchestra.run.smithi040.stdout:client14536 2024-02-16T19:22:30.569 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:22:30.569 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:22:30.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:30.625 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-16T19:22:30.688 INFO:teuthology.orchestra.run.smithi040.stdout:client14536 2024-02-16T19:22:30.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:30.689 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-16T19:22:30.753 INFO:teuthology.orchestra.run.smithi040.stdout:client14536 2024-02-16T19:22:30.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:30.753 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715.client14536/status') 2024-02-16T19:22:30.824 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.14536 (3)192.168.0.1:0/3779740512 2024-02-16T19:22:30.824 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2024-02-16T19:22:30.825 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2024-02-16T19:22:30.825 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2024-02-16T19:22:30.825 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 2.1684e-05 s, 3.2 MB/s 2024-02-16T19:22:30.826 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-02-16T19:22:30.826 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-16T19:22:30.826 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-16T19:22:30.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-16T19:22:31.132 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:31.548 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:31 smithi040 ceph-mon[29201]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-16T19:22:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:31 smithi163 ceph-mon[36452]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-16T19:22:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 -- 172.21.15.40:0/2016801171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5858101f90 msgr2=0x7f5858102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:32.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 --2- 172.21.15.40:0/2016801171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5858101f90 0x7f5858102360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5848004940 tx=0x7f5848009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 -- 172.21.15.40:0/2016801171 shutdown_connections 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 --2- 172.21.15.40:0/2016801171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58580fc670 0x7f58580fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 --2- 172.21.15.40:0/2016801171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5858101f90 0x7f5858102360 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 -- 172.21.15.40:0/2016801171 >> 172.21.15.40:0/2016801171 conn(0x7f5858071500 msgr2=0x7f5858071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 -- 172.21.15.40:0/2016801171 shutdown_connections 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 -- 172.21.15.40:0/2016801171 wait complete. 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 Processor -- start 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 -- start start 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58580fc670 0x7f585810c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5858101f90 0x7f585810ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585810d230 con 0x7f58580fc670 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.746+0000 7f585cd94700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585810d3a0 con 0x7f5858101f90 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58580fc670 0x7f585810c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:32.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58580fc670 0x7f585810c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37696/0 (socket says 172.21.15.40:37696) 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585659c700 1 -- 172.21.15.40:0/703382603 learned_addr learned my addr 172.21.15.40:0/703382603 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f5855d9b700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5858101f90 0x7f585810ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585659c700 1 -- 172.21.15.40:0/703382603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5858101f90 msgr2=0x7f585810ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585659c700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5858101f90 0x7f585810ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585659c700 1 -- 172.21.15.40:0/703382603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5848005040 con 0x7f58580fc670 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585659c700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58580fc670 0x7f585810c7b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5848009100 tx=0x7f5848009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f584effd700 1 -- 172.21.15.40:0/703382603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f584800e040 con 0x7f58580fc670 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f584effd700 1 -- 172.21.15.40:0/703382603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5848013420 con 0x7f58580fc670 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585810d620 con 0x7f58580fc670 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f584effd700 1 -- 172.21.15.40:0/703382603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f584800bee0 con 0x7f58580fc670 2024-02-16T19:22:32.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.747+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585811a1d0 con 0x7f58580fc670 2024-02-16T19:22:32.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.748+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58580754c0 con 0x7f58580fc670 2024-02-16T19:22:32.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.749+0000 7f584effd700 1 -- 172.21.15.40:0/703382603 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f584800ba10 con 0x7f58580fc670 2024-02-16T19:22:32.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.749+0000 7f584effd700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f58440692a0 0x7f584406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:32.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.749+0000 7f584effd700 1 -- 172.21.15.40:0/703382603 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5848088350 con 0x7f58580fc670 2024-02-16T19:22:32.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.749+0000 7f5855d9b700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f58440692a0 0x7f584406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:32.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.750+0000 7f5855d9b700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f58440692a0 0x7f584406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58580fc020 tx=0x7f584000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:32.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.754+0000 7f584effd700 1 -- 172.21.15.40:0/703382603 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f58480580c0 con 0x7f58580fc670 2024-02-16T19:22:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.989+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58581043b0 con 0x7f58580fc670 2024-02-16T19:22:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.989+0000 7f584effd700 1 -- 172.21.15.40:0/703382603 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5848057c50 con 0x7f58580fc670 2024-02-16T19:22:32.990 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-16T19:22:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f58440692a0 msgr2=0x7f584406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f58440692a0 0x7f584406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58580fc020 tx=0x7f584000b040 comp rx=0 tx=0).stop 2024-02-16T19:22:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58580fc670 msgr2=0x7f585810c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58580fc670 0x7f585810c7b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5848009100 tx=0x7f5848009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 shutdown_connections 2024-02-16T19:22:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f58440692a0 0x7f584406b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58580fc670 0x7f585810c7b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 --2- 172.21.15.40:0/703382603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5858101f90 0x7f585810ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 >> 172.21.15.40:0/703382603 conn(0x7f5858071500 msgr2=0x7f58580f9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 shutdown_connections 2024-02-16T19:22:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:32.991+0000 7f585cd94700 1 -- 172.21.15.40:0/703382603 wait complete. 2024-02-16T19:22:33.634 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:33 smithi040 ceph-mon[29201]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 0 op/s 2024-02-16T19:22:33.634 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:33 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/703382603' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T19:22:33.664 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-16T19:22:33.664 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-16T19:22:33.664 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2024-02-16T19:22:33.664 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-16T19:22:33.664 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-16T19:22:33.665 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-16T19:22:33.665 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-16T19:22:33.665 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-16T19:22:33.665 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-16T19:22:33.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:33.665 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2024-02-16T19:22:33.684 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42266sec preferred_lft 42266sec 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-02-16T19:22:33.685 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-16T19:22:33.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:33.686 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-02-16T19:22:33.686 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-02-16T19:22:33.686 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add name ceph-brx type bridge 2024-02-16T19:22:33.686 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr flush dev ceph-brx 2024-02-16T19:22:33.686 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set ceph-brx up 2024-02-16T19:22:33.686 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-16T19:22:33.686 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-02-16T19:22:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:33 smithi163 ceph-mon[36452]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 0 op/s 2024-02-16T19:22:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:33 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/703382603' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-16T19:22:33.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:33.847 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-16T19:22:33.914 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-02-16T19:22:33.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:33.916 DEBUG:teuthology.orchestra.run.smithi163:> route 2024-02-16T19:22:33.996 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2024-02-16T19:22:33.997 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-16T19:22:33.997 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-16T19:22:33.997 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-16T19:22:33.997 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-16T19:22:33.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:33.998 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-02-16T19:22:33.998 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-02-16T19:22:33.998 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-16T19:22:33.998 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-16T19:22:33.998 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-16T19:22:33.998 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-02-16T19:22:34.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:34.309 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-02-16T19:22:34.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:34.324 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2024-02-16T19:22:34.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:34.379 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-02-16T19:22:34.379 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-02-16T19:22:34.379 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T19:22:34.379 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-16T19:22:34.379 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-02-16T19:22:34.469 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-16T19:22:34.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:34.469 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-02-16T19:22:34.469 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-02-16T19:22:34.469 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-16T19:22:34.469 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-16T19:22:34.469 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-16T19:22:34.469 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-16T19:22:34.469 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-16T19:22:34.469 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-02-16T19:22:34.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:34.591 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-02-16T19:22:34.591 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-02-16T19:22:34.591 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2024-02-16T19:22:34.591 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.0 master ceph-brx 2024-02-16T19:22:34.592 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-02-16T19:22:34.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T19:22:34.634 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-16T19:22:34.689 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-16T19:22:34.866 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-16T19:22:34.866 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-16T19:22:34.882 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-02-16T19:22:34.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:34.882 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-02-16T19:22:34.972 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-02-16T19:22:34.972 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-02-16T19:22:34.972 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-02-16T19:22:35.078 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-02-16T19:22:35.079 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-02-16T19:22:35.079 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-02-16T19:22:35.079 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.163:3300" 2024-02-16T19:22:35.079 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-16T19:22:35.083 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-02-16T19:22:35.083 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-02-16T19:22:35.083 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-02-16T19:22:35.083 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.163:3300" 2024-02-16T19:22:35.084 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-16T19:22:35.084 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.163:6789" 2024-02-16T19:22:35.084 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.163:6789:/ 2024-02-16T19:22:35.084 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8c23bd6c-ccff-11ee-95ba-87774f69a715" will pass to kernel 2024-02-16T19:22:35.084 INFO:teuthology.orchestra.run.smithi163.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-16T19:22:35.084 INFO:teuthology.orchestra.run.smithi163.stdout: You just mounted a file system that supports labels which does not 2024-02-16T19:22:35.084 INFO:teuthology.orchestra.run.smithi163.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-16T19:22:35.085 INFO:teuthology.orchestra.run.smithi163.stdout: applications will generate AVC messages and not be allowed access to 2024-02-16T19:22:35.085 INFO:teuthology.orchestra.run.smithi163.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-16T19:22:35.087 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-16T19:22:35.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:35.087 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-16T19:22:35.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:35.114 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-16T19:22:35.170 INFO:teuthology.orchestra.run.smithi163.stdout:client24309 2024-02-16T19:22:35.170 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:22:35.170 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:22:35.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:35.226 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-16T19:22:35.282 INFO:teuthology.orchestra.run.smithi163.stdout:client24309 2024-02-16T19:22:35.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:35.283 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-16T19:22:35.339 INFO:teuthology.orchestra.run.smithi163.stdout:client24309 2024-02-16T19:22:35.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:22:35.340 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715.client24309/status') 2024-02-16T19:22:35.407 INFO:teuthology.orchestra.run.smithi163.stdout:instance: client.24309 (3)192.168.0.1:0/3707514684 2024-02-16T19:22:35.407 INFO:teuthology.orchestra.run.smithi163.stdout:blocklisted: false 2024-02-16T19:22:35.407 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records in 2024-02-16T19:22:35.407 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records out 2024-02-16T19:22:35.407 INFO:teuthology.orchestra.run.smithi163.stderr:70 bytes copied, 2.475e-05 s, 2.8 MB/s 2024-02-16T19:22:35.409 INFO:teuthology.run_tasks:Running task print... 2024-02-16T19:22:35.420 INFO:teuthology.task.print:**** done client 2024-02-16T19:22:35.420 INFO:teuthology.run_tasks:Running task parallel... 2024-02-16T19:22:35.457 INFO:teuthology.task.parallel:starting parallel... 2024-02-16T19:22:35.457 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-16T19:22:35.458 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T19:22:35.458 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:22:35.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-02-16T19:22:35.459 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-16T19:22:35.459 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-16T19:22:35.463 INFO:tasks.workunit:Pulling workunits from ref ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T19:22:35.463 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-16T19:22:35.463 INFO:tasks.workunit:timeout=3h 2024-02-16T19:22:35.463 INFO:tasks.workunit:cleanup=True 2024-02-16T19:22:35.464 DEBUG:teuthology.orchestra.run.smithi040:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-16T19:22:35.484 INFO:teuthology.orchestra.run.smithi040.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-16T19:22:35.484 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-16T19:22:35.484 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-02-16T19:22:35.484 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:35.484 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-16T19:22:35.484 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-16T19:22:35.484 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-02-16 19:22:02.300759971 +0000 2024-02-16T19:22:35.485 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-02-16 19:22:30.501548544 +0000 2024-02-16T19:22:35.485 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-02-16 19:22:02.300759971 +0000 2024-02-16T19:22:35.485 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-16T19:22:35.485 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-16T19:22:35.557 DEBUG:teuthology.orchestra.run.smithi163:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-02-16 19:22:35.550483022 +0000 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-02-16 19:22:35.550483022 +0000 2024-02-16T19:22:35.574 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-02-16 19:22:02.300759971 +0000 2024-02-16T19:22:35.574 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-16T19:22:35.574 DEBUG:teuthology.orchestra.run.smithi163:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-16T19:22:35.637 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:35 smithi163 ceph-mon[36452]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:35.647 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T19:22:35.648 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout ed7ab725e1a43615009dfb74a42bf33d80081bc8 2024-02-16T19:22:35.663 INFO:tasks.workunit.client.0.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-16T19:22:35.702 INFO:tasks.workunit.client.1.smithi163.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-16T19:22:35.743 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:35.860 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:35 smithi040 ceph-mon[29201]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:36.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:36 smithi163 ceph-mon[36452]: pgmap v197: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-16T19:22:36.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:36 smithi040 ceph-mon[29201]: pgmap v197: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-16T19:22:37.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.392+0000 7f6fedc97700 1 -- 172.21.15.40:0/4289097512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe80fda80 msgr2=0x7f6fe80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:37.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.392+0000 7f6fedc97700 1 --2- 172.21.15.40:0/4289097512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe80fda80 0x7f6fe80ffe60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8004940 tx=0x7f6fd8009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:37.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.393+0000 7f6fedc97700 1 -- 172.21.15.40:0/4289097512 shutdown_connections 2024-02-16T19:22:37.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.393+0000 7f6fedc97700 1 --2- 172.21.15.40:0/4289097512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fe81003a0 0x7f6fe8102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:37.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.393+0000 7f6fedc97700 1 --2- 172.21.15.40:0/4289097512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe80fda80 0x7f6fe80ffe60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.393+0000 7f6fedc97700 1 -- 172.21.15.40:0/4289097512 >> 172.21.15.40:0/4289097512 conn(0x7f6fe80f74e0 msgr2=0x7f6fe80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.393+0000 7f6fedc97700 1 -- 172.21.15.40:0/4289097512 shutdown_connections 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.393+0000 7f6fedc97700 1 -- 172.21.15.40:0/4289097512 wait complete. 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.393+0000 7f6fedc97700 1 Processor -- start 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.393+0000 7f6fedc97700 1 -- start start 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fedc97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe80fda80 0x7f6fe810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fedc97700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fe81003a0 0x7f6fe810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fedc97700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe810e470 con 0x7f6fe80fda80 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fedc97700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe810ed10 con 0x7f6fe81003a0 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fe77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe80fda80 0x7f6fe810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fe77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe80fda80 0x7f6fe810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37712/0 (socket says 172.21.15.40:37712) 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fe77fe700 1 -- 172.21.15.40:0/3035278152 learned_addr learned my addr 172.21.15.40:0/3035278152 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fe6ffd700 1 --2- 172.21.15.40:0/3035278152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fe81003a0 0x7f6fe810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:37.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fe77fe700 1 -- 172.21.15.40:0/3035278152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fe81003a0 msgr2=0x7f6fe810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fe77fe700 1 --2- 172.21.15.40:0/3035278152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fe81003a0 0x7f6fe810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:37.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fe77fe700 1 -- 172.21.15.40:0/3035278152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8005040 con 0x7f6fe80fda80 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.394+0000 7f6fe77fe700 1 --2- 172.21.15.40:0/3035278152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fe80fda80 0x7f6fe810d930 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80093b0 tx=0x7f6fd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.395+0000 7f6fcffff700 1 -- 172.21.15.40:0/3035278152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fd800e040 con 0x7f6fe80fda80 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.395+0000 7f6fcffff700 1 -- 172.21.15.40:0/3035278152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fd800bcc0 con 0x7f6fe80fda80 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.395+0000 7f6fedc97700 1 -- 172.21.15.40:0/3035278152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe810ef90 con 0x7f6fe80fda80 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.395+0000 7f6fcffff700 1 -- 172.21.15.40:0/3035278152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fd801c630 con 0x7f6fe80fda80 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.395+0000 7f6fedc97700 1 -- 172.21.15.40:0/3035278152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe810f4d0 con 0x7f6fe80fda80 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.396+0000 7f6fedc97700 1 -- 172.21.15.40:0/3035278152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe804bed0 con 0x7f6fe80fda80 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.396+0000 7f6fcffff700 1 -- 172.21.15.40:0/3035278152 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6fd8013980 con 0x7f6fe80fda80 2024-02-16T19:22:37.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.396+0000 7f6fcffff700 1 --2- 172.21.15.40:0/3035278152 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6fd00692a0 0x7f6fd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.397+0000 7f6fcffff700 1 -- 172.21.15.40:0/3035278152 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6fd8088570 con 0x7f6fe80fda80 2024-02-16T19:22:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.397+0000 7f6fe6ffd700 1 --2- 172.21.15.40:0/3035278152 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6fd00692a0 0x7f6fd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.397+0000 7f6fe6ffd700 1 --2- 172.21.15.40:0/3035278152 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6fd00692a0 0x7f6fd006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc004f40 tx=0x7f6fdc00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:37.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.401+0000 7f6fcffff700 1 -- 172.21.15.40:0/3035278152 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6fd80582e0 con 0x7f6fe80fda80 2024-02-16T19:22:37.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.605+0000 7f6fedc97700 1 -- 172.21.15.40:0/3035278152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f6fe8105a10 con 0x7f6fe80fda80 2024-02-16T19:22:37.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:37.605+0000 7f6fcffff700 1 -- 172.21.15.40:0/3035278152 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f6fd8057e70 con 0x7f6fe80fda80 2024-02-16T19:22:37.608 INFO:teuthology.orchestra.run.smithi040.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-02-16T19:22:38.259 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T19:22:38.260 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:22:38.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-16T19:22:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:39 smithi040 ceph-mon[29201]: pgmap v198: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:22:39.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:39 smithi163 ceph-mon[36452]: pgmap v198: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.275+0000 7f27d9f6f700 1 -- 172.21.15.40:0/1448707677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d41000e0 msgr2=0x7f27d4102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.275+0000 7f27d9f6f700 1 --2- 172.21.15.40:0/1448707677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d41000e0 0x7f27d4102520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27c8004940 tx=0x7f27c8009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.275+0000 7f27d9f6f700 1 -- 172.21.15.40:0/1448707677 shutdown_connections 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.275+0000 7f27d9f6f700 1 --2- 172.21.15.40:0/1448707677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d41000e0 0x7f27d4102520 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.275+0000 7f27d9f6f700 1 --2- 172.21.15.40:0/1448707677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27d4066c50 0x7f27d40ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.275+0000 7f27d9f6f700 1 -- 172.21.15.40:0/1448707677 >> 172.21.15.40:0/1448707677 conn(0x7f27d40f7520 msgr2=0x7f27d40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.275+0000 7f27d9f6f700 1 -- 172.21.15.40:0/1448707677 shutdown_connections 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.275+0000 7f27d9f6f700 1 -- 172.21.15.40:0/1448707677 wait complete. 2024-02-16T19:22:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d9f6f700 1 Processor -- start 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d9f6f700 1 -- start start 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d9f6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d4066c50 0x7f27d410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d9f6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27d41000e0 0x7f27d410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d9f6f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d410e420 con 0x7f27d4066c50 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d9f6f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d410ecc0 con 0x7f27d41000e0 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d4066c50 0x7f27d410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27d41000e0 0x7f27d410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d4066c50 0x7f27d410d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45226/0 (socket says 172.21.15.40:45226) 2024-02-16T19:22:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d37fe700 1 -- 172.21.15.40:0/3295941767 learned_addr learned my addr 172.21.15.40:0/3295941767 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d37fe700 1 -- 172.21.15.40:0/3295941767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27d41000e0 msgr2=0x7f27d410de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d37fe700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27d41000e0 0x7f27d410de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d37fe700 1 -- 172.21.15.40:0/3295941767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c8005040 con 0x7f27d4066c50 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.276+0000 7f27d2ffd700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27d41000e0 0x7f27d410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.277+0000 7f27d37fe700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d4066c50 0x7f27d410d8e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f27c400eb40 tx=0x7f27c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.277+0000 7f27bbfff700 1 -- 172.21.15.40:0/3295941767 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27c4007cc0 con 0x7f27d4066c50 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.277+0000 7f27bbfff700 1 -- 172.21.15.40:0/3295941767 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27c400f040 con 0x7f27d4066c50 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.277+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d410ef40 con 0x7f27d4066c50 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.277+0000 7f27bbfff700 1 -- 172.21.15.40:0/3295941767 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27c400ca10 con 0x7f27d4066c50 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.277+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d410f480 con 0x7f27d4066c50 2024-02-16T19:22:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.278+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d404bed0 con 0x7f27d4066c50 2024-02-16T19:22:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.278+0000 7f27bbfff700 1 -- 172.21.15.40:0/3295941767 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f27c400cb70 con 0x7f27d4066c50 2024-02-16T19:22:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.279+0000 7f27bbfff700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f27bc0692a0 0x7f27bc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.279+0000 7f27bbfff700 1 -- 172.21.15.40:0/3295941767 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f27c4086d20 con 0x7f27d4066c50 2024-02-16T19:22:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.279+0000 7f27d2ffd700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f27bc0692a0 0x7f27bc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.279+0000 7f27d2ffd700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f27bc0692a0 0x7f27bc06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f27c8000c00 tx=0x7f27c8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:40.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.284+0000 7f27bbfff700 1 -- 172.21.15.40:0/3295941767 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f27c404dbd0 con 0x7f27d4066c50 2024-02-16T19:22:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:40.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.507+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f27d4105a10 con 0x7f27d4066c50 2024-02-16T19:22:40.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.507+0000 7f27bbfff700 1 -- 172.21.15.40:0/3295941767 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f27c40568b0 con 0x7f27d4066c50 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f27bc0692a0 msgr2=0x7f27bc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f27bc0692a0 0x7f27bc06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f27c8000c00 tx=0x7f27c8009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d4066c50 msgr2=0x7f27d410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d4066c50 0x7f27d410d8e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f27c400eb40 tx=0x7f27c400b040 comp rx=0 tx=0).stop 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 shutdown_connections 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f27bc0692a0 0x7f27bc06b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27d4066c50 0x7f27d410d8e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 --2- 172.21.15.40:0/3295941767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27d41000e0 0x7f27d410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.509+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 >> 172.21.15.40:0/3295941767 conn(0x7f27d40f7520 msgr2=0x7f27d40f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.510+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 shutdown_connections 2024-02-16T19:22:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:40.510+0000 7f27d9f6f700 1 -- 172.21.15.40:0/3295941767 wait complete. 2024-02-16T19:22:41.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-16T19:22:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:41.600 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:41 smithi040 ceph-mon[29201]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-02-16T19:22:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:41 smithi163 ceph-mon[36452]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-02-16T19:22:43.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.373+0000 7f4eeed55700 1 -- 172.21.15.40:0/1134829610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fbe40 msgr2=0x7f4ee80fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:43.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.373+0000 7f4eeed55700 1 --2- 172.21.15.40:0/1134829610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fbe40 0x7f4ee80fc250 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4edc004940 tx=0x7f4edc009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:43.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.373+0000 7f4eeed55700 1 -- 172.21.15.40:0/1134829610 shutdown_connections 2024-02-16T19:22:43.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.373+0000 7f4eeed55700 1 --2- 172.21.15.40:0/1134829610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee80fcfb0 0x7f4ee80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:43.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.373+0000 7f4eeed55700 1 --2- 172.21.15.40:0/1134829610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fbe40 0x7f4ee80fc250 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:43.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.373+0000 7f4eeed55700 1 -- 172.21.15.40:0/1134829610 >> 172.21.15.40:0/1134829610 conn(0x7f4ee80f74e0 msgr2=0x7f4ee80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.373+0000 7f4eeed55700 1 -- 172.21.15.40:0/1134829610 shutdown_connections 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.374+0000 7f4eeed55700 1 -- 172.21.15.40:0/1134829610 wait complete. 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.374+0000 7f4eeed55700 1 Processor -- start 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.374+0000 7f4eeed55700 1 -- start start 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4eeed55700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee80fbe40 0x7f4ee8110610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4eeed55700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fcfb0 0x7f4ee8112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4eeed55700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8066460 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4eeed55700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80665a0 con 0x7f4ee80fbe40 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4ee7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fcfb0 0x7f4ee8112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4ee7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fcfb0 0x7f4ee8112b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45246/0 (socket says 172.21.15.40:45246) 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4ee7fff700 1 -- 172.21.15.40:0/692536193 learned_addr learned my addr 172.21.15.40:0/692536193 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4eecaf1700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee80fbe40 0x7f4ee8110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4ee7fff700 1 -- 172.21.15.40:0/692536193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee80fbe40 msgr2=0x7f4ee8110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4ee7fff700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee80fbe40 0x7f4ee8110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4ee7fff700 1 -- 172.21.15.40:0/692536193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc005040 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4eecaf1700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee80fbe40 0x7f4ee8110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4ee7fff700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fcfb0 0x7f4ee8112b60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400eb70 tx=0x7f4ed400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.375+0000 7f4ee57fa700 1 -- 172.21.15.40:0/692536193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed4007cc0 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.376+0000 7f4ee57fa700 1 -- 172.21.15.40:0/692536193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ed400f040 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.376+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee8113160 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.376+0000 7f4ee57fa700 1 -- 172.21.15.40:0/692536193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed4009640 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.376+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee81136a0 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.377+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee804bed0 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.377+0000 7f4ee57fa700 1 -- 172.21.15.40:0/692536193 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4ed40097a0 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.377+0000 7f4ee57fa700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4ed80692a0 0x7f4ed806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.378+0000 7f4ee57fa700 1 -- 172.21.15.40:0/692536193 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4ed4086d40 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.378+0000 7f4eecaf1700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4ed80692a0 0x7f4ed806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.378+0000 7f4eecaf1700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4ed80692a0 0x7f4ed806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009b30 tx=0x7f4edc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:43.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.382+0000 7f4ee57fa700 1 -- 172.21.15.40:0/692536193 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4ed4053020 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.383 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:43 smithi040 ceph-mon[29201]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:22:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.616+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f4ee8113b30 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.616+0000 7f4ee57fa700 1 -- 172.21.15.40:0/692536193 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f4ed4056640 con 0x7f4ee80fcfb0 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.618+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4ed80692a0 msgr2=0x7f4ed806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.618+0000 7f4eeed55700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4ed80692a0 0x7f4ed806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009b30 tx=0x7f4edc009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.618+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fcfb0 msgr2=0x7f4ee8112b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.618+0000 7f4eeed55700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fcfb0 0x7f4ee8112b60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400eb70 tx=0x7f4ed400b040 comp rx=0 tx=0).stop 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.618+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 shutdown_connections 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.619+0000 7f4eeed55700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4ed80692a0 0x7f4ed806b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.619+0000 7f4eeed55700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee80fbe40 0x7f4ee8110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.619+0000 7f4eeed55700 1 --2- 172.21.15.40:0/692536193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80fcfb0 0x7f4ee8112b60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.619+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 >> 172.21.15.40:0/692536193 conn(0x7f4ee80f74e0 msgr2=0x7f4ee8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.619+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 shutdown_connections 2024-02-16T19:22:43.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:43.619+0000 7f4eeed55700 1 -- 172.21.15.40:0/692536193 wait complete. 2024-02-16T19:22:43.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:43 smithi163 ceph-mon[36452]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:22:44.441 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-16T19:22:44.709 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:45 smithi040 ceph-mon[29201]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:22:45.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:45 smithi163 ceph-mon[36452]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.557+0000 7fc0f220d700 1 -- 172.21.15.40:0/1395428468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fc1c0 msgr2=0x7fc0ec0fc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.557+0000 7fc0f220d700 1 --2- 172.21.15.40:0/1395428468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fc1c0 0x7fc0ec0fc630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0004940 tx=0x7fc0e0009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.557+0000 7fc0f220d700 1 -- 172.21.15.40:0/1395428468 shutdown_connections 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.557+0000 7fc0f220d700 1 --2- 172.21.15.40:0/1395428468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fc1c0 0x7fc0ec0fc630 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.557+0000 7fc0f220d700 1 --2- 172.21.15.40:0/1395428468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec0fb870 0x7fc0ec0fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.557+0000 7fc0f220d700 1 -- 172.21.15.40:0/1395428468 >> 172.21.15.40:0/1395428468 conn(0x7fc0ec0f74e0 msgr2=0x7fc0ec0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.557+0000 7fc0f220d700 1 -- 172.21.15.40:0/1395428468 shutdown_connections 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0f220d700 1 -- 172.21.15.40:0/1395428468 wait complete. 2024-02-16T19:22:46.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0f220d700 1 Processor -- start 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0f220d700 1 -- start start 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0f220d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fb870 0x7fc0ec10fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0f220d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec0fc1c0 0x7fc0ec110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0f220d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec110620 con 0x7fc0ec0fb870 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0f220d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec110ec0 con 0x7fc0ec0fc1c0 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fb870 0x7fc0ec10fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fb870 0x7fc0ec10fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45250/0 (socket says 172.21.15.40:45250) 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0eb7fe700 1 -- 172.21.15.40:0/1925102378 learned_addr learned my addr 172.21.15.40:0/1925102378 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.558+0000 7fc0eaffd700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec0fc1c0 0x7fc0ec110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0eb7fe700 1 -- 172.21.15.40:0/1925102378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec0fc1c0 msgr2=0x7fc0ec110020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0eb7fe700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec0fc1c0 0x7fc0ec110020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0eb7fe700 1 -- 172.21.15.40:0/1925102378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0005040 con 0x7fc0ec0fb870 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0eb7fe700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fb870 0x7fc0ec10fae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc002a80 tx=0x7fc0dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:46.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0d3fff700 1 -- 172.21.15.40:0/1925102378 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0dc007cc0 con 0x7fc0ec0fb870 2024-02-16T19:22:46.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec1111a0 con 0x7fc0ec0fb870 2024-02-16T19:22:46.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0d3fff700 1 -- 172.21.15.40:0/1925102378 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0dc00f040 con 0x7fc0ec0fb870 2024-02-16T19:22:46.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0d3fff700 1 -- 172.21.15.40:0/1925102378 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0dc00dac0 con 0x7fc0ec0fb870 2024-02-16T19:22:46.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.559+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec1116e0 con 0x7fc0ec0fb870 2024-02-16T19:22:46.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.560+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec04bed0 con 0x7fc0ec0fb870 2024-02-16T19:22:46.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.561+0000 7fc0d3fff700 1 -- 172.21.15.40:0/1925102378 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc0dc00dc20 con 0x7fc0ec0fb870 2024-02-16T19:22:46.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.561+0000 7fc0d3fff700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc0d40692a0 0x7fc0d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:46.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.561+0000 7fc0d3fff700 1 -- 172.21.15.40:0/1925102378 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc0dc086f60 con 0x7fc0ec0fb870 2024-02-16T19:22:46.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.561+0000 7fc0eaffd700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc0d40692a0 0x7fc0d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:46.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.561+0000 7fc0eaffd700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc0d40692a0 0x7fc0d406b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0004f90 tx=0x7fc0e0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:46.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.566+0000 7fc0d3fff700 1 -- 172.21.15.40:0/1925102378 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc0dc056c50 con 0x7fc0ec0fb870 2024-02-16T19:22:46.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.759+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fc0ec107c00 con 0x7fc0ec0fb870 2024-02-16T19:22:46.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.759+0000 7fc0d3fff700 1 -- 172.21.15.40:0/1925102378 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fc0dc0567e0 con 0x7fc0ec0fb870 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc0d40692a0 msgr2=0x7fc0d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc0d40692a0 0x7fc0d406b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0004f90 tx=0x7fc0e0009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fb870 msgr2=0x7fc0ec10fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fb870 0x7fc0ec10fae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc002a80 tx=0x7fc0dc0046b0 comp rx=0 tx=0).stop 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 shutdown_connections 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fc0d40692a0 0x7fc0d406b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0ec0fb870 0x7fc0ec10fae0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 --2- 172.21.15.40:0/1925102378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0ec0fc1c0 0x7fc0ec110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 >> 172.21.15.40:0/1925102378 conn(0x7fc0ec0f74e0 msgr2=0x7fc0ec103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 shutdown_connections 2024-02-16T19:22:46.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:46.761+0000 7fc0f220d700 1 -- 172.21.15.40:0/1925102378 wait complete. 2024-02-16T19:22:47.517 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:47 smithi040 ceph-mon[29201]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:22:47.541 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-16T19:22:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:47 smithi163 ceph-mon[36452]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-16T19:22:47.814 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:49.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:49 smithi040 ceph-mon[29201]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T19:22:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:49 smithi163 ceph-mon[36452]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T19:22:50.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.841+0000 7f5d803d2700 1 -- 172.21.15.40:0/3391088512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d780fda80 msgr2=0x7f5d780ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:50.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.841+0000 7f5d803d2700 1 --2- 172.21.15.40:0/3391088512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d780fda80 0x7f5d780ffe60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5d74004940 tx=0x7f5d74009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.841+0000 7f5d803d2700 1 -- 172.21.15.40:0/3391088512 shutdown_connections 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.841+0000 7f5d803d2700 1 --2- 172.21.15.40:0/3391088512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d781003a0 0x7f5d78102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.841+0000 7f5d803d2700 1 --2- 172.21.15.40:0/3391088512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d780fda80 0x7f5d780ffe60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.841+0000 7f5d803d2700 1 -- 172.21.15.40:0/3391088512 >> 172.21.15.40:0/3391088512 conn(0x7f5d780f74e0 msgr2=0x7f5d780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.841+0000 7f5d803d2700 1 -- 172.21.15.40:0/3391088512 shutdown_connections 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.841+0000 7f5d803d2700 1 -- 172.21.15.40:0/3391088512 wait complete. 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d803d2700 1 Processor -- start 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d803d2700 1 -- start start 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d803d2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d780fda80 0x7f5d780fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d803d2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d781003a0 0x7f5d780fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d803d2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780fbf90 con 0x7f5d781003a0 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d803d2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780fc100 con 0x7f5d780fda80 2024-02-16T19:22:50.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d7d96d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d781003a0 0x7f5d780fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d7d96d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d781003a0 0x7f5d780fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48938/0 (socket says 172.21.15.40:48938) 2024-02-16T19:22:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d7d96d700 1 -- 172.21.15.40:0/2936804086 learned_addr learned my addr 172.21.15.40:0/2936804086 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:50.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d7e16e700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d780fda80 0x7f5d780fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:50.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d7d96d700 1 -- 172.21.15.40:0/2936804086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d780fda80 msgr2=0x7f5d780fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d7d96d700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d780fda80 0x7f5d780fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.842+0000 7f5d7d96d700 1 -- 172.21.15.40:0/2936804086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74005040 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.843+0000 7f5d7d96d700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d781003a0 0x7f5d780fb990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5d68002a80 tx=0x7f5d680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.843+0000 7f5d6effd700 1 -- 172.21.15.40:0/2936804086 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d68007cc0 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.843+0000 7f5d6effd700 1 -- 172.21.15.40:0/2936804086 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d6800f040 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.843+0000 7f5d6effd700 1 -- 172.21.15.40:0/2936804086 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d6800dac0 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.843+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780fc3e0 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.843+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d781158f0 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.844+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d78107a50 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.845+0000 7f5d6effd700 1 -- 172.21.15.40:0/2936804086 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5d6800dc20 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.845+0000 7f5d6effd700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5d640692a0 0x7f5d6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.845+0000 7f5d6effd700 1 -- 172.21.15.40:0/2936804086 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5d68086d60 con 0x7f5d781003a0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.845+0000 7f5d7e16e700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5d640692a0 0x7f5d6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:50.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.846+0000 7f5d7e16e700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5d640692a0 0x7f5d6406b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d74004890 tx=0x7f5d74009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:50.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:50.850+0000 7f5d6effd700 1 -- 172.21.15.40:0/2936804086 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5d68056ad0 con 0x7f5d781003a0 2024-02-16T19:22:51.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.067+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d7802b450 con 0x7f5d640692a0 2024-02-16T19:22:51.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.073+0000 7f5d6effd700 1 -- 172.21.15.40:0/2936804086 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f5d7802b450 con 0x7f5d640692a0 2024-02-16T19:22:51.074 INFO:teuthology.orchestra.run.smithi040.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:22:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5d640692a0 msgr2=0x7f5d6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5d640692a0 0x7f5d6406b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d74004890 tx=0x7f5d74009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d781003a0 msgr2=0x7f5d780fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d781003a0 0x7f5d780fb990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5d68002a80 tx=0x7f5d680046b0 comp rx=0 tx=0).stop 2024-02-16T19:22:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 shutdown_connections 2024-02-16T19:22:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f5d640692a0 0x7f5d6406b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:51.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d780fda80 0x7f5d780fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:51.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 --2- 172.21.15.40:0/2936804086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d781003a0 0x7f5d780fb990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:51.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 >> 172.21.15.40:0/2936804086 conn(0x7f5d780f74e0 msgr2=0x7f5d780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:51.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 shutdown_connections 2024-02-16T19:22:51.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:51.075+0000 7f5d803d2700 1 -- 172.21.15.40:0/2936804086 wait complete. 2024-02-16T19:22:51.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:51 smithi040 ceph-mon[29201]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-16T19:22:51.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:51 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:51.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:51 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:51 smithi163 ceph-mon[36452]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-16T19:22:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:51 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:51 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:22:51.876 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-16T19:22:51.877 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:22:51.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-02-16T19:22:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:22:52.582 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:52 smithi040 ceph-mon[29201]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:52.582 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:52 smithi040 ceph-mon[29201]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:22:52.582 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:52 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:52.582 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:52 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:52.582 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:52 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:52.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:52 smithi163 ceph-mon[36452]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:52.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:52 smithi163 ceph-mon[36452]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:22:52.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:52 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:22:52.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:52 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:22:52.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:52 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:22:53.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:53 smithi040 ceph-mon[29201]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:22:53.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:53 smithi040 ceph-mon[29201]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:22:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:53 smithi163 ceph-mon[36452]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:22:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:53 smithi163 ceph-mon[36452]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.107+0000 7f2fd80a8700 1 -- 172.21.15.40:0/2055976420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 msgr2=0x7f2fd00fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.107+0000 7f2fd80a8700 1 --2- 172.21.15.40:0/2055976420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 0x7f2fd00fc250 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc004940 tx=0x7f2fcc009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.107+0000 7f2fd80a8700 1 -- 172.21.15.40:0/2055976420 shutdown_connections 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.107+0000 7f2fd80a8700 1 --2- 172.21.15.40:0/2055976420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00fcfb0 0x7f2fd00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.107+0000 7f2fd80a8700 1 --2- 172.21.15.40:0/2055976420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 0x7f2fd00fc250 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.107+0000 7f2fd80a8700 1 -- 172.21.15.40:0/2055976420 >> 172.21.15.40:0/2055976420 conn(0x7f2fd00f7520 msgr2=0x7f2fd00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.107+0000 7f2fd80a8700 1 -- 172.21.15.40:0/2055976420 shutdown_connections 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.108+0000 7f2fd80a8700 1 -- 172.21.15.40:0/2055976420 wait complete. 2024-02-16T19:22:54.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.108+0000 7f2fd80a8700 1 Processor -- start 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.108+0000 7f2fd80a8700 1 -- start start 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.108+0000 7f2fd80a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 0x7f2fd010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.108+0000 7f2fd80a8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00fcfb0 0x7f2fd010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.108+0000 7f2fd80a8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd010e3f0 con 0x7f2fd00fbe40 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.108+0000 7f2fd80a8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd010ec90 con 0x7f2fd00fcfb0 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.108+0000 7f2fd5e44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 0x7f2fd010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd5e44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 0x7f2fd010d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48966/0 (socket says 172.21.15.40:48966) 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd5e44700 1 -- 172.21.15.40:0/4182938256 learned_addr learned my addr 172.21.15.40:0/4182938256 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd5643700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00fcfb0 0x7f2fd010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd5e44700 1 -- 172.21.15.40:0/4182938256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00fcfb0 msgr2=0x7f2fd010ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd5e44700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00fcfb0 0x7f2fd010ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd5e44700 1 -- 172.21.15.40:0/4182938256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc005040 con 0x7f2fd00fbe40 2024-02-16T19:22:54.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd5e44700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 0x7f2fd010d8b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00b3b0 tx=0x7f2fcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:54.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fc67fc700 1 -- 172.21.15.40:0/4182938256 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fcc00e040 con 0x7f2fd00fbe40 2024-02-16T19:22:54.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fc67fc700 1 -- 172.21.15.40:0/4182938256 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fcc00bd90 con 0x7f2fd00fbe40 2024-02-16T19:22:54.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd010ef10 con 0x7f2fd00fbe40 2024-02-16T19:22:54.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fc67fc700 1 -- 172.21.15.40:0/4182938256 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fcc01c630 con 0x7f2fd00fbe40 2024-02-16T19:22:54.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.109+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd010f450 con 0x7f2fd00fbe40 2024-02-16T19:22:54.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.111+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd004bed0 con 0x7f2fd00fbe40 2024-02-16T19:22:54.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.111+0000 7f2fc67fc700 1 -- 172.21.15.40:0/4182938256 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2fcc013980 con 0x7f2fd00fbe40 2024-02-16T19:22:54.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.112+0000 7f2fc67fc700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fbc0692b0 0x7f2fbc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.112+0000 7f2fc67fc700 1 -- 172.21.15.40:0/4182938256 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2fcc088670 con 0x7f2fd00fbe40 2024-02-16T19:22:54.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.112+0000 7f2fd5643700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fbc0692b0 0x7f2fbc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.113+0000 7f2fd5643700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fbc0692b0 0x7f2fbc06b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009fd0 tx=0x7f2fc000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:54.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.118+0000 7f2fc67fc700 1 -- 172.21.15.40:0/4182938256 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2fcc0583e0 con 0x7f2fd00fbe40 2024-02-16T19:22:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.442+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fd010f0a0 con 0x7f2fbc0692b0 2024-02-16T19:22:54.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.443+0000 7f2fc67fc700 1 -- 172.21.15.40:0/4182938256 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f2fd010f0a0 con 0x7f2fbc0692b0 2024-02-16T19:22:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fbc0692b0 msgr2=0x7f2fbc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fbc0692b0 0x7f2fbc06b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009fd0 tx=0x7f2fc000b040 comp rx=0 tx=0).stop 2024-02-16T19:22:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 msgr2=0x7f2fd010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 0x7f2fd010d8b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00b3b0 tx=0x7f2fcc009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 shutdown_connections 2024-02-16T19:22:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f2fbc0692b0 0x7f2fbc06b760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fd00fbe40 0x7f2fd010d8b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 --2- 172.21.15.40:0/4182938256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00fcfb0 0x7f2fd010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.445+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 >> 172.21.15.40:0/4182938256 conn(0x7f2fd00f7520 msgr2=0x7f2fd0065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.446+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 shutdown_connections 2024-02-16T19:22:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.446+0000 7f2fd80a8700 1 -- 172.21.15.40:0/4182938256 wait complete. 2024-02-16T19:22:54.458 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:22:54.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.578+0000 7f9f2600b700 1 -- 172.21.15.40:0/729235190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 msgr2=0x7f9f201007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.578+0000 7f9f2600b700 1 --2- 172.21.15.40:0/729235190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 0x7f9f201007b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9f14004940 tx=0x7f9f14009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.578+0000 7f9f2600b700 1 -- 172.21.15.40:0/729235190 shutdown_connections 2024-02-16T19:22:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.578+0000 7f9f2600b700 1 --2- 172.21.15.40:0/729235190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 0x7f9f201007b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.578+0000 7f9f2600b700 1 --2- 172.21.15.40:0/729235190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f200ff150 0x7f9f200ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.578+0000 7f9f2600b700 1 -- 172.21.15.40:0/729235190 >> 172.21.15.40:0/729235190 conn(0x7f9f200fa920 msgr2=0x7f9f200fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.578+0000 7f9f2600b700 1 -- 172.21.15.40:0/729235190 shutdown_connections 2024-02-16T19:22:54.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.578+0000 7f9f2600b700 1 -- 172.21.15.40:0/729235190 wait complete. 2024-02-16T19:22:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.579+0000 7f9f2600b700 1 Processor -- start 2024-02-16T19:22:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.579+0000 7f9f2600b700 1 -- start start 2024-02-16T19:22:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.579+0000 7f9f2600b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f200ff150 0x7f9f2010fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.579+0000 7f9f2600b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 0x7f9f20110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.579+0000 7f9f2600b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20110640 con 0x7f9f20100360 2024-02-16T19:22:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.579+0000 7f9f2600b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20110ee0 con 0x7f9f200ff150 2024-02-16T19:22:54.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 0x7f9f20110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 0x7f9f20110040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48990/0 (socket says 172.21.15.40:48990) 2024-02-16T19:22:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f200ff150 0x7f9f2010fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1effd700 1 -- 172.21.15.40:0/252182797 learned_addr learned my addr 172.21.15.40:0/252182797 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1effd700 1 -- 172.21.15.40:0/252182797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f200ff150 msgr2=0x7f9f2010fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1effd700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f200ff150 0x7f9f2010fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1effd700 1 -- 172.21.15.40:0/252182797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14005040 con 0x7f9f20100360 2024-02-16T19:22:54.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1f7fe700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f200ff150 0x7f9f2010fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:22:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f1effd700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 0x7f9f20110040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f07fff700 1 -- 172.21.15.40:0/252182797 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f14021070 con 0x7f9f20100360 2024-02-16T19:22:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f20111160 con 0x7f9f20100360 2024-02-16T19:22:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.580+0000 7f9f07fff700 1 -- 172.21.15.40:0/252182797 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f1401c400 con 0x7f9f20100360 2024-02-16T19:22:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.581+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f20111640 con 0x7f9f20100360 2024-02-16T19:22:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.581+0000 7f9f07fff700 1 -- 172.21.15.40:0/252182797 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f1400e070 con 0x7f9f20100360 2024-02-16T19:22:54.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.582+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2004bed0 con 0x7f9f20100360 2024-02-16T19:22:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.582+0000 7f9f07fff700 1 -- 172.21.15.40:0/252182797 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9f1401ca50 con 0x7f9f20100360 2024-02-16T19:22:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.583+0000 7f9f07fff700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9f080692a0 0x7f9f0806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.583+0000 7f9f07fff700 1 -- 172.21.15.40:0/252182797 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9f140883e0 con 0x7f9f20100360 2024-02-16T19:22:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.583+0000 7f9f1f7fe700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9f080692a0 0x7f9f0806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.583+0000 7f9f1f7fe700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9f080692a0 0x7f9f0806b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009b80 tx=0x7f9f1000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:54.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.587+0000 7f9f07fff700 1 -- 172.21.15.40:0/252182797 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9f14058150 con 0x7f9f20100360 2024-02-16T19:22:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.853+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f20111a50 con 0x7f9f080692a0 2024-02-16T19:22:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.855+0000 7f9f07fff700 1 -- 172.21.15.40:0/252182797 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f9f20111a50 con 0x7f9f080692a0 2024-02-16T19:22:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.857+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9f080692a0 msgr2=0x7f9f0806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.857+0000 7f9f2600b700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9f080692a0 0x7f9f0806b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009b80 tx=0x7f9f1000b040 comp rx=0 tx=0).stop 2024-02-16T19:22:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.857+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 msgr2=0x7f9f20110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.857+0000 7f9f2600b700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 0x7f9f20110040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f14009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.858+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 shutdown_connections 2024-02-16T19:22:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.858+0000 7f9f2600b700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f9f080692a0 0x7f9f0806b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.858+0000 7f9f2600b700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f200ff150 0x7f9f2010fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.858+0000 7f9f2600b700 1 --2- 172.21.15.40:0/252182797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f20100360 0x7f9f20110040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.858+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 >> 172.21.15.40:0/252182797 conn(0x7f9f200fa920 msgr2=0x7f9f20103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.858+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 shutdown_connections 2024-02-16T19:22:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.858+0000 7f9f2600b700 1 -- 172.21.15.40:0/252182797 wait complete. 2024-02-16T19:22:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.975+0000 7f3fa877f700 1 -- 172.21.15.40:0/4197535956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 msgr2=0x7f3fa00fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.975+0000 7f3fa877f700 1 --2- 172.21.15.40:0/4197535956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 0x7f3fa00fc250 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c004940 tx=0x7f3f9c009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.975+0000 7f3fa877f700 1 -- 172.21.15.40:0/4197535956 shutdown_connections 2024-02-16T19:22:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.975+0000 7f3fa877f700 1 --2- 172.21.15.40:0/4197535956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fa00fcfb0 0x7f3fa00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.975+0000 7f3fa877f700 1 --2- 172.21.15.40:0/4197535956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 0x7f3fa00fc250 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.975+0000 7f3fa877f700 1 -- 172.21.15.40:0/4197535956 >> 172.21.15.40:0/4197535956 conn(0x7f3fa00f74e0 msgr2=0x7f3fa00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.975+0000 7f3fa877f700 1 -- 172.21.15.40:0/4197535956 shutdown_connections 2024-02-16T19:22:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.975+0000 7f3fa877f700 1 -- 172.21.15.40:0/4197535956 wait complete. 2024-02-16T19:22:54.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa877f700 1 Processor -- start 2024-02-16T19:22:54.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa877f700 1 -- start start 2024-02-16T19:22:54.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa877f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 0x7f3fa00fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa877f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fa00fcfb0 0x7f3fa00fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa877f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00fe980 con 0x7f3fa00fbe40 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa877f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00feaf0 con 0x7f3fa00fcfb0 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa651b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 0x7f3fa00fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa651b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 0x7f3fa00fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49006/0 (socket says 172.21.15.40:49006) 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa651b700 1 -- 172.21.15.40:0/3036057068 learned_addr learned my addr 172.21.15.40:0/3036057068 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa5d1a700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fa00fcfb0 0x7f3fa00fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa651b700 1 -- 172.21.15.40:0/3036057068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fa00fcfb0 msgr2=0x7f3fa00fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa651b700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fa00fcfb0 0x7f3fa00fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa651b700 1 -- 172.21.15.40:0/3036057068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c005040 con 0x7f3fa00fbe40 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.976+0000 7f3fa651b700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 0x7f3fa00fff00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000c00 tx=0x7f3f9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.977+0000 7f3f96ffd700 1 -- 172.21.15.40:0/3036057068 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9c021070 con 0x7f3fa00fbe40 2024-02-16T19:22:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.977+0000 7f3f96ffd700 1 -- 172.21.15.40:0/3036057068 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f9c01c400 con 0x7f3fa00fbe40 2024-02-16T19:22:54.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.977+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa00fed70 con 0x7f3fa00fbe40 2024-02-16T19:22:54.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.977+0000 7f3f96ffd700 1 -- 172.21.15.40:0/3036057068 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9c00e070 con 0x7f3fa00fbe40 2024-02-16T19:22:54.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.977+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa01158e0 con 0x7f3fa00fbe40 2024-02-16T19:22:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.978+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa004bed0 con 0x7f3fa00fbe40 2024-02-16T19:22:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.978+0000 7f3f96ffd700 1 -- 172.21.15.40:0/3036057068 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3f9c01ca50 con 0x7f3fa00fbe40 2024-02-16T19:22:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.979+0000 7f3f96ffd700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3f8c0692b0 0x7f3f8c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.979+0000 7f3f96ffd700 1 -- 172.21.15.40:0/3036057068 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3f9c088400 con 0x7f3fa00fbe40 2024-02-16T19:22:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.979+0000 7f3fa5d1a700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3f8c0692b0 0x7f3f8c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.979+0000 7f3fa5d1a700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3f8c0692b0 0x7f3f8c06b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009b80 tx=0x7f3f9000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:54.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:54.985+0000 7f3f96ffd700 1 -- 172.21.15.40:0/3036057068 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3f9c058170 con 0x7f3fa00fbe40 2024-02-16T19:22:55.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.235+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fa0065dd0 con 0x7f3f8c0692b0 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.248+0000 7f3f96ffd700 1 -- 172.21.15.40:0/3036057068 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f3fa0065dd0 con 0x7f3f8c0692b0 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (5m) 28s ago 6m 25.1M - 0.25.0 c8568f914cd2 aadccae1ccec 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (6m) 28s ago 6m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (5m) 33s ago 5m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (4m) 28s ago 6m 84.5M - 9.4.7 954c08fa6188 8447b5580554 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (47s) 28s ago 46s 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (54s) 28s ago 53s 10.4M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:22:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (44s) 33s ago 44s 14.4M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (51s) 33s ago 51s 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:9283 running (7m) 28s ago 7m 482M - 17.2.7-513-gb88fae66 870a0944ccb8 4e9e9bb4389a 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283 running (5m) 33s ago 5m 436M - 17.2.7-513-gb88fae66 870a0944ccb8 06e26641f085 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (7m) 28s ago 7m 44.9M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (5m) 33s ago 5m 35.4M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (6m) 28s ago 6m 14.0M - 1.5.0 0da6a335fe13 61d16f3f00d4 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (5m) 33s ago 5m 16.7M - 1.5.0 0da6a335fe13 c45ebf41f1b8 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (4m) 28s ago 4m 47.9M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (4m) 28s ago 4m 46.4M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (3m) 28s ago 3m 45.3M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (3m) 33s ago 3m 44.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (2m) 33s ago 2m 41.1M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (2m) 33s ago 2m 44.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:22:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (4m) 28s ago 5m 51.4M - 2.43.0 a07b618ecd1d 557bd885dbc5 2024-02-16T19:22:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3f8c0692b0 msgr2=0x7f3f8c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3f8c0692b0 0x7f3f8c06b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009b80 tx=0x7f3f9000b040 comp rx=0 tx=0).stop 2024-02-16T19:22:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 msgr2=0x7f3fa00fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 0x7f3fa00fff00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000c00 tx=0x7f3f9c009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 shutdown_connections 2024-02-16T19:22:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3f8c0692b0 0x7f3f8c06b760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fa00fbe40 0x7f3fa00fff00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 --2- 172.21.15.40:0/3036057068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fa00fcfb0 0x7f3fa00fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 >> 172.21.15.40:0/3036057068 conn(0x7f3fa00f74e0 msgr2=0x7f3fa00656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 shutdown_connections 2024-02-16T19:22:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.250+0000 7f3fa877f700 1 -- 172.21.15.40:0/3036057068 wait complete. 2024-02-16T19:22:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.368+0000 7f45859ec700 1 -- 172.21.15.40:0/2959938088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 msgr2=0x7f4580102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:55.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.368+0000 7f45859ec700 1 --2- 172.21.15.40:0/2959938088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 0x7f4580102520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4574004940 tx=0x7f4574009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.368+0000 7f45859ec700 1 -- 172.21.15.40:0/2959938088 shutdown_connections 2024-02-16T19:22:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.368+0000 7f45859ec700 1 --2- 172.21.15.40:0/2959938088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 0x7f4580102520 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.368+0000 7f45859ec700 1 --2- 172.21.15.40:0/2959938088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800663c0 0x7f45800ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.368+0000 7f45859ec700 1 -- 172.21.15.40:0/2959938088 >> 172.21.15.40:0/2959938088 conn(0x7f45800f74e0 msgr2=0x7f45800f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.368+0000 7f45859ec700 1 -- 172.21.15.40:0/2959938088 shutdown_connections 2024-02-16T19:22:55.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.368+0000 7f45859ec700 1 -- 172.21.15.40:0/2959938088 wait complete. 2024-02-16T19:22:55.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f45859ec700 1 Processor -- start 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f45859ec700 1 -- start start 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f45859ec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800663c0 0x7f458010d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f45859ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 0x7f458010dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f45859ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458010e4e0 con 0x7f45801000e0 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f45859ec700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458010ed80 con 0x7f45800663c0 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f457e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 0x7f458010dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f457effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800663c0 0x7f458010d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f457e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 0x7f458010dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49030/0 (socket says 172.21.15.40:49030) 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.369+0000 7f457e7fc700 1 -- 172.21.15.40:0/2323826642 learned_addr learned my addr 172.21.15.40:0/2323826642 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f457e7fc700 1 -- 172.21.15.40:0/2323826642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800663c0 msgr2=0x7f458010d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:55.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f457e7fc700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800663c0 0x7f458010d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f457e7fc700 1 -- 172.21.15.40:0/2323826642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4574005040 con 0x7f45801000e0 2024-02-16T19:22:55.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f457effd700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800663c0 0x7f458010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:22:55.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f457e7fc700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 0x7f458010dee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4574000c00 tx=0x7f4574009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:55.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f45677fe700 1 -- 172.21.15.40:0/2323826642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457400e040 con 0x7f45801000e0 2024-02-16T19:22:55.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458010f000 con 0x7f45801000e0 2024-02-16T19:22:55.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f45677fe700 1 -- 172.21.15.40:0/2323826642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f457400bbf0 con 0x7f45801000e0 2024-02-16T19:22:55.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f45677fe700 1 -- 172.21.15.40:0/2323826642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457401c630 con 0x7f45801000e0 2024-02-16T19:22:55.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.370+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458010f540 con 0x7f45801000e0 2024-02-16T19:22:55.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.372+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4580107a50 con 0x7f45801000e0 2024-02-16T19:22:55.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.372+0000 7f45677fe700 1 -- 172.21.15.40:0/2323826642 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f457400bd60 con 0x7f45801000e0 2024-02-16T19:22:55.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.373+0000 7f45677fe700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f45680692a0 0x7f456806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:55.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.373+0000 7f45677fe700 1 -- 172.21.15.40:0/2323826642 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4574088560 con 0x7f45801000e0 2024-02-16T19:22:55.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.373+0000 7f457effd700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f45680692a0 0x7f456806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:55.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.373+0000 7f457effd700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f45680692a0 0x7f456806b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f457000f7c0 tx=0x7f4570011040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:55.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.381+0000 7f45677fe700 1 -- 172.21.15.40:0/2323826642 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f45740582d0 con 0x7f45801000e0 2024-02-16T19:22:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:55 smithi040 ceph-mon[29201]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:55 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.726+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f458002b2d0 con 0x7f45801000e0 2024-02-16T19:22:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.726+0000 7f45677fe700 1 -- 172.21.15.40:0/2323826642 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f4574057e60 con 0x7f45801000e0 2024-02-16T19:22:55.728 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:22:55.729 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:22:55.730 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:22:55.730 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:22:55.730 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:22:55.730 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 14 2024-02-16T19:22:55.730 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:22:55.730 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:22:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.729+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f45680692a0 msgr2=0x7f456806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.729+0000 7f45859ec700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f45680692a0 0x7f456806b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f457000f7c0 tx=0x7f4570011040 comp rx=0 tx=0).stop 2024-02-16T19:22:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.729+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 msgr2=0x7f458010dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.729+0000 7f45859ec700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 0x7f458010dee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4574000c00 tx=0x7f4574009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.730+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 shutdown_connections 2024-02-16T19:22:55.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.730+0000 7f45859ec700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f45680692a0 0x7f456806b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.730+0000 7f45859ec700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800663c0 0x7f458010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.730+0000 7f45859ec700 1 --2- 172.21.15.40:0/2323826642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45801000e0 0x7f458010dee0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.730+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 >> 172.21.15.40:0/2323826642 conn(0x7f45800f74e0 msgr2=0x7f45800f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.730+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 shutdown_connections 2024-02-16T19:22:55.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.730+0000 7f45859ec700 1 -- 172.21.15.40:0/2323826642 wait complete. 2024-02-16T19:22:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:55 smithi163 ceph-mon[36452]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:55 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:22:55.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.856+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3204379468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780fbe40 msgr2=0x7f3e780fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.856+0000 7f3e7eb16700 1 --2- 172.21.15.40:0/3204379468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780fbe40 0x7f3e780fc250 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c004940 tx=0x7f3e6c009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.856+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3204379468 shutdown_connections 2024-02-16T19:22:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.856+0000 7f3e7eb16700 1 --2- 172.21.15.40:0/3204379468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e780fcfb0 0x7f3e780fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.856+0000 7f3e7eb16700 1 --2- 172.21.15.40:0/3204379468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780fbe40 0x7f3e780fc250 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.856+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3204379468 >> 172.21.15.40:0/3204379468 conn(0x7f3e780f74e0 msgr2=0x7f3e780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.857+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3204379468 shutdown_connections 2024-02-16T19:22:55.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.857+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3204379468 wait complete. 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.857+0000 7f3e7eb16700 1 Processor -- start 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.857+0000 7f3e7eb16700 1 -- start start 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e7eb16700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e780fcfb0 0x7f3e78110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e7eb16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780663b0 0x7f3e78112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e7eb16700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78113260 con 0x7f3e780663b0 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e7eb16700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e781133d0 con 0x7f3e780fcfb0 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780663b0 0x7f3e78112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780663b0 0x7f3e78112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49054/0 (socket says 172.21.15.40:49054) 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e77fff700 1 -- 172.21.15.40:0/3811385458 learned_addr learned my addr 172.21.15.40:0/3811385458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:55.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e7c8b2700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e780fcfb0 0x7f3e78110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:55.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e77fff700 1 -- 172.21.15.40:0/3811385458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e780fcfb0 msgr2=0x7f3e78110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:55.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e77fff700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e780fcfb0 0x7f3e78110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:55.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e77fff700 1 -- 172.21.15.40:0/3811385458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c005040 con 0x7f3e780663b0 2024-02-16T19:22:55.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e77fff700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780663b0 0x7f3e78112c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3e64002a80 tx=0x7f3e640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.858+0000 7f3e757fa700 1 -- 172.21.15.40:0/3811385458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e64007cc0 con 0x7f3e780663b0 2024-02-16T19:22:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.859+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e781136b0 con 0x7f3e780663b0 2024-02-16T19:22:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.859+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e78113bc0 con 0x7f3e780663b0 2024-02-16T19:22:55.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.859+0000 7f3e757fa700 1 -- 172.21.15.40:0/3811385458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e6400f040 con 0x7f3e780663b0 2024-02-16T19:22:55.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.859+0000 7f3e757fa700 1 -- 172.21.15.40:0/3811385458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6400dac0 con 0x7f3e780663b0 2024-02-16T19:22:55.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.860+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7804bed0 con 0x7f3e780663b0 2024-02-16T19:22:55.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.861+0000 7f3e757fa700 1 -- 172.21.15.40:0/3811385458 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3e6400dcc0 con 0x7f3e780663b0 2024-02-16T19:22:55.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.861+0000 7f3e757fa700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3e680692b0 0x7f3e6806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:55.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.861+0000 7f3e757fa700 1 -- 172.21.15.40:0/3811385458 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3e64086e80 con 0x7f3e780663b0 2024-02-16T19:22:55.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.862+0000 7f3e7c8b2700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3e680692b0 0x7f3e6806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:55.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.862+0000 7f3e7c8b2700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3e680692b0 0x7f3e6806b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00b3b0 tx=0x7f3e6c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:55.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:55.868+0000 7f3e757fa700 1 -- 172.21.15.40:0/3811385458 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3e640530e0 con 0x7f3e780663b0 2024-02-16T19:22:56.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.166+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3e78104930 con 0x7f3e780663b0 2024-02-16T19:22:56.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.167+0000 7f3e757fa700 1 -- 172.21.15.40:0/3811385458 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1820 (secure 0 0 0) 0x7f3e64056700 con 0x7f3e780663b0 2024-02-16T19:22:56.168 INFO:teuthology.orchestra.run.smithi040.stdout:e17 2024-02-16T19:22:56.168 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:22:56.168 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:22:56.168 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:22:56.168 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:56.168 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 17 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:22:17.019628+0000 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:22:56.169 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:22:56.170 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:56.171 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:56.171 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:22:56.171 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:22:56.171 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:22:56.171 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.169+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3e680692b0 msgr2=0x7f3e6806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.169+0000 7f3e7eb16700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3e680692b0 0x7f3e6806b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00b3b0 tx=0x7f3e6c009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.169+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780663b0 msgr2=0x7f3e78112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.169+0000 7f3e7eb16700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780663b0 0x7f3e78112c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3e64002a80 tx=0x7f3e640046b0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.170+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 shutdown_connections 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.170+0000 7f3e7eb16700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f3e680692b0 0x7f3e6806b760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.170+0000 7f3e7eb16700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e780fcfb0 0x7f3e78110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.170+0000 7f3e7eb16700 1 --2- 172.21.15.40:0/3811385458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e780663b0 0x7f3e78112c80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.170+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 >> 172.21.15.40:0/3811385458 conn(0x7f3e780f74e0 msgr2=0x7f3e78100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:56.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.170+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 shutdown_connections 2024-02-16T19:22:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.170+0000 7f3e7eb16700 1 -- 172.21.15.40:0/3811385458 wait complete. 2024-02-16T19:22:56.173 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-02-16T19:22:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.300+0000 7f1404700700 1 -- 172.21.15.40:0/2129069252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc1000c0 msgr2=0x7f13fc102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.300+0000 7f1404700700 1 --2- 172.21.15.40:0/2129069252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc1000c0 0x7f13fc102520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13ec004940 tx=0x7f13ec009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.300+0000 7f1404700700 1 -- 172.21.15.40:0/2129069252 shutdown_connections 2024-02-16T19:22:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.300+0000 7f1404700700 1 --2- 172.21.15.40:0/2129069252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc1000c0 0x7f13fc102520 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.300+0000 7f1404700700 1 --2- 172.21.15.40:0/2129069252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13fc0663e0 0x7f13fc0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.300+0000 7f1404700700 1 -- 172.21.15.40:0/2129069252 >> 172.21.15.40:0/2129069252 conn(0x7f13fc0f74e0 msgr2=0x7f13fc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.300+0000 7f1404700700 1 -- 172.21.15.40:0/2129069252 shutdown_connections 2024-02-16T19:22:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.300+0000 7f1404700700 1 -- 172.21.15.40:0/2129069252 wait complete. 2024-02-16T19:22:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f1404700700 1 Processor -- start 2024-02-16T19:22:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f1404700700 1 -- start start 2024-02-16T19:22:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f1404700700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc0663e0 0x7f13fc0fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f1404700700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13fc1000c0 0x7f13fc0fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f1404700700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0fbff0 con 0x7f13fc0663e0 2024-02-16T19:22:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f1404700700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0fc160 con 0x7f13fc1000c0 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f140249c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc0663e0 0x7f13fc0fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f140249c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc0663e0 0x7f13fc0fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49072/0 (socket says 172.21.15.40:49072) 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f140249c700 1 -- 172.21.15.40:0/3482762680 learned_addr learned my addr 172.21.15.40:0/3482762680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.301+0000 7f1401c9b700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13fc1000c0 0x7f13fc0fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f140249c700 1 -- 172.21.15.40:0/3482762680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13fc1000c0 msgr2=0x7f13fc0fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f140249c700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13fc1000c0 0x7f13fc0fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f140249c700 1 -- 172.21.15.40:0/3482762680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec005040 con 0x7f13fc0663e0 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f1401c9b700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13fc1000c0 0x7f13fc0fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f140249c700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc0663e0 0x7f13fc0fd570 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f13f800eb40 tx=0x7f13f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f13f2ffd700 1 -- 172.21.15.40:0/3482762680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13f8007cc0 con 0x7f13fc0663e0 2024-02-16T19:22:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f13f2ffd700 1 -- 172.21.15.40:0/3482762680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13f800f040 con 0x7f13fc0663e0 2024-02-16T19:22:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f13f2ffd700 1 -- 172.21.15.40:0/3482762680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13f800ca10 con 0x7f13fc0663e0 2024-02-16T19:22:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13fc0fc3e0 con 0x7f13fc0663e0 2024-02-16T19:22:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.302+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc115900 con 0x7f13fc0663e0 2024-02-16T19:22:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.303+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13fc04bed0 con 0x7f13fc0663e0 2024-02-16T19:22:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.304+0000 7f13f2ffd700 1 -- 172.21.15.40:0/3482762680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f13f800cb70 con 0x7f13fc0663e0 2024-02-16T19:22:56.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.304+0000 7f13f2ffd700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f13e80692b0 0x7f13e806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:56.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.304+0000 7f13f2ffd700 1 -- 172.21.15.40:0/3482762680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f13f8086cf0 con 0x7f13fc0663e0 2024-02-16T19:22:56.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.304+0000 7f1401c9b700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f13e80692b0 0x7f13e806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:56.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.305+0000 7f1401c9b700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f13e80692b0 0x7f13e806b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f13ec013740 tx=0x7f13ec009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:56.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.312+0000 7f13f2ffd700 1 -- 172.21.15.40:0/3482762680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f13f804dbd0 con 0x7f13fc0663e0 2024-02-16T19:22:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.573+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13fc100d60 con 0x7f13e80692b0 2024-02-16T19:22:56.574 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:56 smithi040 ceph-mon[29201]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:56.574 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:56 smithi040 ceph-mon[29201]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:56.575 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:56 smithi040 ceph-mon[29201]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:56.575 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:56 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2323826642' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:22:56.575 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:56 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3811385458' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:22:56.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.574+0000 7f13f2ffd700 1 -- 172.21.15.40:0/3482762680 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f13fc100d60 con 0x7f13e80692b0 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [], 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "", 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 image", 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:22:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:22:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f13e80692b0 msgr2=0x7f13e806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f13e80692b0 0x7f13e806b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f13ec013740 tx=0x7f13ec009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc0663e0 msgr2=0x7f13fc0fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc0663e0 0x7f13fc0fd570 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f13f800eb40 tx=0x7f13f800b040 comp rx=0 tx=0).stop 2024-02-16T19:22:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 shutdown_connections 2024-02-16T19:22:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f13e80692b0 0x7f13e806b760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13fc0663e0 0x7f13fc0fd570 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 --2- 172.21.15.40:0/3482762680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13fc1000c0 0x7f13fc0fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 >> 172.21.15.40:0/3482762680 conn(0x7f13fc0f74e0 msgr2=0x7f13fc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 shutdown_connections 2024-02-16T19:22:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.577+0000 7f1404700700 1 -- 172.21.15.40:0/3482762680 wait complete. 2024-02-16T19:22:56.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.706+0000 7f03e9f90700 1 -- 172.21.15.40:0/1952954593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fc7f0 msgr2=0x7f03e40fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.706+0000 7f03e9f90700 1 --2- 172.21.15.40:0/1952954593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fc7f0 0x7f03e40fcc40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f03d8004940 tx=0x7f03d8009e30 comp rx=0 tx=0).stop 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.706+0000 7f03e9f90700 1 -- 172.21.15.40:0/1952954593 shutdown_connections 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.706+0000 7f03e9f90700 1 --2- 172.21.15.40:0/1952954593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fc7f0 0x7f03e40fcc40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.706+0000 7f03e9f90700 1 --2- 172.21.15.40:0/1952954593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e40fb5e0 0x7f03e40fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.706+0000 7f03e9f90700 1 -- 172.21.15.40:0/1952954593 >> 172.21.15.40:0/1952954593 conn(0x7f03e40f6ca0 msgr2=0x7f03e40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.707+0000 7f03e9f90700 1 -- 172.21.15.40:0/1952954593 shutdown_connections 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.707+0000 7f03e9f90700 1 -- 172.21.15.40:0/1952954593 wait complete. 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.707+0000 7f03e9f90700 1 Processor -- start 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.707+0000 7f03e9f90700 1 -- start start 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e9f90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fb5e0 0x7f03e4110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e9f90700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e40fc7f0 0x7f03e4112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e9f90700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40fe7b0 con 0x7f03e40fb5e0 2024-02-16T19:22:56.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e9f90700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40fe8f0 con 0x7f03e40fc7f0 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fb5e0 0x7f03e4110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e40fc7f0 0x7f03e4112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fb5e0 0x7f03e4110670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49078/0 (socket says 172.21.15.40:49078) 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e37fe700 1 -- 172.21.15.40:0/4244168454 learned_addr learned my addr 172.21.15.40:0/4244168454 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e2ffd700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e40fc7f0 0x7f03e4112bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.40:53514/0 (socket says 172.21.15.40:53514) 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e37fe700 1 -- 172.21.15.40:0/4244168454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e40fc7f0 msgr2=0x7f03e4112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e37fe700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e40fc7f0 0x7f03e4112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e37fe700 1 -- 172.21.15.40:0/4244168454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d8005040 con 0x7f03e40fb5e0 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e2ffd700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e40fc7f0 0x7f03e4112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:22:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.708+0000 7f03e37fe700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fb5e0 0x7f03e4110670 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f03d400eb70 tx=0x7f03d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:56.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.709+0000 7f03cbfff700 1 -- 172.21.15.40:0/4244168454 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d4007cc0 con 0x7f03e40fb5e0 2024-02-16T19:22:56.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.709+0000 7f03cbfff700 1 -- 172.21.15.40:0/4244168454 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03d400f040 con 0x7f03e40fb5e0 2024-02-16T19:22:56.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.709+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e410c4b0 con 0x7f03e40fb5e0 2024-02-16T19:22:56.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.709+0000 7f03cbfff700 1 -- 172.21.15.40:0/4244168454 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d400cb20 con 0x7f03e40fb5e0 2024-02-16T19:22:56.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.709+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e410c9f0 con 0x7f03e40fb5e0 2024-02-16T19:22:56.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.710+0000 7f03cbfff700 1 -- 172.21.15.40:0/4244168454 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f03d400cc80 con 0x7f03e40fb5e0 2024-02-16T19:22:56.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.710+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e404bed0 con 0x7f03e40fb5e0 2024-02-16T19:22:56.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.711+0000 7f03cbfff700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03cc0692a0 0x7f03cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:22:56.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.711+0000 7f03cbfff700 1 -- 172.21.15.40:0/4244168454 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f03d4086b80 con 0x7f03e40fb5e0 2024-02-16T19:22:56.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.711+0000 7f03e2ffd700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03cc0692a0 0x7f03cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:22:56.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.711+0000 7f03e2ffd700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03cc0692a0 0x7f03cc06b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f03d800b0d0 tx=0x7f03d801c040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:22:56.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:56.716+0000 7f03cbfff700 1 -- 172.21.15.40:0/4244168454 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f03d404dbd0 con 0x7f03e40fb5e0 2024-02-16T19:22:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:56 smithi163 ceph-mon[36452]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:56 smithi163 ceph-mon[36452]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:56 smithi163 ceph-mon[36452]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:56 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2323826642' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:22:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:56 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3811385458' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:22:57.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.061+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f03e410d0c0 con 0x7f03e40fb5e0 2024-02-16T19:22:57.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.062+0000 7f03cbfff700 1 -- 172.21.15.40:0/4244168454 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f03d4052f10 con 0x7f03e40fb5e0 2024-02-16T19:22:57.063 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:22:57.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03cc0692a0 msgr2=0x7f03cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03cc0692a0 0x7f03cc06b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f03d800b0d0 tx=0x7f03d801c040 comp rx=0 tx=0).stop 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fb5e0 msgr2=0x7f03e4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fb5e0 0x7f03e4110670 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f03d400eb70 tx=0x7f03d400b040 comp rx=0 tx=0).stop 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 shutdown_connections 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f03cc0692a0 0x7f03cc06b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03e40fb5e0 0x7f03e4110670 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 --2- 172.21.15.40:0/4244168454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e40fc7f0 0x7f03e4112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 >> 172.21.15.40:0/4244168454 conn(0x7f03e40f6ca0 msgr2=0x7f03e40656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 shutdown_connections 2024-02-16T19:22:57.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:22:57.064+0000 7f03e9f90700 1 -- 172.21.15.40:0/4244168454 wait complete. 2024-02-16T19:22:57.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:57 smithi040 ceph-mon[29201]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:57.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:57 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/4244168454' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:22:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:57 smithi163 ceph-mon[36452]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:57 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/4244168454' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:22:58.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:58 smithi040 ceph-mon[29201]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:58.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:58 smithi163 ceph-mon[36452]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:22:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:22:59 smithi040 ceph-mon[29201]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:22:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:22:59 smithi163 ceph-mon[36452]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:01 smithi040 ceph-mon[29201]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:01 smithi163 ceph-mon[36452]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:03 smithi040 ceph-mon[29201]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:03 smithi163 ceph-mon[36452]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:05.539 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:05 smithi163 ceph-mon[36452]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:05 smithi040 ceph-mon[29201]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:07 smithi040 ceph-mon[29201]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:07 smithi163 ceph-mon[36452]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:09 smithi040 ceph-mon[29201]: pgmap v213: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:09 smithi163 ceph-mon[36452]: pgmap v213: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:23:10.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:10.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:11 smithi040 ceph-mon[29201]: pgmap v214: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:23:11.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:11 smithi163 ceph-mon[36452]: pgmap v214: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:23:13.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:13 smithi040 ceph-mon[29201]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:23:13.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:13 smithi163 ceph-mon[36452]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:23:15.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:15 smithi040 ceph-mon[29201]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:23:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:15 smithi163 ceph-mon[36452]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-16T19:23:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:17 smithi040 ceph-mon[29201]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T19:23:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:17 smithi163 ceph-mon[36452]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T19:23:19.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:19 smithi040 ceph-mon[29201]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T19:23:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:19 smithi163 ceph-mon[36452]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T19:23:21.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:21 smithi040 ceph-mon[29201]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T19:23:21.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:21 smithi163 ceph-mon[36452]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-16T19:23:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:23 smithi040 ceph-mon[29201]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:23:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:23 smithi163 ceph-mon[36452]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:23:24.404 INFO:tasks.workunit.client.1.smithi163.stderr:Note: switching to 'ed7ab725e1a43615009dfb74a42bf33d80081bc8'. 2024-02-16T19:23:24.404 INFO:tasks.workunit.client.1.smithi163.stderr: 2024-02-16T19:23:24.404 INFO:tasks.workunit.client.1.smithi163.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-16T19:23:24.404 INFO:tasks.workunit.client.1.smithi163.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-16T19:23:24.404 INFO:tasks.workunit.client.1.smithi163.stderr:state without impacting any branches by switching back to a branch. 2024-02-16T19:23:24.404 INFO:tasks.workunit.client.1.smithi163.stderr: 2024-02-16T19:23:24.404 INFO:tasks.workunit.client.1.smithi163.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr: 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr: git switch -c 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr: 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr:Or undo this operation with: 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr: 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr: git switch - 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr: 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr: 2024-02-16T19:23:24.405 INFO:tasks.workunit.client.1.smithi163.stderr:HEAD is now at ed7ab725e1a qa: test fuse/kclient for mds upgrade seq 2024-02-16T19:23:24.409 DEBUG:teuthology.orchestra.run.smithi163:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-16T19:23:24.468 INFO:tasks.workunit.client.1.smithi163.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-16T19:23:24.471 INFO:tasks.workunit.client.1.smithi163.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-16T19:23:24.471 INFO:tasks.workunit.client.1.smithi163.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-16T19:23:25.239 INFO:tasks.workunit.client.1.smithi163.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-16T19:23:25.310 INFO:tasks.workunit.client.1.smithi163.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-16T19:23:25.353 INFO:tasks.workunit.client.1.smithi163.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-16T19:23:25.354 INFO:tasks.workunit.client.1.smithi163.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-16T19:23:25.355 INFO:tasks.workunit.client.1.smithi163.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-16T19:23:25.393 INFO:tasks.workunit.client.1.smithi163.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-16T19:23:25.397 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:23:25.397 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-16T19:23:25.453 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-16T19:23:25.454 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-16T19:23:25.454 DEBUG:teuthology.orchestra.run.smithi163:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed7ab725e1a43615009dfb74a42bf33d80081bc8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-16T19:23:25.513 INFO:tasks.workunit.client.1.smithi163.stderr:+ mkdir -p fsstress 2024-02-16T19:23:25.515 INFO:tasks.workunit.client.1.smithi163.stderr:+ pushd fsstress 2024-02-16T19:23:25.515 INFO:tasks.workunit.client.1.smithi163.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T19:23:25.515 INFO:tasks.workunit.client.1.smithi163.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-16T19:23:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:25 smithi040 ceph-mon[29201]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:23:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:25 smithi163 ceph-mon[36452]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:23:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:26.046 INFO:tasks.workunit.client.1.smithi163.stderr:+ tar xzf ltp-full.tgz 2024-02-16T19:23:27.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.177+0000 7fe55a67d700 1 -- 172.21.15.40:0/2328667891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 msgr2=0x7fe5540ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.177+0000 7fe55a67d700 1 --2- 172.21.15.40:0/2328667891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 0x7fe5540ffaf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe544004940 tx=0x7fe544009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.177+0000 7fe55a67d700 1 -- 172.21.15.40:0/2328667891 shutdown_connections 2024-02-16T19:23:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.177+0000 7fe55a67d700 1 --2- 172.21.15.40:0/2328667891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5541000c0 0x7fe554102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.177+0000 7fe55a67d700 1 --2- 172.21.15.40:0/2328667891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 0x7fe5540ffaf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.177+0000 7fe55a67d700 1 -- 172.21.15.40:0/2328667891 >> 172.21.15.40:0/2328667891 conn(0x7fe5540f74e0 msgr2=0x7fe5540f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.177+0000 7fe55a67d700 1 -- 172.21.15.40:0/2328667891 shutdown_connections 2024-02-16T19:23:27.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.177+0000 7fe55a67d700 1 -- 172.21.15.40:0/2328667891 wait complete. 2024-02-16T19:23:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.178+0000 7fe55a67d700 1 Processor -- start 2024-02-16T19:23:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.178+0000 7fe55a67d700 1 -- start start 2024-02-16T19:23:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.178+0000 7fe55a67d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 0x7fe55410d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.178+0000 7fe55a67d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5541000c0 0x7fe55410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.178+0000 7fe55a67d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55410e450 con 0x7fe5540663e0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.178+0000 7fe55a67d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55410ecf0 con 0x7fe5541000c0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe553fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 0x7fe55410d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe553fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 0x7fe55410d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38878/0 (socket says 172.21.15.40:38878) 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe553fff700 1 -- 172.21.15.40:0/1568100481 learned_addr learned my addr 172.21.15.40:0/1568100481 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe5537fe700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5541000c0 0x7fe55410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe553fff700 1 -- 172.21.15.40:0/1568100481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5541000c0 msgr2=0x7fe55410de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe553fff700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5541000c0 0x7fe55410de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe553fff700 1 -- 172.21.15.40:0/1568100481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544005040 con 0x7fe5540663e0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe553fff700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 0x7fe55410d910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe544000c00 tx=0x7fe544009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.179+0000 7fe550ff9700 1 -- 172.21.15.40:0/1568100481 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe544021070 con 0x7fe5540663e0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.180+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55410ef70 con 0x7fe5540663e0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.180+0000 7fe550ff9700 1 -- 172.21.15.40:0/1568100481 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe54401c400 con 0x7fe5540663e0 2024-02-16T19:23:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.180+0000 7fe550ff9700 1 -- 172.21.15.40:0/1568100481 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe54400e070 con 0x7fe5540663e0 2024-02-16T19:23:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.180+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55410f450 con 0x7fe5540663e0 2024-02-16T19:23:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.181+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55404bed0 con 0x7fe5540663e0 2024-02-16T19:23:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.182+0000 7fe550ff9700 1 -- 172.21.15.40:0/1568100481 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe54401ca50 con 0x7fe5540663e0 2024-02-16T19:23:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.183+0000 7fe550ff9700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fe53c0692a0 0x7fe53c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.183+0000 7fe550ff9700 1 -- 172.21.15.40:0/1568100481 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe544088df0 con 0x7fe5540663e0 2024-02-16T19:23:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.183+0000 7fe5537fe700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fe53c0692a0 0x7fe53c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.183+0000 7fe5537fe700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fe53c0692a0 0x7fe53c06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe548009b80 tx=0x7fe54800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:27.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.186+0000 7fe550ff9700 1 -- 172.21.15.40:0/1568100481 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe544058c00 con 0x7fe5540663e0 2024-02-16T19:23:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.402+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe554100d20 con 0x7fe53c0692a0 2024-02-16T19:23:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.404+0000 7fe550ff9700 1 -- 172.21.15.40:0/1568100481 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fe554100d20 con 0x7fe53c0692a0 2024-02-16T19:23:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fe53c0692a0 msgr2=0x7fe53c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fe53c0692a0 0x7fe53c06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe548009b80 tx=0x7fe54800b040 comp rx=0 tx=0).stop 2024-02-16T19:23:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 msgr2=0x7fe55410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 0x7fe55410d910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe544000c00 tx=0x7fe544009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 shutdown_connections 2024-02-16T19:23:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fe53c0692a0 0x7fe53c06b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5540663e0 0x7fe55410d910 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 --2- 172.21.15.40:0/1568100481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5541000c0 0x7fe55410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 >> 172.21.15.40:0/1568100481 conn(0x7fe5540f74e0 msgr2=0x7fe5540f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 shutdown_connections 2024-02-16T19:23:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.406+0000 7fe55a67d700 1 -- 172.21.15.40:0/1568100481 wait complete. 2024-02-16T19:23:27.418 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:23:27.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.522+0000 7fef0211d700 1 -- 172.21.15.40:0/2832417708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 msgr2=0x7feefc0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.522+0000 7fef0211d700 1 --2- 172.21.15.40:0/2832417708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 0x7feefc0fc250 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feeec004940 tx=0x7feeec009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:27.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.522+0000 7fef0211d700 1 -- 172.21.15.40:0/2832417708 shutdown_connections 2024-02-16T19:23:27.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.522+0000 7fef0211d700 1 --2- 172.21.15.40:0/2832417708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feefc0fcfb0 0x7feefc0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.522+0000 7fef0211d700 1 --2- 172.21.15.40:0/2832417708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 0x7feefc0fc250 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.522+0000 7fef0211d700 1 -- 172.21.15.40:0/2832417708 >> 172.21.15.40:0/2832417708 conn(0x7feefc0f74e0 msgr2=0x7feefc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:27.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:27 smithi040 ceph-mon[29201]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:23:27.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:23:27.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:27.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:27 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:23:27.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.522+0000 7fef0211d700 1 -- 172.21.15.40:0/2832417708 shutdown_connections 2024-02-16T19:23:27.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.522+0000 7fef0211d700 1 -- 172.21.15.40:0/2832417708 wait complete. 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.523+0000 7fef0211d700 1 Processor -- start 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7fef0211d700 1 -- start start 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7fef0211d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 0x7feefc0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7fef0211d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feefc0fcfb0 0x7feefc0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7fef0211d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0fe980 con 0x7feefc0fbe40 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7fef0211d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0feaf0 con 0x7feefc0fcfb0 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7feefaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feefc0fcfb0 0x7feefc0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7feefb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 0x7feefc0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7feefb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 0x7feefc0fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39094/0 (socket says 172.21.15.40:39094) 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7feefaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feefc0fcfb0 0x7feefc0fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.40:35780/0 (socket says 172.21.15.40:35780) 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7feefb7fe700 1 -- 172.21.15.40:0/1095598302 learned_addr learned my addr 172.21.15.40:0/1095598302 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7feefb7fe700 1 -- 172.21.15.40:0/1095598302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feefc0fcfb0 msgr2=0x7feefc0fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.524+0000 7feefb7fe700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feefc0fcfb0 0x7feefc0fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.525+0000 7feefb7fe700 1 -- 172.21.15.40:0/1095598302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec005040 con 0x7feefc0fbe40 2024-02-16T19:23:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.525+0000 7feefaffd700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feefc0fcfb0 0x7feefc0fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:23:27.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.525+0000 7feefb7fe700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 0x7feefc0fff00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feeec000c00 tx=0x7feeec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:27.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.525+0000 7feee3fff700 1 -- 172.21.15.40:0/1095598302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeec021070 con 0x7feefc0fbe40 2024-02-16T19:23:27.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.525+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc0fed70 con 0x7feefc0fbe40 2024-02-16T19:23:27.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.525+0000 7feee3fff700 1 -- 172.21.15.40:0/1095598302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeec01c400 con 0x7feefc0fbe40 2024-02-16T19:23:27.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.525+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc1158e0 con 0x7feefc0fbe40 2024-02-16T19:23:27.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.526+0000 7feee3fff700 1 -- 172.21.15.40:0/1095598302 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeec00e070 con 0x7feefc0fbe40 2024-02-16T19:23:27.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.527+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc04bed0 con 0x7feefc0fbe40 2024-02-16T19:23:27.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.528+0000 7feee3fff700 1 -- 172.21.15.40:0/1095598302 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7feeec013cd0 con 0x7feefc0fbe40 2024-02-16T19:23:27.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.528+0000 7feee3fff700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7feee4069310 0x7feee406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.528+0000 7feee3fff700 1 -- 172.21.15.40:0/1095598302 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7feeec088300 con 0x7feefc0fbe40 2024-02-16T19:23:27.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.528+0000 7feefaffd700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7feee4069310 0x7feee406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.529+0000 7feefaffd700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7feee4069310 0x7feee406b7c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feef0009b80 tx=0x7feef000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:27.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.537+0000 7feee3fff700 1 -- 172.21.15.40:0/1095598302 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7feeec058070 con 0x7feefc0fbe40 2024-02-16T19:23:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:27 smithi163 ceph-mon[36452]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-16T19:23:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:23:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:23:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:27 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:23:27.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.827+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feefc065dd0 con 0x7feee4069310 2024-02-16T19:23:27.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.828+0000 7feee3fff700 1 -- 172.21.15.40:0/1095598302 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7feefc065dd0 con 0x7feee4069310 2024-02-16T19:23:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7feee4069310 msgr2=0x7feee406b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7feee4069310 0x7feee406b7c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feef0009b80 tx=0x7feef000b040 comp rx=0 tx=0).stop 2024-02-16T19:23:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 msgr2=0x7feefc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 0x7feefc0fff00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feeec000c00 tx=0x7feeec009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 shutdown_connections 2024-02-16T19:23:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7feee4069310 0x7feee406b7c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feefc0fbe40 0x7feefc0fff00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 --2- 172.21.15.40:0/1095598302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feefc0fcfb0 0x7feefc0fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.830+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 >> 172.21.15.40:0/1095598302 conn(0x7feefc0f74e0 msgr2=0x7feefc0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:27.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.831+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 shutdown_connections 2024-02-16T19:23:27.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.831+0000 7fef0211d700 1 -- 172.21.15.40:0/1095598302 wait complete. 2024-02-16T19:23:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.949+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2179112236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fb220 msgr2=0x7fbfe40fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.949+0000 7fbfeb9f5700 1 --2- 172.21.15.40:0/2179112236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fb220 0x7fbfe40fb630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0004940 tx=0x7fbfe0009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.950+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2179112236 shutdown_connections 2024-02-16T19:23:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.950+0000 7fbfeb9f5700 1 --2- 172.21.15.40:0/2179112236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe40fe2b0 0x7fbfe40fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.950+0000 7fbfeb9f5700 1 --2- 172.21.15.40:0/2179112236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fb220 0x7fbfe40fb630 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.950+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2179112236 >> 172.21.15.40:0/2179112236 conn(0x7fbfe40f6ca0 msgr2=0x7fbfe40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:27.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.950+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2179112236 shutdown_connections 2024-02-16T19:23:27.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.950+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2179112236 wait complete. 2024-02-16T19:23:27.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.950+0000 7fbfeb9f5700 1 Processor -- start 2024-02-16T19:23:27.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.950+0000 7fbfeb9f5700 1 -- start start 2024-02-16T19:23:27.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfeb9f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe40fb220 0x7fbfe410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfeb9f5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fe2b0 0x7fbfe410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfeb9f5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe410e4e0 con 0x7fbfe40fe2b0 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfeb9f5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe410ed80 con 0x7fbfe40fb220 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe8f90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fe2b0 0x7fbfe410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe9791700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe40fb220 0x7fbfe410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe9791700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe40fb220 0x7fbfe410d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.40:35800/0 (socket says 172.21.15.40:35800) 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe9791700 1 -- 172.21.15.40:0/2820547844 learned_addr learned my addr 172.21.15.40:0/2820547844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe9791700 1 -- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fe2b0 msgr2=0x7fbfe410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe9791700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fe2b0 0x7fbfe410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe9791700 1 -- 172.21.15.40:0/2820547844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0005040 con 0x7fbfe40fb220 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe8f90700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fe2b0 0x7fbfe410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.951+0000 7fbfe9791700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe40fb220 0x7fbfe410d9a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009fd0 tx=0x7fbfe0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.952+0000 7fbfd9ffb700 1 -- 172.21.15.40:0/2820547844 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe0021070 con 0x7fbfe40fb220 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.952+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe410f000 con 0x7fbfe40fb220 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.952+0000 7fbfd9ffb700 1 -- 172.21.15.40:0/2820547844 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfe001c400 con 0x7fbfe40fb220 2024-02-16T19:23:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.952+0000 7fbfd9ffb700 1 -- 172.21.15.40:0/2820547844 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe000e070 con 0x7fbfe40fb220 2024-02-16T19:23:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.952+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe410f4e0 con 0x7fbfe40fb220 2024-02-16T19:23:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.953+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe404bed0 con 0x7fbfe40fb220 2024-02-16T19:23:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.954+0000 7fbfd9ffb700 1 -- 172.21.15.40:0/2820547844 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbfe001ca90 con 0x7fbfe40fb220 2024-02-16T19:23:27.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.954+0000 7fbfd9ffb700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbfd00692a0 0x7fbfd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:27.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.955+0000 7fbfd9ffb700 1 -- 172.21.15.40:0/2820547844 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbfe00883f0 con 0x7fbfe40fb220 2024-02-16T19:23:27.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.955+0000 7fbfe8f90700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbfd00692a0 0x7fbfd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:27.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.955+0000 7fbfe8f90700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbfd00692a0 0x7fbfd006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4004600 tx=0x7fbfd400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:27.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:27.959+0000 7fbfd9ffb700 1 -- 172.21.15.40:0/2820547844 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbfe0058140 con 0x7fbfe40fb220 2024-02-16T19:23:28.122 INFO:tasks.workunit.client.0.smithi040.stderr:Note: switching to 'ed7ab725e1a43615009dfb74a42bf33d80081bc8'. 2024-02-16T19:23:28.122 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-02-16T19:23:28.122 INFO:tasks.workunit.client.0.smithi040.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr:state without impacting any branches by switching back to a branch. 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr: git switch -c 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr:Or undo this operation with: 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr: git switch - 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-02-16T19:23:28.123 INFO:tasks.workunit.client.0.smithi040.stderr:HEAD is now at ed7ab725e1a qa: test fuse/kclient for mds upgrade seq 2024-02-16T19:23:28.129 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-16T19:23:28.158 INFO:tasks.workunit.client.0.smithi040.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-16T19:23:28.160 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-16T19:23:28.160 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-16T19:23:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.232+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfe410f940 con 0x7fbfd00692a0 2024-02-16T19:23:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.243+0000 7fbfd9ffb700 1 -- 172.21.15.40:0/2820547844 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fbfe410f940 con 0x7fbfd00692a0 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (5m) 61s ago 7m 25.1M - 0.25.0 c8568f914cd2 aadccae1ccec 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (7m) 61s ago 7m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (6m) 66s ago 6m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (5m) 61s ago 6m 84.5M - 9.4.7 954c08fa6188 8447b5580554 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (80s) 61s ago 79s 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (87s) 61s ago 86s 10.4M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (77s) 66s ago 77s 14.4M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (84s) 66s ago 84s 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:9283 running (8m) 61s ago 8m 482M - 17.2.7-513-gb88fae66 870a0944ccb8 4e9e9bb4389a 2024-02-16T19:23:28.245 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283 running (6m) 66s ago 5m 436M - 17.2.7-513-gb88fae66 870a0944ccb8 06e26641f085 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (8m) 61s ago 8m 44.9M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (5m) 66s ago 5m 35.4M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (7m) 61s ago 7m 14.0M - 1.5.0 0da6a335fe13 61d16f3f00d4 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (6m) 66s ago 6m 16.7M - 1.5.0 0da6a335fe13 c45ebf41f1b8 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (5m) 61s ago 5m 47.9M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (4m) 61s ago 4m 46.4M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (4m) 61s ago 4m 45.3M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (3m) 66s ago 3m 44.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (3m) 66s ago 3m 41.1M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (2m) 66s ago 2m 44.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:23:28.246 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (5m) 61s ago 6m 51.4M - 2.43.0 a07b618ecd1d 557bd885dbc5 2024-02-16T19:23:28.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbfd00692a0 msgr2=0x7fbfd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:28.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbfd00692a0 0x7fbfd006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4004600 tx=0x7fbfd400b040 comp rx=0 tx=0).stop 2024-02-16T19:23:28.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe40fb220 msgr2=0x7fbfe410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:28.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe40fb220 0x7fbfe410d9a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009fd0 tx=0x7fbfe0009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 shutdown_connections 2024-02-16T19:23:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbfd00692a0 0x7fbfd006b750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe40fb220 0x7fbfe410d9a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 --2- 172.21.15.40:0/2820547844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe40fe2b0 0x7fbfe410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 >> 172.21.15.40:0/2820547844 conn(0x7fbfe40f6ca0 msgr2=0x7fbfe41014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 shutdown_connections 2024-02-16T19:23:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.245+0000 7fbfeb9f5700 1 -- 172.21.15.40:0/2820547844 wait complete. 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 -- 172.21.15.40:0/2400368735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43940fda80 msgr2=0x7f43940ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 --2- 172.21.15.40:0/2400368735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43940fda80 0x7f43940ffe60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4390004940 tx=0x7f4390009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 -- 172.21.15.40:0/2400368735 shutdown_connections 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 --2- 172.21.15.40:0/2400368735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43941003a0 0x7f4394102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 --2- 172.21.15.40:0/2400368735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43940fda80 0x7f43940ffe60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 -- 172.21.15.40:0/2400368735 >> 172.21.15.40:0/2400368735 conn(0x7f43940f74e0 msgr2=0x7f43940f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 -- 172.21.15.40:0/2400368735 shutdown_connections 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 -- 172.21.15.40:0/2400368735 wait complete. 2024-02-16T19:23:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 Processor -- start 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.354+0000 7f439ba75700 1 -- start start 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f439ba75700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43940fda80 0x7f43940fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f439ba75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43941003a0 0x7f43940fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f439ba75700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940fbf90 con 0x7f43941003a0 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f439ba75700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940fc100 con 0x7f43940fda80 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399010700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43941003a0 0x7f43940fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399811700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43940fda80 0x7f43940fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399010700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43941003a0 0x7f43940fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39114/0 (socket says 172.21.15.40:39114) 2024-02-16T19:23:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399010700 1 -- 172.21.15.40:0/2701001828 learned_addr learned my addr 172.21.15.40:0/2701001828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:23:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399010700 1 -- 172.21.15.40:0/2701001828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43940fda80 msgr2=0x7f43940fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399010700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43940fda80 0x7f43940fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399010700 1 -- 172.21.15.40:0/2701001828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4390005040 con 0x7f43941003a0 2024-02-16T19:23:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399811700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43940fda80 0x7f43940fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:23:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.355+0000 7f4399010700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43941003a0 0x7f43940fb990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f438400eb40 tx=0x7f438400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.356+0000 7f438a7fc700 1 -- 172.21.15.40:0/2701001828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4384007cc0 con 0x7f43941003a0 2024-02-16T19:23:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.356+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43940fc380 con 0x7f43941003a0 2024-02-16T19:23:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.356+0000 7f438a7fc700 1 -- 172.21.15.40:0/2701001828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f438400f040 con 0x7f43941003a0 2024-02-16T19:23:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.356+0000 7f438a7fc700 1 -- 172.21.15.40:0/2701001828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f438400ca10 con 0x7f43941003a0 2024-02-16T19:23:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.356+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43941158f0 con 0x7f43941003a0 2024-02-16T19:23:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.357+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4394107a50 con 0x7f43941003a0 2024-02-16T19:23:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.357+0000 7f438a7fc700 1 -- 172.21.15.40:0/2701001828 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f438400cc80 con 0x7f43941003a0 2024-02-16T19:23:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.358+0000 7f438a7fc700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43800692a0 0x7f438006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.358+0000 7f438a7fc700 1 -- 172.21.15.40:0/2701001828 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4384086dd0 con 0x7f43941003a0 2024-02-16T19:23:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.358+0000 7f4399811700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43800692a0 0x7f438006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.358+0000 7f4399811700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43800692a0 0x7f438006b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4390000c00 tx=0x7f4390009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.362+0000 7f438a7fc700 1 -- 172.21.15.40:0/2701001828 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f438404dbd0 con 0x7f43941003a0 2024-02-16T19:23:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:28 smithi040 ceph-mon[29201]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-16T19:23:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:28 smithi040 ceph-mon[29201]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T19:23:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:28 smithi040 ceph-mon[29201]: Upgrade: Need to upgrade myself (mgr.smithi040.jzygzo) 2024-02-16T19:23:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:28 smithi040 ceph-mon[29201]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi163 2024-02-16T19:23:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:28 smithi040 ceph-mon[29201]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.667+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f439402b340 con 0x7f43941003a0 2024-02-16T19:23:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.667+0000 7f438a7fc700 1 -- 172.21.15.40:0/2701001828 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f4384056b70 con 0x7f43941003a0 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:23:28.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:23:28.670 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:23:28.670 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 14 2024-02-16T19:23:28.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:23:28.670 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43800692a0 msgr2=0x7f438006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43800692a0 0x7f438006b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4390000c00 tx=0x7f4390009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43941003a0 msgr2=0x7f43940fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43941003a0 0x7f43940fb990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f438400eb40 tx=0x7f438400b040 comp rx=0 tx=0).stop 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 shutdown_connections 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43800692a0 0x7f438006b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43940fda80 0x7f43940fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 --2- 172.21.15.40:0/2701001828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43941003a0 0x7f43940fb990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 >> 172.21.15.40:0/2701001828 conn(0x7f43940f74e0 msgr2=0x7f43940f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.670+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 shutdown_connections 2024-02-16T19:23:28.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.671+0000 7f439ba75700 1 -- 172.21.15.40:0/2701001828 wait complete. 2024-02-16T19:23:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:28 smithi163 ceph-mon[36452]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-16T19:23:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:28 smithi163 ceph-mon[36452]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-16T19:23:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:28 smithi163 ceph-mon[36452]: Upgrade: Need to upgrade myself (mgr.smithi040.jzygzo) 2024-02-16T19:23:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:28 smithi163 ceph-mon[36452]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi163 2024-02-16T19:23:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:28 smithi163 ceph-mon[36452]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.785+0000 7f44078c2700 1 -- 172.21.15.40:0/3676202636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 msgr2=0x7f44000fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.785+0000 7f44078c2700 1 --2- 172.21.15.40:0/3676202636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 0x7f44000fd290 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f43fc004940 tx=0x7f43fc009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.785+0000 7f44078c2700 1 -- 172.21.15.40:0/3676202636 shutdown_connections 2024-02-16T19:23:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.785+0000 7f44078c2700 1 --2- 172.21.15.40:0/3676202636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44000fe080 0x7f44000fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.785+0000 7f44078c2700 1 --2- 172.21.15.40:0/3676202636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 0x7f44000fd290 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.785+0000 7f44078c2700 1 -- 172.21.15.40:0/3676202636 >> 172.21.15.40:0/3676202636 conn(0x7f44000f8590 msgr2=0x7f44000fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.785+0000 7f44078c2700 1 -- 172.21.15.40:0/3676202636 shutdown_connections 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.785+0000 7f44078c2700 1 -- 172.21.15.40:0/3676202636 wait complete. 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f44078c2700 1 Processor -- start 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f44078c2700 1 -- start start 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f44078c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 0x7f4400071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f44078c2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44000fe080 0x7f44000702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f44078c2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000708e0 con 0x7f44000fce80 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f44078c2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4400070a50 con 0x7f44000fe080 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f440565e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 0x7f4400071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f440565e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 0x7f4400071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39128/0 (socket says 172.21.15.40:39128) 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f440565e700 1 -- 172.21.15.40:0/3688947020 learned_addr learned my addr 172.21.15.40:0/3688947020 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.786+0000 7f4404e5d700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44000fe080 0x7f44000702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f440565e700 1 -- 172.21.15.40:0/3688947020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44000fe080 msgr2=0x7f44000702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f440565e700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44000fe080 0x7f44000702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f440565e700 1 -- 172.21.15.40:0/3688947020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc005040 con 0x7f44000fce80 2024-02-16T19:23:28.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f440565e700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 0x7f4400071e60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009fd0 tx=0x7f43fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:28.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f43f5ffb700 1 -- 172.21.15.40:0/3688947020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43fc021070 con 0x7f44000fce80 2024-02-16T19:23:28.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f43f5ffb700 1 -- 172.21.15.40:0/3688947020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43fc01c400 con 0x7f44000fce80 2024-02-16T19:23:28.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f43f5ffb700 1 -- 172.21.15.40:0/3688947020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43fc00e070 con 0x7f44000fce80 2024-02-16T19:23:28.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4400070cd0 con 0x7f44000fce80 2024-02-16T19:23:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.787+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4400115830 con 0x7f44000fce80 2024-02-16T19:23:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.788+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440004bed0 con 0x7f44000fce80 2024-02-16T19:23:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.789+0000 7f43f5ffb700 1 -- 172.21.15.40:0/3688947020 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f43fc01ca90 con 0x7f44000fce80 2024-02-16T19:23:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.790+0000 7f43f5ffb700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43ec0692a0 0x7f43ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.790+0000 7f43f5ffb700 1 -- 172.21.15.40:0/3688947020 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f43fc088330 con 0x7f44000fce80 2024-02-16T19:23:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.790+0000 7f4404e5d700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43ec0692a0 0x7f43ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:28.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.790+0000 7f4404e5d700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43ec0692a0 0x7f43ec06b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43f0004600 tx=0x7f43f000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:28.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:28.794+0000 7f43f5ffb700 1 -- 172.21.15.40:0/3688947020 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f43fc058140 con 0x7f44000fce80 2024-02-16T19:23:29.027 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-16T19:23:29.080 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-16T19:23:29.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.079+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4400115cd0 con 0x7f44000fce80 2024-02-16T19:23:29.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.080+0000 7f43f5ffb700 1 -- 172.21.15.40:0/3688947020 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1820 (secure 0 0 0) 0x7f43fc004490 con 0x7f44000fce80 2024-02-16T19:23:29.081 INFO:teuthology.orchestra.run.smithi040.stdout:e17 2024-02-16T19:23:29.081 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:23:29.081 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 17 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:22:17.019628+0000 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:23:29.082 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:23:29.083 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:23:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:23:29.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.083+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43ec0692a0 msgr2=0x7f43ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.083+0000 7f44078c2700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43ec0692a0 0x7f43ec06b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43f0004600 tx=0x7f43f000b040 comp rx=0 tx=0).stop 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.083+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 msgr2=0x7f4400071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.083+0000 7f44078c2700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 0x7f4400071e60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009fd0 tx=0x7f43fc009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.083+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 shutdown_connections 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.083+0000 7f44078c2700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43ec0692a0 0x7f43ec06b750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.084+0000 7f44078c2700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44000fce80 0x7f4400071e60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.084+0000 7f44078c2700 1 --2- 172.21.15.40:0/3688947020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44000fe080 0x7f44000702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.084+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 >> 172.21.15.40:0/3688947020 conn(0x7f44000f8590 msgr2=0x7f4400101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.084+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 shutdown_connections 2024-02-16T19:23:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.084+0000 7f44078c2700 1 -- 172.21.15.40:0/3688947020 wait complete. 2024-02-16T19:23:29.087 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2024-02-16T19:23:29.138 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-16T19:23:29.140 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-16T19:23:29.140 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-16T19:23:29.181 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-16T19:23:29.184 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:23:29.184 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-16T19:23:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.200+0000 7f6e7fc55700 1 -- 172.21.15.40:0/1764836934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fcfb0 msgr2=0x7f6e780fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.200+0000 7f6e7fc55700 1 --2- 172.21.15.40:0/1764836934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fcfb0 0x7f6e780fd420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6e68004940 tx=0x7f6e68009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.200+0000 7f6e7fc55700 1 -- 172.21.15.40:0/1764836934 shutdown_connections 2024-02-16T19:23:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.200+0000 7f6e7fc55700 1 --2- 172.21.15.40:0/1764836934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fcfb0 0x7f6e780fd420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.200+0000 7f6e7fc55700 1 --2- 172.21.15.40:0/1764836934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e780fbe40 0x7f6e780fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.200+0000 7f6e7fc55700 1 -- 172.21.15.40:0/1764836934 >> 172.21.15.40:0/1764836934 conn(0x7f6e780f74e0 msgr2=0x7f6e780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.200+0000 7f6e7fc55700 1 -- 172.21.15.40:0/1764836934 shutdown_connections 2024-02-16T19:23:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.200+0000 7f6e7fc55700 1 -- 172.21.15.40:0/1764836934 wait complete. 2024-02-16T19:23:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7fc55700 1 Processor -- start 2024-02-16T19:23:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7fc55700 1 -- start start 2024-02-16T19:23:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7fc55700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fbe40 0x7f6e7810c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7fc55700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e780fcfb0 0x7f6e7810cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7fc55700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7810d350 con 0x7f6e780fbe40 2024-02-16T19:23:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7fc55700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7810dbf0 con 0x7f6e780fcfb0 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7d9f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fbe40 0x7f6e7810c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7d9f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fbe40 0x7f6e7810c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39154/0 (socket says 172.21.15.40:39154) 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7d9f1700 1 -- 172.21.15.40:0/3967534743 learned_addr learned my addr 172.21.15.40:0/3967534743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7d1f0700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e780fcfb0 0x7f6e7810cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7d9f1700 1 -- 172.21.15.40:0/3967534743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e780fcfb0 msgr2=0x7f6e7810cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.201+0000 7f6e7d9f1700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e780fcfb0 0x7f6e7810cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.202+0000 7f6e7d9f1700 1 -- 172.21.15.40:0/3967534743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68005040 con 0x7f6e780fbe40 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.202+0000 7f6e7d9f1700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fbe40 0x7f6e7810c810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400eb70 tx=0x7f6e7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.202+0000 7f6e6e7fc700 1 -- 172.21.15.40:0/3967534743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e74007ea0 con 0x7f6e780fbe40 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.202+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7810de70 con 0x7f6e780fbe40 2024-02-16T19:23:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.202+0000 7f6e6e7fc700 1 -- 172.21.15.40:0/3967534743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e740095a0 con 0x7f6e780fbe40 2024-02-16T19:23:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.202+0000 7f6e6e7fc700 1 -- 172.21.15.40:0/3967534743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e7400dde0 con 0x7f6e780fbe40 2024-02-16T19:23:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.202+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7810e3b0 con 0x7f6e780fbe40 2024-02-16T19:23:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.203+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7804bed0 con 0x7f6e780fbe40 2024-02-16T19:23:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.204+0000 7f6e6e7fc700 1 -- 172.21.15.40:0/3967534743 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6e74009710 con 0x7f6e780fbe40 2024-02-16T19:23:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.204+0000 7f6e6e7fc700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6e640692a0 0x7f6e6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.204+0000 7f6e6e7fc700 1 -- 172.21.15.40:0/3967534743 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6e74015020 con 0x7f6e780fbe40 2024-02-16T19:23:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.204+0000 7f6e7d1f0700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6e640692a0 0x7f6e6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.205+0000 7f6e7d1f0700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6e640692a0 0x7f6e6406b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009b30 tx=0x7f6e6801c040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:29.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.212+0000 7f6e6e7fc700 1 -- 172.21.15.40:0/3967534743 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6e74057760 con 0x7f6e780fbe40 2024-02-16T19:23:29.217 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-16T19:23:29.218 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-16T19:23:29.218 DEBUG:teuthology.orchestra.run.smithi040:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed7ab725e1a43615009dfb74a42bf33d80081bc8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-16T19:23:29.279 INFO:tasks.workunit.client.0.smithi040.stderr:+ mkdir -p fsstress 2024-02-16T19:23:29.280 INFO:tasks.workunit.client.0.smithi040.stderr:+ pushd fsstress 2024-02-16T19:23:29.281 INFO:tasks.workunit.client.0.smithi040.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T19:23:29.281 INFO:tasks.workunit.client.0.smithi040.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-16T19:23:29.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:29 smithi040 ceph-mon[29201]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:29.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:29 smithi040 ceph-mon[29201]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-02-16T19:23:29.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:29 smithi040 ceph-mon[29201]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:29.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:29 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2701001828' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:23:29.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:29 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3688947020' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:23:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.451+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e7810e7c0 con 0x7f6e640692a0 2024-02-16T19:23:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.452+0000 7f6e6e7fc700 1 -- 172.21.15.40:0/3967534743 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f6e7810e7c0 con 0x7f6e640692a0 2024-02-16T19:23:29.453 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:23:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:23:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:23:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:23:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [], 2024-02-16T19:23:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "0/21 daemons upgraded", 2024-02-16T19:23:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 image on host smithi163", 2024-02-16T19:23:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:23:29.454 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:23:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6e640692a0 msgr2=0x7f6e6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6e640692a0 0x7f6e6406b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009b30 tx=0x7f6e6801c040 comp rx=0 tx=0).stop 2024-02-16T19:23:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fbe40 msgr2=0x7f6e7810c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fbe40 0x7f6e7810c810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400eb70 tx=0x7f6e7400b040 comp rx=0 tx=0).stop 2024-02-16T19:23:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 shutdown_connections 2024-02-16T19:23:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f6e640692a0 0x7f6e6406b750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e780fbe40 0x7f6e7810c810 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 --2- 172.21.15.40:0/3967534743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e780fcfb0 0x7f6e7810cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 >> 172.21.15.40:0/3967534743 conn(0x7f6e780f74e0 msgr2=0x7f6e78100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 shutdown_connections 2024-02-16T19:23:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.454+0000 7f6e7fc55700 1 -- 172.21.15.40:0/3967534743 wait complete. 2024-02-16T19:23:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 -- 172.21.15.40:0/241640224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb780fda80 msgr2=0x7fbb780ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 --2- 172.21.15.40:0/241640224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb780fda80 0x7fbb780ffe60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbb74004940 tx=0x7fbb74009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 -- 172.21.15.40:0/241640224 shutdown_connections 2024-02-16T19:23:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 --2- 172.21.15.40:0/241640224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb781003a0 0x7fbb78102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 --2- 172.21.15.40:0/241640224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb780fda80 0x7fbb780ffe60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 -- 172.21.15.40:0/241640224 >> 172.21.15.40:0/241640224 conn(0x7fbb780f74e0 msgr2=0x7fbb780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 -- 172.21.15.40:0/241640224 shutdown_connections 2024-02-16T19:23:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 -- 172.21.15.40:0/241640224 wait complete. 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.571+0000 7fbb801d7700 1 Processor -- start 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb801d7700 1 -- start start 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb801d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb780fda80 0x7fbb780fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb801d7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb781003a0 0x7fbb780fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb801d7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780fbf90 con 0x7fbb781003a0 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb801d7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780fc100 con 0x7fbb780fda80 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb7d772700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb781003a0 0x7fbb780fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb7d772700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb781003a0 0x7fbb780fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39164/0 (socket says 172.21.15.40:39164) 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb7d772700 1 -- 172.21.15.40:0/1620806372 learned_addr learned my addr 172.21.15.40:0/1620806372 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb7df73700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb780fda80 0x7fbb780fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb7d772700 1 -- 172.21.15.40:0/1620806372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb780fda80 msgr2=0x7fbb780fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb7d772700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb780fda80 0x7fbb780fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb7d772700 1 -- 172.21.15.40:0/1620806372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74005040 con 0x7fbb781003a0 2024-02-16T19:23:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.572+0000 7fbb7df73700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb780fda80 0x7fbb780fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:23:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.573+0000 7fbb7d772700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb781003a0 0x7fbb780fb990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800eb40 tx=0x7fbb6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.573+0000 7fbb6e7fc700 1 -- 172.21.15.40:0/1620806372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb68007cc0 con 0x7fbb781003a0 2024-02-16T19:23:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.573+0000 7fbb6e7fc700 1 -- 172.21.15.40:0/1620806372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb6800f040 con 0x7fbb781003a0 2024-02-16T19:23:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.573+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb780fc380 con 0x7fbb781003a0 2024-02-16T19:23:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.573+0000 7fbb6e7fc700 1 -- 172.21.15.40:0/1620806372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb6800ca10 con 0x7fbb781003a0 2024-02-16T19:23:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.573+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb781158f0 con 0x7fbb781003a0 2024-02-16T19:23:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.574+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb78107a50 con 0x7fbb781003a0 2024-02-16T19:23:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.575+0000 7fbb6e7fc700 1 -- 172.21.15.40:0/1620806372 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbb6800cb70 con 0x7fbb781003a0 2024-02-16T19:23:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.575+0000 7fbb6e7fc700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbb640692a0 0x7fbb6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.576+0000 7fbb6e7fc700 1 -- 172.21.15.40:0/1620806372 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbb68086cf0 con 0x7fbb781003a0 2024-02-16T19:23:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.576+0000 7fbb7df73700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbb640692a0 0x7fbb6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.576+0000 7fbb7df73700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbb640692a0 0x7fbb6406b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb74000c00 tx=0x7fbb74009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:29.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.580+0000 7fbb6e7fc700 1 -- 172.21.15.40:0/1620806372 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbb6804dbd0 con 0x7fbb781003a0 2024-02-16T19:23:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:29 smithi163 ceph-mon[36452]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:29 smithi163 ceph-mon[36452]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-02-16T19:23:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:29 smithi163 ceph-mon[36452]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:29 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2701001828' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:23:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:29 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3688947020' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:23:29.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.880+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbb78105a10 con 0x7fbb781003a0 2024-02-16T19:23:29.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.880+0000 7fbb6e7fc700 1 -- 172.21.15.40:0/1620806372 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbb68056b10 con 0x7fbb781003a0 2024-02-16T19:23:29.882 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:23:29.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.883+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbb640692a0 msgr2=0x7fbb6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbb640692a0 0x7fbb6406b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb74000c00 tx=0x7fbb74009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:29.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb781003a0 msgr2=0x7fbb780fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb781003a0 0x7fbb780fb990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800eb40 tx=0x7fbb6800b040 comp rx=0 tx=0).stop 2024-02-16T19:23:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 shutdown_connections 2024-02-16T19:23:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbb640692a0 0x7fbb6406b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb780fda80 0x7fbb780fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 --2- 172.21.15.40:0/1620806372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb781003a0 0x7fbb780fb990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 >> 172.21.15.40:0/1620806372 conn(0x7fbb780f74e0 msgr2=0x7fbb780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 shutdown_connections 2024-02-16T19:23:29.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:29.884+0000 7fbb801d7700 1 -- 172.21.15.40:0/1620806372 wait complete. 2024-02-16T19:23:30.164 INFO:tasks.workunit.client.0.smithi040.stderr:+ tar xzf ltp-full.tgz 2024-02-16T19:23:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:30 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1620806372' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:23:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:30 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1620806372' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:23:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:31 smithi040 ceph-mon[29201]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:31 smithi040 ceph-mon[29201]: pgmap v224: 65 pgs: 65 active+clean; 6.3 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 489 KiB/s wr, 1 op/s 2024-02-16T19:23:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:31 smithi163 ceph-mon[36452]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:23:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:31 smithi163 ceph-mon[36452]: pgmap v224: 65 pgs: 65 active+clean; 6.3 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 489 KiB/s wr, 1 op/s 2024-02-16T19:23:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:33 smithi040 ceph-mon[29201]: pgmap v225: 65 pgs: 65 active+clean; 10 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 845 KiB/s wr, 1 op/s 2024-02-16T19:23:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:33 smithi163 ceph-mon[36452]: pgmap v225: 65 pgs: 65 active+clean; 10 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 845 KiB/s wr, 1 op/s 2024-02-16T19:23:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:34 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:23:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:34 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:23:35.638 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:35 smithi163 ceph-mon[36452]: pgmap v226: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s wr, 214 op/s 2024-02-16T19:23:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:35 smithi040 ceph-mon[29201]: pgmap v226: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s wr, 214 op/s 2024-02-16T19:23:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:37 smithi040 ceph-mon[29201]: pgmap v227: 65 pgs: 65 active+clean; 64 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.6 MiB/s wr, 526 op/s 2024-02-16T19:23:37.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:37 smithi163 ceph-mon[36452]: pgmap v227: 65 pgs: 65 active+clean; 64 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.6 MiB/s wr, 526 op/s 2024-02-16T19:23:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:39 smithi040 ceph-mon[29201]: pgmap v228: 65 pgs: 65 active+clean; 75 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.5 MiB/s wr, 581 op/s 2024-02-16T19:23:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:39 smithi163 ceph-mon[36452]: pgmap v228: 65 pgs: 65 active+clean; 75 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.5 MiB/s wr, 581 op/s 2024-02-16T19:23:40.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:41.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:41 smithi040 ceph-mon[29201]: pgmap v229: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7.3 MiB/s wr, 595 op/s 2024-02-16T19:23:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:41 smithi163 ceph-mon[36452]: pgmap v229: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7.3 MiB/s wr, 595 op/s 2024-02-16T19:23:43.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:43 smithi040 ceph-mon[29201]: pgmap v230: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.9 MiB/s wr, 595 op/s 2024-02-16T19:23:43.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:43 smithi163 ceph-mon[36452]: pgmap v230: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.9 MiB/s wr, 595 op/s 2024-02-16T19:23:45.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:45 smithi040 ceph-mon[29201]: pgmap v231: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.6 MiB/s wr, 594 op/s 2024-02-16T19:23:45.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:45 smithi163 ceph-mon[36452]: pgmap v231: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.6 MiB/s wr, 594 op/s 2024-02-16T19:23:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:47 smithi040 ceph-mon[29201]: pgmap v232: 65 pgs: 65 active+clean; 91 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 5.7 MiB/s wr, 383 op/s 2024-02-16T19:23:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:47 smithi163 ceph-mon[36452]: pgmap v232: 65 pgs: 65 active+clean; 91 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 5.7 MiB/s wr, 383 op/s 2024-02-16T19:23:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:49 smithi040 ceph-mon[29201]: pgmap v233: 65 pgs: 65 active+clean; 98 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 161 op/s 2024-02-16T19:23:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:49 smithi163 ceph-mon[36452]: pgmap v233: 65 pgs: 65 active+clean; 98 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 161 op/s 2024-02-16T19:23:51.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:51 smithi040 ceph-mon[29201]: pgmap v234: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s wr, 146 op/s 2024-02-16T19:23:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:51 smithi163 ceph-mon[36452]: pgmap v234: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s wr, 146 op/s 2024-02-16T19:23:52.866 INFO:tasks.workunit.client.1.smithi163.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-16T19:23:52.866 INFO:tasks.workunit.client.1.smithi163.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T19:23:52.866 INFO:tasks.workunit.client.1.smithi163.stderr:+ make 2024-02-16T19:23:52.948 INFO:tasks.workunit.client.1.smithi163.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-16T19:23:53.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:53 smithi040 ceph-mon[29201]: pgmap v235: 65 pgs: 65 active+clean; 115 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 198 op/s 2024-02-16T19:23:53.735 INFO:tasks.workunit.client.1.smithi163.stderr:++ readlink -f fsstress 2024-02-16T19:23:53.735 INFO:tasks.workunit.client.1.smithi163.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-16T19:23:53.736 INFO:tasks.workunit.client.1.smithi163.stderr:+ popd 2024-02-16T19:23:53.736 INFO:tasks.workunit.client.1.smithi163.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-16T19:23:53.736 INFO:tasks.workunit.client.1.smithi163.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-16T19:23:53.736 INFO:tasks.workunit.client.1.smithi163.stderr:+ popd 2024-02-16T19:23:53.736 INFO:tasks.workunit.client.1.smithi163.stderr:++ mktemp -d -p . 2024-02-16T19:23:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:53 smithi163 ceph-mon[36452]: pgmap v235: 65 pgs: 65 active+clean; 115 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 198 op/s 2024-02-16T19:23:53.737 INFO:tasks.workunit.client.1.smithi163.stderr:+ T=./tmp.R3cIKKTS6p 2024-02-16T19:23:53.737 INFO:tasks.workunit.client.1.smithi163.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.R3cIKKTS6p -l 1 -n 1000 -p 10 -v 2024-02-16T19:23:53.738 INFO:tasks.workunit.client.1.smithi163.stdout:seed = 1707405772 2024-02-16T19:23:53.742 INFO:tasks.workunit.client.1.smithi163.stdout:0/0: mknod c0 0 2024-02-16T19:23:53.743 INFO:tasks.workunit.client.1.smithi163.stdout:0/1: creat f1 x:0 0 0 2024-02-16T19:23:53.743 INFO:tasks.workunit.client.1.smithi163.stdout:0/2: write f1 [1019954,20850] 0 2024-02-16T19:23:53.744 INFO:tasks.workunit.client.1.smithi163.stdout:3/0: link - no file 2024-02-16T19:23:53.744 INFO:tasks.workunit.client.1.smithi163.stdout:2/0: symlink l0 0 2024-02-16T19:23:53.744 INFO:tasks.workunit.client.1.smithi163.stdout:2/1: truncate - no filename 2024-02-16T19:23:53.745 INFO:tasks.workunit.client.1.smithi163.stdout:2/2: dread - no filename 2024-02-16T19:23:53.745 INFO:tasks.workunit.client.1.smithi163.stdout:2/3: read - no filename 2024-02-16T19:23:53.745 INFO:tasks.workunit.client.1.smithi163.stdout:4/0: chown . 31411056 1 2024-02-16T19:23:53.745 INFO:tasks.workunit.client.1.smithi163.stdout:4/1: dread - no filename 2024-02-16T19:23:53.745 INFO:tasks.workunit.client.1.smithi163.stdout:4/2: dread - no filename 2024-02-16T19:23:53.745 INFO:tasks.workunit.client.1.smithi163.stdout:4/3: read - no filename 2024-02-16T19:23:53.745 INFO:tasks.workunit.client.1.smithi163.stdout:4/4: unlink - no file 2024-02-16T19:23:53.745 INFO:tasks.workunit.client.1.smithi163.stdout:4/5: chown . 103 1 2024-02-16T19:23:53.746 INFO:tasks.workunit.client.1.smithi163.stdout:3/1: mkdir d0 0 2024-02-16T19:23:53.746 INFO:tasks.workunit.client.1.smithi163.stdout:2/4: rename l0 to l1 0 2024-02-16T19:23:53.746 INFO:tasks.workunit.client.1.smithi163.stdout:2/5: dread - no filename 2024-02-16T19:23:53.747 INFO:tasks.workunit.client.1.smithi163.stdout:4/6: symlink l0 0 2024-02-16T19:23:53.748 INFO:tasks.workunit.client.1.smithi163.stdout:4/7: truncate - no filename 2024-02-16T19:23:53.749 INFO:tasks.workunit.client.1.smithi163.stdout:3/2: creat d0/f1 x:0 0 0 2024-02-16T19:23:53.749 INFO:tasks.workunit.client.1.smithi163.stdout:3/3: dread - d0/f1 zero size 2024-02-16T19:23:53.750 INFO:tasks.workunit.client.1.smithi163.stdout:4/8: mknod c1 0 2024-02-16T19:23:53.750 INFO:tasks.workunit.client.1.smithi163.stdout:4/9: write - no filename 2024-02-16T19:23:53.750 INFO:tasks.workunit.client.1.smithi163.stdout:5/0: write - no filename 2024-02-16T19:23:53.750 INFO:tasks.workunit.client.1.smithi163.stdout:5/1: rename - no filename 2024-02-16T19:23:53.750 INFO:tasks.workunit.client.1.smithi163.stdout:4/10: stat c1 0 2024-02-16T19:23:53.750 INFO:tasks.workunit.client.1.smithi163.stdout:5/2: fdatasync - no filename 2024-02-16T19:23:53.750 INFO:tasks.workunit.client.1.smithi163.stdout:5/3: dwrite - no filename 2024-02-16T19:23:53.751 INFO:tasks.workunit.client.1.smithi163.stdout:5/4: chown . 321 1 2024-02-16T19:23:53.751 INFO:tasks.workunit.client.1.smithi163.stdout:5/5: read - no filename 2024-02-16T19:23:53.752 INFO:tasks.workunit.client.1.smithi163.stdout:5/6: mkdir d0 0 2024-02-16T19:23:53.753 INFO:tasks.workunit.client.1.smithi163.stdout:2/6: mkdir d2 0 2024-02-16T19:23:53.753 INFO:tasks.workunit.client.1.smithi163.stdout:2/7: write - no filename 2024-02-16T19:23:53.753 INFO:tasks.workunit.client.1.smithi163.stdout:2/8: dwrite - no filename 2024-02-16T19:23:53.755 INFO:tasks.workunit.client.1.smithi163.stdout:6/0: mkdir d0 0 2024-02-16T19:23:53.755 INFO:tasks.workunit.client.1.smithi163.stdout:6/1: fsync - no filename 2024-02-16T19:23:53.755 INFO:tasks.workunit.client.1.smithi163.stdout:6/2: dwrite - no filename 2024-02-16T19:23:53.755 INFO:tasks.workunit.client.1.smithi163.stdout:6/3: readlink - no filename 2024-02-16T19:23:53.755 INFO:tasks.workunit.client.1.smithi163.stdout:6/4: dread - no filename 2024-02-16T19:23:53.755 INFO:tasks.workunit.client.1.smithi163.stdout:5/7: creat d0/f1 x:0 0 0 2024-02-16T19:23:53.757 INFO:tasks.workunit.client.1.smithi163.stdout:7/0: dwrite - no filename 2024-02-16T19:23:53.759 INFO:tasks.workunit.client.1.smithi163.stdout:8/0: write - no filename 2024-02-16T19:23:53.762 INFO:tasks.workunit.client.1.smithi163.stdout:8/1: mkdir d0 0 2024-02-16T19:23:53.762 INFO:tasks.workunit.client.1.smithi163.stdout:8/2: stat d0 0 2024-02-16T19:23:53.765 INFO:tasks.workunit.client.1.smithi163.stdout:9/0: creat f0 x:0 0 0 2024-02-16T19:23:53.767 INFO:tasks.workunit.client.1.smithi163.stdout:8/3: creat d0/f1 x:0 0 0 2024-02-16T19:23:53.769 INFO:tasks.workunit.client.1.smithi163.stdout:9/1: rename f0 to f1 0 2024-02-16T19:23:53.769 INFO:tasks.workunit.client.1.smithi163.stdout:9/2: creat f2 x:0 0 0 2024-02-16T19:23:53.769 INFO:tasks.workunit.client.1.smithi163.stdout:9/3: creat f3 x:0 0 0 2024-02-16T19:23:53.770 INFO:tasks.workunit.client.1.smithi163.stdout:8/4: rename d0/f1 to d0/f2 0 2024-02-16T19:23:53.780 INFO:tasks.workunit.client.1.smithi163.stdout:9/4: mknod c4 0 2024-02-16T19:23:53.780 INFO:tasks.workunit.client.1.smithi163.stdout:9/5: truncate f3 419350 0 2024-02-16T19:23:53.781 INFO:tasks.workunit.client.1.smithi163.stdout:9/6: mkdir d5 0 2024-02-16T19:23:54.203 INFO:tasks.workunit.client.1.smithi163.stdout:2/9: rename l1 to d2/l3 0 2024-02-16T19:23:54.346 INFO:tasks.workunit.client.1.smithi163.stdout:3/4: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:23:54.434 INFO:tasks.workunit.client.1.smithi163.stdout:2/10: link d2/l3 d2/l4 0 2024-02-16T19:23:54.541 INFO:tasks.workunit.client.1.smithi163.stdout:8/5: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:23:54.541 INFO:tasks.workunit.client.1.smithi163.stdout:8/6: fdatasync d0/f2 0 2024-02-16T19:23:54.541 INFO:tasks.workunit.client.1.smithi163.stdout:8/7: chown d0/f2 1673773648 1 2024-02-16T19:23:54.612 INFO:tasks.workunit.client.1.smithi163.stdout:9/7: rename f3 to d5/f6 0 2024-02-16T19:23:54.947 INFO:tasks.workunit.client.1.smithi163.stdout:2/11: unlink d2/l4 0 2024-02-16T19:23:54.947 INFO:tasks.workunit.client.1.smithi163.stdout:2/12: write - no filename 2024-02-16T19:23:55.158 INFO:tasks.workunit.client.1.smithi163.stdout:3/5: write d0/f1 [390939,56843] 0 2024-02-16T19:23:55.158 INFO:tasks.workunit.client.1.smithi163.stdout:3/6: creat d0/f2 x:0 0 0 2024-02-16T19:23:55.158 INFO:tasks.workunit.client.1.smithi163.stdout:3/7: chown d0/f2 3170695 1 2024-02-16T19:23:55.158 INFO:tasks.workunit.client.1.smithi163.stdout:3/8: write d0/f1 [3436702,39924] 0 2024-02-16T19:23:55.192 INFO:tasks.workunit.client.1.smithi163.stdout:0/3: dwrite f1 [0,4194304] 0 2024-02-16T19:23:55.192 INFO:tasks.workunit.client.1.smithi163.stdout:0/4: fdatasync f1 0 2024-02-16T19:23:55.203 INFO:tasks.workunit.client.1.smithi163.stdout:0/5: dread f1 [0,4194304] 0 2024-02-16T19:23:55.209 INFO:tasks.workunit.client.1.smithi163.stdout:0/6: write f1 [3422311,83051] 0 2024-02-16T19:23:55.229 INFO:tasks.workunit.client.1.smithi163.stdout:8/8: link d0/f2 d0/f3 0 2024-02-16T19:23:55.243 INFO:tasks.workunit.client.1.smithi163.stdout:0/7: dread f1 [0,4194304] 0 2024-02-16T19:23:55.243 INFO:tasks.workunit.client.1.smithi163.stdout:0/8: fsync f1 0 2024-02-16T19:23:55.247 INFO:tasks.workunit.client.1.smithi163.stdout:0/9: dread f1 [0,4194304] 0 2024-02-16T19:23:55.503 INFO:tasks.workunit.client.1.smithi163.stdout:9/8: symlink d5/l7 0 2024-02-16T19:23:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:55 smithi040 ceph-mon[29201]: pgmap v236: 65 pgs: 65 active+clean; 125 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s wr, 201 op/s 2024-02-16T19:23:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:55 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:55 smithi163 ceph-mon[36452]: pgmap v236: 65 pgs: 65 active+clean; 125 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s wr, 201 op/s 2024-02-16T19:23:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:55 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:23:56.195 INFO:tasks.workunit.client.0.smithi040.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-16T19:23:56.201 INFO:tasks.workunit.client.0.smithi040.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T19:23:56.201 INFO:tasks.workunit.client.0.smithi040.stderr:+ make 2024-02-16T19:23:56.207 INFO:tasks.workunit.client.1.smithi163.stdout:2/13: creat d2/f5 x:0 0 0 2024-02-16T19:23:56.207 INFO:tasks.workunit.client.1.smithi163.stdout:2/14: read - d2/f5 zero size 2024-02-16T19:23:56.207 INFO:tasks.workunit.client.1.smithi163.stdout:2/15: chown d2 156 1 2024-02-16T19:23:56.207 INFO:tasks.workunit.client.1.smithi163.stdout:2/16: creat d2/f6 x:0 0 0 2024-02-16T19:23:56.207 INFO:tasks.workunit.client.1.smithi163.stdout:2/17: dread - d2/f6 zero size 2024-02-16T19:23:56.207 INFO:tasks.workunit.client.1.smithi163.stdout:2/18: chown d2/f6 0 1 2024-02-16T19:23:56.524 INFO:tasks.workunit.client.1.smithi163.stdout:0/10: rename c0 to c2 0 2024-02-16T19:23:56.524 INFO:tasks.workunit.client.1.smithi163.stdout:0/11: chown f1 541327 1 2024-02-16T19:23:56.524 INFO:tasks.workunit.client.1.smithi163.stdout:0/12: stat f1 0 2024-02-16T19:23:56.550 INFO:tasks.workunit.client.0.smithi040.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-16T19:23:56.594 INFO:tasks.workunit.client.1.smithi163.stdout:9/9: link c4 d5/c8 0 2024-02-16T19:23:56.595 INFO:tasks.workunit.client.1.smithi163.stdout:2/19: mknod d2/c7 0 2024-02-16T19:23:56.597 INFO:tasks.workunit.client.1.smithi163.stdout:8/9: unlink d0/f3 0 2024-02-16T19:23:56.598 INFO:tasks.workunit.client.1.smithi163.stdout:0/13: rename f1 to f3 0 2024-02-16T19:23:56.599 INFO:tasks.workunit.client.1.smithi163.stdout:9/10: rmdir d5 39 2024-02-16T19:23:56.600 INFO:tasks.workunit.client.1.smithi163.stdout:2/20: symlink d2/l8 0 2024-02-16T19:23:56.600 INFO:tasks.workunit.client.1.smithi163.stdout:2/21: creat d2/f9 x:0 0 0 2024-02-16T19:23:56.627 INFO:tasks.workunit.client.1.smithi163.stdout:8/10: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:23:56.627 INFO:tasks.workunit.client.1.smithi163.stdout:8/11: creat d0/f4 x:0 0 0 2024-02-16T19:23:56.627 INFO:tasks.workunit.client.1.smithi163.stdout:9/11: dwrite f2 [0,4194304] 0 2024-02-16T19:23:56.627 INFO:tasks.workunit.client.1.smithi163.stdout:2/22: dwrite d2/f9 [0,4194304] 0 2024-02-16T19:23:56.629 INFO:tasks.workunit.client.1.smithi163.stdout:8/12: unlink d0/f2 0 2024-02-16T19:23:56.629 INFO:tasks.workunit.client.1.smithi163.stdout:8/13: dread - d0/f4 zero size 2024-02-16T19:23:56.629 INFO:tasks.workunit.client.1.smithi163.stdout:8/14: creat d0/f5 x:0 0 0 2024-02-16T19:23:56.629 INFO:tasks.workunit.client.1.smithi163.stdout:8/15: creat d0/f6 x:0 0 0 2024-02-16T19:23:56.629 INFO:tasks.workunit.client.1.smithi163.stdout:8/16: dread - d0/f4 zero size 2024-02-16T19:23:56.632 INFO:tasks.workunit.client.1.smithi163.stdout:9/12: creat d5/f9 x:0 0 0 2024-02-16T19:23:56.632 INFO:tasks.workunit.client.1.smithi163.stdout:9/13: fdatasync f2 0 2024-02-16T19:23:56.632 INFO:tasks.workunit.client.1.smithi163.stdout:9/14: write d5/f9 [730139,19467] 0 2024-02-16T19:23:56.632 INFO:tasks.workunit.client.1.smithi163.stdout:9/15: fdatasync f2 0 2024-02-16T19:23:56.637 INFO:tasks.workunit.client.1.smithi163.stdout:8/17: mkdir d0/d7 0 2024-02-16T19:23:56.637 INFO:tasks.workunit.client.1.smithi163.stdout:8/18: fdatasync d0/f5 0 2024-02-16T19:23:56.639 INFO:tasks.workunit.client.1.smithi163.stdout:9/16: rename f1 to d5/fa 0 2024-02-16T19:23:56.639 INFO:tasks.workunit.client.1.smithi163.stdout:9/17: write d5/f6 [545754,28790] 0 2024-02-16T19:23:56.639 INFO:tasks.workunit.client.1.smithi163.stdout:9/18: write d5/f6 [1318205,93340] 0 2024-02-16T19:23:56.640 INFO:tasks.workunit.client.1.smithi163.stdout:8/19: symlink d0/d7/l8 0 2024-02-16T19:23:56.641 INFO:tasks.workunit.client.1.smithi163.stdout:9/19: mknod d5/cb 0 2024-02-16T19:23:56.642 INFO:tasks.workunit.client.1.smithi163.stdout:9/20: chown d5/f6 23 1 2024-02-16T19:23:56.642 INFO:tasks.workunit.client.1.smithi163.stdout:9/21: write d5/f6 [2121624,81920] 0 2024-02-16T19:23:56.662 INFO:tasks.workunit.client.1.smithi163.stdout:9/22: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:23:56.664 INFO:tasks.workunit.client.1.smithi163.stdout:9/23: rename d5/c8 to d5/cc 0 2024-02-16T19:23:56.671 INFO:tasks.workunit.client.1.smithi163.stdout:9/24: write f2 [2802886,98274] 0 2024-02-16T19:23:56.682 INFO:tasks.workunit.client.1.smithi163.stdout:9/25: write d5/f9 [1610954,105763] 0 2024-02-16T19:23:56.684 INFO:tasks.workunit.client.1.smithi163.stdout:9/26: fsync f2 0 2024-02-16T19:23:56.828 INFO:tasks.workunit.client.0.smithi040.stderr:++ readlink -f fsstress 2024-02-16T19:23:56.829 INFO:tasks.workunit.client.0.smithi040.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-16T19:23:56.829 INFO:tasks.workunit.client.0.smithi040.stderr:+ popd 2024-02-16T19:23:56.830 INFO:tasks.workunit.client.0.smithi040.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-16T19:23:56.830 INFO:tasks.workunit.client.0.smithi040.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-16T19:23:56.830 INFO:tasks.workunit.client.0.smithi040.stderr:+ popd 2024-02-16T19:23:56.830 INFO:tasks.workunit.client.0.smithi040.stderr:++ mktemp -d -p . 2024-02-16T19:23:56.831 INFO:tasks.workunit.client.0.smithi040.stderr:+ T=./tmp.MFeDXfBFbL 2024-02-16T19:23:56.832 INFO:tasks.workunit.client.0.smithi040.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.MFeDXfBFbL -l 1 -n 1000 -p 10 -v 2024-02-16T19:23:56.833 INFO:tasks.workunit.client.0.smithi040.stdout:seed = 1707278600 2024-02-16T19:23:56.834 INFO:tasks.workunit.client.0.smithi040.stdout:0/0: dread - no filename 2024-02-16T19:23:56.834 INFO:tasks.workunit.client.0.smithi040.stdout:0/1: chown . 195233 1 2024-02-16T19:23:56.835 INFO:tasks.workunit.client.0.smithi040.stdout:0/2: symlink l0 0 2024-02-16T19:23:56.835 INFO:tasks.workunit.client.0.smithi040.stdout:0/3: fdatasync - no filename 2024-02-16T19:23:56.836 INFO:tasks.workunit.client.0.smithi040.stdout:2/0: stat - no entries 2024-02-16T19:23:56.836 INFO:tasks.workunit.client.0.smithi040.stdout:2/1: fsync - no filename 2024-02-16T19:23:56.836 INFO:tasks.workunit.client.0.smithi040.stdout:2/2: rmdir - no directory 2024-02-16T19:23:56.836 INFO:tasks.workunit.client.0.smithi040.stdout:2/3: getdents . 0 2024-02-16T19:23:56.836 INFO:tasks.workunit.client.0.smithi040.stdout:2/4: fsync - no filename 2024-02-16T19:23:56.836 INFO:tasks.workunit.client.0.smithi040.stdout:2/5: dread - no filename 2024-02-16T19:23:56.836 INFO:tasks.workunit.client.0.smithi040.stdout:2/6: rename - no filename 2024-02-16T19:23:56.837 INFO:tasks.workunit.client.0.smithi040.stdout:1/0: creat f0 x:0 0 0 2024-02-16T19:23:56.837 INFO:tasks.workunit.client.0.smithi040.stdout:0/4: creat f1 x:0 0 0 2024-02-16T19:23:56.838 INFO:tasks.workunit.client.0.smithi040.stdout:2/7: mknod c0 0 2024-02-16T19:23:56.840 INFO:tasks.workunit.client.0.smithi040.stdout:3/0: creat f0 x:0 0 0 2024-02-16T19:23:56.840 INFO:tasks.workunit.client.0.smithi040.stdout:2/8: mknod c1 0 2024-02-16T19:23:56.841 INFO:tasks.workunit.client.0.smithi040.stdout:5/0: chown . 1386897 1 2024-02-16T19:23:56.842 INFO:tasks.workunit.client.0.smithi040.stdout:4/0: creat f0 x:0 0 0 2024-02-16T19:23:56.842 INFO:tasks.workunit.client.0.smithi040.stdout:3/1: unlink f0 0 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/2: dread - no filename 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/3: chown . 165 1 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/4: write - no filename 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/5: rmdir - no directory 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/6: link - no file 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/7: dread - no filename 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/8: fsync - no filename 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/9: write - no filename 2024-02-16T19:23:56.843 INFO:tasks.workunit.client.0.smithi040.stdout:3/10: dread - no filename 2024-02-16T19:23:56.844 INFO:tasks.workunit.client.0.smithi040.stdout:2/9: mknod c2 0 2024-02-16T19:23:56.844 INFO:tasks.workunit.client.0.smithi040.stdout:2/10: dwrite - no filename 2024-02-16T19:23:56.844 INFO:tasks.workunit.client.0.smithi040.stdout:2/11: fdatasync - no filename 2024-02-16T19:23:56.844 INFO:tasks.workunit.client.0.smithi040.stdout:2/12: dread - no filename 2024-02-16T19:23:56.844 INFO:tasks.workunit.client.0.smithi040.stdout:2/13: dwrite - no filename 2024-02-16T19:23:56.844 INFO:tasks.workunit.client.0.smithi040.stdout:2/14: chown c2 0 1 2024-02-16T19:23:56.845 INFO:tasks.workunit.client.0.smithi040.stdout:6/0: dwrite - no filename 2024-02-16T19:23:56.847 INFO:tasks.workunit.client.0.smithi040.stdout:5/1: creat f0 x:0 0 0 2024-02-16T19:23:56.847 INFO:tasks.workunit.client.0.smithi040.stdout:5/2: chown f0 0 1 2024-02-16T19:23:56.848 INFO:tasks.workunit.client.0.smithi040.stdout:4/1: symlink l1 0 2024-02-16T19:23:56.848 INFO:tasks.workunit.client.0.smithi040.stdout:4/2: stat f0 0 2024-02-16T19:23:56.848 INFO:tasks.workunit.client.0.smithi040.stdout:4/3: creat f2 x:0 0 0 2024-02-16T19:23:56.848 INFO:tasks.workunit.client.0.smithi040.stdout:4/4: read - f0 zero size 2024-02-16T19:23:56.848 INFO:tasks.workunit.client.0.smithi040.stdout:4/5: write f2 [401403,29258] 0 2024-02-16T19:23:56.848 INFO:tasks.workunit.client.0.smithi040.stdout:4/6: write f0 [400771,60963] 0 2024-02-16T19:23:56.849 INFO:tasks.workunit.client.0.smithi040.stdout:4/7: creat f3 x:0 0 0 2024-02-16T19:23:56.849 INFO:tasks.workunit.client.0.smithi040.stdout:4/8: dread - f3 zero size 2024-02-16T19:23:56.849 INFO:tasks.workunit.client.0.smithi040.stdout:3/11: mkdir d1 0 2024-02-16T19:23:56.849 INFO:tasks.workunit.client.0.smithi040.stdout:3/12: truncate - no filename 2024-02-16T19:23:56.849 INFO:tasks.workunit.client.0.smithi040.stdout:3/13: chown d1 178 1 2024-02-16T19:23:56.849 INFO:tasks.workunit.client.0.smithi040.stdout:3/14: chown d1 2 1 2024-02-16T19:23:56.850 INFO:tasks.workunit.client.0.smithi040.stdout:2/15: symlink l3 0 2024-02-16T19:23:56.850 INFO:tasks.workunit.client.0.smithi040.stdout:7/0: read - no filename 2024-02-16T19:23:56.851 INFO:tasks.workunit.client.0.smithi040.stdout:7/1: rename - no filename 2024-02-16T19:23:56.853 INFO:tasks.workunit.client.0.smithi040.stdout:6/1: mkdir d0 0 2024-02-16T19:23:56.857 INFO:tasks.workunit.client.0.smithi040.stdout:2/16: link c0 c4 0 2024-02-16T19:23:56.857 INFO:tasks.workunit.client.0.smithi040.stdout:8/0: chown . 14 1 2024-02-16T19:23:56.860 INFO:tasks.workunit.client.0.smithi040.stdout:6/2: mkdir d0/d1 0 2024-02-16T19:23:56.860 INFO:tasks.workunit.client.0.smithi040.stdout:6/3: truncate - no filename 2024-02-16T19:23:56.860 INFO:tasks.workunit.client.0.smithi040.stdout:6/4: write - no filename 2024-02-16T19:23:56.860 INFO:tasks.workunit.client.0.smithi040.stdout:6/5: chown d0 86 1 2024-02-16T19:23:56.864 INFO:tasks.workunit.client.0.smithi040.stdout:4/9: rename f3 to f4 0 2024-02-16T19:23:56.864 INFO:tasks.workunit.client.0.smithi040.stdout:3/15: creat d1/f2 x:0 0 0 2024-02-16T19:23:56.864 INFO:tasks.workunit.client.0.smithi040.stdout:7/2: mkdir d0 0 2024-02-16T19:23:56.864 INFO:tasks.workunit.client.0.smithi040.stdout:7/3: dwrite - no filename 2024-02-16T19:23:56.868 INFO:tasks.workunit.client.0.smithi040.stdout:2/17: unlink c4 0 2024-02-16T19:23:56.868 INFO:tasks.workunit.client.0.smithi040.stdout:2/18: dwrite - no filename 2024-02-16T19:23:56.868 INFO:tasks.workunit.client.0.smithi040.stdout:9/0: chown . 61130629 1 2024-02-16T19:23:56.869 INFO:tasks.workunit.client.0.smithi040.stdout:9/1: link - no file 2024-02-16T19:23:56.869 INFO:tasks.workunit.client.0.smithi040.stdout:9/2: truncate - no filename 2024-02-16T19:23:56.869 INFO:tasks.workunit.client.0.smithi040.stdout:8/1: creat f0 x:0 0 0 2024-02-16T19:23:56.869 INFO:tasks.workunit.client.0.smithi040.stdout:6/6: creat d0/f2 x:0 0 0 2024-02-16T19:23:56.869 INFO:tasks.workunit.client.0.smithi040.stdout:6/7: truncate d0/f2 842358 0 2024-02-16T19:23:56.881 INFO:tasks.workunit.client.0.smithi040.stdout:7/4: rmdir d0 0 2024-02-16T19:23:56.881 INFO:tasks.workunit.client.0.smithi040.stdout:7/5: link - no file 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/19: mkdir d5 0 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/20: dread - no filename 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/21: dread - no filename 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/22: getdents d5 0 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/23: write - no filename 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/24: write - no filename 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/25: write - no filename 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/26: dwrite - no filename 2024-02-16T19:23:56.882 INFO:tasks.workunit.client.0.smithi040.stdout:2/27: readlink l3 0 2024-02-16T19:23:56.883 INFO:tasks.workunit.client.0.smithi040.stdout:9/3: creat f0 x:0 0 0 2024-02-16T19:23:56.883 INFO:tasks.workunit.client.0.smithi040.stdout:9/4: write f0 [302496,35325] 0 2024-02-16T19:23:56.883 INFO:tasks.workunit.client.0.smithi040.stdout:2/28: mknod d5/c6 0 2024-02-16T19:23:56.885 INFO:tasks.workunit.client.0.smithi040.stdout:9/5: symlink l1 0 2024-02-16T19:23:56.885 INFO:tasks.workunit.client.0.smithi040.stdout:9/6: truncate f0 1372439 0 2024-02-16T19:23:56.886 INFO:tasks.workunit.client.0.smithi040.stdout:9/7: readlink l1 0 2024-02-16T19:23:56.886 INFO:tasks.workunit.client.0.smithi040.stdout:9/8: truncate f0 1703044 0 2024-02-16T19:23:56.886 INFO:tasks.workunit.client.0.smithi040.stdout:2/29: mkdir d5/d7 0 2024-02-16T19:23:56.935 INFO:tasks.workunit.client.1.smithi163.stdout:0/14: rename c2 to c4 0 2024-02-16T19:23:56.978 INFO:tasks.workunit.client.1.smithi163.stdout:3/9: truncate d0/f1 387977 0 2024-02-16T19:23:56.978 INFO:tasks.workunit.client.1.smithi163.stdout:3/10: chown d0/f2 42614 1 2024-02-16T19:23:56.981 INFO:tasks.workunit.client.1.smithi163.stdout:2/23: rmdir d2 39 2024-02-16T19:23:56.989 INFO:tasks.workunit.client.1.smithi163.stdout:9/27: getdents d5 0 2024-02-16T19:23:56.989 INFO:tasks.workunit.client.1.smithi163.stdout:9/28: write d5/fa [873516,50123] 0 2024-02-16T19:23:56.992 INFO:tasks.workunit.client.1.smithi163.stdout:2/24: getdents d2 0 2024-02-16T19:23:57.003 INFO:tasks.workunit.client.0.smithi040.stdout:1/1: dwrite f0 [0,4194304] 0 2024-02-16T19:23:57.004 INFO:tasks.workunit.client.0.smithi040.stdout:5/3: getdents . 0 2024-02-16T19:23:57.014 INFO:tasks.workunit.client.0.smithi040.stdout:4/10: fsync f4 0 2024-02-16T19:23:57.038 INFO:tasks.workunit.client.0.smithi040.stdout:6/8: truncate d0/f2 263385 0 2024-02-16T19:23:57.088 INFO:tasks.workunit.client.1.smithi163.stdout:9/29: read d5/f6 [1850198,96463] 0 2024-02-16T19:23:57.117 INFO:tasks.workunit.client.0.smithi040.stdout:5/4: mkdir d1 0 2024-02-16T19:23:57.222 INFO:tasks.workunit.client.1.smithi163.stdout:2/25: dwrite d2/f6 [0,4194304] 0 2024-02-16T19:23:57.225 INFO:tasks.workunit.client.1.smithi163.stdout:2/26: rmdir d2 39 2024-02-16T19:23:57.227 INFO:tasks.workunit.client.1.smithi163.stdout:2/27: symlink d2/la 0 2024-02-16T19:23:57.399 INFO:tasks.workunit.client.1.smithi163.stdout:0/15: rename c4 to c5 0 2024-02-16T19:23:57.401 INFO:tasks.workunit.client.1.smithi163.stdout:0/16: truncate f3 1365654 0 2024-02-16T19:23:57.401 INFO:tasks.workunit.client.1.smithi163.stdout:0/17: stat f3 0 2024-02-16T19:23:57.401 INFO:tasks.workunit.client.1.smithi163.stdout:0/18: creat f6 x:0 0 0 2024-02-16T19:23:57.495 INFO:tasks.workunit.client.1.smithi163.stdout:2/28: dwrite d2/f6 [0,4194304] 0 2024-02-16T19:23:57.497 INFO:tasks.workunit.client.1.smithi163.stdout:2/29: creat d2/fb x:0 0 0 2024-02-16T19:23:57.498 INFO:tasks.workunit.client.1.smithi163.stdout:2/30: link d2/f5 d2/fc 0 2024-02-16T19:23:57.498 INFO:tasks.workunit.client.1.smithi163.stdout:2/31: read - d2/f5 zero size 2024-02-16T19:23:57.642 INFO:tasks.workunit.client.1.smithi163.stdout:2/32: dwrite d2/f5 [0,4194304] 0 2024-02-16T19:23:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:57 smithi040 ceph-mon[29201]: pgmap v237: 65 pgs: 65 active+clean; 151 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 5.7 MiB/s wr, 408 op/s 2024-02-16T19:23:57.644 INFO:tasks.workunit.client.1.smithi163.stdout:2/33: mknod d2/cd 0 2024-02-16T19:23:57.644 INFO:tasks.workunit.client.1.smithi163.stdout:2/34: dread - d2/fb zero size 2024-02-16T19:23:57.659 INFO:tasks.workunit.client.1.smithi163.stdout:3/11: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:23:57.659 INFO:tasks.workunit.client.1.smithi163.stdout:3/12: chown d0/f2 308629 1 2024-02-16T19:23:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:57 smithi163 ceph-mon[36452]: pgmap v237: 65 pgs: 65 active+clean; 151 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 5.7 MiB/s wr, 408 op/s 2024-02-16T19:23:57.752 INFO:tasks.workunit.client.1.smithi163.stdout:2/35: dwrite d2/f5 [0,4194304] 0 2024-02-16T19:23:57.848 INFO:tasks.workunit.client.1.smithi163.stdout:9/30: dwrite d5/fa [0,4194304] 0 2024-02-16T19:23:58.105 INFO:tasks.workunit.client.1.smithi163.stdout:3/13: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:23:58.261 INFO:tasks.workunit.client.1.smithi163.stdout:3/14: dread d0/f1 [0,4194304] 0 2024-02-16T19:23:58.269 INFO:tasks.workunit.client.1.smithi163.stdout:9/31: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:23:58.269 INFO:tasks.workunit.client.1.smithi163.stdout:9/32: chown d5 26056 1 2024-02-16T19:23:58.271 INFO:tasks.workunit.client.1.smithi163.stdout:9/33: symlink d5/ld 0 2024-02-16T19:23:58.283 INFO:tasks.workunit.client.1.smithi163.stdout:9/34: write d5/f9 [3785833,30784] 0 2024-02-16T19:23:58.284 INFO:tasks.workunit.client.1.smithi163.stdout:9/35: mkdir d5/de 0 2024-02-16T19:23:58.343 INFO:tasks.workunit.client.1.smithi163.stdout:2/36: getdents d2 0 2024-02-16T19:23:58.380 INFO:tasks.workunit.client.1.smithi163.stdout:0/19: dread f3 [0,4194304] 0 2024-02-16T19:23:58.380 INFO:tasks.workunit.client.1.smithi163.stdout:0/20: creat f7 x:0 0 0 2024-02-16T19:23:58.380 INFO:tasks.workunit.client.1.smithi163.stdout:0/21: creat f8 x:0 0 0 2024-02-16T19:23:58.397 INFO:tasks.workunit.client.1.smithi163.stdout:2/37: dwrite d2/f5 [0,4194304] 0 2024-02-16T19:23:58.397 INFO:tasks.workunit.client.1.smithi163.stdout:2/38: fsync d2/fb 0 2024-02-16T19:23:58.399 INFO:tasks.workunit.client.1.smithi163.stdout:2/39: mknod d2/ce 0 2024-02-16T19:23:58.400 INFO:tasks.workunit.client.1.smithi163.stdout:2/40: readlink d2/l8 0 2024-02-16T19:23:58.401 INFO:tasks.workunit.client.1.smithi163.stdout:2/41: creat d2/ff x:0 0 0 2024-02-16T19:23:58.605 INFO:tasks.workunit.client.1.smithi163.stdout:3/15: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:23:58.607 INFO:tasks.workunit.client.1.smithi163.stdout:0/22: dwrite f6 [0,4194304] 0 2024-02-16T19:23:58.609 INFO:tasks.workunit.client.0.smithi040.stdout:6/9: dread d0/f2 [0,4194304] 0 2024-02-16T19:23:58.609 INFO:tasks.workunit.client.1.smithi163.stdout:3/16: symlink d0/l3 0 2024-02-16T19:23:58.615 INFO:tasks.workunit.client.1.smithi163.stdout:2/42: dwrite d2/f6 [0,4194304] 0 2024-02-16T19:23:58.615 INFO:tasks.workunit.client.1.smithi163.stdout:2/43: chown d2/f9 701 1 2024-02-16T19:23:58.615 INFO:tasks.workunit.client.1.smithi163.stdout:2/44: chown d2/f9 45214 1 2024-02-16T19:23:58.626 INFO:tasks.workunit.client.1.smithi163.stdout:9/36: dwrite d5/fa [0,4194304] 0 2024-02-16T19:23:58.628 INFO:tasks.workunit.client.1.smithi163.stdout:2/45: dread d2/f6 [0,4194304] 0 2024-02-16T19:23:58.638 INFO:tasks.workunit.client.1.smithi163.stdout:9/37: dread d5/fa [0,4194304] 0 2024-02-16T19:23:58.640 INFO:tasks.workunit.client.1.smithi163.stdout:9/38: creat d5/de/ff x:0 0 0 2024-02-16T19:23:58.645 INFO:tasks.workunit.client.1.smithi163.stdout:9/39: write d5/f9 [3801694,67333] 0 2024-02-16T19:23:58.647 INFO:tasks.workunit.client.1.smithi163.stdout:9/40: link d5/ld d5/l10 0 2024-02-16T19:23:58.653 INFO:tasks.workunit.client.1.smithi163.stdout:2/46: dwrite d2/f5 [0,4194304] 0 2024-02-16T19:23:58.653 INFO:tasks.workunit.client.1.smithi163.stdout:2/47: creat d2/f10 x:0 0 0 2024-02-16T19:23:58.653 INFO:tasks.workunit.client.1.smithi163.stdout:2/48: readlink d2/la 0 2024-02-16T19:23:58.660 INFO:tasks.workunit.client.1.smithi163.stdout:2/49: unlink d2/f6 0 2024-02-16T19:23:58.660 INFO:tasks.workunit.client.1.smithi163.stdout:2/50: dread - d2/fb zero size 2024-02-16T19:23:58.660 INFO:tasks.workunit.client.1.smithi163.stdout:2/51: fsync d2/f9 0 2024-02-16T19:23:58.660 INFO:tasks.workunit.client.1.smithi163.stdout:2/52: truncate d2/f9 2610981 0 2024-02-16T19:23:58.660 INFO:tasks.workunit.client.1.smithi163.stdout:2/53: creat d2/f11 x:0 0 0 2024-02-16T19:23:58.662 INFO:tasks.workunit.client.0.smithi040.stdout:9/9: dwrite f0 [0,4194304] 0 2024-02-16T19:23:58.667 INFO:tasks.workunit.client.1.smithi163.stdout:0/23: dwrite f8 [0,4194304] 0 2024-02-16T19:23:58.683 INFO:tasks.workunit.client.1.smithi163.stdout:2/54: dwrite d2/f10 [0,4194304] 0 2024-02-16T19:23:58.687 INFO:tasks.workunit.client.1.smithi163.stdout:2/55: rename d2/la to d2/l12 0 2024-02-16T19:23:58.687 INFO:tasks.workunit.client.1.smithi163.stdout:2/56: dread - d2/f11 zero size 2024-02-16T19:23:58.688 INFO:tasks.workunit.client.1.smithi163.stdout:2/57: truncate d2/fb 14771 0 2024-02-16T19:23:58.688 INFO:tasks.workunit.client.1.smithi163.stdout:2/58: chown d2 92 1 2024-02-16T19:23:58.688 INFO:tasks.workunit.client.1.smithi163.stdout:2/59: write d2/f11 [205404,124522] 0 2024-02-16T19:23:58.688 INFO:tasks.workunit.client.1.smithi163.stdout:2/60: creat d2/f13 x:0 0 0 2024-02-16T19:23:58.690 INFO:tasks.workunit.client.1.smithi163.stdout:2/61: rename d2/l3 to d2/l14 0 2024-02-16T19:23:58.690 INFO:tasks.workunit.client.1.smithi163.stdout:2/62: dread - d2/ff zero size 2024-02-16T19:23:58.691 INFO:tasks.workunit.client.1.smithi163.stdout:2/63: creat d2/f15 x:0 0 0 2024-02-16T19:23:58.691 INFO:tasks.workunit.client.1.smithi163.stdout:2/64: creat d2/f16 x:0 0 0 2024-02-16T19:23:58.695 INFO:tasks.workunit.client.1.smithi163.stdout:2/65: unlink d2/c7 0 2024-02-16T19:23:58.695 INFO:tasks.workunit.client.1.smithi163.stdout:2/66: write d2/fb [550048,126279] 0 2024-02-16T19:23:58.697 INFO:tasks.workunit.client.1.smithi163.stdout:2/67: mkdir d2/d17 0 2024-02-16T19:23:58.697 INFO:tasks.workunit.client.1.smithi163.stdout:2/68: chown d2/fc 31073582 1 2024-02-16T19:23:58.697 INFO:tasks.workunit.client.1.smithi163.stdout:2/69: chown d2/cd 7 1 2024-02-16T19:23:58.697 INFO:tasks.workunit.client.1.smithi163.stdout:2/70: write d2/f13 [88914,96845] 0 2024-02-16T19:23:58.697 INFO:tasks.workunit.client.1.smithi163.stdout:2/71: creat d2/f18 x:0 0 0 2024-02-16T19:23:58.697 INFO:tasks.workunit.client.1.smithi163.stdout:2/72: write d2/f16 [932176,61308] 0 2024-02-16T19:23:58.699 INFO:tasks.workunit.client.1.smithi163.stdout:2/73: stat d2/l14 0 2024-02-16T19:23:58.721 INFO:tasks.workunit.client.1.smithi163.stdout:2/74: dwrite d2/f10 [0,4194304] 0 2024-02-16T19:23:58.722 INFO:tasks.workunit.client.1.smithi163.stdout:2/75: write d2/f16 [1761214,28987] 0 2024-02-16T19:23:58.722 INFO:tasks.workunit.client.1.smithi163.stdout:2/76: chown d2/l12 1 1 2024-02-16T19:23:58.722 INFO:tasks.workunit.client.1.smithi163.stdout:2/77: chown d2/l8 0 1 2024-02-16T19:23:58.723 INFO:tasks.workunit.client.1.smithi163.stdout:2/78: rmdir d2/d17 0 2024-02-16T19:23:58.723 INFO:tasks.workunit.client.1.smithi163.stdout:2/79: creat d2/f19 x:0 0 0 2024-02-16T19:23:58.723 INFO:tasks.workunit.client.1.smithi163.stdout:2/80: truncate d2/ff 562274 0 2024-02-16T19:23:58.725 INFO:tasks.workunit.client.1.smithi163.stdout:2/81: rename d2/f10 to d2/f1a 0 2024-02-16T19:23:58.830 INFO:tasks.workunit.client.1.smithi163.stdout:2/82: dwrite d2/fb [0,4194304] 0 2024-02-16T19:23:58.854 INFO:tasks.workunit.client.1.smithi163.stdout:2/83: dwrite d2/f1a [0,4194304] 0 2024-02-16T19:23:58.882 INFO:tasks.workunit.client.1.smithi163.stdout:2/84: dwrite d2/fc [0,4194304] 0 2024-02-16T19:23:58.884 INFO:tasks.workunit.client.1.smithi163.stdout:2/85: mknod d2/c1b 0 2024-02-16T19:23:58.884 INFO:tasks.workunit.client.1.smithi163.stdout:2/86: chown d2/ce 16828 1 2024-02-16T19:23:58.904 INFO:tasks.workunit.client.1.smithi163.stdout:2/87: dwrite d2/f13 [0,4194304] 0 2024-02-16T19:23:58.904 INFO:tasks.workunit.client.1.smithi163.stdout:2/88: readlink d2/l8 0 2024-02-16T19:23:58.906 INFO:tasks.workunit.client.1.smithi163.stdout:2/89: rename d2/f13 to d2/f1c 0 2024-02-16T19:23:58.907 INFO:tasks.workunit.client.1.smithi163.stdout:2/90: symlink d2/l1d 0 2024-02-16T19:23:58.907 INFO:tasks.workunit.client.1.smithi163.stdout:2/91: write d2/f18 [766386,106801] 0 2024-02-16T19:23:58.908 INFO:tasks.workunit.client.1.smithi163.stdout:9/41: dread f2 [0,4194304] 0 2024-02-16T19:23:59.063 INFO:tasks.workunit.client.1.smithi163.stdout:2/92: dread d2/f16 [0,4194304] 0 2024-02-16T19:23:59.064 INFO:tasks.workunit.client.1.smithi163.stdout:9/42: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:23:59.064 INFO:tasks.workunit.client.1.smithi163.stdout:9/43: fsync d5/de/ff 0 2024-02-16T19:23:59.065 INFO:tasks.workunit.client.1.smithi163.stdout:9/44: chown d5/l10 2 1 2024-02-16T19:23:59.065 INFO:tasks.workunit.client.1.smithi163.stdout:9/45: write d5/de/ff [984895,78171] 0 2024-02-16T19:23:59.065 INFO:tasks.workunit.client.1.smithi163.stdout:9/46: readlink d5/ld 0 2024-02-16T19:23:59.071 INFO:tasks.workunit.client.1.smithi163.stdout:9/47: dread d5/de/ff [0,4194304] 0 2024-02-16T19:23:59.072 INFO:tasks.workunit.client.1.smithi163.stdout:9/48: write f2 [3927637,123565] 0 2024-02-16T19:23:59.075 INFO:tasks.workunit.client.1.smithi163.stdout:9/49: symlink d5/de/l11 0 2024-02-16T19:23:59.076 INFO:tasks.workunit.client.1.smithi163.stdout:9/50: symlink d5/l12 0 2024-02-16T19:23:59.076 INFO:tasks.workunit.client.1.smithi163.stdout:9/51: rmdir d5 39 2024-02-16T19:23:59.076 INFO:tasks.workunit.client.1.smithi163.stdout:9/52: rename d5/de to d5/de/d13 22 2024-02-16T19:23:59.078 INFO:tasks.workunit.client.1.smithi163.stdout:2/93: dwrite d2/f16 [0,4194304] 0 2024-02-16T19:23:59.089 INFO:tasks.workunit.client.1.smithi163.stdout:2/94: truncate d2/f16 2321192 0 2024-02-16T19:23:59.098 INFO:tasks.workunit.client.1.smithi163.stdout:2/95: write d2/fc [2347944,99112] 0 2024-02-16T19:23:59.104 INFO:tasks.workunit.client.1.smithi163.stdout:9/53: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:23:59.104 INFO:tasks.workunit.client.1.smithi163.stdout:9/54: write d5/f6 [2787261,8087] 0 2024-02-16T19:23:59.106 INFO:tasks.workunit.client.1.smithi163.stdout:9/55: creat d5/f14 x:0 0 0 2024-02-16T19:23:59.106 INFO:tasks.workunit.client.1.smithi163.stdout:9/56: creat d5/f15 x:0 0 0 2024-02-16T19:23:59.109 INFO:tasks.workunit.client.0.smithi040.stdout:4/11: dwrite f2 [0,4194304] 0 2024-02-16T19:23:59.109 INFO:tasks.workunit.client.0.smithi040.stdout:4/12: fdatasync f4 0 2024-02-16T19:23:59.125 INFO:tasks.workunit.client.1.smithi163.stdout:2/96: dwrite d2/ff [0,4194304] 0 2024-02-16T19:23:59.125 INFO:tasks.workunit.client.1.smithi163.stdout:2/97: creat d2/f1e x:0 0 0 2024-02-16T19:23:59.125 INFO:tasks.workunit.client.1.smithi163.stdout:2/98: creat d2/f1f x:0 0 0 2024-02-16T19:23:59.125 INFO:tasks.workunit.client.1.smithi163.stdout:9/57: dwrite d5/fa [0,4194304] 0 2024-02-16T19:23:59.125 INFO:tasks.workunit.client.1.smithi163.stdout:9/58: write d5/f6 [3040442,111673] 0 2024-02-16T19:23:59.125 INFO:tasks.workunit.client.1.smithi163.stdout:9/59: chown c4 495470378 1 2024-02-16T19:23:59.125 INFO:tasks.workunit.client.1.smithi163.stdout:9/60: readlink d5/de/l11 0 2024-02-16T19:23:59.127 INFO:tasks.workunit.client.1.smithi163.stdout:2/99: unlink d2/cd 0 2024-02-16T19:23:59.127 INFO:tasks.workunit.client.1.smithi163.stdout:2/100: dread - d2/f19 zero size 2024-02-16T19:23:59.132 INFO:tasks.workunit.client.1.smithi163.stdout:2/101: getdents d2 0 2024-02-16T19:23:59.132 INFO:tasks.workunit.client.1.smithi163.stdout:2/102: dread - d2/f15 zero size 2024-02-16T19:23:59.133 INFO:tasks.workunit.client.1.smithi163.stdout:2/103: symlink d2/l20 0 2024-02-16T19:23:59.133 INFO:tasks.workunit.client.1.smithi163.stdout:2/104: write d2/f5 [20957,28744] 0 2024-02-16T19:23:59.134 INFO:tasks.workunit.client.1.smithi163.stdout:2/105: mknod d2/c21 0 2024-02-16T19:23:59.151 INFO:tasks.workunit.client.1.smithi163.stdout:2/106: dwrite d2/fb [0,4194304] 0 2024-02-16T19:23:59.151 INFO:tasks.workunit.client.1.smithi163.stdout:2/107: write d2/f5 [652125,113568] 0 2024-02-16T19:23:59.151 INFO:tasks.workunit.client.1.smithi163.stdout:9/61: dwrite d5/f14 [0,4194304] 0 2024-02-16T19:23:59.158 INFO:tasks.workunit.client.1.smithi163.stdout:9/62: truncate d5/f6 423250 0 2024-02-16T19:23:59.159 INFO:tasks.workunit.client.1.smithi163.stdout:9/63: dread - d5/f15 zero size 2024-02-16T19:23:59.159 INFO:tasks.workunit.client.1.smithi163.stdout:9/64: write d5/f15 [811885,73643] 0 2024-02-16T19:23:59.160 INFO:tasks.workunit.client.1.smithi163.stdout:9/65: getdents d5 0 2024-02-16T19:23:59.168 INFO:tasks.workunit.client.1.smithi163.stdout:9/66: dread f2 [0,4194304] 0 2024-02-16T19:23:59.174 INFO:tasks.workunit.client.1.smithi163.stdout:2/108: dwrite d2/f15 [0,4194304] 0 2024-02-16T19:23:59.174 INFO:tasks.workunit.client.1.smithi163.stdout:9/67: truncate d5/fa 3262885 0 2024-02-16T19:23:59.176 INFO:tasks.workunit.client.1.smithi163.stdout:2/109: rename d2/c21 to d2/c22 0 2024-02-16T19:23:59.177 INFO:tasks.workunit.client.1.smithi163.stdout:9/68: mkdir d5/de/d16 0 2024-02-16T19:23:59.177 INFO:tasks.workunit.client.1.smithi163.stdout:9/69: getdents d5/de/d16 0 2024-02-16T19:23:59.177 INFO:tasks.workunit.client.1.smithi163.stdout:9/70: creat d5/de/f17 x:0 0 0 2024-02-16T19:23:59.177 INFO:tasks.workunit.client.1.smithi163.stdout:9/71: dread - d5/de/f17 zero size 2024-02-16T19:23:59.179 INFO:tasks.workunit.client.1.smithi163.stdout:2/110: creat d2/f23 x:0 0 0 2024-02-16T19:23:59.181 INFO:tasks.workunit.client.1.smithi163.stdout:9/72: creat d5/f18 x:0 0 0 2024-02-16T19:23:59.181 INFO:tasks.workunit.client.1.smithi163.stdout:9/73: chown d5/de/l11 782418 1 2024-02-16T19:23:59.181 INFO:tasks.workunit.client.1.smithi163.stdout:2/111: symlink d2/l24 0 2024-02-16T19:23:59.182 INFO:tasks.workunit.client.1.smithi163.stdout:9/74: link f2 d5/f19 0 2024-02-16T19:23:59.182 INFO:tasks.workunit.client.1.smithi163.stdout:9/75: creat d5/f1a x:0 0 0 2024-02-16T19:23:59.183 INFO:tasks.workunit.client.1.smithi163.stdout:2/112: mknod d2/c25 0 2024-02-16T19:23:59.183 INFO:tasks.workunit.client.1.smithi163.stdout:2/113: readlink d2/l1d 0 2024-02-16T19:23:59.184 INFO:tasks.workunit.client.1.smithi163.stdout:2/114: fdatasync d2/ff 0 2024-02-16T19:23:59.184 INFO:tasks.workunit.client.1.smithi163.stdout:2/115: stat d2 0 2024-02-16T19:23:59.184 INFO:tasks.workunit.client.1.smithi163.stdout:2/116: chown d2/f9 120778 1 2024-02-16T19:23:59.185 INFO:tasks.workunit.client.1.smithi163.stdout:9/76: creat d5/de/d16/f1b x:0 0 0 2024-02-16T19:23:59.186 INFO:tasks.workunit.client.1.smithi163.stdout:2/117: mkdir d2/d26 0 2024-02-16T19:23:59.186 INFO:tasks.workunit.client.1.smithi163.stdout:9/77: truncate d5/fa 616092 0 2024-02-16T19:23:59.187 INFO:tasks.workunit.client.1.smithi163.stdout:2/118: creat d2/d26/f27 x:0 0 0 2024-02-16T19:23:59.188 INFO:tasks.workunit.client.1.smithi163.stdout:9/78: mknod d5/de/c1c 0 2024-02-16T19:23:59.189 INFO:tasks.workunit.client.1.smithi163.stdout:2/119: fsync d2/f18 0 2024-02-16T19:23:59.189 INFO:tasks.workunit.client.1.smithi163.stdout:9/79: mknod d5/de/d16/c1d 0 2024-02-16T19:23:59.190 INFO:tasks.workunit.client.1.smithi163.stdout:9/80: readlink d5/de/l11 0 2024-02-16T19:23:59.193 INFO:tasks.workunit.client.1.smithi163.stdout:2/120: rename d2/f11 to d2/d26/f28 0 2024-02-16T19:23:59.194 INFO:tasks.workunit.client.1.smithi163.stdout:9/81: truncate d5/f14 2284367 0 2024-02-16T19:23:59.194 INFO:tasks.workunit.client.1.smithi163.stdout:9/82: creat d5/f1e x:0 0 0 2024-02-16T19:23:59.195 INFO:tasks.workunit.client.1.smithi163.stdout:2/121: mknod d2/d26/c29 0 2024-02-16T19:23:59.195 INFO:tasks.workunit.client.1.smithi163.stdout:2/122: fdatasync d2/d26/f27 0 2024-02-16T19:23:59.199 INFO:tasks.workunit.client.1.smithi163.stdout:2/123: mkdir d2/d2a 0 2024-02-16T19:23:59.199 INFO:tasks.workunit.client.1.smithi163.stdout:2/124: creat d2/d26/f2b x:0 0 0 2024-02-16T19:23:59.199 INFO:tasks.workunit.client.1.smithi163.stdout:2/125: chown d2/d26 0 1 2024-02-16T19:23:59.201 INFO:tasks.workunit.client.1.smithi163.stdout:9/83: getdents d5/de/d16 0 2024-02-16T19:23:59.201 INFO:tasks.workunit.client.1.smithi163.stdout:9/84: write d5/f19 [2258731,107811] 0 2024-02-16T19:23:59.202 INFO:tasks.workunit.client.1.smithi163.stdout:2/126: mknod d2/d2a/c2c 0 2024-02-16T19:23:59.202 INFO:tasks.workunit.client.1.smithi163.stdout:2/127: write d2/d26/f27 [432014,66496] 0 2024-02-16T19:23:59.204 INFO:tasks.workunit.client.1.smithi163.stdout:2/128: mkdir d2/d2a/d2d 0 2024-02-16T19:23:59.204 INFO:tasks.workunit.client.1.smithi163.stdout:2/129: fdatasync d2/fb 0 2024-02-16T19:23:59.204 INFO:tasks.workunit.client.1.smithi163.stdout:2/130: dread - d2/f1f zero size 2024-02-16T19:23:59.205 INFO:tasks.workunit.client.1.smithi163.stdout:2/131: creat d2/d2a/d2d/f2e x:0 0 0 2024-02-16T19:23:59.205 INFO:tasks.workunit.client.1.smithi163.stdout:2/132: creat d2/d26/f2f x:0 0 0 2024-02-16T19:23:59.205 INFO:tasks.workunit.client.1.smithi163.stdout:9/85: write d5/f14 [3257872,20442] 0 2024-02-16T19:23:59.207 INFO:tasks.workunit.client.1.smithi163.stdout:9/86: dread d5/fa [0,4194304] 0 2024-02-16T19:23:59.207 INFO:tasks.workunit.client.1.smithi163.stdout:9/87: chown d5/l7 1548 1 2024-02-16T19:23:59.207 INFO:tasks.workunit.client.1.smithi163.stdout:9/88: write d5/f6 [1467839,77357] 0 2024-02-16T19:23:59.209 INFO:tasks.workunit.client.1.smithi163.stdout:9/89: mknod d5/c1f 0 2024-02-16T19:23:59.210 INFO:tasks.workunit.client.1.smithi163.stdout:2/133: getdents d2 0 2024-02-16T19:23:59.210 INFO:tasks.workunit.client.1.smithi163.stdout:2/134: chown d2/d2a/c2c 5 1 2024-02-16T19:23:59.210 INFO:tasks.workunit.client.1.smithi163.stdout:2/135: truncate d2/f19 541020 0 2024-02-16T19:23:59.212 INFO:tasks.workunit.client.1.smithi163.stdout:9/90: symlink d5/de/d16/l20 0 2024-02-16T19:23:59.220 INFO:tasks.workunit.client.1.smithi163.stdout:9/91: dread d5/f6 [0,4194304] 0 2024-02-16T19:23:59.222 INFO:tasks.workunit.client.1.smithi163.stdout:9/92: creat d5/de/d16/f21 x:0 0 0 2024-02-16T19:23:59.223 INFO:tasks.workunit.client.1.smithi163.stdout:9/93: symlink d5/de/l22 0 2024-02-16T19:23:59.224 INFO:tasks.workunit.client.1.smithi163.stdout:9/94: symlink d5/l23 0 2024-02-16T19:23:59.225 INFO:tasks.workunit.client.1.smithi163.stdout:9/95: mknod d5/de/c24 0 2024-02-16T19:23:59.227 INFO:tasks.workunit.client.1.smithi163.stdout:9/96: link d5/f18 d5/de/d16/f25 0 2024-02-16T19:23:59.229 INFO:tasks.workunit.client.1.smithi163.stdout:9/97: unlink d5/l23 0 2024-02-16T19:23:59.230 INFO:tasks.workunit.client.1.smithi163.stdout:9/98: symlink d5/de/l26 0 2024-02-16T19:23:59.230 INFO:tasks.workunit.client.1.smithi163.stdout:9/99: chown d5/de/d16/f1b 122089959 1 2024-02-16T19:23:59.232 INFO:tasks.workunit.client.1.smithi163.stdout:9/100: unlink d5/fa 0 2024-02-16T19:23:59.232 INFO:tasks.workunit.client.1.smithi163.stdout:9/101: stat d5/de/l26 0 2024-02-16T19:23:59.233 INFO:tasks.workunit.client.1.smithi163.stdout:9/102: symlink d5/l27 0 2024-02-16T19:23:59.244 INFO:tasks.workunit.client.1.smithi163.stdout:9/103: dread d5/f9 [0,4194304] 0 2024-02-16T19:23:59.244 INFO:tasks.workunit.client.1.smithi163.stdout:9/104: dread - d5/f1a zero size 2024-02-16T19:23:59.244 INFO:tasks.workunit.client.1.smithi163.stdout:9/105: creat d5/de/f28 x:0 0 0 2024-02-16T19:23:59.249 INFO:tasks.workunit.client.1.smithi163.stdout:9/106: rename d5/cc to d5/c29 0 2024-02-16T19:23:59.249 INFO:tasks.workunit.client.1.smithi163.stdout:9/107: truncate d5/de/d16/f1b 786299 0 2024-02-16T19:23:59.249 INFO:tasks.workunit.client.1.smithi163.stdout:9/108: readlink d5/de/d16/l20 0 2024-02-16T19:23:59.249 INFO:tasks.workunit.client.1.smithi163.stdout:9/109: chown d5/cb 26824 1 2024-02-16T19:23:59.250 INFO:tasks.workunit.client.1.smithi163.stdout:9/110: rename d5/de/d16/f1b to d5/de/d16/f2a 0 2024-02-16T19:23:59.253 INFO:tasks.workunit.client.1.smithi163.stdout:9/111: dread d5/f19 [0,4194304] 0 2024-02-16T19:23:59.268 INFO:tasks.workunit.client.1.smithi163.stdout:9/112: rmdir d5/de/d16 39 2024-02-16T19:23:59.288 INFO:tasks.workunit.client.1.smithi163.stdout:9/113: dwrite d5/f18 [0,4194304] 0 2024-02-16T19:23:59.310 INFO:tasks.workunit.client.1.smithi163.stdout:9/114: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:23:59.310 INFO:tasks.workunit.client.1.smithi163.stdout:9/115: chown d5/f1e 22885507 1 2024-02-16T19:23:59.310 INFO:tasks.workunit.client.1.smithi163.stdout:9/116: chown d5/f15 690 1 2024-02-16T19:23:59.310 INFO:tasks.workunit.client.1.smithi163.stdout:9/117: creat d5/f2b x:0 0 0 2024-02-16T19:23:59.310 INFO:tasks.workunit.client.1.smithi163.stdout:9/118: creat d5/f2c x:0 0 0 2024-02-16T19:23:59.310 INFO:tasks.workunit.client.1.smithi163.stdout:9/119: read d5/de/d16/f2a [334352,116547] 0 2024-02-16T19:23:59.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/120: dwrite d5/de/f17 [0,4194304] 0 2024-02-16T19:23:59.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/121: write d5/f2c [747456,50495] 0 2024-02-16T19:23:59.329 INFO:tasks.workunit.client.1.smithi163.stdout:9/122: mknod d5/de/c2d 0 2024-02-16T19:23:59.329 INFO:tasks.workunit.client.1.smithi163.stdout:9/123: creat d5/f2e x:0 0 0 2024-02-16T19:23:59.329 INFO:tasks.workunit.client.1.smithi163.stdout:9/124: creat d5/f2f x:0 0 0 2024-02-16T19:23:59.332 INFO:tasks.workunit.client.1.smithi163.stdout:9/125: mkdir d5/de/d30 0 2024-02-16T19:23:59.333 INFO:tasks.workunit.client.1.smithi163.stdout:9/126: mkdir d5/de/d30/d31 0 2024-02-16T19:23:59.335 INFO:tasks.workunit.client.1.smithi163.stdout:9/127: dread d5/f15 [0,4194304] 0 2024-02-16T19:23:59.352 INFO:tasks.workunit.client.1.smithi163.stdout:9/128: dwrite d5/f14 [0,4194304] 0 2024-02-16T19:23:59.353 INFO:tasks.workunit.client.1.smithi163.stdout:9/129: mknod d5/de/d16/c32 0 2024-02-16T19:23:59.354 INFO:tasks.workunit.client.1.smithi163.stdout:9/130: write d5/f15 [1662622,47426] 0 2024-02-16T19:23:59.355 INFO:tasks.workunit.client.1.smithi163.stdout:9/131: mknod d5/de/d16/c33 0 2024-02-16T19:23:59.356 INFO:tasks.workunit.client.1.smithi163.stdout:9/132: rename d5/de/c2d to d5/de/d30/d31/c34 0 2024-02-16T19:23:59.358 INFO:tasks.workunit.client.1.smithi163.stdout:9/133: mkdir d5/de/d16/d35 0 2024-02-16T19:23:59.359 INFO:tasks.workunit.client.1.smithi163.stdout:9/134: creat d5/de/d30/f36 x:0 0 0 2024-02-16T19:23:59.361 INFO:tasks.workunit.client.1.smithi163.stdout:9/135: unlink d5/de/d16/f21 0 2024-02-16T19:23:59.363 INFO:tasks.workunit.client.1.smithi163.stdout:9/136: rename d5/de/c1c to d5/de/d30/d31/c37 0 2024-02-16T19:23:59.365 INFO:tasks.workunit.client.1.smithi163.stdout:9/137: unlink d5/de/d16/c33 0 2024-02-16T19:23:59.365 INFO:tasks.workunit.client.1.smithi163.stdout:9/138: stat d5/de/d16/d35 0 2024-02-16T19:23:59.365 INFO:tasks.workunit.client.1.smithi163.stdout:9/139: stat d5/f9 0 2024-02-16T19:23:59.366 INFO:tasks.workunit.client.1.smithi163.stdout:9/140: mknod d5/de/c38 0 2024-02-16T19:23:59.367 INFO:tasks.workunit.client.1.smithi163.stdout:9/141: mknod d5/de/d30/c39 0 2024-02-16T19:23:59.367 INFO:tasks.workunit.client.1.smithi163.stdout:9/142: write d5/f19 [2420230,64234] 0 2024-02-16T19:23:59.370 INFO:tasks.workunit.client.1.smithi163.stdout:9/143: truncate d5/de/f17 428344 0 2024-02-16T19:23:59.378 INFO:tasks.workunit.client.1.smithi163.stdout:9/144: getdents d5/de/d30/d31 0 2024-02-16T19:23:59.387 INFO:tasks.workunit.client.1.smithi163.stdout:9/145: write d5/f18 [1374166,1104] 0 2024-02-16T19:23:59.387 INFO:tasks.workunit.client.1.smithi163.stdout:9/146: dread - d5/f2f zero size 2024-02-16T19:23:59.387 INFO:tasks.workunit.client.1.smithi163.stdout:9/147: stat d5/f18 0 2024-02-16T19:23:59.388 INFO:tasks.workunit.client.1.smithi163.stdout:9/148: write d5/f2c [479037,39362] 0 2024-02-16T19:23:59.389 INFO:tasks.workunit.client.1.smithi163.stdout:9/149: rename d5/de/f17 to d5/de/d16/f3a 0 2024-02-16T19:23:59.389 INFO:tasks.workunit.client.1.smithi163.stdout:9/150: chown d5/c1f 1 1 2024-02-16T19:23:59.392 INFO:tasks.workunit.client.1.smithi163.stdout:9/151: getdents d5/de 0 2024-02-16T19:23:59.392 INFO:tasks.workunit.client.1.smithi163.stdout:9/152: chown d5/de/d30/f36 5752 1 2024-02-16T19:23:59.408 INFO:tasks.workunit.client.1.smithi163.stdout:9/153: dwrite d5/f14 [0,4194304] 0 2024-02-16T19:23:59.417 INFO:tasks.workunit.client.1.smithi163.stdout:9/154: write d5/f9 [934287,98863] 0 2024-02-16T19:23:59.438 INFO:tasks.workunit.client.1.smithi163.stdout:9/155: dwrite d5/f15 [0,4194304] 0 2024-02-16T19:23:59.440 INFO:tasks.workunit.client.1.smithi163.stdout:9/156: link d5/de/d16/f25 d5/de/d16/d35/f3b 0 2024-02-16T19:23:59.441 INFO:tasks.workunit.client.1.smithi163.stdout:9/157: creat d5/de/d16/f3c x:0 0 0 2024-02-16T19:23:59.444 INFO:tasks.workunit.client.1.smithi163.stdout:9/158: rename d5/de/d16/d35 to d5/de/d30/d3d 0 2024-02-16T19:23:59.446 INFO:tasks.workunit.client.1.smithi163.stdout:9/159: read d5/de/ff [325659,78597] 0 2024-02-16T19:23:59.447 INFO:tasks.workunit.client.1.smithi163.stdout:9/160: link d5/f15 d5/de/d30/f3e 0 2024-02-16T19:23:59.448 INFO:tasks.workunit.client.1.smithi163.stdout:9/161: mknod d5/de/d16/c3f 0 2024-02-16T19:23:59.450 INFO:tasks.workunit.client.1.smithi163.stdout:9/162: rename f2 to d5/f40 0 2024-02-16T19:23:59.450 INFO:tasks.workunit.client.1.smithi163.stdout:9/163: read - d5/de/d30/f36 zero size 2024-02-16T19:23:59.450 INFO:tasks.workunit.client.1.smithi163.stdout:9/164: write d5/f2f [815343,51108] 0 2024-02-16T19:23:59.451 INFO:tasks.workunit.client.1.smithi163.stdout:9/165: getdents d5/de/d30/d3d 0 2024-02-16T19:23:59.451 INFO:tasks.workunit.client.1.smithi163.stdout:9/166: readlink d5/de/l22 0 2024-02-16T19:23:59.451 INFO:tasks.workunit.client.1.smithi163.stdout:9/167: symlink d5/de/d30/d3d/l41 0 2024-02-16T19:23:59.452 INFO:tasks.workunit.client.1.smithi163.stdout:9/168: symlink d5/de/d30/d31/l42 0 2024-02-16T19:23:59.452 INFO:tasks.workunit.client.1.smithi163.stdout:9/169: write d5/de/d16/f3a [714531,21671] 0 2024-02-16T19:23:59.471 INFO:tasks.workunit.client.1.smithi163.stdout:9/170: dwrite d5/de/ff [0,4194304] 0 2024-02-16T19:23:59.472 INFO:tasks.workunit.client.1.smithi163.stdout:9/171: dread d5/de/d16/f2a [0,4194304] 0 2024-02-16T19:23:59.473 INFO:tasks.workunit.client.1.smithi163.stdout:9/172: mkdir d5/de/d30/d43 0 2024-02-16T19:23:59.493 INFO:tasks.workunit.client.1.smithi163.stdout:9/173: dwrite d5/de/d30/f3e [0,4194304] 0 2024-02-16T19:23:59.500 INFO:tasks.workunit.client.1.smithi163.stdout:9/174: rmdir d5 39 2024-02-16T19:23:59.506 INFO:tasks.workunit.client.1.smithi163.stdout:9/175: dread d5/f19 [0,4194304] 0 2024-02-16T19:23:59.506 INFO:tasks.workunit.client.1.smithi163.stdout:9/176: chown d5/de/d30/d3d/l41 15780313 1 2024-02-16T19:23:59.507 INFO:tasks.workunit.client.1.smithi163.stdout:9/177: symlink d5/de/d30/d31/l44 0 2024-02-16T19:23:59.510 INFO:tasks.workunit.client.1.smithi163.stdout:9/178: truncate d5/f40 86258 0 2024-02-16T19:23:59.512 INFO:tasks.workunit.client.1.smithi163.stdout:9/179: unlink d5/de/d16/f3a 0 2024-02-16T19:23:59.513 INFO:tasks.workunit.client.1.smithi163.stdout:9/180: symlink d5/de/d30/d43/l45 0 2024-02-16T19:23:59.513 INFO:tasks.workunit.client.1.smithi163.stdout:9/181: dread - d5/f1a zero size 2024-02-16T19:23:59.513 INFO:tasks.workunit.client.1.smithi163.stdout:9/182: chown d5/de/d16/f25 1504334 1 2024-02-16T19:23:59.527 INFO:tasks.workunit.client.1.smithi163.stdout:9/183: dwrite d5/f19 [0,4194304] 0 2024-02-16T19:23:59.543 INFO:tasks.workunit.client.1.smithi163.stdout:9/184: dwrite d5/de/d30/f36 [0,4194304] 0 2024-02-16T19:23:59.544 INFO:tasks.workunit.client.1.smithi163.stdout:9/185: mkdir d5/de/d16/d46 0 2024-02-16T19:23:59.545 INFO:tasks.workunit.client.1.smithi163.stdout:9/186: mkdir d5/de/d16/d47 0 2024-02-16T19:23:59.546 INFO:tasks.workunit.client.1.smithi163.stdout:9/187: symlink d5/de/d16/d47/l48 0 2024-02-16T19:23:59.546 INFO:tasks.workunit.client.1.smithi163.stdout:9/188: chown d5/f6 13 1 2024-02-16T19:23:59.546 INFO:tasks.workunit.client.1.smithi163.stdout:9/189: chown d5/de/f28 419466 1 2024-02-16T19:23:59.551 INFO:tasks.workunit.client.1.smithi163.stdout:9/190: dread d5/f2f [0,4194304] 0 2024-02-16T19:23:59.559 INFO:tasks.workunit.client.1.smithi163.stdout:9/191: dread d5/de/d30/f36 [0,4194304] 0 2024-02-16T19:23:59.560 INFO:tasks.workunit.client.1.smithi163.stdout:9/192: mkdir d5/de/d30/d3d/d49 0 2024-02-16T19:23:59.561 INFO:tasks.workunit.client.1.smithi163.stdout:9/193: symlink d5/de/d16/l4a 0 2024-02-16T19:23:59.561 INFO:tasks.workunit.client.1.smithi163.stdout:9/194: stat d5/de/d30/d3d/d49 0 2024-02-16T19:23:59.561 INFO:tasks.workunit.client.1.smithi163.stdout:9/195: stat d5/de/d30/d31/c34 0 2024-02-16T19:23:59.563 INFO:tasks.workunit.client.1.smithi163.stdout:9/196: dread d5/f6 [0,4194304] 0 2024-02-16T19:23:59.587 INFO:tasks.workunit.client.1.smithi163.stdout:9/197: dwrite d5/f2f [0,4194304] 0 2024-02-16T19:23:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:23:59 smithi040 ceph-mon[29201]: pgmap v238: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 341 KiB/s rd, 9.0 MiB/s wr, 724 op/s 2024-02-16T19:23:59.660 INFO:tasks.workunit.client.1.smithi163.stdout:9/198: dwrite d5/f2b [0,4194304] 0 2024-02-16T19:23:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:23:59 smithi163 ceph-mon[36452]: pgmap v238: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 341 KiB/s rd, 9.0 MiB/s wr, 724 op/s 2024-02-16T19:23:59.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.992+0000 7f958373e700 1 -- 172.21.15.40:0/3933394832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 msgr2=0x7f957c0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.992+0000 7f958373e700 1 --2- 172.21.15.40:0/3933394832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 0x7f957c0fc250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9578004940 tx=0x7f9578009e30 comp rx=0 tx=0).stop 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 -- 172.21.15.40:0/3933394832 shutdown_connections 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 --2- 172.21.15.40:0/3933394832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f957c0fcfb0 0x7f957c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 --2- 172.21.15.40:0/3933394832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 0x7f957c0fc250 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 -- 172.21.15.40:0/3933394832 >> 172.21.15.40:0/3933394832 conn(0x7f957c0f7520 msgr2=0x7f957c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 -- 172.21.15.40:0/3933394832 shutdown_connections 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 -- 172.21.15.40:0/3933394832 wait complete. 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 Processor -- start 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 -- start start 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 0x7f957c10d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f957c0fcfb0 0x7f957c10dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c10e3e0 con 0x7f957c0fbe40 2024-02-16T19:23:59.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f958373e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c10ec80 con 0x7f957c0fcfb0 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.993+0000 7f95814da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 0x7f957c10d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f95814da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 0x7f957c10d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53250/0 (socket says 172.21.15.40:53250) 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f95814da700 1 -- 172.21.15.40:0/2189485776 learned_addr learned my addr 172.21.15.40:0/2189485776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f9580cd9700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f957c0fcfb0 0x7f957c10dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f95814da700 1 -- 172.21.15.40:0/2189485776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f957c0fcfb0 msgr2=0x7f957c10dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f95814da700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f957c0fcfb0 0x7f957c10dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f95814da700 1 -- 172.21.15.40:0/2189485776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578005040 con 0x7f957c0fbe40 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f95814da700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 0x7f957c10d8a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f95780093b0 tx=0x7f9578009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f9571ffb700 1 -- 172.21.15.40:0/2189485776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f957800e040 con 0x7f957c0fbe40 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f9571ffb700 1 -- 172.21.15.40:0/2189485776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f957800bcc0 con 0x7f957c0fbe40 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c10ef00 con 0x7f957c0fbe40 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f9571ffb700 1 -- 172.21.15.40:0/2189485776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f957801c630 con 0x7f957c0fbe40 2024-02-16T19:23:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.994+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c10f440 con 0x7f957c0fbe40 2024-02-16T19:23:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.995+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957c04bed0 con 0x7f957c0fbe40 2024-02-16T19:23:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.996+0000 7f9571ffb700 1 -- 172.21.15.40:0/2189485776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9578013980 con 0x7f957c0fbe40 2024-02-16T19:23:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.996+0000 7f9571ffb700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f95680692a0 0x7f956806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:23:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.996+0000 7f9571ffb700 1 -- 172.21.15.40:0/2189485776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9578088590 con 0x7f957c0fbe40 2024-02-16T19:23:59.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.996+0000 7f9580cd9700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f95680692a0 0x7f956806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:23:59.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:23:59.997+0000 7f9580cd9700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f95680692a0 0x7f956806b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f956c009fd0 tx=0x7f956c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:00.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.001+0000 7f9571ffb700 1 -- 172.21.15.40:0/2189485776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9578058300 con 0x7f957c0fbe40 2024-02-16T19:24:00.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.227+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f957c10f090 con 0x7f95680692a0 2024-02-16T19:24:00.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.228+0000 7f9571ffb700 1 -- 172.21.15.40:0/2189485776 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f957c10f090 con 0x7f95680692a0 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f95680692a0 msgr2=0x7f956806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f95680692a0 0x7f956806b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f956c009fd0 tx=0x7f956c00b040 comp rx=0 tx=0).stop 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 msgr2=0x7f957c10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 0x7f957c10d8a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f95780093b0 tx=0x7f9578009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 shutdown_connections 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f95680692a0 0x7f956806b750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f957c0fbe40 0x7f957c10d8a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 --2- 172.21.15.40:0/2189485776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f957c0fcfb0 0x7f957c10dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 >> 172.21.15.40:0/2189485776 conn(0x7f957c0f7520 msgr2=0x7f957c065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 shutdown_connections 2024-02-16T19:24:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.230+0000 7f958373e700 1 -- 172.21.15.40:0/2189485776 wait complete. 2024-02-16T19:24:00.242 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:24:00.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 -- 172.21.15.40:0/268255458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c1000e0 msgr2=0x7f8f3c102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 --2- 172.21.15.40:0/268255458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c1000e0 0x7f8f3c102520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f30004940 tx=0x7f8f30009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 -- 172.21.15.40:0/268255458 shutdown_connections 2024-02-16T19:24:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 --2- 172.21.15.40:0/268255458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c1000e0 0x7f8f3c102520 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 --2- 172.21.15.40:0/268255458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c0663c0 0x7f8f3c0ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 -- 172.21.15.40:0/268255458 >> 172.21.15.40:0/268255458 conn(0x7f8f3c0f74e0 msgr2=0x7f8f3c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 -- 172.21.15.40:0/268255458 shutdown_connections 2024-02-16T19:24:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 -- 172.21.15.40:0/268255458 wait complete. 2024-02-16T19:24:00.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.334+0000 7f8f41dad700 1 Processor -- start 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f41dad700 1 -- start start 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f41dad700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c0663c0 0x7f8f3c10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f41dad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c1000e0 0x7f8f3c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f41dad700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c10e470 con 0x7f8f3c0663c0 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f41dad700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c10ed10 con 0x7f8f3c1000e0 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f3b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c0663c0 0x7f8f3c10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f3b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c0663c0 0x7f8f3c10d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53262/0 (socket says 172.21.15.40:53262) 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f3b7fe700 1 -- 172.21.15.40:0/1762342917 learned_addr learned my addr 172.21.15.40:0/1762342917 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f3affd700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c1000e0 0x7f8f3c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f3b7fe700 1 -- 172.21.15.40:0/1762342917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c1000e0 msgr2=0x7f8f3c10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f3b7fe700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c1000e0 0x7f8f3c10de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.335+0000 7f8f3b7fe700 1 -- 172.21.15.40:0/1762342917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f30005040 con 0x7f8f3c0663c0 2024-02-16T19:24:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.336+0000 7f8f3b7fe700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c0663c0 0x7f8f3c10d930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00eb40 tx=0x7f8f2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.336+0000 7f8f23fff700 1 -- 172.21.15.40:0/1762342917 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f2c007cc0 con 0x7f8f3c0663c0 2024-02-16T19:24:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.336+0000 7f8f23fff700 1 -- 172.21.15.40:0/1762342917 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f2c00f040 con 0x7f8f3c0663c0 2024-02-16T19:24:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.336+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c10ef90 con 0x7f8f3c0663c0 2024-02-16T19:24:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.336+0000 7f8f23fff700 1 -- 172.21.15.40:0/1762342917 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f2c00ca10 con 0x7f8f3c0663c0 2024-02-16T19:24:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.336+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c10f4d0 con 0x7f8f3c0663c0 2024-02-16T19:24:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.337+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c04bed0 con 0x7f8f3c0663c0 2024-02-16T19:24:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.337+0000 7f8f23fff700 1 -- 172.21.15.40:0/1762342917 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8f2c00cb70 con 0x7f8f3c0663c0 2024-02-16T19:24:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.338+0000 7f8f23fff700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8f240692a0 0x7f8f2406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.338+0000 7f8f23fff700 1 -- 172.21.15.40:0/1762342917 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8f2c086cf0 con 0x7f8f3c0663c0 2024-02-16T19:24:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.338+0000 7f8f3affd700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8f240692a0 0x7f8f2406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.338+0000 7f8f3affd700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8f240692a0 0x7f8f2406b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8f30000c00 tx=0x7f8f30009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:00.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.342+0000 7f8f23fff700 1 -- 172.21.15.40:0/1762342917 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8f2c04dbd0 con 0x7f8f3c0663c0 2024-02-16T19:24:00.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.552+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f3c100d60 con 0x7f8f240692a0 2024-02-16T19:24:00.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.554+0000 7f8f23fff700 1 -- 172.21.15.40:0/1762342917 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f8f3c100d60 con 0x7f8f240692a0 2024-02-16T19:24:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8f240692a0 msgr2=0x7f8f2406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8f240692a0 0x7f8f2406b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8f30000c00 tx=0x7f8f30009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c0663c0 msgr2=0x7f8f3c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c0663c0 0x7f8f3c10d930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00eb40 tx=0x7f8f2c00b040 comp rx=0 tx=0).stop 2024-02-16T19:24:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 shutdown_connections 2024-02-16T19:24:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f8f240692a0 0x7f8f2406b750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f3c0663c0 0x7f8f3c10d930 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 --2- 172.21.15.40:0/1762342917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c1000e0 0x7f8f3c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 >> 172.21.15.40:0/1762342917 conn(0x7f8f3c0f74e0 msgr2=0x7f8f3c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:00.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 shutdown_connections 2024-02-16T19:24:00.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.556+0000 7f8f41dad700 1 -- 172.21.15.40:0/1762342917 wait complete. 2024-02-16T19:24:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.658+0000 7f164d49a700 1 -- 172.21.15.40:0/2336958198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 msgr2=0x7f1648102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.658+0000 7f164d49a700 1 --2- 172.21.15.40:0/2336958198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 0x7f1648102780 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f163c004940 tx=0x7f163c009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.658+0000 7f164d49a700 1 -- 172.21.15.40:0/2336958198 shutdown_connections 2024-02-16T19:24:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.658+0000 7f164d49a700 1 --2- 172.21.15.40:0/2336958198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 0x7f1648102780 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.658+0000 7f164d49a700 1 --2- 172.21.15.40:0/2336958198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16480fda80 0x7f16480ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.658+0000 7f164d49a700 1 -- 172.21.15.40:0/2336958198 >> 172.21.15.40:0/2336958198 conn(0x7f16480f7520 msgr2=0x7f16480f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.658+0000 7f164d49a700 1 -- 172.21.15.40:0/2336958198 shutdown_connections 2024-02-16T19:24:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.658+0000 7f164d49a700 1 -- 172.21.15.40:0/2336958198 wait complete. 2024-02-16T19:24:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f164d49a700 1 Processor -- start 2024-02-16T19:24:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f164d49a700 1 -- start start 2024-02-16T19:24:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f164d49a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16480fda80 0x7f164810fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f164d49a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 0x7f1648110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f164d49a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1648110680 con 0x7f16481003a0 2024-02-16T19:24:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f164d49a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1648110f20 con 0x7f16480fda80 2024-02-16T19:24:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f16467fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 0x7f1648110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f16467fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 0x7f1648110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53280/0 (socket says 172.21.15.40:53280) 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f16467fc700 1 -- 172.21.15.40:0/3960938488 learned_addr learned my addr 172.21.15.40:0/3960938488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.659+0000 7f1646ffd700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16480fda80 0x7f164810fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f16467fc700 1 -- 172.21.15.40:0/3960938488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16480fda80 msgr2=0x7f164810fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f16467fc700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16480fda80 0x7f164810fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f16467fc700 1 -- 172.21.15.40:0/3960938488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c005040 con 0x7f16481003a0 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f16467fc700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 0x7f1648110080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f163c009fd0 tx=0x7f163c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f162f7fe700 1 -- 172.21.15.40:0/3960938488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163c00e040 con 0x7f16481003a0 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16481111a0 con 0x7f16481003a0 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f162f7fe700 1 -- 172.21.15.40:0/3960938488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f163c00bcc0 con 0x7f16481003a0 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f162f7fe700 1 -- 172.21.15.40:0/3960938488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163c01c630 con 0x7f16481003a0 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.660+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16481116e0 con 0x7f16481003a0 2024-02-16T19:24:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.661+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164804bed0 con 0x7f16481003a0 2024-02-16T19:24:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.661+0000 7f162f7fe700 1 -- 172.21.15.40:0/3960938488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f163c013420 con 0x7f16481003a0 2024-02-16T19:24:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.662+0000 7f162f7fe700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16300692b0 0x7f163006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.662+0000 7f162f7fe700 1 -- 172.21.15.40:0/3960938488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f163c088500 con 0x7f16481003a0 2024-02-16T19:24:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.662+0000 7f1646ffd700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16300692b0 0x7f163006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.662+0000 7f1646ffd700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16300692b0 0x7f163006b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1638004f40 tx=0x7f163800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:00.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.666+0000 7f162f7fe700 1 -- 172.21.15.40:0/3960938488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f163c058310 con 0x7f16481003a0 2024-02-16T19:24:00.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.880+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1648111b40 con 0x7f16300692b0 2024-02-16T19:24:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.890+0000 7f162f7fe700 1 -- 172.21.15.40:0/3960938488 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f1648111b40 con 0x7f16300692b0 2024-02-16T19:24:00.891 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:24:00.891 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (6m) 94s ago 7m 25.1M - 0.25.0 c8568f914cd2 aadccae1ccec 2024-02-16T19:24:00.891 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (7m) 94s ago 7m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:24:00.891 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (6m) 99s ago 6m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (6m) 94s ago 7m 84.5M - 9.4.7 954c08fa6188 8447b5580554 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (113s) 94s ago 112s 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (119s) 94s ago 119s 10.4M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (110s) 99s ago 109s 14.4M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (117s) 99s ago 117s 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:9283 running (8m) 94s ago 8m 482M - 17.2.7-513-gb88fae66 870a0944ccb8 4e9e9bb4389a 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283 running (6m) 99s ago 6m 436M - 17.2.7-513-gb88fae66 870a0944ccb8 06e26641f085 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (9m) 94s ago 9m 44.9M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (6m) 99s ago 6m 35.4M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (7m) 94s ago 7m 14.0M - 1.5.0 0da6a335fe13 61d16f3f00d4 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (6m) 99s ago 6m 16.7M - 1.5.0 0da6a335fe13 c45ebf41f1b8 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (5m) 94s ago 5m 47.9M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (5m) 94s ago 5m 46.4M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:24:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (4m) 94s ago 4m 45.3M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:24:00.893 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (4m) 99s ago 4m 44.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:24:00.893 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (3m) 99s ago 3m 41.1M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:24:00.893 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (3m) 99s ago 3m 44.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:24:00.893 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (5m) 94s ago 6m 51.4M - 2.43.0 a07b618ecd1d 557bd885dbc5 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16300692b0 msgr2=0x7f163006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16300692b0 0x7f163006b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1638004f40 tx=0x7f163800b040 comp rx=0 tx=0).stop 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 msgr2=0x7f1648110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 0x7f1648110080 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f163c009fd0 tx=0x7f163c009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 shutdown_connections 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f16300692b0 0x7f163006b760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16480fda80 0x7f164810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 --2- 172.21.15.40:0/3960938488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16481003a0 0x7f1648110080 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 >> 172.21.15.40:0/3960938488 conn(0x7f16480f7520 msgr2=0x7f16480fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 shutdown_connections 2024-02-16T19:24:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.892+0000 7f164d49a700 1 -- 172.21.15.40:0/3960938488 wait complete. 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 -- 172.21.15.40:0/2374076295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 msgr2=0x7f43e40fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 --2- 172.21.15.40:0/2374076295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 0x7f43e40fd420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f43e0004940 tx=0x7f43e0009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 -- 172.21.15.40:0/2374076295 shutdown_connections 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 --2- 172.21.15.40:0/2374076295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 0x7f43e40fd420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 --2- 172.21.15.40:0/2374076295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e40fbe40 0x7f43e40fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 -- 172.21.15.40:0/2374076295 >> 172.21.15.40:0/2374076295 conn(0x7f43e40f74e0 msgr2=0x7f43e40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 -- 172.21.15.40:0/2374076295 shutdown_connections 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 -- 172.21.15.40:0/2374076295 wait complete. 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 Processor -- start 2024-02-16T19:24:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.995+0000 7f43ec537700 1 -- start start 2024-02-16T19:24:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43ec537700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e40fbe40 0x7f43e410d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43ec537700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 0x7f43e410ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43ec537700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e410e3f0 con 0x7f43e40fcfb0 2024-02-16T19:24:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43ec537700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e410ec90 con 0x7f43e40fbe40 2024-02-16T19:24:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43e9ad2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 0x7f43e410ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43e9ad2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 0x7f43e410ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53308/0 (socket says 172.21.15.40:53308) 2024-02-16T19:24:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43e9ad2700 1 -- 172.21.15.40:0/1046699021 learned_addr learned my addr 172.21.15.40:0/1046699021 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:00.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43ea2d3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e40fbe40 0x7f43e410d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43e9ad2700 1 -- 172.21.15.40:0/1046699021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e40fbe40 msgr2=0x7f43e410d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43e9ad2700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e40fbe40 0x7f43e410d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43e9ad2700 1 -- 172.21.15.40:0/1046699021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0005040 con 0x7f43e40fcfb0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43ea2d3700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e40fbe40 0x7f43e410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43e9ad2700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 0x7f43e410ddf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f43e00093b0 tx=0x7f43e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43daffd700 1 -- 172.21.15.40:0/1046699021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43e000e040 con 0x7f43e40fcfb0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.996+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e410ef10 con 0x7f43e40fcfb0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.997+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e410f450 con 0x7f43e40fcfb0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.997+0000 7f43daffd700 1 -- 172.21.15.40:0/1046699021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43e000bcc0 con 0x7f43e40fcfb0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.997+0000 7f43daffd700 1 -- 172.21.15.40:0/1046699021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43e001c630 con 0x7f43e40fcfb0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.997+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e404bed0 con 0x7f43e40fcfb0 2024-02-16T19:24:01.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.998+0000 7f43daffd700 1 -- 172.21.15.40:0/1046699021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f43e0013980 con 0x7f43e40fcfb0 2024-02-16T19:24:01.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.998+0000 7f43daffd700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43d00692a0 0x7f43d006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:01.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.998+0000 7f43daffd700 1 -- 172.21.15.40:0/1046699021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f43e0088540 con 0x7f43e40fcfb0 2024-02-16T19:24:01.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.998+0000 7f43ea2d3700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43d00692a0 0x7f43d006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:01.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:00.999+0000 7f43ea2d3700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43d00692a0 0x7f43d006b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009fd0 tx=0x7f43d400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:01.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.003+0000 7f43daffd700 1 -- 172.21.15.40:0/1046699021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f43e00582b0 con 0x7f43e40fcfb0 2024-02-16T19:24:01.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.280+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f43e410f7e0 con 0x7f43e40fcfb0 2024-02-16T19:24:01.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.281+0000 7f43daffd700 1 -- 172.21.15.40:0/1046699021 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f43e0057e40 con 0x7f43e40fcfb0 2024-02-16T19:24:01.282 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:24:01.282 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:24:01.282 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 14 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:24:01.283 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:24:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43d00692a0 msgr2=0x7f43d006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43d00692a0 0x7f43d006b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009fd0 tx=0x7f43d400b040 comp rx=0 tx=0).stop 2024-02-16T19:24:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 msgr2=0x7f43e410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 0x7f43e410ddf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f43e00093b0 tx=0x7f43e0009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 shutdown_connections 2024-02-16T19:24:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f43d00692a0 0x7f43d006b750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e40fbe40 0x7f43e410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 --2- 172.21.15.40:0/1046699021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e40fcfb0 0x7f43e410ddf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 >> 172.21.15.40:0/1046699021 conn(0x7f43e40f74e0 msgr2=0x7f43e40656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 shutdown_connections 2024-02-16T19:24:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.283+0000 7f43ec537700 1 -- 172.21.15.40:0/1046699021 wait complete. 2024-02-16T19:24:01.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 -- 172.21.15.40:0/1147808657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a101000c0 msgr2=0x7f4a10102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 --2- 172.21.15.40:0/1147808657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a101000c0 0x7f4a10102520 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4a00004940 tx=0x7f4a00009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 -- 172.21.15.40:0/1147808657 shutdown_connections 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 --2- 172.21.15.40:0/1147808657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a101000c0 0x7f4a10102520 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 --2- 172.21.15.40:0/1147808657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a10066c70 0x7f4a100ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 -- 172.21.15.40:0/1147808657 >> 172.21.15.40:0/1147808657 conn(0x7f4a100f7520 msgr2=0x7f4a100f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 -- 172.21.15.40:0/1147808657 shutdown_connections 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 -- 172.21.15.40:0/1147808657 wait complete. 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 Processor -- start 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 -- start start 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a10066c70 0x7f4a100fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.386+0000 7f4a17ebf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a101000c0 0x7f4a100fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a17ebf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100fbff0 con 0x7f4a10066c70 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a17ebf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100fc160 con 0x7f4a101000c0 2024-02-16T19:24:01.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a15c5b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a10066c70 0x7f4a100fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a15c5b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a10066c70 0x7f4a100fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53322/0 (socket says 172.21.15.40:53322) 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a15c5b700 1 -- 172.21.15.40:0/3247551200 learned_addr learned my addr 172.21.15.40:0/3247551200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a1545a700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a101000c0 0x7f4a100fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a15c5b700 1 -- 172.21.15.40:0/3247551200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a101000c0 msgr2=0x7f4a100fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a15c5b700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a101000c0 0x7f4a100fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a15c5b700 1 -- 172.21.15.40:0/3247551200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00005040 con 0x7f4a10066c70 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a15c5b700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a10066c70 0x7f4a100fd570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00eb40 tx=0x7f4a0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a067fc700 1 -- 172.21.15.40:0/3247551200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a0c007cc0 con 0x7f4a10066c70 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a100fc3e0 con 0x7f4a10066c70 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a067fc700 1 -- 172.21.15.40:0/3247551200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a0c00f040 con 0x7f4a10066c70 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a067fc700 1 -- 172.21.15.40:0/3247551200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a0c00ca10 con 0x7f4a10066c70 2024-02-16T19:24:01.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.387+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a10115900 con 0x7f4a10066c70 2024-02-16T19:24:01.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.389+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1004bed0 con 0x7f4a10066c70 2024-02-16T19:24:01.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.389+0000 7f4a067fc700 1 -- 172.21.15.40:0/3247551200 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4a0c00cc80 con 0x7f4a10066c70 2024-02-16T19:24:01.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.389+0000 7f4a067fc700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f49fc0692a0 0x7f49fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:01.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.389+0000 7f4a067fc700 1 -- 172.21.15.40:0/3247551200 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4a0c086dd0 con 0x7f4a10066c70 2024-02-16T19:24:01.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.389+0000 7f4a1545a700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f49fc0692a0 0x7f49fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:01.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.390+0000 7f4a1545a700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f49fc0692a0 0x7f49fc06b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a00009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:01.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.395+0000 7f4a067fc700 1 -- 172.21.15.40:0/3247551200 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4a0c04dbd0 con 0x7f4a10066c70 2024-02-16T19:24:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.642+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4a10115c70 con 0x7f4a10066c70 2024-02-16T19:24:01.643 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:01 smithi040 ceph-mon[29201]: pgmap v239: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 341 KiB/s rd, 8.4 MiB/s wr, 635 op/s 2024-02-16T19:24:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:01 smithi040 ceph-mon[29201]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:01 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1046699021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:24:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.642+0000 7f4a067fc700 1 -- 172.21.15.40:0/3247551200 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f4a0c056b70 con 0x7f4a10066c70 2024-02-16T19:24:01.644 INFO:teuthology.orchestra.run.smithi040.stdout:e18 2024-02-16T19:24:01.644 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:24:01.644 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:24:01.644 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:24:01.644 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 18 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:23:33.336776+0000 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:24:01.645 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:24:01.646 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:24:01.647 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:24:01.647 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:24:01.647 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:24:01.647 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:24:01.647 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:24:01.647 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f49fc0692a0 msgr2=0x7f49fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f49fc0692a0 0x7f49fc06b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a00009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a10066c70 msgr2=0x7f4a100fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a10066c70 0x7f4a100fd570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00eb40 tx=0x7f4a0c00b040 comp rx=0 tx=0).stop 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 shutdown_connections 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f49fc0692a0 0x7f49fc06b750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a10066c70 0x7f4a100fd570 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 --2- 172.21.15.40:0/3247551200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a101000c0 0x7f4a100fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 >> 172.21.15.40:0/3247551200 conn(0x7f4a100f7520 msgr2=0x7f4a100f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 shutdown_connections 2024-02-16T19:24:01.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.645+0000 7f4a17ebf700 1 -- 172.21.15.40:0/3247551200 wait complete. 2024-02-16T19:24:01.649 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 18 2024-02-16T19:24:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:01 smithi163 ceph-mon[36452]: pgmap v239: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 341 KiB/s rd, 8.4 MiB/s wr, 635 op/s 2024-02-16T19:24:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:01 smithi163 ceph-mon[36452]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:01 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1046699021' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:24:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.748+0000 7f81c8373700 1 -- 172.21.15.40:0/1636134622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c01003a0 msgr2=0x7f81c0102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.748+0000 7f81c8373700 1 --2- 172.21.15.40:0/1636134622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c01003a0 0x7f81c0102780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f81b4004940 tx=0x7f81b4009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.748+0000 7f81c8373700 1 -- 172.21.15.40:0/1636134622 shutdown_connections 2024-02-16T19:24:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.748+0000 7f81c8373700 1 --2- 172.21.15.40:0/1636134622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c01003a0 0x7f81c0102780 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.748+0000 7f81c8373700 1 --2- 172.21.15.40:0/1636134622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81c00fda80 0x7f81c00ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.748+0000 7f81c8373700 1 -- 172.21.15.40:0/1636134622 >> 172.21.15.40:0/1636134622 conn(0x7f81c00f74e0 msgr2=0x7f81c00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.748+0000 7f81c8373700 1 -- 172.21.15.40:0/1636134622 shutdown_connections 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.748+0000 7f81c8373700 1 -- 172.21.15.40:0/1636134622 wait complete. 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c8373700 1 Processor -- start 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c8373700 1 -- start start 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c8373700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c00fda80 0x7f81c010fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c8373700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81c01003a0 0x7f81c0110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c8373700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c0110640 con 0x7f81c00fda80 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c8373700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c0110ee0 con 0x7f81c01003a0 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c610f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c00fda80 0x7f81c010fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c610f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c00fda80 0x7f81c010fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53334/0 (socket says 172.21.15.40:53334) 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c610f700 1 -- 172.21.15.40:0/1930803321 learned_addr learned my addr 172.21.15.40:0/1930803321 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c590e700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81c01003a0 0x7f81c0110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c610f700 1 -- 172.21.15.40:0/1930803321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81c01003a0 msgr2=0x7f81c0110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c610f700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81c01003a0 0x7f81c0110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c610f700 1 -- 172.21.15.40:0/1930803321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4005040 con 0x7f81c00fda80 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.749+0000 7f81c610f700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c00fda80 0x7f81c010fb00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f81bc002a80 tx=0x7f81bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.750+0000 7f81b2ffd700 1 -- 172.21.15.40:0/1930803321 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81bc007cc0 con 0x7f81c00fda80 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.750+0000 7f81b2ffd700 1 -- 172.21.15.40:0/1930803321 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81bc00f040 con 0x7f81c00fda80 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.750+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c01111c0 con 0x7f81c00fda80 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.750+0000 7f81b2ffd700 1 -- 172.21.15.40:0/1930803321 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81bc00dac0 con 0x7f81c00fda80 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.750+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c0111700 con 0x7f81c00fda80 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.751+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c004bed0 con 0x7f81c00fda80 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.751+0000 7f81b2ffd700 1 -- 172.21.15.40:0/1930803321 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f81bc00dc20 con 0x7f81c00fda80 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.751+0000 7f81b2ffd700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f81ac0692a0 0x7f81ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.752+0000 7f81b2ffd700 1 -- 172.21.15.40:0/1930803321 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f81bc086f60 con 0x7f81c00fda80 2024-02-16T19:24:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.752+0000 7f81c590e700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f81ac0692a0 0x7f81ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.752+0000 7f81c590e700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f81ac0692a0 0x7f81ac06b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f81b4004f90 tx=0x7f81b4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:01.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.756+0000 7f81b2ffd700 1 -- 172.21.15.40:0/1930803321 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f81bc056c50 con 0x7f81c00fda80 2024-02-16T19:24:01.787 INFO:tasks.workunit.client.0.smithi040.stdout:5/5: symlink d1/l2 0 2024-02-16T19:24:01.787 INFO:tasks.workunit.client.0.smithi040.stdout:5/6: read - f0 zero size 2024-02-16T19:24:01.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.969+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81c0111c00 con 0x7f81ac0692a0 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.970+0000 7f81b2ffd700 1 -- 172.21.15.40:0/1930803321 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f81c0111c00 con 0x7f81ac0692a0 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [], 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "0/21 daemons upgraded", 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 image on host smithi163", 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:24:01.971 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:24:01.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f81ac0692a0 msgr2=0x7f81ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f81ac0692a0 0x7f81ac06b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f81b4004f90 tx=0x7f81b4009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:01.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c00fda80 msgr2=0x7f81c010fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:01.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c00fda80 0x7f81c010fb00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f81bc002a80 tx=0x7f81bc0046b0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 shutdown_connections 2024-02-16T19:24:01.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f81ac0692a0 0x7f81ac06b750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81c00fda80 0x7f81c010fb00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 --2- 172.21.15.40:0/1930803321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81c01003a0 0x7f81c0110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:01.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 >> 172.21.15.40:0/1930803321 conn(0x7f81c00f74e0 msgr2=0x7f81c00fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:01.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 shutdown_connections 2024-02-16T19:24:01.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:01.972+0000 7f81c8373700 1 -- 172.21.15.40:0/1930803321 wait complete. 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.076+0000 7f7c20b97700 1 -- 172.21.15.40:0/1906986830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fcef0 msgr2=0x7f7c1c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.076+0000 7f7c20b97700 1 --2- 172.21.15.40:0/1906986830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fcef0 0x7f7c1c0fd320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c004940 tx=0x7f7c0c009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.076+0000 7f7c20b97700 1 -- 172.21.15.40:0/1906986830 shutdown_connections 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.076+0000 7f7c20b97700 1 --2- 172.21.15.40:0/1906986830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c1c0fe0e0 0x7f7c1c0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.076+0000 7f7c20b97700 1 --2- 172.21.15.40:0/1906986830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fcef0 0x7f7c1c0fd320 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.076+0000 7f7c20b97700 1 -- 172.21.15.40:0/1906986830 >> 172.21.15.40:0/1906986830 conn(0x7f7c1c0f8620 msgr2=0x7f7c1c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c20b97700 1 -- 172.21.15.40:0/1906986830 shutdown_connections 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c20b97700 1 -- 172.21.15.40:0/1906986830 wait complete. 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c20b97700 1 Processor -- start 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c20b97700 1 -- start start 2024-02-16T19:24:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c20b97700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c1c0fcef0 0x7f7c1c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c20b97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fe0e0 0x7f7c1c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c20b97700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c10e4b0 con 0x7f7c1c0fe0e0 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c20b97700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c10ed50 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c19d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fe0e0 0x7f7c1c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c19d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fe0e0 0x7f7c1c10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53360/0 (socket says 172.21.15.40:53360) 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c19d9b700 1 -- 172.21.15.40:0/699668320 learned_addr learned my addr 172.21.15.40:0/699668320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.077+0000 7f7c1a59c700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c1c0fcef0 0x7f7c1c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c1a59c700 1 -- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fe0e0 msgr2=0x7f7c1c10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c1a59c700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fe0e0 0x7f7c1c10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c1a59c700 1 -- 172.21.15.40:0/699668320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c005040 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c19d9b700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fe0e0 0x7f7c1c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:24:02.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c1a59c700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c1c0fcef0 0x7f7c1c10d970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009fd0 tx=0x7f7c0c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c0affd700 1 -- 172.21.15.40:0/699668320 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c0c00e040 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c0affd700 1 -- 172.21.15.40:0/699668320 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c0c00bbf0 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1c10efd0 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c0affd700 1 -- 172.21.15.40:0/699668320 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c0c013660 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.078+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c10f510 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.079+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1c04bed0 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.080+0000 7f7c0affd700 1 -- 172.21.15.40:0/699668320 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7c0c00bd60 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.080+0000 7f7c0affd700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c040692a0 0x7f7c0406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.080+0000 7f7c0affd700 1 -- 172.21.15.40:0/699668320 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7c0c087a40 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.080+0000 7f7c19d9b700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c040692a0 0x7f7c0406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.080+0000 7f7c19d9b700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c040692a0 0x7f7c0406b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c100041d0 tx=0x7f7c1000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.084+0000 7f7c0affd700 1 -- 172.21.15.40:0/699668320 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7c0c057790 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.375+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7c1c105a30 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.375+0000 7f7c0affd700 1 -- 172.21.15.40:0/699668320 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7c0c021090 con 0x7f7c1c0fcef0 2024-02-16T19:24:02.376 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:24:02.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c040692a0 msgr2=0x7f7c0406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:02.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c040692a0 0x7f7c0406b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c100041d0 tx=0x7f7c1000b040 comp rx=0 tx=0).stop 2024-02-16T19:24:02.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c1c0fcef0 msgr2=0x7f7c1c10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:02.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c1c0fcef0 0x7f7c1c10d970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009fd0 tx=0x7f7c0c009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:02.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 shutdown_connections 2024-02-16T19:24:02.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7c040692a0 0x7f7c0406b750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:02.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c1c0fcef0 0x7f7c1c10d970 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:02.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 --2- 172.21.15.40:0/699668320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c1c0fe0e0 0x7f7c1c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:02.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 >> 172.21.15.40:0/699668320 conn(0x7f7c1c0f8620 msgr2=0x7f7c1c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:02.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 shutdown_connections 2024-02-16T19:24:02.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:02.377+0000 7f7c20b97700 1 -- 172.21.15.40:0/699668320 wait complete. 2024-02-16T19:24:02.530 INFO:tasks.workunit.client.0.smithi040.stdout:4/13: mknod c5 0 2024-02-16T19:24:02.530 INFO:tasks.workunit.client.0.smithi040.stdout:4/14: creat f6 x:0 0 0 2024-02-16T19:24:02.531 INFO:tasks.workunit.client.0.smithi040.stdout:1/2: rename f0 to f1 0 2024-02-16T19:24:02.531 INFO:tasks.workunit.client.0.smithi040.stdout:1/3: write f1 [4395743,21625] 0 2024-02-16T19:24:02.532 INFO:tasks.workunit.client.0.smithi040.stdout:1/4: creat f2 x:0 0 0 2024-02-16T19:24:02.532 INFO:tasks.workunit.client.0.smithi040.stdout:1/5: creat f3 x:0 0 0 2024-02-16T19:24:02.532 INFO:tasks.workunit.client.0.smithi040.stdout:5/7: creat d1/f3 x:0 0 0 2024-02-16T19:24:02.534 INFO:tasks.workunit.client.0.smithi040.stdout:6/10: getdents d0 0 2024-02-16T19:24:02.535 INFO:tasks.workunit.client.0.smithi040.stdout:6/11: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:02.550 INFO:tasks.workunit.client.0.smithi040.stdout:1/6: dwrite f2 [0,4194304] 0 2024-02-16T19:24:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:02 smithi040 ceph-mon[29201]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:02 smithi040 ceph-mon[29201]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:02 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3247551200' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:24:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:02 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/699668320' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:24:02.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:02 smithi163 ceph-mon[36452]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:02.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:02 smithi163 ceph-mon[36452]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:02.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:02 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3247551200' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:24:02.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:02 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/699668320' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:24:03.139 INFO:tasks.workunit.client.0.smithi040.stdout:4/15: rename f2 to f7 0 2024-02-16T19:24:03.143 INFO:tasks.workunit.client.0.smithi040.stdout:6/12: mknod d0/d1/c3 0 2024-02-16T19:24:03.143 INFO:tasks.workunit.client.0.smithi040.stdout:6/13: readlink - no filename 2024-02-16T19:24:03.147 INFO:tasks.workunit.client.0.smithi040.stdout:6/14: creat d0/f4 x:0 0 0 2024-02-16T19:24:03.147 INFO:tasks.workunit.client.0.smithi040.stdout:6/15: write d0/f4 [902532,75910] 0 2024-02-16T19:24:03.148 INFO:tasks.workunit.client.0.smithi040.stdout:1/7: rename f2 to f4 0 2024-02-16T19:24:03.148 INFO:tasks.workunit.client.0.smithi040.stdout:1/8: dread - f3 zero size 2024-02-16T19:24:03.148 INFO:tasks.workunit.client.0.smithi040.stdout:1/9: write f3 [569299,118190] 0 2024-02-16T19:24:03.148 INFO:tasks.workunit.client.0.smithi040.stdout:1/10: creat f5 x:0 0 0 2024-02-16T19:24:03.148 INFO:tasks.workunit.client.0.smithi040.stdout:1/11: creat f6 x:0 0 0 2024-02-16T19:24:03.148 INFO:tasks.workunit.client.0.smithi040.stdout:1/12: creat f7 x:0 0 0 2024-02-16T19:24:03.149 INFO:tasks.workunit.client.0.smithi040.stdout:6/16: unlink d0/f2 0 2024-02-16T19:24:03.150 INFO:tasks.workunit.client.0.smithi040.stdout:6/17: read d0/f4 [310477,14948] 0 2024-02-16T19:24:03.150 INFO:tasks.workunit.client.0.smithi040.stdout:6/18: creat d0/f5 x:0 0 0 2024-02-16T19:24:03.150 INFO:tasks.workunit.client.0.smithi040.stdout:6/19: chown d0/d1 6815394 1 2024-02-16T19:24:03.150 INFO:tasks.workunit.client.0.smithi040.stdout:6/20: dread - d0/f5 zero size 2024-02-16T19:24:03.156 INFO:tasks.workunit.client.0.smithi040.stdout:1/13: write f4 [2469111,32640] 0 2024-02-16T19:24:03.157 INFO:tasks.workunit.client.0.smithi040.stdout:1/14: rmdir - no directory 2024-02-16T19:24:03.157 INFO:tasks.workunit.client.0.smithi040.stdout:6/21: dread d0/f4 [0,4194304] 0 2024-02-16T19:24:03.157 INFO:tasks.workunit.client.0.smithi040.stdout:1/15: rename f6 to f8 0 2024-02-16T19:24:03.158 INFO:tasks.workunit.client.0.smithi040.stdout:6/22: dread d0/f4 [0,4194304] 0 2024-02-16T19:24:03.563 INFO:tasks.workunit.client.0.smithi040.stdout:4/16: getdents . 0 2024-02-16T19:24:03.563 INFO:tasks.workunit.client.0.smithi040.stdout:4/17: chown f0 7 1 2024-02-16T19:24:03.563 INFO:tasks.workunit.client.0.smithi040.stdout:4/18: write f6 [79422,24251] 0 2024-02-16T19:24:03.563 INFO:tasks.workunit.client.0.smithi040.stdout:4/19: rmdir - no directory 2024-02-16T19:24:03.563 INFO:tasks.workunit.client.0.smithi040.stdout:4/20: getdents . 0 2024-02-16T19:24:03.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/21: creat f8 x:0 0 0 2024-02-16T19:24:03.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/22: creat f9 x:0 0 0 2024-02-16T19:24:03.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/23: creat fa x:0 0 0 2024-02-16T19:24:03.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/24: write f4 [60022,46509] 0 2024-02-16T19:24:03.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/25: creat fb x:0 0 0 2024-02-16T19:24:03.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/26: truncate fa 541602 0 2024-02-16T19:24:03.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/27: truncate f0 1014858 0 2024-02-16T19:24:03.567 INFO:tasks.workunit.client.0.smithi040.stdout:1/16: rename f4 to f9 0 2024-02-16T19:24:03.568 INFO:tasks.workunit.client.0.smithi040.stdout:1/17: stat f7 0 2024-02-16T19:24:03.568 INFO:tasks.workunit.client.0.smithi040.stdout:1/18: creat fa x:0 0 0 2024-02-16T19:24:03.568 INFO:tasks.workunit.client.0.smithi040.stdout:1/19: chown f7 1034 1 2024-02-16T19:24:03.568 INFO:tasks.workunit.client.0.smithi040.stdout:1/20: rmdir - no directory 2024-02-16T19:24:03.568 INFO:tasks.workunit.client.0.smithi040.stdout:4/28: mknod cc 0 2024-02-16T19:24:03.570 INFO:tasks.workunit.client.0.smithi040.stdout:4/29: mkdir dd 0 2024-02-16T19:24:03.570 INFO:tasks.workunit.client.0.smithi040.stdout:4/30: fdatasync fb 0 2024-02-16T19:24:03.571 INFO:tasks.workunit.client.0.smithi040.stdout:1/21: link fa fb 0 2024-02-16T19:24:03.572 INFO:tasks.workunit.client.0.smithi040.stdout:1/22: write f7 [249045,59261] 0 2024-02-16T19:24:03.572 INFO:tasks.workunit.client.0.smithi040.stdout:1/23: chown f9 0 1 2024-02-16T19:24:03.572 INFO:tasks.workunit.client.0.smithi040.stdout:4/31: mkdir dd/de 0 2024-02-16T19:24:03.573 INFO:tasks.workunit.client.0.smithi040.stdout:1/24: mknod cc 0 2024-02-16T19:24:03.574 INFO:tasks.workunit.client.0.smithi040.stdout:1/25: rename f3 to fd 0 2024-02-16T19:24:03.574 INFO:tasks.workunit.client.0.smithi040.stdout:1/26: chown f7 5754981 1 2024-02-16T19:24:03.589 INFO:tasks.workunit.client.0.smithi040.stdout:4/32: dwrite f9 [0,4194304] 0 2024-02-16T19:24:03.589 INFO:tasks.workunit.client.0.smithi040.stdout:4/33: fsync f0 0 2024-02-16T19:24:03.596 INFO:tasks.workunit.client.0.smithi040.stdout:1/27: dwrite f5 [0,4194304] 0 2024-02-16T19:24:03.604 INFO:tasks.workunit.client.0.smithi040.stdout:1/28: symlink le 0 2024-02-16T19:24:03.610 INFO:tasks.workunit.client.0.smithi040.stdout:4/34: dwrite f6 [0,4194304] 0 2024-02-16T19:24:03.611 INFO:tasks.workunit.client.0.smithi040.stdout:4/35: mkdir dd/df 0 2024-02-16T19:24:03.613 INFO:tasks.workunit.client.0.smithi040.stdout:4/36: rmdir dd/df 0 2024-02-16T19:24:03.613 INFO:tasks.workunit.client.0.smithi040.stdout:4/37: mkdir dd/de/d10 0 2024-02-16T19:24:03.619 INFO:tasks.workunit.client.0.smithi040.stdout:1/29: getdents . 0 2024-02-16T19:24:03.619 INFO:tasks.workunit.client.0.smithi040.stdout:1/30: write f8 [311421,80136] 0 2024-02-16T19:24:03.620 INFO:tasks.workunit.client.0.smithi040.stdout:1/31: mknod cf 0 2024-02-16T19:24:03.620 INFO:tasks.workunit.client.0.smithi040.stdout:1/32: fsync fa 0 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:0/5: sync 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:9/10: sync 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:2/30: sync 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:7/6: sync 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:7/7: write - no filename 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:3/16: sync 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:8/2: sync 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:3/17: chown d1 43534169 1 2024-02-16T19:24:03.630 INFO:tasks.workunit.client.0.smithi040.stdout:6/23: sync 2024-02-16T19:24:03.631 INFO:tasks.workunit.client.0.smithi040.stdout:0/6: truncate f1 228678 0 2024-02-16T19:24:03.631 INFO:tasks.workunit.client.0.smithi040.stdout:3/18: dread - d1/f2 zero size 2024-02-16T19:24:03.631 INFO:tasks.workunit.client.0.smithi040.stdout:5/8: sync 2024-02-16T19:24:03.631 INFO:tasks.workunit.client.0.smithi040.stdout:8/3: unlink f0 0 2024-02-16T19:24:03.632 INFO:tasks.workunit.client.0.smithi040.stdout:9/11: truncate f0 3914513 0 2024-02-16T19:24:03.632 INFO:tasks.workunit.client.0.smithi040.stdout:9/12: chown f0 35 1 2024-02-16T19:24:03.633 INFO:tasks.workunit.client.0.smithi040.stdout:2/31: mkdir d5/d8 0 2024-02-16T19:24:03.633 INFO:tasks.workunit.client.0.smithi040.stdout:2/32: write - no filename 2024-02-16T19:24:03.633 INFO:tasks.workunit.client.0.smithi040.stdout:2/33: truncate - no filename 2024-02-16T19:24:03.633 INFO:tasks.workunit.client.0.smithi040.stdout:2/34: write - no filename 2024-02-16T19:24:03.633 INFO:tasks.workunit.client.0.smithi040.stdout:2/35: readlink l3 0 2024-02-16T19:24:03.633 INFO:tasks.workunit.client.0.smithi040.stdout:2/36: truncate - no filename 2024-02-16T19:24:03.634 INFO:tasks.workunit.client.0.smithi040.stdout:0/7: unlink l0 0 2024-02-16T19:24:03.634 INFO:tasks.workunit.client.0.smithi040.stdout:0/8: creat f2 x:0 0 0 2024-02-16T19:24:03.634 INFO:tasks.workunit.client.0.smithi040.stdout:6/24: symlink d0/d1/l6 0 2024-02-16T19:24:03.636 INFO:tasks.workunit.client.0.smithi040.stdout:5/9: unlink d1/f3 0 2024-02-16T19:24:03.636 INFO:tasks.workunit.client.0.smithi040.stdout:5/10: creat d1/f4 x:0 0 0 2024-02-16T19:24:03.636 INFO:tasks.workunit.client.0.smithi040.stdout:7/8: getdents . 0 2024-02-16T19:24:03.636 INFO:tasks.workunit.client.0.smithi040.stdout:7/9: fdatasync - no filename 2024-02-16T19:24:03.637 INFO:tasks.workunit.client.0.smithi040.stdout:8/4: mknod c1 0 2024-02-16T19:24:03.637 INFO:tasks.workunit.client.0.smithi040.stdout:8/5: creat f2 x:0 0 0 2024-02-16T19:24:03.638 INFO:tasks.workunit.client.0.smithi040.stdout:2/37: mknod d5/d8/c9 0 2024-02-16T19:24:03.639 INFO:tasks.workunit.client.0.smithi040.stdout:4/38: dwrite fb [0,4194304] 0 2024-02-16T19:24:03.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/33: dwrite f5 [0,4194304] 0 2024-02-16T19:24:03.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/34: fsync f5 0 2024-02-16T19:24:03.644 INFO:tasks.workunit.client.0.smithi040.stdout:7/10: creat f1 x:0 0 0 2024-02-16T19:24:03.649 INFO:tasks.workunit.client.0.smithi040.stdout:2/38: creat d5/d7/fa x:0 0 0 2024-02-16T19:24:03.653 INFO:tasks.workunit.client.0.smithi040.stdout:3/19: dwrite d1/f2 [0,4194304] 0 2024-02-16T19:24:03.653 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:03 smithi163 ceph-mon[36452]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:03.653 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:03 smithi163 ceph-mon[36452]: pgmap v240: 65 pgs: 65 active+clean; 271 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 937 op/s 2024-02-16T19:24:03.654 INFO:tasks.workunit.client.0.smithi040.stdout:4/39: rename f6 to dd/f11 0 2024-02-16T19:24:03.657 INFO:tasks.workunit.client.0.smithi040.stdout:1/35: symlink l10 0 2024-02-16T19:24:03.657 INFO:tasks.workunit.client.0.smithi040.stdout:1/36: write f7 [1235533,103921] 0 2024-02-16T19:24:03.657 INFO:tasks.workunit.client.0.smithi040.stdout:0/9: link f2 f3 0 2024-02-16T19:24:03.660 INFO:tasks.workunit.client.0.smithi040.stdout:8/6: dwrite f2 [0,4194304] 0 2024-02-16T19:24:03.660 INFO:tasks.workunit.client.0.smithi040.stdout:7/11: creat f2 x:0 0 0 2024-02-16T19:24:03.661 INFO:tasks.workunit.client.0.smithi040.stdout:6/25: dwrite d0/f4 [0,4194304] 0 2024-02-16T19:24:03.661 INFO:tasks.workunit.client.0.smithi040.stdout:6/26: readlink d0/d1/l6 0 2024-02-16T19:24:03.662 INFO:tasks.workunit.client.0.smithi040.stdout:2/39: symlink d5/d8/lb 0 2024-02-16T19:24:03.668 INFO:tasks.workunit.client.0.smithi040.stdout:3/20: rename d1/f2 to d1/f3 0 2024-02-16T19:24:03.668 INFO:tasks.workunit.client.0.smithi040.stdout:5/11: rmdir d1 39 2024-02-16T19:24:03.668 INFO:tasks.workunit.client.0.smithi040.stdout:5/12: dread - f0 zero size 2024-02-16T19:24:03.668 INFO:tasks.workunit.client.0.smithi040.stdout:5/13: chown d1 1 1 2024-02-16T19:24:03.668 INFO:tasks.workunit.client.0.smithi040.stdout:5/14: write f0 [230031,83919] 0 2024-02-16T19:24:03.668 INFO:tasks.workunit.client.0.smithi040.stdout:5/15: readlink d1/l2 0 2024-02-16T19:24:03.672 INFO:tasks.workunit.client.0.smithi040.stdout:1/37: unlink fa 0 2024-02-16T19:24:03.672 INFO:tasks.workunit.client.0.smithi040.stdout:9/13: dread f0 [0,4194304] 0 2024-02-16T19:24:03.675 INFO:tasks.workunit.client.0.smithi040.stdout:3/21: dread d1/f3 [0,4194304] 0 2024-02-16T19:24:03.676 INFO:tasks.workunit.client.0.smithi040.stdout:3/22: chown d1/f3 1765585 1 2024-02-16T19:24:03.676 INFO:tasks.workunit.client.0.smithi040.stdout:3/23: chown d1 26179737 1 2024-02-16T19:24:03.678 INFO:tasks.workunit.client.0.smithi040.stdout:4/40: link dd/f11 dd/de/f12 0 2024-02-16T19:24:03.681 INFO:tasks.workunit.client.0.smithi040.stdout:9/14: write f0 [1050267,67940] 0 2024-02-16T19:24:03.681 INFO:tasks.workunit.client.0.smithi040.stdout:0/10: dwrite f2 [0,4194304] 0 2024-02-16T19:24:03.681 INFO:tasks.workunit.client.0.smithi040.stdout:9/15: write f0 [1252749,76436] 0 2024-02-16T19:24:03.681 INFO:tasks.workunit.client.0.smithi040.stdout:0/11: fdatasync f3 0 2024-02-16T19:24:03.682 INFO:tasks.workunit.client.0.smithi040.stdout:2/40: link c0 d5/d8/cc 0 2024-02-16T19:24:03.684 INFO:tasks.workunit.client.0.smithi040.stdout:5/16: mknod d1/c5 0 2024-02-16T19:24:03.685 INFO:tasks.workunit.client.0.smithi040.stdout:1/38: truncate f1 597379 0 2024-02-16T19:24:03.687 INFO:tasks.workunit.client.0.smithi040.stdout:3/24: mkdir d1/d4 0 2024-02-16T19:24:03.688 INFO:tasks.workunit.client.0.smithi040.stdout:6/27: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:24:03.690 INFO:tasks.workunit.client.0.smithi040.stdout:1/39: dread f8 [0,4194304] 0 2024-02-16T19:24:03.695 INFO:tasks.workunit.client.0.smithi040.stdout:8/7: write f2 [4388471,113588] 0 2024-02-16T19:24:03.695 INFO:tasks.workunit.client.0.smithi040.stdout:8/8: creat f3 x:0 0 0 2024-02-16T19:24:03.696 INFO:tasks.workunit.client.0.smithi040.stdout:0/12: mknod c4 0 2024-02-16T19:24:03.699 INFO:tasks.workunit.client.0.smithi040.stdout:2/41: mkdir d5/d7/dd 0 2024-02-16T19:24:03.700 INFO:tasks.workunit.client.0.smithi040.stdout:5/17: mknod d1/c6 0 2024-02-16T19:24:03.700 INFO:tasks.workunit.client.0.smithi040.stdout:5/18: dread - d1/f4 zero size 2024-02-16T19:24:03.700 INFO:tasks.workunit.client.0.smithi040.stdout:5/19: readlink d1/l2 0 2024-02-16T19:24:03.701 INFO:tasks.workunit.client.0.smithi040.stdout:3/25: mknod d1/c5 0 2024-02-16T19:24:03.703 INFO:tasks.workunit.client.0.smithi040.stdout:7/12: link f2 f3 0 2024-02-16T19:24:03.703 INFO:tasks.workunit.client.0.smithi040.stdout:7/13: fdatasync f3 0 2024-02-16T19:24:03.706 INFO:tasks.workunit.client.0.smithi040.stdout:4/41: write f7 [282622,39338] 0 2024-02-16T19:24:03.706 INFO:tasks.workunit.client.0.smithi040.stdout:4/42: chown f9 0 1 2024-02-16T19:24:03.707 INFO:tasks.workunit.client.0.smithi040.stdout:3/26: write d1/f3 [228859,66587] 0 2024-02-16T19:24:03.707 INFO:tasks.workunit.client.0.smithi040.stdout:3/27: creat d1/f6 x:0 0 0 2024-02-16T19:24:03.707 INFO:tasks.workunit.client.0.smithi040.stdout:3/28: chown d1/c5 43609828 1 2024-02-16T19:24:03.714 INFO:tasks.workunit.client.0.smithi040.stdout:9/16: unlink f0 0 2024-02-16T19:24:03.717 INFO:tasks.workunit.client.0.smithi040.stdout:0/13: mknod c5 0 2024-02-16T19:24:03.717 INFO:tasks.workunit.client.0.smithi040.stdout:0/14: creat f6 x:0 0 0 2024-02-16T19:24:03.718 INFO:tasks.workunit.client.0.smithi040.stdout:4/43: write dd/f11 [3579935,83896] 0 2024-02-16T19:24:03.719 INFO:tasks.workunit.client.0.smithi040.stdout:4/44: stat fb 0 2024-02-16T19:24:03.719 INFO:tasks.workunit.client.0.smithi040.stdout:4/45: chown fb 13012974 1 2024-02-16T19:24:03.721 INFO:tasks.workunit.client.0.smithi040.stdout:1/40: rename le to l11 0 2024-02-16T19:24:03.722 INFO:tasks.workunit.client.0.smithi040.stdout:2/42: mknod d5/d7/ce 0 2024-02-16T19:24:03.725 INFO:tasks.workunit.client.0.smithi040.stdout:3/29: mkdir d1/d7 0 2024-02-16T19:24:03.726 INFO:tasks.workunit.client.0.smithi040.stdout:9/17: rename l1 to l2 0 2024-02-16T19:24:03.726 INFO:tasks.workunit.client.0.smithi040.stdout:9/18: chown l2 830 1 2024-02-16T19:24:03.726 INFO:tasks.workunit.client.0.smithi040.stdout:9/19: write - no filename 2024-02-16T19:24:03.726 INFO:tasks.workunit.client.0.smithi040.stdout:9/20: dread - no filename 2024-02-16T19:24:03.726 INFO:tasks.workunit.client.0.smithi040.stdout:9/21: write - no filename 2024-02-16T19:24:03.727 INFO:tasks.workunit.client.0.smithi040.stdout:0/15: write f1 [448464,71000] 0 2024-02-16T19:24:03.728 INFO:tasks.workunit.client.0.smithi040.stdout:4/46: unlink l1 0 2024-02-16T19:24:03.730 INFO:tasks.workunit.client.0.smithi040.stdout:5/20: dwrite d1/f4 [0,4194304] 0 2024-02-16T19:24:03.730 INFO:tasks.workunit.client.0.smithi040.stdout:7/14: dwrite f3 [0,4194304] 0 2024-02-16T19:24:03.730 INFO:tasks.workunit.client.0.smithi040.stdout:7/15: write f1 [352256,12435] 0 2024-02-16T19:24:03.730 INFO:tasks.workunit.client.0.smithi040.stdout:4/47: write f0 [619935,123379] 0 2024-02-16T19:24:03.731 INFO:tasks.workunit.client.0.smithi040.stdout:8/9: dwrite f3 [0,4194304] 0 2024-02-16T19:24:03.732 INFO:tasks.workunit.client.0.smithi040.stdout:3/30: rename d1/c5 to d1/d4/c8 0 2024-02-16T19:24:03.734 INFO:tasks.workunit.client.0.smithi040.stdout:9/22: mkdir d3 0 2024-02-16T19:24:03.734 INFO:tasks.workunit.client.0.smithi040.stdout:9/23: dwrite - no filename 2024-02-16T19:24:03.736 INFO:tasks.workunit.client.0.smithi040.stdout:0/16: rename f1 to f7 0 2024-02-16T19:24:03.737 INFO:tasks.workunit.client.0.smithi040.stdout:0/17: chown f6 866 1 2024-02-16T19:24:03.737 INFO:tasks.workunit.client.0.smithi040.stdout:0/18: write f6 [520640,35516] 0 2024-02-16T19:24:03.741 INFO:tasks.workunit.client.0.smithi040.stdout:5/21: creat d1/f7 x:0 0 0 2024-02-16T19:24:03.741 INFO:tasks.workunit.client.0.smithi040.stdout:5/22: fdatasync d1/f7 0 2024-02-16T19:24:03.741 INFO:tasks.workunit.client.0.smithi040.stdout:5/23: readlink d1/l2 0 2024-02-16T19:24:03.744 INFO:tasks.workunit.client.0.smithi040.stdout:4/48: mkdir dd/de/d10/d13 0 2024-02-16T19:24:03.744 INFO:tasks.workunit.client.0.smithi040.stdout:8/10: dwrite f2 [0,4194304] 0 2024-02-16T19:24:03.744 INFO:tasks.workunit.client.0.smithi040.stdout:8/11: creat f4 x:0 0 0 2024-02-16T19:24:03.750 INFO:tasks.workunit.client.0.smithi040.stdout:1/41: creat f12 x:0 0 0 2024-02-16T19:24:03.753 INFO:tasks.workunit.client.0.smithi040.stdout:3/31: rmdir d1/d4 39 2024-02-16T19:24:03.756 INFO:tasks.workunit.client.0.smithi040.stdout:9/24: mkdir d3/d4 0 2024-02-16T19:24:03.756 INFO:tasks.workunit.client.0.smithi040.stdout:9/25: fsync - no filename 2024-02-16T19:24:03.756 INFO:tasks.workunit.client.0.smithi040.stdout:9/26: chown l2 49298 1 2024-02-16T19:24:03.757 INFO:tasks.workunit.client.0.smithi040.stdout:0/19: rename c4 to c8 0 2024-02-16T19:24:03.758 INFO:tasks.workunit.client.0.smithi040.stdout:0/20: creat f9 x:0 0 0 2024-02-16T19:24:03.759 INFO:tasks.workunit.client.0.smithi040.stdout:4/49: mknod dd/de/c14 0 2024-02-16T19:24:03.761 INFO:tasks.workunit.client.0.smithi040.stdout:3/32: dread d1/f3 [0,4194304] 0 2024-02-16T19:24:03.761 INFO:tasks.workunit.client.0.smithi040.stdout:3/33: dread - d1/f6 zero size 2024-02-16T19:24:03.765 INFO:tasks.workunit.client.0.smithi040.stdout:9/27: creat d3/d4/f5 x:0 0 0 2024-02-16T19:24:03.765 INFO:tasks.workunit.client.0.smithi040.stdout:9/28: truncate d3/d4/f5 448730 0 2024-02-16T19:24:03.765 INFO:tasks.workunit.client.0.smithi040.stdout:9/29: stat d3/d4 0 2024-02-16T19:24:03.765 INFO:tasks.workunit.client.0.smithi040.stdout:9/30: write d3/d4/f5 [847114,54808] 0 2024-02-16T19:24:03.767 INFO:tasks.workunit.client.0.smithi040.stdout:5/24: dwrite d1/f7 [0,4194304] 0 2024-02-16T19:24:03.767 INFO:tasks.workunit.client.0.smithi040.stdout:8/12: dwrite f4 [0,4194304] 0 2024-02-16T19:24:03.768 INFO:tasks.workunit.client.0.smithi040.stdout:0/21: mknod ca 0 2024-02-16T19:24:03.778 INFO:tasks.workunit.client.0.smithi040.stdout:9/31: dread d3/d4/f5 [0,4194304] 0 2024-02-16T19:24:03.779 INFO:tasks.workunit.client.0.smithi040.stdout:9/32: rename d3 to d3/d6 22 2024-02-16T19:24:03.779 INFO:tasks.workunit.client.0.smithi040.stdout:9/33: write d3/d4/f5 [884300,83030] 0 2024-02-16T19:24:03.786 INFO:tasks.workunit.client.0.smithi040.stdout:3/34: dwrite d1/f6 [0,4194304] 0 2024-02-16T19:24:03.786 INFO:tasks.workunit.client.0.smithi040.stdout:3/35: chown d1/f6 27168848 1 2024-02-16T19:24:03.786 INFO:tasks.workunit.client.0.smithi040.stdout:3/36: chown d1/d7 4981423 1 2024-02-16T19:24:03.786 INFO:tasks.workunit.client.0.smithi040.stdout:3/37: readlink - no filename 2024-02-16T19:24:03.786 INFO:tasks.workunit.client.0.smithi040.stdout:9/34: mknod d3/d4/c7 0 2024-02-16T19:24:03.788 INFO:tasks.workunit.client.0.smithi040.stdout:4/50: getdents dd/de 0 2024-02-16T19:24:03.792 INFO:tasks.workunit.client.0.smithi040.stdout:3/38: link d1/f6 d1/d7/f9 0 2024-02-16T19:24:03.794 INFO:tasks.workunit.client.0.smithi040.stdout:1/42: link cf c13 0 2024-02-16T19:24:03.794 INFO:tasks.workunit.client.0.smithi040.stdout:0/22: dread f3 [0,4194304] 0 2024-02-16T19:24:03.794 INFO:tasks.workunit.client.0.smithi040.stdout:0/23: fsync f6 0 2024-02-16T19:24:03.795 INFO:tasks.workunit.client.0.smithi040.stdout:4/51: creat dd/de/d10/d13/f15 x:0 0 0 2024-02-16T19:24:03.795 INFO:tasks.workunit.client.0.smithi040.stdout:4/52: truncate f0 1467353 0 2024-02-16T19:24:03.796 INFO:tasks.workunit.client.0.smithi040.stdout:3/39: creat d1/d4/fa x:0 0 0 2024-02-16T19:24:03.797 INFO:tasks.workunit.client.0.smithi040.stdout:1/43: fdatasync f7 0 2024-02-16T19:24:03.798 INFO:tasks.workunit.client.0.smithi040.stdout:3/40: mkdir d1/d4/db 0 2024-02-16T19:24:03.798 INFO:tasks.workunit.client.0.smithi040.stdout:3/41: write d1/d4/fa [559937,47672] 0 2024-02-16T19:24:03.800 INFO:tasks.workunit.client.0.smithi040.stdout:1/44: creat f14 x:0 0 0 2024-02-16T19:24:03.800 INFO:tasks.workunit.client.0.smithi040.stdout:0/24: dread f3 [0,4194304] 0 2024-02-16T19:24:03.800 INFO:tasks.workunit.client.0.smithi040.stdout:5/25: dwrite d1/f7 [0,4194304] 0 2024-02-16T19:24:03.800 INFO:tasks.workunit.client.0.smithi040.stdout:9/35: fdatasync d3/d4/f5 0 2024-02-16T19:24:03.800 INFO:tasks.workunit.client.0.smithi040.stdout:9/36: fsync d3/d4/f5 0 2024-02-16T19:24:03.801 INFO:tasks.workunit.client.0.smithi040.stdout:7/16: dwrite f3 [4194304,4194304] 0 2024-02-16T19:24:03.801 INFO:tasks.workunit.client.0.smithi040.stdout:8/13: dwrite f4 [0,4194304] 0 2024-02-16T19:24:03.801 INFO:tasks.workunit.client.0.smithi040.stdout:8/14: creat f5 x:0 0 0 2024-02-16T19:24:03.801 INFO:tasks.workunit.client.0.smithi040.stdout:8/15: chown f4 5408 1 2024-02-16T19:24:03.801 INFO:tasks.workunit.client.0.smithi040.stdout:1/45: unlink f7 0 2024-02-16T19:24:03.802 INFO:tasks.workunit.client.0.smithi040.stdout:7/17: dread f1 [0,4194304] 0 2024-02-16T19:24:03.803 INFO:tasks.workunit.client.0.smithi040.stdout:0/25: mkdir db 0 2024-02-16T19:24:03.803 INFO:tasks.workunit.client.0.smithi040.stdout:0/26: write f9 [456787,30404] 0 2024-02-16T19:24:03.807 INFO:tasks.workunit.client.0.smithi040.stdout:0/27: dread f6 [0,4194304] 0 2024-02-16T19:24:03.808 INFO:tasks.workunit.client.0.smithi040.stdout:0/28: dread f6 [0,4194304] 0 2024-02-16T19:24:03.808 INFO:tasks.workunit.client.0.smithi040.stdout:9/37: rename d3/d4/f5 to d3/d4/f8 0 2024-02-16T19:24:03.811 INFO:tasks.workunit.client.0.smithi040.stdout:9/38: write d3/d4/f8 [325691,125194] 0 2024-02-16T19:24:03.811 INFO:tasks.workunit.client.0.smithi040.stdout:4/53: dwrite f9 [0,4194304] 0 2024-02-16T19:24:03.811 INFO:tasks.workunit.client.0.smithi040.stdout:4/54: stat c5 0 2024-02-16T19:24:03.814 INFO:tasks.workunit.client.0.smithi040.stdout:8/16: mkdir d6 0 2024-02-16T19:24:03.818 INFO:tasks.workunit.client.0.smithi040.stdout:5/26: getdents d1 0 2024-02-16T19:24:03.819 INFO:tasks.workunit.client.0.smithi040.stdout:3/42: dwrite d1/d7/f9 [0,4194304] 0 2024-02-16T19:24:03.820 INFO:tasks.workunit.client.0.smithi040.stdout:0/29: link c8 db/cc 0 2024-02-16T19:24:03.822 INFO:tasks.workunit.client.0.smithi040.stdout:4/55: rename dd/de/c14 to dd/de/c16 0 2024-02-16T19:24:03.823 INFO:tasks.workunit.client.0.smithi040.stdout:8/17: mkdir d6/d7 0 2024-02-16T19:24:03.823 INFO:tasks.workunit.client.0.smithi040.stdout:8/18: fsync f4 0 2024-02-16T19:24:03.824 INFO:tasks.workunit.client.0.smithi040.stdout:7/18: mknod c4 0 2024-02-16T19:24:03.829 INFO:tasks.workunit.client.0.smithi040.stdout:9/39: getdents d3/d4 0 2024-02-16T19:24:03.832 INFO:tasks.workunit.client.0.smithi040.stdout:8/19: creat d6/f8 x:0 0 0 2024-02-16T19:24:03.833 INFO:tasks.workunit.client.0.smithi040.stdout:8/20: creat d6/f9 x:0 0 0 2024-02-16T19:24:03.833 INFO:tasks.workunit.client.0.smithi040.stdout:8/21: readlink - no filename 2024-02-16T19:24:03.834 INFO:tasks.workunit.client.0.smithi040.stdout:6/28: sync 2024-02-16T19:24:03.834 INFO:tasks.workunit.client.0.smithi040.stdout:6/29: fsync d0/f5 0 2024-02-16T19:24:03.837 INFO:tasks.workunit.client.0.smithi040.stdout:7/19: symlink l5 0 2024-02-16T19:24:03.837 INFO:tasks.workunit.client.0.smithi040.stdout:7/20: fsync f3 0 2024-02-16T19:24:03.839 INFO:tasks.workunit.client.0.smithi040.stdout:6/30: rename d0/d1/l6 to d0/d1/l7 0 2024-02-16T19:24:03.839 INFO:tasks.workunit.client.0.smithi040.stdout:6/31: chown d0/d1/l7 151463 1 2024-02-16T19:24:03.842 INFO:tasks.workunit.client.0.smithi040.stdout:6/32: mknod d0/d1/c8 0 2024-02-16T19:24:03.843 INFO:tasks.workunit.client.0.smithi040.stdout:4/56: link dd/de/c16 dd/de/d10/d13/c17 0 2024-02-16T19:24:03.843 INFO:tasks.workunit.client.0.smithi040.stdout:4/57: chown dd 3377293 1 2024-02-16T19:24:03.843 INFO:tasks.workunit.client.0.smithi040.stdout:4/58: write fa [896543,8491] 0 2024-02-16T19:24:03.843 INFO:tasks.workunit.client.0.smithi040.stdout:4/59: write dd/de/d10/d13/f15 [647493,29921] 0 2024-02-16T19:24:03.845 INFO:tasks.workunit.client.0.smithi040.stdout:4/60: truncate f7 451566 0 2024-02-16T19:24:03.848 INFO:tasks.workunit.client.0.smithi040.stdout:3/43: dread d1/d7/f9 [0,4194304] 0 2024-02-16T19:24:03.848 INFO:tasks.workunit.client.0.smithi040.stdout:2/43: sync 2024-02-16T19:24:03.849 INFO:tasks.workunit.client.0.smithi040.stdout:3/44: mkdir d1/dc 0 2024-02-16T19:24:03.855 INFO:tasks.workunit.client.0.smithi040.stdout:3/45: dread d1/d7/f9 [0,4194304] 0 2024-02-16T19:24:03.855 INFO:tasks.workunit.client.0.smithi040.stdout:6/33: dread d0/f4 [0,4194304] 0 2024-02-16T19:24:03.856 INFO:tasks.workunit.client.0.smithi040.stdout:1/46: dwrite fd [0,4194304] 0 2024-02-16T19:24:03.858 INFO:tasks.workunit.client.0.smithi040.stdout:0/30: dwrite f6 [0,4194304] 0 2024-02-16T19:24:03.864 INFO:tasks.workunit.client.0.smithi040.stdout:0/31: write f3 [3258914,86893] 0 2024-02-16T19:24:03.868 INFO:tasks.workunit.client.0.smithi040.stdout:8/22: dwrite f2 [0,4194304] 0 2024-02-16T19:24:03.870 INFO:tasks.workunit.client.0.smithi040.stdout:5/27: dwrite d1/f7 [4194304,4194304] 0 2024-02-16T19:24:03.873 INFO:tasks.workunit.client.0.smithi040.stdout:4/61: dwrite f4 [0,4194304] 0 2024-02-16T19:24:03.873 INFO:tasks.workunit.client.0.smithi040.stdout:4/62: read dd/de/f12 [2632576,68334] 0 2024-02-16T19:24:03.875 INFO:tasks.workunit.client.0.smithi040.stdout:7/21: dwrite f3 [4194304,4194304] 0 2024-02-16T19:24:03.876 INFO:tasks.workunit.client.0.smithi040.stdout:7/22: read f1 [222475,49241] 0 2024-02-16T19:24:03.879 INFO:tasks.workunit.client.0.smithi040.stdout:2/44: symlink d5/d7/dd/lf 0 2024-02-16T19:24:03.882 INFO:tasks.workunit.client.0.smithi040.stdout:9/40: dwrite d3/d4/f8 [0,4194304] 0 2024-02-16T19:24:03.882 INFO:tasks.workunit.client.0.smithi040.stdout:9/41: stat d3/d4/f8 0 2024-02-16T19:24:03.888 INFO:tasks.workunit.client.0.smithi040.stdout:8/23: dwrite f2 [0,4194304] 0 2024-02-16T19:24:03.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:03 smithi040 ceph-mon[29201]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:03.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:03 smithi040 ceph-mon[29201]: pgmap v240: 65 pgs: 65 active+clean; 271 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 937 op/s 2024-02-16T19:24:03.902 INFO:tasks.workunit.client.0.smithi040.stdout:6/34: creat d0/d1/f9 x:0 0 0 2024-02-16T19:24:03.902 INFO:tasks.workunit.client.0.smithi040.stdout:6/35: chown d0/d1/f9 27715 1 2024-02-16T19:24:03.903 INFO:tasks.workunit.client.0.smithi040.stdout:1/47: unlink c13 0 2024-02-16T19:24:03.903 INFO:tasks.workunit.client.0.smithi040.stdout:1/48: chown f14 62783650 1 2024-02-16T19:24:03.907 INFO:tasks.workunit.client.0.smithi040.stdout:3/46: dwrite d1/d4/fa [0,4194304] 0 2024-02-16T19:24:03.907 INFO:tasks.workunit.client.0.smithi040.stdout:3/47: chown d1 24 1 2024-02-16T19:24:03.913 INFO:tasks.workunit.client.0.smithi040.stdout:4/63: dwrite dd/de/d10/d13/f15 [0,4194304] 0 2024-02-16T19:24:03.913 INFO:tasks.workunit.client.0.smithi040.stdout:2/45: dwrite d5/d7/fa [0,4194304] 0 2024-02-16T19:24:03.914 INFO:tasks.workunit.client.0.smithi040.stdout:0/32: dwrite f2 [0,4194304] 0 2024-02-16T19:24:03.914 INFO:tasks.workunit.client.0.smithi040.stdout:0/33: truncate f7 1292821 0 2024-02-16T19:24:03.919 INFO:tasks.workunit.client.0.smithi040.stdout:3/48: write d1/d4/fa [2103006,4043] 0 2024-02-16T19:24:03.925 INFO:tasks.workunit.client.0.smithi040.stdout:2/46: dread d5/d7/fa [0,4194304] 0 2024-02-16T19:24:03.927 INFO:tasks.workunit.client.0.smithi040.stdout:3/49: dread d1/f6 [0,4194304] 0 2024-02-16T19:24:03.938 INFO:tasks.workunit.client.0.smithi040.stdout:5/28: truncate d1/f4 2897379 0 2024-02-16T19:24:03.943 INFO:tasks.workunit.client.0.smithi040.stdout:9/42: symlink d3/d4/l9 0 2024-02-16T19:24:03.943 INFO:tasks.workunit.client.0.smithi040.stdout:9/43: fsync d3/d4/f8 0 2024-02-16T19:24:03.943 INFO:tasks.workunit.client.0.smithi040.stdout:8/24: symlink d6/d7/la 0 2024-02-16T19:24:03.944 INFO:tasks.workunit.client.0.smithi040.stdout:8/25: truncate f2 5230238 0 2024-02-16T19:24:03.944 INFO:tasks.workunit.client.0.smithi040.stdout:8/26: write d6/f8 [838779,115765] 0 2024-02-16T19:24:03.944 INFO:tasks.workunit.client.0.smithi040.stdout:6/36: rename d0/d1/f9 to d0/d1/fa 0 2024-02-16T19:24:03.945 INFO:tasks.workunit.client.0.smithi040.stdout:1/49: stat l10 0 2024-02-16T19:24:03.949 INFO:tasks.workunit.client.0.smithi040.stdout:3/50: dwrite d1/d7/f9 [0,4194304] 0 2024-02-16T19:24:03.950 INFO:tasks.workunit.client.0.smithi040.stdout:4/64: rename c5 to dd/de/d10/c18 0 2024-02-16T19:24:03.954 INFO:tasks.workunit.client.0.smithi040.stdout:0/34: getdents db 0 2024-02-16T19:24:03.956 INFO:tasks.workunit.client.0.smithi040.stdout:8/27: dread f4 [0,4194304] 0 2024-02-16T19:24:03.956 INFO:tasks.workunit.client.0.smithi040.stdout:8/28: write f5 [556597,38975] 0 2024-02-16T19:24:03.963 INFO:tasks.workunit.client.0.smithi040.stdout:5/29: rmdir d1 39 2024-02-16T19:24:03.963 INFO:tasks.workunit.client.0.smithi040.stdout:5/30: readlink d1/l2 0 2024-02-16T19:24:03.963 INFO:tasks.workunit.client.0.smithi040.stdout:0/35: dread f2 [0,4194304] 0 2024-02-16T19:24:03.963 INFO:tasks.workunit.client.0.smithi040.stdout:0/36: fdatasync f6 0 2024-02-16T19:24:03.963 INFO:tasks.workunit.client.0.smithi040.stdout:7/23: mknod c6 0 2024-02-16T19:24:03.964 INFO:tasks.workunit.client.0.smithi040.stdout:9/44: symlink d3/d4/la 0 2024-02-16T19:24:03.965 INFO:tasks.workunit.client.0.smithi040.stdout:6/37: link d0/d1/l7 d0/lb 0 2024-02-16T19:24:03.965 INFO:tasks.workunit.client.0.smithi040.stdout:6/38: stat d0/d1/fa 0 2024-02-16T19:24:03.969 INFO:tasks.workunit.client.0.smithi040.stdout:6/39: write d0/f4 [1895944,71291] 0 2024-02-16T19:24:03.969 INFO:tasks.workunit.client.0.smithi040.stdout:6/40: write d0/d1/fa [155587,86897] 0 2024-02-16T19:24:03.971 INFO:tasks.workunit.client.0.smithi040.stdout:4/65: creat dd/f19 x:0 0 0 2024-02-16T19:24:03.971 INFO:tasks.workunit.client.0.smithi040.stdout:4/66: write f7 [1317515,82712] 0 2024-02-16T19:24:03.971 INFO:tasks.workunit.client.0.smithi040.stdout:7/24: write f3 [2427314,147] 0 2024-02-16T19:24:03.971 INFO:tasks.workunit.client.0.smithi040.stdout:7/25: stat f1 0 2024-02-16T19:24:03.973 INFO:tasks.workunit.client.0.smithi040.stdout:8/29: symlink d6/lb 0 2024-02-16T19:24:03.973 INFO:tasks.workunit.client.0.smithi040.stdout:8/30: chown d6/lb 414475160 1 2024-02-16T19:24:03.974 INFO:tasks.workunit.client.0.smithi040.stdout:5/31: unlink d1/c5 0 2024-02-16T19:24:03.975 INFO:tasks.workunit.client.0.smithi040.stdout:0/37: creat db/fd x:0 0 0 2024-02-16T19:24:03.979 INFO:tasks.workunit.client.0.smithi040.stdout:6/41: dread d0/f4 [0,4194304] 0 2024-02-16T19:24:03.979 INFO:tasks.workunit.client.0.smithi040.stdout:1/50: getdents . 0 2024-02-16T19:24:03.979 INFO:tasks.workunit.client.0.smithi040.stdout:7/26: dread f3 [0,4194304] 0 2024-02-16T19:24:03.979 INFO:tasks.workunit.client.0.smithi040.stdout:7/27: fdatasync f3 0 2024-02-16T19:24:03.982 INFO:tasks.workunit.client.0.smithi040.stdout:8/31: symlink d6/lc 0 2024-02-16T19:24:03.982 INFO:tasks.workunit.client.0.smithi040.stdout:4/67: creat dd/de/f1a x:0 0 0 2024-02-16T19:24:03.984 INFO:tasks.workunit.client.0.smithi040.stdout:9/45: dwrite d3/d4/f8 [0,4194304] 0 2024-02-16T19:24:03.984 INFO:tasks.workunit.client.0.smithi040.stdout:6/42: mknod d0/d1/cc 0 2024-02-16T19:24:03.985 INFO:tasks.workunit.client.0.smithi040.stdout:7/28: creat f7 x:0 0 0 2024-02-16T19:24:03.985 INFO:tasks.workunit.client.0.smithi040.stdout:7/29: chown f2 72685 1 2024-02-16T19:24:03.986 INFO:tasks.workunit.client.0.smithi040.stdout:7/30: dread - f7 zero size 2024-02-16T19:24:03.988 INFO:tasks.workunit.client.0.smithi040.stdout:8/32: symlink d6/d7/ld 0 2024-02-16T19:24:04.000 INFO:tasks.workunit.client.0.smithi040.stdout:6/43: symlink d0/d1/ld 0 2024-02-16T19:24:04.000 INFO:tasks.workunit.client.0.smithi040.stdout:6/44: creat d0/fe x:0 0 0 2024-02-16T19:24:04.001 INFO:tasks.workunit.client.0.smithi040.stdout:8/33: dread f2 [0,4194304] 0 2024-02-16T19:24:04.001 INFO:tasks.workunit.client.0.smithi040.stdout:0/38: dwrite f7 [0,4194304] 0 2024-02-16T19:24:04.001 INFO:tasks.workunit.client.0.smithi040.stdout:0/39: write db/fd [759503,64804] 0 2024-02-16T19:24:04.001 INFO:tasks.workunit.client.0.smithi040.stdout:7/31: creat f8 x:0 0 0 2024-02-16T19:24:04.001 INFO:tasks.workunit.client.0.smithi040.stdout:7/32: fdatasync f3 0 2024-02-16T19:24:04.003 INFO:tasks.workunit.client.0.smithi040.stdout:0/40: dread f9 [0,4194304] 0 2024-02-16T19:24:04.007 INFO:tasks.workunit.client.0.smithi040.stdout:8/34: symlink d6/d7/le 0 2024-02-16T19:24:04.010 INFO:tasks.workunit.client.0.smithi040.stdout:1/51: dwrite fd [0,4194304] 0 2024-02-16T19:24:04.010 INFO:tasks.workunit.client.0.smithi040.stdout:1/52: getdents . 0 2024-02-16T19:24:04.010 INFO:tasks.workunit.client.0.smithi040.stdout:0/41: mkdir db/de 0 2024-02-16T19:24:04.011 INFO:tasks.workunit.client.0.smithi040.stdout:8/35: rmdir d6/d7 39 2024-02-16T19:24:04.013 INFO:tasks.workunit.client.0.smithi040.stdout:7/33: rename f8 to f9 0 2024-02-16T19:24:04.013 INFO:tasks.workunit.client.0.smithi040.stdout:7/34: dread - f9 zero size 2024-02-16T19:24:04.013 INFO:tasks.workunit.client.0.smithi040.stdout:7/35: truncate f7 448776 0 2024-02-16T19:24:04.015 INFO:tasks.workunit.client.0.smithi040.stdout:1/53: mknod c15 0 2024-02-16T19:24:04.015 INFO:tasks.workunit.client.0.smithi040.stdout:1/54: dread - f12 zero size 2024-02-16T19:24:04.017 INFO:tasks.workunit.client.0.smithi040.stdout:0/42: dread f6 [0,4194304] 0 2024-02-16T19:24:04.020 INFO:tasks.workunit.client.0.smithi040.stdout:6/45: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:24:04.020 INFO:tasks.workunit.client.0.smithi040.stdout:6/46: creat d0/ff x:0 0 0 2024-02-16T19:24:04.020 INFO:tasks.workunit.client.0.smithi040.stdout:6/47: fsync d0/f4 0 2024-02-16T19:24:04.024 INFO:tasks.workunit.client.0.smithi040.stdout:6/48: mknod d0/d1/c10 0 2024-02-16T19:24:04.024 INFO:tasks.workunit.client.0.smithi040.stdout:6/49: creat d0/d1/f11 x:0 0 0 2024-02-16T19:24:04.024 INFO:tasks.workunit.client.0.smithi040.stdout:9/46: dwrite d3/d4/f8 [0,4194304] 0 2024-02-16T19:24:04.028 INFO:tasks.workunit.client.0.smithi040.stdout:6/50: mkdir d0/d1/d12 0 2024-02-16T19:24:04.028 INFO:tasks.workunit.client.0.smithi040.stdout:6/51: truncate d0/fe 999141 0 2024-02-16T19:24:04.028 INFO:tasks.workunit.client.0.smithi040.stdout:3/51: truncate d1/d7/f9 1449254 0 2024-02-16T19:24:04.029 INFO:tasks.workunit.client.0.smithi040.stdout:3/52: fdatasync d1/d7/f9 0 2024-02-16T19:24:04.029 INFO:tasks.workunit.client.0.smithi040.stdout:6/52: symlink d0/l13 0 2024-02-16T19:24:04.034 INFO:tasks.workunit.client.0.smithi040.stdout:5/32: getdents d1 0 2024-02-16T19:24:04.036 INFO:tasks.workunit.client.0.smithi040.stdout:5/33: mknod d1/c8 0 2024-02-16T19:24:04.037 INFO:tasks.workunit.client.0.smithi040.stdout:8/36: dwrite f3 [0,4194304] 0 2024-02-16T19:24:04.040 INFO:tasks.workunit.client.0.smithi040.stdout:8/37: rename d6/d7/ld to d6/lf 0 2024-02-16T19:24:04.040 INFO:tasks.workunit.client.0.smithi040.stdout:8/38: fsync f3 0 2024-02-16T19:24:04.040 INFO:tasks.workunit.client.0.smithi040.stdout:4/68: rmdir dd/de 39 2024-02-16T19:24:04.042 INFO:tasks.workunit.client.0.smithi040.stdout:5/34: link d1/c6 d1/c9 0 2024-02-16T19:24:04.042 INFO:tasks.workunit.client.0.smithi040.stdout:0/43: dwrite f9 [0,4194304] 0 2024-02-16T19:24:04.044 INFO:tasks.workunit.client.0.smithi040.stdout:4/69: creat dd/de/d10/f1b x:0 0 0 2024-02-16T19:24:04.044 INFO:tasks.workunit.client.0.smithi040.stdout:8/39: write f4 [3041120,18005] 0 2024-02-16T19:24:04.045 INFO:tasks.workunit.client.0.smithi040.stdout:9/47: dwrite d3/d4/f8 [0,4194304] 0 2024-02-16T19:24:04.045 INFO:tasks.workunit.client.0.smithi040.stdout:9/48: chown d3/d4/la 18617794 1 2024-02-16T19:24:04.047 INFO:tasks.workunit.client.0.smithi040.stdout:5/35: dread f0 [0,4194304] 0 2024-02-16T19:24:04.047 INFO:tasks.workunit.client.0.smithi040.stdout:9/49: mknod d3/cb 0 2024-02-16T19:24:04.047 INFO:tasks.workunit.client.0.smithi040.stdout:9/50: readlink d3/d4/l9 0 2024-02-16T19:24:04.047 INFO:tasks.workunit.client.0.smithi040.stdout:9/51: write d3/d4/f8 [5127048,89249] 0 2024-02-16T19:24:04.050 INFO:tasks.workunit.client.0.smithi040.stdout:5/36: mkdir d1/da 0 2024-02-16T19:24:04.052 INFO:tasks.workunit.client.0.smithi040.stdout:0/44: dread f9 [0,4194304] 0 2024-02-16T19:24:04.053 INFO:tasks.workunit.client.0.smithi040.stdout:0/45: chown c5 1112080310 1 2024-02-16T19:24:04.053 INFO:tasks.workunit.client.0.smithi040.stdout:0/46: write db/fd [10812,81884] 0 2024-02-16T19:24:04.053 INFO:tasks.workunit.client.0.smithi040.stdout:0/47: readlink - no filename 2024-02-16T19:24:04.053 INFO:tasks.workunit.client.0.smithi040.stdout:0/48: creat db/ff x:0 0 0 2024-02-16T19:24:04.055 INFO:tasks.workunit.client.0.smithi040.stdout:0/49: mkdir db/de/d10 0 2024-02-16T19:24:04.056 INFO:tasks.workunit.client.0.smithi040.stdout:7/36: getdents . 0 2024-02-16T19:24:04.057 INFO:tasks.workunit.client.0.smithi040.stdout:0/50: link f7 db/f11 0 2024-02-16T19:24:04.058 INFO:tasks.workunit.client.0.smithi040.stdout:4/70: dwrite dd/de/d10/f1b [0,4194304] 0 2024-02-16T19:24:04.058 INFO:tasks.workunit.client.0.smithi040.stdout:7/37: creat fa x:0 0 0 2024-02-16T19:24:04.059 INFO:tasks.workunit.client.0.smithi040.stdout:1/55: write f12 [825162,47422] 0 2024-02-16T19:24:04.060 INFO:tasks.workunit.client.0.smithi040.stdout:5/37: write d1/f7 [1644515,96009] 0 2024-02-16T19:24:04.060 INFO:tasks.workunit.client.0.smithi040.stdout:7/38: creat fb x:0 0 0 2024-02-16T19:24:04.062 INFO:tasks.workunit.client.0.smithi040.stdout:4/71: dread dd/f11 [0,4194304] 0 2024-02-16T19:24:04.062 INFO:tasks.workunit.client.0.smithi040.stdout:7/39: write f1 [1133187,33326] 0 2024-02-16T19:24:04.062 INFO:tasks.workunit.client.0.smithi040.stdout:7/40: truncate f1 1345573 0 2024-02-16T19:24:04.064 INFO:tasks.workunit.client.0.smithi040.stdout:7/41: creat fc x:0 0 0 2024-02-16T19:24:04.064 INFO:tasks.workunit.client.0.smithi040.stdout:7/42: fdatasync fc 0 2024-02-16T19:24:04.065 INFO:tasks.workunit.client.0.smithi040.stdout:1/56: read f1 [11835,64779] 0 2024-02-16T19:24:04.066 INFO:tasks.workunit.client.0.smithi040.stdout:7/43: stat f9 0 2024-02-16T19:24:04.066 INFO:tasks.workunit.client.0.smithi040.stdout:4/72: dread dd/de/f12 [0,4194304] 0 2024-02-16T19:24:04.066 INFO:tasks.workunit.client.0.smithi040.stdout:3/53: truncate d1/f6 2131460 0 2024-02-16T19:24:04.067 INFO:tasks.workunit.client.0.smithi040.stdout:5/38: read d1/f4 [82338,124311] 0 2024-02-16T19:24:04.068 INFO:tasks.workunit.client.0.smithi040.stdout:2/47: sync 2024-02-16T19:24:04.068 INFO:tasks.workunit.client.0.smithi040.stdout:1/57: creat f16 x:0 0 0 2024-02-16T19:24:04.068 INFO:tasks.workunit.client.0.smithi040.stdout:1/58: dread - f14 zero size 2024-02-16T19:24:04.068 INFO:tasks.workunit.client.0.smithi040.stdout:1/59: fdatasync f5 0 2024-02-16T19:24:04.068 INFO:tasks.workunit.client.0.smithi040.stdout:1/60: chown fb 1998216 1 2024-02-16T19:24:04.068 INFO:tasks.workunit.client.0.smithi040.stdout:1/61: fdatasync f16 0 2024-02-16T19:24:04.069 INFO:tasks.workunit.client.0.smithi040.stdout:1/62: chown f16 1722654 1 2024-02-16T19:24:04.069 INFO:tasks.workunit.client.0.smithi040.stdout:6/53: rmdir d0 39 2024-02-16T19:24:04.069 INFO:tasks.workunit.client.0.smithi040.stdout:6/54: rename d0 to d0/d14 22 2024-02-16T19:24:04.069 INFO:tasks.workunit.client.0.smithi040.stdout:6/55: chown d0/f4 170 1 2024-02-16T19:24:04.070 INFO:tasks.workunit.client.0.smithi040.stdout:4/73: unlink dd/de/d10/c18 0 2024-02-16T19:24:04.070 INFO:tasks.workunit.client.0.smithi040.stdout:4/74: chown dd/de/f12 0 1 2024-02-16T19:24:04.072 INFO:tasks.workunit.client.0.smithi040.stdout:0/51: read f3 [630313,84625] 0 2024-02-16T19:24:04.073 INFO:tasks.workunit.client.0.smithi040.stdout:3/54: getdents d1/d7 0 2024-02-16T19:24:04.074 INFO:tasks.workunit.client.0.smithi040.stdout:8/40: truncate f3 3107896 0 2024-02-16T19:24:04.076 INFO:tasks.workunit.client.0.smithi040.stdout:2/48: mkdir d5/d7/d10 0 2024-02-16T19:24:04.080 INFO:tasks.workunit.client.0.smithi040.stdout:1/63: mkdir d17 0 2024-02-16T19:24:04.080 INFO:tasks.workunit.client.0.smithi040.stdout:1/64: chown f16 3 1 2024-02-16T19:24:04.081 INFO:tasks.workunit.client.0.smithi040.stdout:0/52: mkdir db/d12 0 2024-02-16T19:24:04.082 INFO:tasks.workunit.client.0.smithi040.stdout:9/52: dwrite d3/d4/f8 [0,4194304] 0 2024-02-16T19:24:04.084 INFO:tasks.workunit.client.0.smithi040.stdout:3/55: truncate d1/d4/fa 3096603 0 2024-02-16T19:24:04.085 INFO:tasks.workunit.client.0.smithi040.stdout:3/56: chown d1/f6 415702 1 2024-02-16T19:24:04.088 INFO:tasks.workunit.client.0.smithi040.stdout:8/41: dread d6/f8 [0,4194304] 0 2024-02-16T19:24:04.088 INFO:tasks.workunit.client.0.smithi040.stdout:5/39: rename d1/c9 to d1/cb 0 2024-02-16T19:24:04.088 INFO:tasks.workunit.client.0.smithi040.stdout:5/40: chown d1/cb 29780152 1 2024-02-16T19:24:04.089 INFO:tasks.workunit.client.0.smithi040.stdout:5/41: read d1/f4 [1948402,45550] 0 2024-02-16T19:24:04.090 INFO:tasks.workunit.client.0.smithi040.stdout:5/42: dread f0 [0,4194304] 0 2024-02-16T19:24:04.092 INFO:tasks.workunit.client.0.smithi040.stdout:1/65: symlink d17/l18 0 2024-02-16T19:24:04.092 INFO:tasks.workunit.client.0.smithi040.stdout:1/66: write f14 [275958,23116] 0 2024-02-16T19:24:04.092 INFO:tasks.workunit.client.0.smithi040.stdout:1/67: stat f12 0 2024-02-16T19:24:04.092 INFO:tasks.workunit.client.0.smithi040.stdout:1/68: chown cf 11 1 2024-02-16T19:24:04.092 INFO:tasks.workunit.client.0.smithi040.stdout:1/69: chown d17 151 1 2024-02-16T19:24:04.093 INFO:tasks.workunit.client.0.smithi040.stdout:0/53: getdents db/de 0 2024-02-16T19:24:04.095 INFO:tasks.workunit.client.0.smithi040.stdout:9/53: dread d3/d4/f8 [0,4194304] 0 2024-02-16T19:24:04.097 INFO:tasks.workunit.client.0.smithi040.stdout:7/44: getdents . 0 2024-02-16T19:24:04.097 INFO:tasks.workunit.client.0.smithi040.stdout:7/45: getdents . 0 2024-02-16T19:24:04.097 INFO:tasks.workunit.client.0.smithi040.stdout:7/46: read f3 [1302468,15982] 0 2024-02-16T19:24:04.099 INFO:tasks.workunit.client.0.smithi040.stdout:3/57: creat d1/dc/fd x:0 0 0 2024-02-16T19:24:04.101 INFO:tasks.workunit.client.0.smithi040.stdout:5/43: creat d1/fc x:0 0 0 2024-02-16T19:24:04.101 INFO:tasks.workunit.client.0.smithi040.stdout:1/70: symlink d17/l19 0 2024-02-16T19:24:04.103 INFO:tasks.workunit.client.0.smithi040.stdout:9/54: creat d3/fc x:0 0 0 2024-02-16T19:24:04.103 INFO:tasks.workunit.client.0.smithi040.stdout:9/55: chown d3/fc 2226 1 2024-02-16T19:24:04.105 INFO:tasks.workunit.client.0.smithi040.stdout:3/58: symlink d1/le 0 2024-02-16T19:24:04.105 INFO:tasks.workunit.client.0.smithi040.stdout:3/59: read - d1/dc/fd zero size 2024-02-16T19:24:04.105 INFO:tasks.workunit.client.0.smithi040.stdout:3/60: creat d1/ff x:0 0 0 2024-02-16T19:24:04.105 INFO:tasks.workunit.client.0.smithi040.stdout:3/61: getdents d1/d7 0 2024-02-16T19:24:04.105 INFO:tasks.workunit.client.0.smithi040.stdout:5/44: truncate f0 33828 0 2024-02-16T19:24:04.105 INFO:tasks.workunit.client.0.smithi040.stdout:5/45: fsync d1/f4 0 2024-02-16T19:24:04.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/56: rename d3/d4/l9 to d3/d4/ld 0 2024-02-16T19:24:04.110 INFO:tasks.workunit.client.0.smithi040.stdout:3/62: mknod d1/d4/c10 0 2024-02-16T19:24:04.111 INFO:tasks.workunit.client.0.smithi040.stdout:5/46: creat d1/fd x:0 0 0 2024-02-16T19:24:04.124 INFO:tasks.workunit.client.0.smithi040.stdout:3/63: write d1/f6 [116705,55030] 0 2024-02-16T19:24:04.124 INFO:tasks.workunit.client.0.smithi040.stdout:3/64: rename d1/dc to d1/dc/d11 22 2024-02-16T19:24:04.124 INFO:tasks.workunit.client.0.smithi040.stdout:3/65: truncate d1/ff 693011 0 2024-02-16T19:24:04.125 INFO:tasks.workunit.client.0.smithi040.stdout:3/66: mknod d1/d4/db/c12 0 2024-02-16T19:24:04.126 INFO:tasks.workunit.client.0.smithi040.stdout:3/67: rename d1/d4/c10 to d1/c13 0 2024-02-16T19:24:04.126 INFO:tasks.workunit.client.0.smithi040.stdout:3/68: getdents d1/d7 0 2024-02-16T19:24:04.126 INFO:tasks.workunit.client.0.smithi040.stdout:2/49: dwrite d5/d7/fa [0,4194304] 0 2024-02-16T19:24:04.127 INFO:tasks.workunit.client.0.smithi040.stdout:3/69: symlink d1/d4/db/l14 0 2024-02-16T19:24:04.127 INFO:tasks.workunit.client.0.smithi040.stdout:2/50: symlink d5/d7/d10/l11 0 2024-02-16T19:24:04.128 INFO:tasks.workunit.client.0.smithi040.stdout:3/70: creat d1/d7/f15 x:0 0 0 2024-02-16T19:24:04.128 INFO:tasks.workunit.client.0.smithi040.stdout:3/71: fsync d1/d4/fa 0 2024-02-16T19:24:04.130 INFO:tasks.workunit.client.0.smithi040.stdout:8/42: dwrite f5 [0,4194304] 0 2024-02-16T19:24:04.130 INFO:tasks.workunit.client.0.smithi040.stdout:0/54: dwrite f6 [4194304,4194304] 0 2024-02-16T19:24:04.130 INFO:tasks.workunit.client.0.smithi040.stdout:0/55: fsync f7 0 2024-02-16T19:24:04.130 INFO:tasks.workunit.client.0.smithi040.stdout:7/47: dwrite fc [0,4194304] 0 2024-02-16T19:24:04.130 INFO:tasks.workunit.client.0.smithi040.stdout:7/48: fsync f7 0 2024-02-16T19:24:04.133 INFO:tasks.workunit.client.0.smithi040.stdout:8/43: dread f2 [0,4194304] 0 2024-02-16T19:24:04.133 INFO:tasks.workunit.client.0.smithi040.stdout:8/44: chown d6/f9 21639 1 2024-02-16T19:24:04.134 INFO:tasks.workunit.client.0.smithi040.stdout:9/57: dwrite d3/fc [0,4194304] 0 2024-02-16T19:24:04.134 INFO:tasks.workunit.client.0.smithi040.stdout:0/56: truncate f9 2965714 0 2024-02-16T19:24:04.137 INFO:tasks.workunit.client.0.smithi040.stdout:2/51: dread d5/d7/fa [0,4194304] 0 2024-02-16T19:24:04.137 INFO:tasks.workunit.client.0.smithi040.stdout:7/49: unlink f3 0 2024-02-16T19:24:04.137 INFO:tasks.workunit.client.0.smithi040.stdout:3/72: dread d1/d7/f9 [0,4194304] 0 2024-02-16T19:24:04.142 INFO:tasks.workunit.client.0.smithi040.stdout:0/57: mknod db/de/d10/c13 0 2024-02-16T19:24:04.142 INFO:tasks.workunit.client.0.smithi040.stdout:2/52: write d5/d7/fa [4102701,106119] 0 2024-02-16T19:24:04.144 INFO:tasks.workunit.client.0.smithi040.stdout:8/45: dread f5 [0,4194304] 0 2024-02-16T19:24:04.144 INFO:tasks.workunit.client.0.smithi040.stdout:3/73: creat d1/d4/db/f16 x:0 0 0 2024-02-16T19:24:04.145 INFO:tasks.workunit.client.0.smithi040.stdout:2/53: fsync d5/d7/fa 0 2024-02-16T19:24:04.145 INFO:tasks.workunit.client.0.smithi040.stdout:2/54: write d5/d7/fa [3067384,27950] 0 2024-02-16T19:24:04.145 INFO:tasks.workunit.client.0.smithi040.stdout:2/55: write d5/d7/fa [1927685,28196] 0 2024-02-16T19:24:04.145 INFO:tasks.workunit.client.0.smithi040.stdout:9/58: getdents d3 0 2024-02-16T19:24:04.147 INFO:tasks.workunit.client.0.smithi040.stdout:0/58: unlink f6 0 2024-02-16T19:24:04.155 INFO:tasks.workunit.client.0.smithi040.stdout:5/47: rmdir d1 39 2024-02-16T19:24:04.155 INFO:tasks.workunit.client.0.smithi040.stdout:5/48: write d1/fd [382356,89310] 0 2024-02-16T19:24:04.157 INFO:tasks.workunit.client.0.smithi040.stdout:5/49: creat d1/fe x:0 0 0 2024-02-16T19:24:04.157 INFO:tasks.workunit.client.0.smithi040.stdout:5/50: chown d1/fe 13 1 2024-02-16T19:24:04.157 INFO:tasks.workunit.client.0.smithi040.stdout:5/51: chown d1 0 1 2024-02-16T19:24:04.164 INFO:tasks.workunit.client.0.smithi040.stdout:5/52: readlink d1/l2 0 2024-02-16T19:24:04.164 INFO:tasks.workunit.client.0.smithi040.stdout:5/53: dread - d1/fe zero size 2024-02-16T19:24:04.166 INFO:tasks.workunit.client.0.smithi040.stdout:5/54: creat d1/da/ff x:0 0 0 2024-02-16T19:24:04.168 INFO:tasks.workunit.client.0.smithi040.stdout:7/50: getdents . 0 2024-02-16T19:24:04.170 INFO:tasks.workunit.client.0.smithi040.stdout:5/55: mknod d1/da/c10 0 2024-02-16T19:24:04.170 INFO:tasks.workunit.client.0.smithi040.stdout:5/56: dread - d1/fe zero size 2024-02-16T19:24:04.174 INFO:tasks.workunit.client.0.smithi040.stdout:3/74: truncate d1/f6 660618 0 2024-02-16T19:24:04.174 INFO:tasks.workunit.client.0.smithi040.stdout:3/75: write d1/d7/f15 [573074,62123] 0 2024-02-16T19:24:04.174 INFO:tasks.workunit.client.0.smithi040.stdout:3/76: write d1/d7/f15 [309439,72363] 0 2024-02-16T19:24:04.175 INFO:tasks.workunit.client.0.smithi040.stdout:2/56: truncate d5/d7/fa 1207986 0 2024-02-16T19:24:04.177 INFO:tasks.workunit.client.0.smithi040.stdout:3/77: getdents d1/d4 0 2024-02-16T19:24:04.179 INFO:tasks.workunit.client.0.smithi040.stdout:8/46: dwrite d6/f8 [0,4194304] 0 2024-02-16T19:24:04.184 INFO:tasks.workunit.client.0.smithi040.stdout:8/47: write f2 [4165002,55458] 0 2024-02-16T19:24:04.184 INFO:tasks.workunit.client.0.smithi040.stdout:8/48: stat f4 0 2024-02-16T19:24:04.184 INFO:tasks.workunit.client.0.smithi040.stdout:8/49: mknod d6/c10 0 2024-02-16T19:24:04.186 INFO:tasks.workunit.client.0.smithi040.stdout:8/50: unlink d6/lc 0 2024-02-16T19:24:04.187 INFO:tasks.workunit.client.0.smithi040.stdout:8/51: mknod d6/d7/c11 0 2024-02-16T19:24:04.187 INFO:tasks.workunit.client.0.smithi040.stdout:8/52: chown d6/d7 567884454 1 2024-02-16T19:24:04.187 INFO:tasks.workunit.client.0.smithi040.stdout:8/53: rmdir d6/d7 39 2024-02-16T19:24:04.197 INFO:tasks.workunit.client.0.smithi040.stdout:8/54: write f3 [1371172,25061] 0 2024-02-16T19:24:04.197 INFO:tasks.workunit.client.0.smithi040.stdout:8/55: chown d6/lb 4335 1 2024-02-16T19:24:04.199 INFO:tasks.workunit.client.0.smithi040.stdout:9/59: dwrite d3/fc [0,4194304] 0 2024-02-16T19:24:04.199 INFO:tasks.workunit.client.0.smithi040.stdout:9/60: stat d3/d4/c7 0 2024-02-16T19:24:04.199 INFO:tasks.workunit.client.0.smithi040.stdout:5/57: dwrite d1/fe [0,4194304] 0 2024-02-16T19:24:04.199 INFO:tasks.workunit.client.0.smithi040.stdout:0/59: dwrite f2 [0,4194304] 0 2024-02-16T19:24:04.199 INFO:tasks.workunit.client.0.smithi040.stdout:7/51: dwrite fa [0,4194304] 0 2024-02-16T19:24:04.207 INFO:tasks.workunit.client.0.smithi040.stdout:0/60: rmdir db/d12 0 2024-02-16T19:24:04.210 INFO:tasks.workunit.client.0.smithi040.stdout:7/52: creat fd x:0 0 0 2024-02-16T19:24:04.212 INFO:tasks.workunit.client.0.smithi040.stdout:5/58: mknod d1/c11 0 2024-02-16T19:24:04.212 INFO:tasks.workunit.client.0.smithi040.stdout:9/61: dread d3/fc [0,4194304] 0 2024-02-16T19:24:04.214 INFO:tasks.workunit.client.0.smithi040.stdout:0/61: mkdir db/d14 0 2024-02-16T19:24:04.217 INFO:tasks.workunit.client.0.smithi040.stdout:7/53: rename c4 to ce 0 2024-02-16T19:24:04.218 INFO:tasks.workunit.client.0.smithi040.stdout:9/62: symlink d3/le 0 2024-02-16T19:24:04.219 INFO:tasks.workunit.client.0.smithi040.stdout:8/56: dwrite f3 [0,4194304] 0 2024-02-16T19:24:04.221 INFO:tasks.workunit.client.0.smithi040.stdout:7/54: link fb ff 0 2024-02-16T19:24:04.221 INFO:tasks.workunit.client.0.smithi040.stdout:7/55: readlink l5 0 2024-02-16T19:24:04.222 INFO:tasks.workunit.client.0.smithi040.stdout:9/63: mknod d3/cf 0 2024-02-16T19:24:04.237 INFO:tasks.workunit.client.0.smithi040.stdout:9/64: write d3/fc [791808,44109] 0 2024-02-16T19:24:04.237 INFO:tasks.workunit.client.0.smithi040.stdout:9/65: write d3/fc [1225343,22441] 0 2024-02-16T19:24:04.260 INFO:tasks.workunit.client.0.smithi040.stdout:2/57: dwrite d5/d7/fa [0,4194304] 0 2024-02-16T19:24:04.261 INFO:tasks.workunit.client.0.smithi040.stdout:3/78: dwrite d1/d7/f9 [0,4194304] 0 2024-02-16T19:24:04.261 INFO:tasks.workunit.client.0.smithi040.stdout:9/66: dwrite d3/d4/f8 [4194304,4194304] 0 2024-02-16T19:24:04.261 INFO:tasks.workunit.client.0.smithi040.stdout:5/59: dwrite d1/fd [0,4194304] 0 2024-02-16T19:24:04.261 INFO:tasks.workunit.client.0.smithi040.stdout:5/60: rename d1/da to d1/da/d12 22 2024-02-16T19:24:04.261 INFO:tasks.workunit.client.0.smithi040.stdout:0/62: dwrite f3 [4194304,4194304] 0 2024-02-16T19:24:04.262 INFO:tasks.workunit.client.0.smithi040.stdout:7/56: truncate fa 2580324 0 2024-02-16T19:24:04.262 INFO:tasks.workunit.client.0.smithi040.stdout:7/57: chown f7 27163564 1 2024-02-16T19:24:04.262 INFO:tasks.workunit.client.0.smithi040.stdout:7/58: dread - ff zero size 2024-02-16T19:24:04.262 INFO:tasks.workunit.client.0.smithi040.stdout:7/59: fdatasync f1 0 2024-02-16T19:24:04.262 INFO:tasks.workunit.client.0.smithi040.stdout:7/60: rmdir - no directory 2024-02-16T19:24:04.262 INFO:tasks.workunit.client.0.smithi040.stdout:7/61: chown f9 0 1 2024-02-16T19:24:04.262 INFO:tasks.workunit.client.0.smithi040.stdout:7/62: dread - fb zero size 2024-02-16T19:24:04.262 INFO:tasks.workunit.client.0.smithi040.stdout:3/79: dread d1/ff [0,4194304] 0 2024-02-16T19:24:04.274 INFO:tasks.workunit.client.0.smithi040.stdout:8/57: truncate f3 829816 0 2024-02-16T19:24:04.275 INFO:tasks.workunit.client.0.smithi040.stdout:2/58: dread d5/d7/fa [0,4194304] 0 2024-02-16T19:24:04.276 INFO:tasks.workunit.client.0.smithi040.stdout:9/67: rename d3/cf to d3/c10 0 2024-02-16T19:24:04.277 INFO:tasks.workunit.client.0.smithi040.stdout:9/68: stat d3/d4/f8 0 2024-02-16T19:24:04.277 INFO:tasks.workunit.client.0.smithi040.stdout:5/61: truncate d1/fe 3190592 0 2024-02-16T19:24:04.278 INFO:tasks.workunit.client.0.smithi040.stdout:0/63: unlink c5 0 2024-02-16T19:24:04.329 INFO:tasks.workunit.client.0.smithi040.stdout:8/58: mkdir d6/d12 0 2024-02-16T19:24:04.329 INFO:tasks.workunit.client.0.smithi040.stdout:8/59: creat d6/f13 x:0 0 0 2024-02-16T19:24:04.330 INFO:tasks.workunit.client.0.smithi040.stdout:7/63: dwrite ff [0,4194304] 0 2024-02-16T19:24:04.330 INFO:tasks.workunit.client.0.smithi040.stdout:7/64: fsync f9 0 2024-02-16T19:24:04.331 INFO:tasks.workunit.client.0.smithi040.stdout:2/59: mkdir d5/d7/d12 0 2024-02-16T19:24:04.334 INFO:tasks.workunit.client.0.smithi040.stdout:9/69: unlink d3/d4/la 0 2024-02-16T19:24:04.334 INFO:tasks.workunit.client.0.smithi040.stdout:9/70: truncate d3/d4/f8 9144211 0 2024-02-16T19:24:04.342 INFO:tasks.workunit.client.0.smithi040.stdout:3/80: dwrite d1/d7/f9 [0,4194304] 0 2024-02-16T19:24:04.342 INFO:tasks.workunit.client.0.smithi040.stdout:3/81: write d1/d4/db/f16 [83646,44651] 0 2024-02-16T19:24:04.342 INFO:tasks.workunit.client.0.smithi040.stdout:9/71: fdatasync d3/fc 0 2024-02-16T19:24:04.342 INFO:tasks.workunit.client.0.smithi040.stdout:5/62: dwrite d1/f7 [0,4194304] 0 2024-02-16T19:24:04.342 INFO:tasks.workunit.client.0.smithi040.stdout:5/63: read d1/f4 [304262,35177] 0 2024-02-16T19:24:04.343 INFO:tasks.workunit.client.0.smithi040.stdout:5/64: dread - d1/da/ff zero size 2024-02-16T19:24:04.344 INFO:tasks.workunit.client.0.smithi040.stdout:9/72: mkdir d3/d4/d11 0 2024-02-16T19:24:04.346 INFO:tasks.workunit.client.0.smithi040.stdout:5/65: creat d1/f13 x:0 0 0 2024-02-16T19:24:04.347 INFO:tasks.workunit.client.0.smithi040.stdout:9/73: mkdir d3/d4/d12 0 2024-02-16T19:24:04.347 INFO:tasks.workunit.client.0.smithi040.stdout:9/74: chown l2 17665813 1 2024-02-16T19:24:04.347 INFO:tasks.workunit.client.0.smithi040.stdout:9/75: fsync d3/fc 0 2024-02-16T19:24:04.349 INFO:tasks.workunit.client.0.smithi040.stdout:3/82: dread d1/f3 [0,4194304] 0 2024-02-16T19:24:04.349 INFO:tasks.workunit.client.0.smithi040.stdout:3/83: chown d1 7099 1 2024-02-16T19:24:04.349 INFO:tasks.workunit.client.0.smithi040.stdout:3/84: creat d1/d4/db/f17 x:0 0 0 2024-02-16T19:24:04.349 INFO:tasks.workunit.client.0.smithi040.stdout:9/76: unlink d3/d4/c7 0 2024-02-16T19:24:04.349 INFO:tasks.workunit.client.0.smithi040.stdout:2/60: dread d5/d7/fa [0,4194304] 0 2024-02-16T19:24:04.351 INFO:tasks.workunit.client.0.smithi040.stdout:5/66: link f0 d1/f14 0 2024-02-16T19:24:04.351 INFO:tasks.workunit.client.0.smithi040.stdout:5/67: chown d1/f14 14775094 1 2024-02-16T19:24:04.351 INFO:tasks.workunit.client.0.smithi040.stdout:5/68: write d1/fd [4285787,25992] 0 2024-02-16T19:24:04.351 INFO:tasks.workunit.client.0.smithi040.stdout:5/69: chown d1/f14 0 1 2024-02-16T19:24:04.352 INFO:tasks.workunit.client.0.smithi040.stdout:8/60: dwrite d6/f9 [0,4194304] 0 2024-02-16T19:24:04.353 INFO:tasks.workunit.client.0.smithi040.stdout:3/85: link d1/f6 d1/d7/f18 0 2024-02-16T19:24:04.353 INFO:tasks.workunit.client.0.smithi040.stdout:3/86: write d1/f3 [428914,82736] 0 2024-02-16T19:24:04.354 INFO:tasks.workunit.client.0.smithi040.stdout:2/61: creat d5/d7/d12/f13 x:0 0 0 2024-02-16T19:24:04.354 INFO:tasks.workunit.client.0.smithi040.stdout:2/62: chown d5/d7/d12 7167 1 2024-02-16T19:24:04.356 INFO:tasks.workunit.client.0.smithi040.stdout:0/64: truncate f3 8225370 0 2024-02-16T19:24:04.357 INFO:tasks.workunit.client.0.smithi040.stdout:8/61: creat d6/d12/f14 x:0 0 0 2024-02-16T19:24:04.360 INFO:tasks.workunit.client.0.smithi040.stdout:3/87: creat d1/d4/f19 x:0 0 0 2024-02-16T19:24:04.360 INFO:tasks.workunit.client.0.smithi040.stdout:7/65: dwrite fd [0,4194304] 0 2024-02-16T19:24:04.362 INFO:tasks.workunit.client.0.smithi040.stdout:8/62: getdents d6/d7 0 2024-02-16T19:24:04.362 INFO:tasks.workunit.client.0.smithi040.stdout:9/77: getdents d3/d4 0 2024-02-16T19:24:04.363 INFO:tasks.workunit.client.0.smithi040.stdout:5/70: dread d1/f7 [4194304,4194304] 0 2024-02-16T19:24:04.363 INFO:tasks.workunit.client.0.smithi040.stdout:5/71: dread - d1/fc zero size 2024-02-16T19:24:04.363 INFO:tasks.workunit.client.0.smithi040.stdout:5/72: chown d1/cb 14925 1 2024-02-16T19:24:04.364 INFO:tasks.workunit.client.0.smithi040.stdout:0/65: write db/f11 [2115572,79957] 0 2024-02-16T19:24:04.364 INFO:tasks.workunit.client.0.smithi040.stdout:0/66: write db/ff [1000526,107494] 0 2024-02-16T19:24:04.364 INFO:tasks.workunit.client.0.smithi040.stdout:3/88: symlink d1/d4/l1a 0 2024-02-16T19:24:04.364 INFO:tasks.workunit.client.0.smithi040.stdout:3/89: write d1/dc/fd [835509,70170] 0 2024-02-16T19:24:04.365 INFO:tasks.workunit.client.0.smithi040.stdout:3/90: chown d1/c13 1708223 1 2024-02-16T19:24:04.367 INFO:tasks.workunit.client.0.smithi040.stdout:7/66: rename f1 to f10 0 2024-02-16T19:24:04.368 INFO:tasks.workunit.client.0.smithi040.stdout:8/63: mkdir d6/d7/d15 0 2024-02-16T19:24:04.368 INFO:tasks.workunit.client.0.smithi040.stdout:9/78: creat d3/d4/d12/f13 x:0 0 0 2024-02-16T19:24:04.370 INFO:tasks.workunit.client.0.smithi040.stdout:3/91: truncate d1/d4/fa 1589840 0 2024-02-16T19:24:04.370 INFO:tasks.workunit.client.0.smithi040.stdout:7/67: rename fd to f11 0 2024-02-16T19:24:04.372 INFO:tasks.workunit.client.0.smithi040.stdout:5/73: dread d1/fe [0,4194304] 0 2024-02-16T19:24:04.372 INFO:tasks.workunit.client.0.smithi040.stdout:3/92: rename d1/d4/db/l14 to d1/l1b 0 2024-02-16T19:24:04.372 INFO:tasks.workunit.client.0.smithi040.stdout:7/68: rename ce to c12 0 2024-02-16T19:24:04.372 INFO:tasks.workunit.client.0.smithi040.stdout:7/69: chown fc 28 1 2024-02-16T19:24:04.373 INFO:tasks.workunit.client.0.smithi040.stdout:5/74: link d1/l2 d1/da/l15 0 2024-02-16T19:24:04.375 INFO:tasks.workunit.client.0.smithi040.stdout:5/75: mknod d1/da/c16 0 2024-02-16T19:24:04.381 INFO:tasks.workunit.client.0.smithi040.stdout:9/79: dread d3/d4/f8 [0,4194304] 0 2024-02-16T19:24:04.383 INFO:tasks.workunit.client.0.smithi040.stdout:9/80: creat d3/f14 x:0 0 0 2024-02-16T19:24:04.383 INFO:tasks.workunit.client.0.smithi040.stdout:9/81: truncate d3/d4/d12/f13 631939 0 2024-02-16T19:24:04.384 INFO:tasks.workunit.client.0.smithi040.stdout:9/82: mkdir d3/d4/d11/d15 0 2024-02-16T19:24:04.385 INFO:tasks.workunit.client.0.smithi040.stdout:7/70: write fb [394877,105059] 0 2024-02-16T19:24:04.385 INFO:tasks.workunit.client.0.smithi040.stdout:9/83: unlink d3/d4/d12/f13 0 2024-02-16T19:24:04.386 INFO:tasks.workunit.client.0.smithi040.stdout:7/71: symlink l13 0 2024-02-16T19:24:04.387 INFO:tasks.workunit.client.0.smithi040.stdout:7/72: symlink l14 0 2024-02-16T19:24:04.387 INFO:tasks.workunit.client.0.smithi040.stdout:7/73: creat f15 x:0 0 0 2024-02-16T19:24:04.403 INFO:tasks.workunit.client.0.smithi040.stdout:4/75: sync 2024-02-16T19:24:04.403 INFO:tasks.workunit.client.0.smithi040.stdout:1/71: sync 2024-02-16T19:24:04.403 INFO:tasks.workunit.client.0.smithi040.stdout:6/56: sync 2024-02-16T19:24:04.404 INFO:tasks.workunit.client.0.smithi040.stdout:4/76: getdents dd/de/d10/d13 0 2024-02-16T19:24:04.404 INFO:tasks.workunit.client.0.smithi040.stdout:4/77: creat dd/de/d10/f1c x:0 0 0 2024-02-16T19:24:04.406 INFO:tasks.workunit.client.0.smithi040.stdout:1/72: creat d17/f1a x:0 0 0 2024-02-16T19:24:04.408 INFO:tasks.workunit.client.0.smithi040.stdout:1/73: symlink d17/l1b 0 2024-02-16T19:24:04.408 INFO:tasks.workunit.client.0.smithi040.stdout:1/74: creat d17/f1c x:0 0 0 2024-02-16T19:24:04.413 INFO:tasks.workunit.client.0.smithi040.stdout:8/64: dwrite d6/d12/f14 [0,4194304] 0 2024-02-16T19:24:04.414 INFO:tasks.workunit.client.0.smithi040.stdout:1/75: mkdir d17/d1d 0 2024-02-16T19:24:04.417 INFO:tasks.workunit.client.0.smithi040.stdout:1/76: symlink d17/d1d/l1e 0 2024-02-16T19:24:04.418 INFO:tasks.workunit.client.0.smithi040.stdout:0/67: write f3 [917345,125999] 0 2024-02-16T19:24:04.420 INFO:tasks.workunit.client.0.smithi040.stdout:1/77: unlink l11 0 2024-02-16T19:24:04.420 INFO:tasks.workunit.client.0.smithi040.stdout:1/78: creat d17/f1f x:0 0 0 2024-02-16T19:24:04.423 INFO:tasks.workunit.client.0.smithi040.stdout:1/79: symlink d17/l20 0 2024-02-16T19:24:04.425 INFO:tasks.workunit.client.0.smithi040.stdout:1/80: creat d17/d1d/f21 x:0 0 0 2024-02-16T19:24:04.425 INFO:tasks.workunit.client.0.smithi040.stdout:1/81: truncate f16 223613 0 2024-02-16T19:24:04.427 INFO:tasks.workunit.client.0.smithi040.stdout:1/82: mknod d17/c22 0 2024-02-16T19:24:04.427 INFO:tasks.workunit.client.0.smithi040.stdout:1/83: chown d17/f1f 849756 1 2024-02-16T19:24:04.427 INFO:tasks.workunit.client.0.smithi040.stdout:1/84: write f1 [1243529,99278] 0 2024-02-16T19:24:04.427 INFO:tasks.workunit.client.0.smithi040.stdout:1/85: read - d17/f1f zero size 2024-02-16T19:24:04.434 INFO:tasks.workunit.client.0.smithi040.stdout:9/84: dwrite d3/fc [0,4194304] 0 2024-02-16T19:24:04.434 INFO:tasks.workunit.client.0.smithi040.stdout:9/85: dread - d3/f14 zero size 2024-02-16T19:24:04.434 INFO:tasks.workunit.client.0.smithi040.stdout:9/86: truncate d3/f14 446968 0 2024-02-16T19:24:04.435 INFO:tasks.workunit.client.0.smithi040.stdout:9/87: symlink d3/l16 0 2024-02-16T19:24:04.439 INFO:tasks.workunit.client.0.smithi040.stdout:4/78: dwrite f9 [0,4194304] 0 2024-02-16T19:24:04.439 INFO:tasks.workunit.client.0.smithi040.stdout:8/65: dwrite d6/f9 [0,4194304] 0 2024-02-16T19:24:04.440 INFO:tasks.workunit.client.0.smithi040.stdout:8/66: getdents d6/d7/d15 0 2024-02-16T19:24:04.441 INFO:tasks.workunit.client.0.smithi040.stdout:3/93: getdents d1/d4/db 0 2024-02-16T19:24:04.448 INFO:tasks.workunit.client.0.smithi040.stdout:0/68: dwrite db/fd [0,4194304] 0 2024-02-16T19:24:04.448 INFO:tasks.workunit.client.0.smithi040.stdout:0/69: chown c8 7266664 1 2024-02-16T19:24:04.449 INFO:tasks.workunit.client.0.smithi040.stdout:1/86: dwrite d17/f1c [0,4194304] 0 2024-02-16T19:24:04.451 INFO:tasks.workunit.client.0.smithi040.stdout:4/79: write f4 [1066232,78843] 0 2024-02-16T19:24:04.452 INFO:tasks.workunit.client.0.smithi040.stdout:0/70: mknod db/c15 0 2024-02-16T19:24:04.452 INFO:tasks.workunit.client.0.smithi040.stdout:0/71: chown ca 0 1 2024-02-16T19:24:04.456 INFO:tasks.workunit.client.0.smithi040.stdout:9/88: read d3/fc [538705,48869] 0 2024-02-16T19:24:04.456 INFO:tasks.workunit.client.0.smithi040.stdout:6/57: dwrite d0/ff [0,4194304] 0 2024-02-16T19:24:04.458 INFO:tasks.workunit.client.0.smithi040.stdout:4/80: truncate fa 25510 0 2024-02-16T19:24:04.458 INFO:tasks.workunit.client.0.smithi040.stdout:4/81: chown dd/de/d10/f1b 10267266 1 2024-02-16T19:24:04.460 INFO:tasks.workunit.client.0.smithi040.stdout:0/72: rename db/d14 to db/d16 0 2024-02-16T19:24:04.460 INFO:tasks.workunit.client.0.smithi040.stdout:0/73: readlink - no filename 2024-02-16T19:24:04.462 INFO:tasks.workunit.client.0.smithi040.stdout:9/89: dread d3/fc [0,4194304] 0 2024-02-16T19:24:04.462 INFO:tasks.workunit.client.0.smithi040.stdout:1/87: rmdir d17 39 2024-02-16T19:24:04.465 INFO:tasks.workunit.client.0.smithi040.stdout:8/67: dwrite f4 [4194304,4194304] 0 2024-02-16T19:24:04.465 INFO:tasks.workunit.client.0.smithi040.stdout:6/58: dread d0/f5 [0,4194304] 0 2024-02-16T19:24:04.466 INFO:tasks.workunit.client.0.smithi040.stdout:6/59: stat d0/d1/fa 0 2024-02-16T19:24:04.473 INFO:tasks.workunit.client.0.smithi040.stdout:4/82: rename f4 to dd/f1d 0 2024-02-16T19:24:04.473 INFO:tasks.workunit.client.0.smithi040.stdout:4/83: fsync dd/de/d10/f1c 0 2024-02-16T19:24:04.475 INFO:tasks.workunit.client.0.smithi040.stdout:9/90: mknod d3/c17 0 2024-02-16T19:24:04.479 INFO:tasks.workunit.client.0.smithi040.stdout:9/91: mkdir d3/d4/d11/d15/d18 0 2024-02-16T19:24:04.480 INFO:tasks.workunit.client.0.smithi040.stdout:8/68: getdents d6/d12 0 2024-02-16T19:24:04.487 INFO:tasks.workunit.client.0.smithi040.stdout:9/92: dread d3/fc [0,4194304] 0 2024-02-16T19:24:04.487 INFO:tasks.workunit.client.0.smithi040.stdout:9/93: chown d3/d4/d11/d15/d18 97 1 2024-02-16T19:24:04.487 INFO:tasks.workunit.client.0.smithi040.stdout:9/94: fdatasync d3/fc 0 2024-02-16T19:24:04.496 INFO:tasks.workunit.client.0.smithi040.stdout:8/69: write d6/d12/f14 [1735964,103757] 0 2024-02-16T19:24:04.496 INFO:tasks.workunit.client.0.smithi040.stdout:0/74: dwrite f3 [4194304,4194304] 0 2024-02-16T19:24:04.496 INFO:tasks.workunit.client.0.smithi040.stdout:6/60: dwrite d0/fe [0,4194304] 0 2024-02-16T19:24:04.496 INFO:tasks.workunit.client.0.smithi040.stdout:4/84: link dd/f1d dd/f1e 0 2024-02-16T19:24:04.497 INFO:tasks.workunit.client.0.smithi040.stdout:1/88: dwrite f1 [0,4194304] 0 2024-02-16T19:24:04.500 INFO:tasks.workunit.client.0.smithi040.stdout:4/85: dread dd/de/f12 [0,4194304] 0 2024-02-16T19:24:04.500 INFO:tasks.workunit.client.0.smithi040.stdout:4/86: truncate dd/de/d10/f1c 493319 0 2024-02-16T19:24:04.500 INFO:tasks.workunit.client.0.smithi040.stdout:4/87: rename dd/de to dd/de/d1f 22 2024-02-16T19:24:04.500 INFO:tasks.workunit.client.0.smithi040.stdout:4/88: readlink - no filename 2024-02-16T19:24:04.500 INFO:tasks.workunit.client.0.smithi040.stdout:8/70: mknod d6/c16 0 2024-02-16T19:24:04.501 INFO:tasks.workunit.client.0.smithi040.stdout:6/61: symlink d0/l15 0 2024-02-16T19:24:04.502 INFO:tasks.workunit.client.0.smithi040.stdout:1/89: symlink d17/l23 0 2024-02-16T19:24:04.502 INFO:tasks.workunit.client.0.smithi040.stdout:0/75: getdents db 0 2024-02-16T19:24:04.504 INFO:tasks.workunit.client.0.smithi040.stdout:8/71: link d6/lb d6/l17 0 2024-02-16T19:24:04.504 INFO:tasks.workunit.client.0.smithi040.stdout:8/72: write d6/f9 [4418219,116886] 0 2024-02-16T19:24:04.505 INFO:tasks.workunit.client.0.smithi040.stdout:6/62: creat d0/d1/f16 x:0 0 0 2024-02-16T19:24:04.505 INFO:tasks.workunit.client.0.smithi040.stdout:6/63: truncate d0/d1/fa 602147 0 2024-02-16T19:24:04.506 INFO:tasks.workunit.client.0.smithi040.stdout:9/95: dwrite d3/f14 [0,4194304] 0 2024-02-16T19:24:04.508 INFO:tasks.workunit.client.0.smithi040.stdout:1/90: rename d17/l19 to d17/d1d/l24 0 2024-02-16T19:24:04.508 INFO:tasks.workunit.client.0.smithi040.stdout:1/91: dread - d17/f1a zero size 2024-02-16T19:24:04.508 INFO:tasks.workunit.client.0.smithi040.stdout:1/92: stat d17/d1d 0 2024-02-16T19:24:04.515 INFO:tasks.workunit.client.0.smithi040.stdout:9/96: creat d3/d4/d11/f19 x:0 0 0 2024-02-16T19:24:04.516 INFO:tasks.workunit.client.0.smithi040.stdout:0/76: write f3 [6610265,15749] 0 2024-02-16T19:24:04.516 INFO:tasks.workunit.client.0.smithi040.stdout:1/93: creat d17/f25 x:0 0 0 2024-02-16T19:24:04.516 INFO:tasks.workunit.client.0.smithi040.stdout:1/94: write f9 [3111751,122369] 0 2024-02-16T19:24:04.516 INFO:tasks.workunit.client.0.smithi040.stdout:1/95: chown cf 558536 1 2024-02-16T19:24:04.517 INFO:tasks.workunit.client.0.smithi040.stdout:1/96: mkdir d17/d26 0 2024-02-16T19:24:04.517 INFO:tasks.workunit.client.0.smithi040.stdout:1/97: truncate d17/d1d/f21 831316 0 2024-02-16T19:24:04.519 INFO:tasks.workunit.client.0.smithi040.stdout:8/73: rmdir d6 39 2024-02-16T19:24:04.520 INFO:tasks.workunit.client.0.smithi040.stdout:6/64: read d0/fe [2124173,43156] 0 2024-02-16T19:24:04.520 INFO:tasks.workunit.client.0.smithi040.stdout:4/89: dwrite f0 [0,4194304] 0 2024-02-16T19:24:04.520 INFO:tasks.workunit.client.0.smithi040.stdout:4/90: chown dd/de/d10/f1b 35 1 2024-02-16T19:24:04.520 INFO:tasks.workunit.client.0.smithi040.stdout:4/91: stat dd/f11 0 2024-02-16T19:24:04.521 INFO:tasks.workunit.client.0.smithi040.stdout:1/98: dread f12 [0,4194304] 0 2024-02-16T19:24:04.521 INFO:tasks.workunit.client.0.smithi040.stdout:8/74: creat d6/d7/d15/f18 x:0 0 0 2024-02-16T19:24:04.523 INFO:tasks.workunit.client.0.smithi040.stdout:6/65: mkdir d0/d17 0 2024-02-16T19:24:04.523 INFO:tasks.workunit.client.0.smithi040.stdout:6/66: dread - d0/d1/f16 zero size 2024-02-16T19:24:04.529 INFO:tasks.workunit.client.0.smithi040.stdout:1/99: rename d17/d1d/l1e to d17/d1d/l27 0 2024-02-16T19:24:04.529 INFO:tasks.workunit.client.0.smithi040.stdout:1/100: getdents d17/d26 0 2024-02-16T19:24:04.531 INFO:tasks.workunit.client.0.smithi040.stdout:8/75: creat d6/d7/f19 x:0 0 0 2024-02-16T19:24:04.531 INFO:tasks.workunit.client.0.smithi040.stdout:0/77: dread f3 [0,4194304] 0 2024-02-16T19:24:04.531 INFO:tasks.workunit.client.0.smithi040.stdout:0/78: truncate f3 8863069 0 2024-02-16T19:24:04.531 INFO:tasks.workunit.client.0.smithi040.stdout:0/79: read f7 [3315905,29285] 0 2024-02-16T19:24:04.531 INFO:tasks.workunit.client.0.smithi040.stdout:0/80: truncate db/ff 1671902 0 2024-02-16T19:24:04.532 INFO:tasks.workunit.client.0.smithi040.stdout:2/63: sync 2024-02-16T19:24:04.533 INFO:tasks.workunit.client.0.smithi040.stdout:8/76: creat d6/f1a x:0 0 0 2024-02-16T19:24:04.534 INFO:tasks.workunit.client.0.smithi040.stdout:6/67: rmdir d0/d1 39 2024-02-16T19:24:04.534 INFO:tasks.workunit.client.0.smithi040.stdout:6/68: fsync d0/ff 0 2024-02-16T19:24:04.534 INFO:tasks.workunit.client.0.smithi040.stdout:6/69: fdatasync d0/d1/fa 0 2024-02-16T19:24:04.535 INFO:tasks.workunit.client.0.smithi040.stdout:2/64: rename d5/d8/lb to d5/d7/l14 0 2024-02-16T19:24:04.536 INFO:tasks.workunit.client.0.smithi040.stdout:2/65: rmdir d5/d8 39 2024-02-16T19:24:04.536 INFO:tasks.workunit.client.0.smithi040.stdout:2/66: creat d5/d7/d12/f15 x:0 0 0 2024-02-16T19:24:04.540 INFO:tasks.workunit.client.0.smithi040.stdout:8/77: rename d6/d7/la to d6/d7/d15/l1b 0 2024-02-16T19:24:04.540 INFO:tasks.workunit.client.0.smithi040.stdout:8/78: fdatasync d6/f8 0 2024-02-16T19:24:04.543 INFO:tasks.workunit.client.0.smithi040.stdout:2/67: write d5/d7/fa [285715,79719] 0 2024-02-16T19:24:04.543 INFO:tasks.workunit.client.0.smithi040.stdout:2/68: stat d5/d7/d12/f13 0 2024-02-16T19:24:04.548 INFO:tasks.workunit.client.0.smithi040.stdout:0/81: write db/fd [2453688,124312] 0 2024-02-16T19:24:04.549 INFO:tasks.workunit.client.0.smithi040.stdout:0/82: mknod db/d16/c17 0 2024-02-16T19:24:04.549 INFO:tasks.workunit.client.0.smithi040.stdout:0/83: chown f3 271434 1 2024-02-16T19:24:04.550 INFO:tasks.workunit.client.0.smithi040.stdout:4/92: dwrite fb [0,4194304] 0 2024-02-16T19:24:04.550 INFO:tasks.workunit.client.0.smithi040.stdout:9/97: dwrite d3/f14 [0,4194304] 0 2024-02-16T19:24:04.552 INFO:tasks.workunit.client.0.smithi040.stdout:1/101: dwrite d17/f25 [0,4194304] 0 2024-02-16T19:24:04.552 INFO:tasks.workunit.client.0.smithi040.stdout:1/102: creat d17/f28 x:0 0 0 2024-02-16T19:24:04.552 INFO:tasks.workunit.client.0.smithi040.stdout:1/103: truncate d17/f25 4790870 0 2024-02-16T19:24:04.556 INFO:tasks.workunit.client.0.smithi040.stdout:0/84: dread f3 [4194304,4194304] 0 2024-02-16T19:24:04.559 INFO:tasks.workunit.client.0.smithi040.stdout:0/85: symlink db/de/d10/l18 0 2024-02-16T19:24:04.559 INFO:tasks.workunit.client.0.smithi040.stdout:0/86: fdatasync f3 0 2024-02-16T19:24:04.561 INFO:tasks.workunit.client.0.smithi040.stdout:0/87: symlink db/de/d10/l19 0 2024-02-16T19:24:04.561 INFO:tasks.workunit.client.0.smithi040.stdout:0/88: write f3 [9530579,40909] 0 2024-02-16T19:24:04.561 INFO:tasks.workunit.client.0.smithi040.stdout:0/89: write f7 [3745118,77332] 0 2024-02-16T19:24:04.561 INFO:tasks.workunit.client.0.smithi040.stdout:0/90: read f3 [1323163,112361] 0 2024-02-16T19:24:04.562 INFO:tasks.workunit.client.0.smithi040.stdout:4/93: dread dd/de/d10/d13/f15 [0,4194304] 0 2024-02-16T19:24:04.562 INFO:tasks.workunit.client.0.smithi040.stdout:4/94: truncate dd/f19 286868 0 2024-02-16T19:24:04.562 INFO:tasks.workunit.client.0.smithi040.stdout:4/95: read - f8 zero size 2024-02-16T19:24:04.562 INFO:tasks.workunit.client.0.smithi040.stdout:6/70: dwrite d0/fe [0,4194304] 0 2024-02-16T19:24:04.563 INFO:tasks.workunit.client.0.smithi040.stdout:0/91: link db/c15 db/de/d10/c1a 0 2024-02-16T19:24:04.563 INFO:tasks.workunit.client.0.smithi040.stdout:0/92: stat db/f11 0 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:0/93: readlink db/de/d10/l19 0 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:0/94: write f2 [7749033,9571] 0 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:0/95: chown db/d16 0 1 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/96: dread fa [0,4194304] 0 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:8/79: getdents d6 0 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:8/80: chown f2 2135511 1 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:8/81: chown c1 0 1 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:9/98: dread d3/f14 [0,4194304] 0 2024-02-16T19:24:04.564 INFO:tasks.workunit.client.0.smithi040.stdout:9/99: fdatasync d3/d4/f8 0 2024-02-16T19:24:04.565 INFO:tasks.workunit.client.0.smithi040.stdout:0/96: creat db/f1b x:0 0 0 2024-02-16T19:24:04.572 INFO:tasks.workunit.client.0.smithi040.stdout:8/82: getdents d6/d7 0 2024-02-16T19:24:04.573 INFO:tasks.workunit.client.0.smithi040.stdout:7/74: sync 2024-02-16T19:24:04.573 INFO:tasks.workunit.client.0.smithi040.stdout:7/75: dread - f15 zero size 2024-02-16T19:24:04.573 INFO:tasks.workunit.client.0.smithi040.stdout:3/94: sync 2024-02-16T19:24:04.573 INFO:tasks.workunit.client.0.smithi040.stdout:3/95: dread - d1/d4/f19 zero size 2024-02-16T19:24:04.573 INFO:tasks.workunit.client.0.smithi040.stdout:5/76: sync 2024-02-16T19:24:04.573 INFO:tasks.workunit.client.0.smithi040.stdout:5/77: read - d1/fc zero size 2024-02-16T19:24:04.575 INFO:tasks.workunit.client.0.smithi040.stdout:3/96: symlink d1/dc/l1c 0 2024-02-16T19:24:04.575 INFO:tasks.workunit.client.0.smithi040.stdout:3/97: creat d1/dc/f1d x:0 0 0 2024-02-16T19:24:04.575 INFO:tasks.workunit.client.0.smithi040.stdout:3/98: chown d1/l1b 7128 1 2024-02-16T19:24:04.577 INFO:tasks.workunit.client.0.smithi040.stdout:4/97: write dd/de/d10/f1b [3744794,6662] 0 2024-02-16T19:24:04.577 INFO:tasks.workunit.client.0.smithi040.stdout:3/99: mknod d1/d4/db/c1e 0 2024-02-16T19:24:04.577 INFO:tasks.workunit.client.0.smithi040.stdout:3/100: stat d1/d4/db 0 2024-02-16T19:24:04.579 INFO:tasks.workunit.client.0.smithi040.stdout:4/98: unlink f9 0 2024-02-16T19:24:04.579 INFO:tasks.workunit.client.0.smithi040.stdout:4/99: creat dd/de/d10/f20 x:0 0 0 2024-02-16T19:24:04.581 INFO:tasks.workunit.client.0.smithi040.stdout:4/100: symlink dd/de/d10/l21 0 2024-02-16T19:24:04.581 INFO:tasks.workunit.client.0.smithi040.stdout:4/101: truncate dd/de/d10/d13/f15 800117 0 2024-02-16T19:24:04.618 INFO:tasks.workunit.client.0.smithi040.stdout:6/71: write d0/ff [3819138,35235] 0 2024-02-16T19:24:04.621 INFO:tasks.workunit.client.0.smithi040.stdout:6/72: creat d0/d1/d12/f18 x:0 0 0 2024-02-16T19:24:04.622 INFO:tasks.workunit.client.0.smithi040.stdout:8/83: write d6/f8 [889363,115965] 0 2024-02-16T19:24:04.622 INFO:tasks.workunit.client.0.smithi040.stdout:8/84: dread - d6/f1a zero size 2024-02-16T19:24:04.622 INFO:tasks.workunit.client.0.smithi040.stdout:8/85: dread - d6/d7/d15/f18 zero size 2024-02-16T19:24:04.623 INFO:tasks.workunit.client.0.smithi040.stdout:6/73: rename d0/lb to d0/l19 0 2024-02-16T19:24:04.626 INFO:tasks.workunit.client.0.smithi040.stdout:7/76: write f11 [1760488,78710] 0 2024-02-16T19:24:04.627 INFO:tasks.workunit.client.0.smithi040.stdout:7/77: creat f16 x:0 0 0 2024-02-16T19:24:04.631 INFO:tasks.workunit.client.0.smithi040.stdout:7/78: rename f2 to f17 0 2024-02-16T19:24:04.632 INFO:tasks.workunit.client.0.smithi040.stdout:7/79: fdatasync f9 0 2024-02-16T19:24:04.632 INFO:tasks.workunit.client.0.smithi040.stdout:7/80: dread - f15 zero size 2024-02-16T19:24:04.635 INFO:tasks.workunit.client.0.smithi040.stdout:6/74: write d0/f5 [2981703,62472] 0 2024-02-16T19:24:04.637 INFO:tasks.workunit.client.0.smithi040.stdout:6/75: creat d0/d17/f1a x:0 0 0 2024-02-16T19:24:04.638 INFO:tasks.workunit.client.0.smithi040.stdout:6/76: symlink d0/d17/l1b 0 2024-02-16T19:24:04.638 INFO:tasks.workunit.client.0.smithi040.stdout:1/104: dwrite fd [0,4194304] 0 2024-02-16T19:24:04.639 INFO:tasks.workunit.client.0.smithi040.stdout:6/77: link d0/d17/l1b d0/l1c 0 2024-02-16T19:24:04.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/105: mkdir d17/d29 0 2024-02-16T19:24:04.640 INFO:tasks.workunit.client.0.smithi040.stdout:6/78: creat d0/f1d x:0 0 0 2024-02-16T19:24:04.641 INFO:tasks.workunit.client.0.smithi040.stdout:6/79: truncate d0/ff 4220858 0 2024-02-16T19:24:04.658 INFO:tasks.workunit.client.0.smithi040.stdout:1/106: dread f5 [0,4194304] 0 2024-02-16T19:24:04.659 INFO:tasks.workunit.client.0.smithi040.stdout:1/107: dread f8 [0,4194304] 0 2024-02-16T19:24:04.661 INFO:tasks.workunit.client.0.smithi040.stdout:4/102: dwrite fb [0,4194304] 0 2024-02-16T19:24:04.663 INFO:tasks.workunit.client.0.smithi040.stdout:0/97: dwrite f7 [0,4194304] 0 2024-02-16T19:24:04.663 INFO:tasks.workunit.client.0.smithi040.stdout:0/98: write f3 [3491299,56128] 0 2024-02-16T19:24:04.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/103: symlink dd/l22 0 2024-02-16T19:24:04.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/104: creat dd/f23 x:0 0 0 2024-02-16T19:24:04.664 INFO:tasks.workunit.client.0.smithi040.stdout:0/99: unlink f2 0 2024-02-16T19:24:04.666 INFO:tasks.workunit.client.0.smithi040.stdout:4/105: symlink dd/de/l24 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:5/78: dwrite d1/fe [4194304,4194304] 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:5/79: stat d1/da/c10 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:8/86: dwrite f3 [0,4194304] 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:5/80: write d1/f14 [436909,56165] 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:8/87: write d6/f8 [1501766,91209] 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:3/101: dwrite d1/dc/f1d [0,4194304] 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:7/81: dwrite f9 [0,4194304] 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:7/82: read ff [2931892,91199] 0 2024-02-16T19:24:04.670 INFO:tasks.workunit.client.0.smithi040.stdout:7/83: chown c6 349585031 1 2024-02-16T19:24:04.673 INFO:tasks.workunit.client.0.smithi040.stdout:8/88: rename d6/f1a to d6/d7/d15/f1c 0 2024-02-16T19:24:04.676 INFO:tasks.workunit.client.0.smithi040.stdout:0/100: write f7 [2332581,3098] 0 2024-02-16T19:24:04.677 INFO:tasks.workunit.client.0.smithi040.stdout:0/101: fdatasync db/ff 0 2024-02-16T19:24:04.677 INFO:tasks.workunit.client.0.smithi040.stdout:0/102: truncate db/f11 5021382 0 2024-02-16T19:24:04.677 INFO:tasks.workunit.client.0.smithi040.stdout:6/80: dwrite d0/d1/fa [0,4194304] 0 2024-02-16T19:24:04.677 INFO:tasks.workunit.client.0.smithi040.stdout:6/81: truncate d0/d1/f11 819216 0 2024-02-16T19:24:04.678 INFO:tasks.workunit.client.0.smithi040.stdout:6/82: write d0/f4 [3767577,95045] 0 2024-02-16T19:24:04.678 INFO:tasks.workunit.client.0.smithi040.stdout:6/83: stat d0/fe 0 2024-02-16T19:24:04.678 INFO:tasks.workunit.client.0.smithi040.stdout:6/84: fsync d0/f5 0 2024-02-16T19:24:04.678 INFO:tasks.workunit.client.0.smithi040.stdout:3/102: symlink d1/d4/db/l1f 0 2024-02-16T19:24:04.681 INFO:tasks.workunit.client.0.smithi040.stdout:7/84: symlink l18 0 2024-02-16T19:24:04.681 INFO:tasks.workunit.client.0.smithi040.stdout:5/81: read d1/fe [4360104,3601] 0 2024-02-16T19:24:04.684 INFO:tasks.workunit.client.0.smithi040.stdout:8/89: mkdir d6/d7/d1d 0 2024-02-16T19:24:04.685 INFO:tasks.workunit.client.0.smithi040.stdout:0/103: symlink db/d16/l1c 0 2024-02-16T19:24:04.686 INFO:tasks.workunit.client.0.smithi040.stdout:0/104: write db/fd [214225,75272] 0 2024-02-16T19:24:04.686 INFO:tasks.workunit.client.0.smithi040.stdout:4/106: dwrite dd/de/d10/f1b [0,4194304] 0 2024-02-16T19:24:04.686 INFO:tasks.workunit.client.0.smithi040.stdout:4/107: dread - dd/de/f1a zero size 2024-02-16T19:24:04.694 INFO:tasks.workunit.client.0.smithi040.stdout:4/108: write dd/de/d10/f1b [2648420,89782] 0 2024-02-16T19:24:04.702 INFO:tasks.workunit.client.0.smithi040.stdout:3/103: rename d1/d4/db/f17 to d1/d4/db/f20 0 2024-02-16T19:24:04.702 INFO:tasks.workunit.client.0.smithi040.stdout:8/90: write f3 [1721629,74686] 0 2024-02-16T19:24:04.703 INFO:tasks.workunit.client.0.smithi040.stdout:3/104: write d1/d4/fa [1147869,74871] 0 2024-02-16T19:24:04.703 INFO:tasks.workunit.client.0.smithi040.stdout:7/85: symlink l19 0 2024-02-16T19:24:04.708 INFO:tasks.workunit.client.0.smithi040.stdout:8/91: symlink d6/l1e 0 2024-02-16T19:24:04.708 INFO:tasks.workunit.client.0.smithi040.stdout:8/92: write f2 [3577161,93800] 0 2024-02-16T19:24:04.708 INFO:tasks.workunit.client.0.smithi040.stdout:8/93: write d6/d7/d15/f1c [309465,85533] 0 2024-02-16T19:24:04.708 INFO:tasks.workunit.client.0.smithi040.stdout:8/94: truncate d6/d7/d15/f18 120680 0 2024-02-16T19:24:04.714 INFO:tasks.workunit.client.0.smithi040.stdout:5/82: dwrite d1/fc [0,4194304] 0 2024-02-16T19:24:04.714 INFO:tasks.workunit.client.0.smithi040.stdout:0/105: dwrite db/ff [0,4194304] 0 2024-02-16T19:24:04.715 INFO:tasks.workunit.client.0.smithi040.stdout:7/86: write f17 [7882585,55339] 0 2024-02-16T19:24:04.718 INFO:tasks.workunit.client.0.smithi040.stdout:5/83: creat d1/f17 x:0 0 0 2024-02-16T19:24:04.723 INFO:tasks.workunit.client.0.smithi040.stdout:7/87: symlink l1a 0 2024-02-16T19:24:04.723 INFO:tasks.workunit.client.0.smithi040.stdout:5/84: dread f0 [0,4194304] 0 2024-02-16T19:24:04.723 INFO:tasks.workunit.client.0.smithi040.stdout:0/106: unlink db/de/d10/l18 0 2024-02-16T19:24:04.726 INFO:tasks.workunit.client.0.smithi040.stdout:5/85: symlink d1/l18 0 2024-02-16T19:24:04.727 INFO:tasks.workunit.client.0.smithi040.stdout:3/105: dread d1/d4/fa [0,4194304] 0 2024-02-16T19:24:04.727 INFO:tasks.workunit.client.0.smithi040.stdout:3/106: chown d1/l1b 586531606 1 2024-02-16T19:24:04.729 INFO:tasks.workunit.client.0.smithi040.stdout:2/69: sync 2024-02-16T19:24:04.729 INFO:tasks.workunit.client.0.smithi040.stdout:9/100: sync 2024-02-16T19:24:04.729 INFO:tasks.workunit.client.0.smithi040.stdout:2/70: fdatasync d5/d7/fa 0 2024-02-16T19:24:04.729 INFO:tasks.workunit.client.0.smithi040.stdout:3/107: dread d1/d7/f15 [0,4194304] 0 2024-02-16T19:24:04.730 INFO:tasks.workunit.client.0.smithi040.stdout:3/108: mknod d1/d7/c21 0 2024-02-16T19:24:04.732 INFO:tasks.workunit.client.0.smithi040.stdout:3/109: symlink d1/d4/db/l22 0 2024-02-16T19:24:04.732 INFO:tasks.workunit.client.0.smithi040.stdout:3/110: creat d1/f23 x:0 0 0 2024-02-16T19:24:04.732 INFO:tasks.workunit.client.0.smithi040.stdout:3/111: fsync d1/d4/f19 0 2024-02-16T19:24:04.732 INFO:tasks.workunit.client.0.smithi040.stdout:3/112: read d1/d4/fa [215919,57405] 0 2024-02-16T19:24:04.732 INFO:tasks.workunit.client.0.smithi040.stdout:3/113: fdatasync d1/dc/fd 0 2024-02-16T19:24:04.740 INFO:tasks.workunit.client.0.smithi040.stdout:3/114: write d1/f6 [4015659,84837] 0 2024-02-16T19:24:04.745 INFO:tasks.workunit.client.0.smithi040.stdout:8/95: dwrite d6/d7/d15/f1c [0,4194304] 0 2024-02-16T19:24:04.745 INFO:tasks.workunit.client.0.smithi040.stdout:2/71: dwrite d5/d7/d12/f13 [0,4194304] 0 2024-02-16T19:24:04.746 INFO:tasks.workunit.client.0.smithi040.stdout:8/96: creat d6/d12/f1f x:0 0 0 2024-02-16T19:24:04.752 INFO:tasks.workunit.client.0.smithi040.stdout:0/107: rename db/de/d10 to db/d16/d1d 0 2024-02-16T19:24:04.753 INFO:tasks.workunit.client.0.smithi040.stdout:0/108: dread db/f11 [4194304,4194304] 0 2024-02-16T19:24:04.754 INFO:tasks.workunit.client.0.smithi040.stdout:5/86: rename d1/fe to d1/da/f19 0 2024-02-16T19:24:04.754 INFO:tasks.workunit.client.0.smithi040.stdout:9/101: rename d3/d4/d11/d15 to d3/d4/d11/d15/d18/d1a 22 2024-02-16T19:24:04.754 INFO:tasks.workunit.client.0.smithi040.stdout:9/102: stat d3/c17 0 2024-02-16T19:24:04.754 INFO:tasks.workunit.client.0.smithi040.stdout:5/87: write d1/f17 [225328,46458] 0 2024-02-16T19:24:04.754 INFO:tasks.workunit.client.0.smithi040.stdout:9/103: fdatasync d3/d4/d11/f19 0 2024-02-16T19:24:04.754 INFO:tasks.workunit.client.0.smithi040.stdout:9/104: readlink d3/d4/ld 0 2024-02-16T19:24:04.755 INFO:tasks.workunit.client.0.smithi040.stdout:9/105: fdatasync d3/d4/f8 0 2024-02-16T19:24:04.756 INFO:tasks.workunit.client.0.smithi040.stdout:3/115: rename d1/d4/db/l1f to d1/dc/l24 0 2024-02-16T19:24:04.757 INFO:tasks.workunit.client.0.smithi040.stdout:5/88: mknod d1/da/c1a 0 2024-02-16T19:24:04.758 INFO:tasks.workunit.client.0.smithi040.stdout:9/106: truncate d3/d4/f8 3370439 0 2024-02-16T19:24:04.758 INFO:tasks.workunit.client.0.smithi040.stdout:9/107: write d3/fc [4698756,43386] 0 2024-02-16T19:24:04.759 INFO:tasks.workunit.client.0.smithi040.stdout:9/108: stat d3/d4/d11/d15/d18 0 2024-02-16T19:24:04.761 INFO:tasks.workunit.client.0.smithi040.stdout:9/109: symlink d3/d4/l1b 0 2024-02-16T19:24:04.764 INFO:tasks.workunit.client.0.smithi040.stdout:5/89: rename d1/l2 to d1/l1b 0 2024-02-16T19:24:04.764 INFO:tasks.workunit.client.0.smithi040.stdout:5/90: truncate d1/f17 315423 0 2024-02-16T19:24:04.765 INFO:tasks.workunit.client.0.smithi040.stdout:9/110: mkdir d3/d1c 0 2024-02-16T19:24:04.770 INFO:tasks.workunit.client.0.smithi040.stdout:8/97: dwrite f4 [4194304,4194304] 0 2024-02-16T19:24:04.775 INFO:tasks.workunit.client.0.smithi040.stdout:5/91: mkdir d1/da/d1c 0 2024-02-16T19:24:04.780 INFO:tasks.workunit.client.0.smithi040.stdout:5/92: dread d1/f14 [0,4194304] 0 2024-02-16T19:24:04.780 INFO:tasks.workunit.client.0.smithi040.stdout:5/93: chown d1/f7 5469400 1 2024-02-16T19:24:04.781 INFO:tasks.workunit.client.0.smithi040.stdout:5/94: creat d1/f1d x:0 0 0 2024-02-16T19:24:04.786 INFO:tasks.workunit.client.0.smithi040.stdout:3/116: dwrite d1/d4/fa [0,4194304] 0 2024-02-16T19:24:04.786 INFO:tasks.workunit.client.0.smithi040.stdout:3/117: readlink d1/d4/db/l22 0 2024-02-16T19:24:04.793 INFO:tasks.workunit.client.0.smithi040.stdout:9/111: dwrite d3/d4/d11/f19 [0,4194304] 0 2024-02-16T19:24:04.796 INFO:tasks.workunit.client.0.smithi040.stdout:2/72: dwrite d5/d7/d12/f13 [4194304,4194304] 0 2024-02-16T19:24:04.796 INFO:tasks.workunit.client.0.smithi040.stdout:2/73: fdatasync d5/d7/d12/f15 0 2024-02-16T19:24:04.796 INFO:tasks.workunit.client.0.smithi040.stdout:2/74: write d5/d7/fa [2320701,119544] 0 2024-02-16T19:24:04.797 INFO:tasks.workunit.client.0.smithi040.stdout:8/98: dwrite d6/d7/d15/f1c [0,4194304] 0 2024-02-16T19:24:04.798 INFO:tasks.workunit.client.0.smithi040.stdout:2/75: mkdir d5/d7/d10/d16 0 2024-02-16T19:24:04.801 INFO:tasks.workunit.client.0.smithi040.stdout:2/76: mknod d5/d7/d10/c17 0 2024-02-16T19:24:04.801 INFO:tasks.workunit.client.0.smithi040.stdout:2/77: getdents d5 0 2024-02-16T19:24:04.802 INFO:tasks.workunit.client.0.smithi040.stdout:2/78: truncate d5/d7/d12/f13 9151519 0 2024-02-16T19:24:04.803 INFO:tasks.workunit.client.0.smithi040.stdout:2/79: unlink d5/d8/c9 0 2024-02-16T19:24:04.803 INFO:tasks.workunit.client.0.smithi040.stdout:1/108: sync 2024-02-16T19:24:04.804 INFO:tasks.workunit.client.0.smithi040.stdout:9/112: dread d3/fc [0,4194304] 0 2024-02-16T19:24:04.804 INFO:tasks.workunit.client.0.smithi040.stdout:2/80: unlink d5/d7/fa 0 2024-02-16T19:24:04.809 INFO:tasks.workunit.client.0.smithi040.stdout:3/118: dwrite d1/d7/f15 [0,4194304] 0 2024-02-16T19:24:04.811 INFO:tasks.workunit.client.0.smithi040.stdout:1/109: dread fd [0,4194304] 0 2024-02-16T19:24:04.811 INFO:tasks.workunit.client.0.smithi040.stdout:8/99: write d6/d7/d15/f1c [3049141,110790] 0 2024-02-16T19:24:04.815 INFO:tasks.workunit.client.0.smithi040.stdout:9/113: write d3/d4/d11/f19 [2954439,17237] 0 2024-02-16T19:24:04.816 INFO:tasks.workunit.client.0.smithi040.stdout:2/81: getdents d5/d7 0 2024-02-16T19:24:04.820 INFO:tasks.workunit.client.0.smithi040.stdout:9/114: creat d3/d4/d11/d15/f1d x:0 0 0 2024-02-16T19:24:04.826 INFO:tasks.workunit.client.0.smithi040.stdout:9/115: dread d3/f14 [0,4194304] 0 2024-02-16T19:24:04.826 INFO:tasks.workunit.client.0.smithi040.stdout:9/116: stat d3/d4/d11 0 2024-02-16T19:24:04.829 INFO:tasks.workunit.client.0.smithi040.stdout:9/117: rmdir d3/d4/d12 0 2024-02-16T19:24:04.838 INFO:tasks.workunit.client.0.smithi040.stdout:3/119: dwrite d1/d7/f9 [4194304,4194304] 0 2024-02-16T19:24:04.840 INFO:tasks.workunit.client.0.smithi040.stdout:9/118: dread d3/d4/d11/f19 [0,4194304] 0 2024-02-16T19:24:04.840 INFO:tasks.workunit.client.0.smithi040.stdout:1/110: dwrite fb [0,4194304] 0 2024-02-16T19:24:04.840 INFO:tasks.workunit.client.0.smithi040.stdout:2/82: dwrite d5/d7/d12/f15 [0,4194304] 0 2024-02-16T19:24:04.842 INFO:tasks.workunit.client.0.smithi040.stdout:3/120: unlink d1/d4/db/f20 0 2024-02-16T19:24:04.846 INFO:tasks.workunit.client.0.smithi040.stdout:9/119: getdents d3/d4 0 2024-02-16T19:24:04.848 INFO:tasks.workunit.client.0.smithi040.stdout:1/111: dread d17/f25 [0,4194304] 0 2024-02-16T19:24:04.848 INFO:tasks.workunit.client.0.smithi040.stdout:1/112: stat d17/l20 0 2024-02-16T19:24:04.851 INFO:tasks.workunit.client.0.smithi040.stdout:2/83: unlink d5/c6 0 2024-02-16T19:24:04.852 INFO:tasks.workunit.client.0.smithi040.stdout:3/121: read d1/d7/f18 [5009092,75896] 0 2024-02-16T19:24:04.853 INFO:tasks.workunit.client.0.smithi040.stdout:2/84: creat d5/f18 x:0 0 0 2024-02-16T19:24:04.861 INFO:tasks.workunit.client.0.smithi040.stdout:6/85: sync 2024-02-16T19:24:04.862 INFO:tasks.workunit.client.0.smithi040.stdout:6/86: truncate d0/d17/f1a 163888 0 2024-02-16T19:24:04.863 INFO:tasks.workunit.client.0.smithi040.stdout:6/87: symlink d0/d1/l1e 0 2024-02-16T19:24:04.863 INFO:tasks.workunit.client.0.smithi040.stdout:6/88: read - d0/f1d zero size 2024-02-16T19:24:04.863 INFO:tasks.workunit.client.0.smithi040.stdout:6/89: chown d0/ff 61631 1 2024-02-16T19:24:04.863 INFO:tasks.workunit.client.0.smithi040.stdout:6/90: chown d0/d1/d12 185470 1 2024-02-16T19:24:04.863 INFO:tasks.workunit.client.0.smithi040.stdout:6/91: write d0/f4 [1111768,25336] 0 2024-02-16T19:24:04.865 INFO:tasks.workunit.client.0.smithi040.stdout:6/92: creat d0/d1/f1f x:0 0 0 2024-02-16T19:24:04.867 INFO:tasks.workunit.client.0.smithi040.stdout:6/93: link d0/ff d0/d1/f20 0 2024-02-16T19:24:04.869 INFO:tasks.workunit.client.0.smithi040.stdout:6/94: unlink d0/d1/f11 0 2024-02-16T19:24:04.869 INFO:tasks.workunit.client.0.smithi040.stdout:6/95: write d0/f1d [720260,84062] 0 2024-02-16T19:24:04.869 INFO:tasks.workunit.client.0.smithi040.stdout:6/96: chown d0/f4 2420830 1 2024-02-16T19:24:04.889 INFO:tasks.workunit.client.0.smithi040.stdout:3/122: write d1/dc/f1d [1755680,62257] 0 2024-02-16T19:24:04.889 INFO:tasks.workunit.client.0.smithi040.stdout:3/123: chown d1/d4/f19 7470 1 2024-02-16T19:24:04.892 INFO:tasks.workunit.client.0.smithi040.stdout:3/124: rename d1/d7/f15 to d1/d4/f25 0 2024-02-16T19:24:04.897 INFO:tasks.workunit.client.0.smithi040.stdout:3/125: dread d1/d7/f18 [4194304,4194304] 0 2024-02-16T19:24:04.897 INFO:tasks.workunit.client.0.smithi040.stdout:3/126: stat d1/dc/f1d 0 2024-02-16T19:24:04.903 INFO:tasks.workunit.client.0.smithi040.stdout:3/127: dread d1/d4/db/f16 [0,4194304] 0 2024-02-16T19:24:04.910 INFO:tasks.workunit.client.0.smithi040.stdout:9/120: dwrite d3/f14 [0,4194304] 0 2024-02-16T19:24:04.910 INFO:tasks.workunit.client.0.smithi040.stdout:9/121: fsync d3/f14 0 2024-02-16T19:24:04.910 INFO:tasks.workunit.client.0.smithi040.stdout:9/122: readlink l2 0 2024-02-16T19:24:04.911 INFO:tasks.workunit.client.0.smithi040.stdout:6/97: dwrite d0/d1/f1f [0,4194304] 0 2024-02-16T19:24:04.911 INFO:tasks.workunit.client.0.smithi040.stdout:6/98: creat d0/d17/f21 x:0 0 0 2024-02-16T19:24:04.911 INFO:tasks.workunit.client.0.smithi040.stdout:2/85: dwrite d5/d7/d12/f13 [0,4194304] 0 2024-02-16T19:24:04.911 INFO:tasks.workunit.client.0.smithi040.stdout:1/113: dwrite d17/f1f [0,4194304] 0 2024-02-16T19:24:04.912 INFO:tasks.workunit.client.0.smithi040.stdout:1/114: dread d17/d1d/f21 [0,4194304] 0 2024-02-16T19:24:04.912 INFO:tasks.workunit.client.0.smithi040.stdout:9/123: link d3/d4/l1b d3/d4/d11/d15/d18/l1e 0 2024-02-16T19:24:04.913 INFO:tasks.workunit.client.0.smithi040.stdout:9/124: creat d3/f1f x:0 0 0 2024-02-16T19:24:04.913 INFO:tasks.workunit.client.0.smithi040.stdout:9/125: truncate d3/f1f 190452 0 2024-02-16T19:24:04.913 INFO:tasks.workunit.client.0.smithi040.stdout:9/126: fdatasync d3/d4/f8 0 2024-02-16T19:24:04.913 INFO:tasks.workunit.client.0.smithi040.stdout:9/127: truncate d3/d4/d11/d15/f1d 900259 0 2024-02-16T19:24:04.920 INFO:tasks.workunit.client.0.smithi040.stdout:1/115: truncate f9 3118605 0 2024-02-16T19:24:04.921 INFO:tasks.workunit.client.0.smithi040.stdout:1/116: chown f14 419327344 1 2024-02-16T19:24:04.921 INFO:tasks.workunit.client.0.smithi040.stdout:1/117: truncate d17/d1d/f21 1019309 0 2024-02-16T19:24:04.923 INFO:tasks.workunit.client.0.smithi040.stdout:9/128: getdents d3/d4 0 2024-02-16T19:24:04.923 INFO:tasks.workunit.client.0.smithi040.stdout:6/99: rmdir d0/d1/d12 39 2024-02-16T19:24:04.924 INFO:tasks.workunit.client.0.smithi040.stdout:1/118: write d17/f25 [1434148,127073] 0 2024-02-16T19:24:04.926 INFO:tasks.workunit.client.0.smithi040.stdout:9/129: rename d3/le to d3/d4/l20 0 2024-02-16T19:24:04.926 INFO:tasks.workunit.client.0.smithi040.stdout:9/130: creat d3/f21 x:0 0 0 2024-02-16T19:24:04.927 INFO:tasks.workunit.client.0.smithi040.stdout:6/100: symlink d0/d1/l22 0 2024-02-16T19:24:04.927 INFO:tasks.workunit.client.0.smithi040.stdout:6/101: rename d0/d1 to d0/d1/d12/d23 22 2024-02-16T19:24:04.927 INFO:tasks.workunit.client.0.smithi040.stdout:2/86: truncate d5/d7/d12/f13 3798140 0 2024-02-16T19:24:04.929 INFO:tasks.workunit.client.0.smithi040.stdout:9/131: symlink d3/l22 0 2024-02-16T19:24:04.930 INFO:tasks.workunit.client.0.smithi040.stdout:6/102: mkdir d0/d17/d24 0 2024-02-16T19:24:04.931 INFO:tasks.workunit.client.0.smithi040.stdout:2/87: creat d5/d7/f19 x:0 0 0 2024-02-16T19:24:04.935 INFO:tasks.workunit.client.0.smithi040.stdout:9/132: creat d3/d4/d11/d15/d18/f23 x:0 0 0 2024-02-16T19:24:04.935 INFO:tasks.workunit.client.0.smithi040.stdout:9/133: creat d3/d4/d11/d15/f24 x:0 0 0 2024-02-16T19:24:04.935 INFO:tasks.workunit.client.0.smithi040.stdout:9/134: dread - d3/f21 zero size 2024-02-16T19:24:04.938 INFO:tasks.workunit.client.0.smithi040.stdout:6/103: rmdir d0/d1 39 2024-02-16T19:24:04.938 INFO:tasks.workunit.client.0.smithi040.stdout:6/104: chown d0/d1 20373616 1 2024-02-16T19:24:04.938 INFO:tasks.workunit.client.0.smithi040.stdout:6/105: chown d0/d17/f1a 0 1 2024-02-16T19:24:04.940 INFO:tasks.workunit.client.0.smithi040.stdout:8/100: sync 2024-02-16T19:24:04.940 INFO:tasks.workunit.client.0.smithi040.stdout:0/109: sync 2024-02-16T19:24:04.940 INFO:tasks.workunit.client.0.smithi040.stdout:5/95: sync 2024-02-16T19:24:04.940 INFO:tasks.workunit.client.0.smithi040.stdout:7/88: sync 2024-02-16T19:24:04.940 INFO:tasks.workunit.client.0.smithi040.stdout:4/109: sync 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:7/89: chown fb 2653 1 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:8/101: read - d6/d7/f19 zero size 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:8/102: write d6/d7/d15/f18 [830027,55958] 0 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:8/103: read - d6/d7/f19 zero size 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:8/104: write f3 [5119944,34000] 0 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:9/135: mknod d3/c25 0 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:9/136: creat d3/d4/d11/d15/d18/f26 x:0 0 0 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:9/137: truncate d3/d4/d11/f19 4793832 0 2024-02-16T19:24:04.941 INFO:tasks.workunit.client.0.smithi040.stdout:9/138: truncate d3/d4/d11/d15/d18/f26 237471 0 2024-02-16T19:24:04.942 INFO:tasks.workunit.client.0.smithi040.stdout:9/139: chown d3/f21 67 1 2024-02-16T19:24:04.942 INFO:tasks.workunit.client.0.smithi040.stdout:9/140: read d3/f1f [178071,14651] 0 2024-02-16T19:24:04.949 INFO:tasks.workunit.client.0.smithi040.stdout:5/96: mkdir d1/da/d1e 0 2024-02-16T19:24:04.952 INFO:tasks.workunit.client.0.smithi040.stdout:7/90: mknod c1b 0 2024-02-16T19:24:04.952 INFO:tasks.workunit.client.0.smithi040.stdout:2/88: dwrite d5/f18 [0,4194304] 0 2024-02-16T19:24:04.953 INFO:tasks.workunit.client.0.smithi040.stdout:8/105: rename d6/lb to d6/l20 0 2024-02-16T19:24:04.953 INFO:tasks.workunit.client.0.smithi040.stdout:8/106: write f3 [6048747,39944] 0 2024-02-16T19:24:04.956 INFO:tasks.workunit.client.0.smithi040.stdout:9/141: symlink d3/d4/d11/d15/d18/l27 0 2024-02-16T19:24:04.960 INFO:tasks.workunit.client.0.smithi040.stdout:2/89: mknod d5/c1a 0 2024-02-16T19:24:04.963 INFO:tasks.workunit.client.0.smithi040.stdout:9/142: getdents d3 0 2024-02-16T19:24:04.963 INFO:tasks.workunit.client.0.smithi040.stdout:9/143: readlink l2 0 2024-02-16T19:24:04.963 INFO:tasks.workunit.client.0.smithi040.stdout:9/144: chown d3/f21 28 1 2024-02-16T19:24:04.968 INFO:tasks.workunit.client.0.smithi040.stdout:9/145: dread d3/d4/d11/f19 [0,4194304] 0 2024-02-16T19:24:04.968 INFO:tasks.workunit.client.0.smithi040.stdout:9/146: fdatasync d3/d4/d11/f19 0 2024-02-16T19:24:04.968 INFO:tasks.workunit.client.0.smithi040.stdout:9/147: write d3/d4/d11/f19 [344074,11921] 0 2024-02-16T19:24:04.969 INFO:tasks.workunit.client.0.smithi040.stdout:9/148: rename d3/d4/l20 to d3/d4/d11/d15/l28 0 2024-02-16T19:24:04.969 INFO:tasks.workunit.client.0.smithi040.stdout:9/149: dread - d3/f21 zero size 2024-02-16T19:24:04.974 INFO:tasks.workunit.client.0.smithi040.stdout:0/110: dwrite db/f1b [0,4194304] 0 2024-02-16T19:24:04.976 INFO:tasks.workunit.client.0.smithi040.stdout:0/111: fdatasync f9 0 2024-02-16T19:24:04.976 INFO:tasks.workunit.client.0.smithi040.stdout:0/112: write f9 [3284272,92999] 0 2024-02-16T19:24:04.977 INFO:tasks.workunit.client.0.smithi040.stdout:4/110: dwrite dd/f19 [0,4194304] 0 2024-02-16T19:24:04.977 INFO:tasks.workunit.client.0.smithi040.stdout:0/113: truncate f7 682753 0 2024-02-16T19:24:04.979 INFO:tasks.workunit.client.0.smithi040.stdout:6/106: dwrite d0/d1/f16 [0,4194304] 0 2024-02-16T19:24:04.979 INFO:tasks.workunit.client.0.smithi040.stdout:5/97: dwrite d1/f4 [0,4194304] 0 2024-02-16T19:24:04.981 INFO:tasks.workunit.client.0.smithi040.stdout:4/111: dread f7 [0,4194304] 0 2024-02-16T19:24:04.981 INFO:tasks.workunit.client.0.smithi040.stdout:0/114: unlink db/d16/l1c 0 2024-02-16T19:24:04.981 INFO:tasks.workunit.client.0.smithi040.stdout:0/115: write db/fd [834649,63979] 0 2024-02-16T19:24:04.983 INFO:tasks.workunit.client.0.smithi040.stdout:7/91: getdents . 0 2024-02-16T19:24:04.983 INFO:tasks.workunit.client.0.smithi040.stdout:7/92: stat f16 0 2024-02-16T19:24:04.983 INFO:tasks.workunit.client.0.smithi040.stdout:7/93: fsync fc 0 2024-02-16T19:24:04.983 INFO:tasks.workunit.client.0.smithi040.stdout:7/94: stat ff 0 2024-02-16T19:24:04.984 INFO:tasks.workunit.client.0.smithi040.stdout:0/116: fsync f9 0 2024-02-16T19:24:04.987 INFO:tasks.workunit.client.0.smithi040.stdout:6/107: creat d0/d1/f25 x:0 0 0 2024-02-16T19:24:04.993 INFO:tasks.workunit.client.0.smithi040.stdout:4/112: symlink dd/de/d10/l25 0 2024-02-16T19:24:04.998 INFO:tasks.workunit.client.0.smithi040.stdout:4/113: creat dd/de/d10/d13/f26 x:0 0 0 2024-02-16T19:24:04.999 INFO:tasks.workunit.client.0.smithi040.stdout:4/114: dread dd/de/d10/f1c [0,4194304] 0 2024-02-16T19:24:04.999 INFO:tasks.workunit.client.0.smithi040.stdout:4/115: readlink dd/de/l24 0 2024-02-16T19:24:04.999 INFO:tasks.workunit.client.0.smithi040.stdout:4/116: fdatasync dd/de/d10/f20 0 2024-02-16T19:24:05.002 INFO:tasks.workunit.client.0.smithi040.stdout:0/117: rmdir db/d16 39 2024-02-16T19:24:05.004 INFO:tasks.workunit.client.0.smithi040.stdout:0/118: mknod db/c1e 0 2024-02-16T19:24:05.005 INFO:tasks.workunit.client.0.smithi040.stdout:0/119: symlink db/de/l1f 0 2024-02-16T19:24:05.005 INFO:tasks.workunit.client.0.smithi040.stdout:0/120: creat db/f20 x:0 0 0 2024-02-16T19:24:05.006 INFO:tasks.workunit.client.0.smithi040.stdout:0/121: truncate f9 647838 0 2024-02-16T19:24:05.007 INFO:tasks.workunit.client.0.smithi040.stdout:0/122: mkdir db/d16/d1d/d21 0 2024-02-16T19:24:05.008 INFO:tasks.workunit.client.0.smithi040.stdout:3/128: sync 2024-02-16T19:24:05.008 INFO:tasks.workunit.client.0.smithi040.stdout:0/123: symlink db/d16/l22 0 2024-02-16T19:24:05.009 INFO:tasks.workunit.client.0.smithi040.stdout:3/129: mknod d1/d4/db/c26 0 2024-02-16T19:24:05.012 INFO:tasks.workunit.client.0.smithi040.stdout:5/98: dwrite d1/f1d [0,4194304] 0 2024-02-16T19:24:05.015 INFO:tasks.workunit.client.0.smithi040.stdout:6/108: dwrite d0/d1/f16 [0,4194304] 0 2024-02-16T19:24:05.015 INFO:tasks.workunit.client.0.smithi040.stdout:3/130: dread d1/f6 [4194304,4194304] 0 2024-02-16T19:24:05.015 INFO:tasks.workunit.client.0.smithi040.stdout:7/95: dwrite f10 [0,4194304] 0 2024-02-16T19:24:05.015 INFO:tasks.workunit.client.0.smithi040.stdout:3/131: chown d1/f23 17375877 1 2024-02-16T19:24:05.016 INFO:tasks.workunit.client.0.smithi040.stdout:3/132: write d1/dc/f1d [3206011,2952] 0 2024-02-16T19:24:05.016 INFO:tasks.workunit.client.0.smithi040.stdout:6/109: write d0/f5 [3943944,37918] 0 2024-02-16T19:24:05.016 INFO:tasks.workunit.client.0.smithi040.stdout:6/110: chown d0/d1/d12 0 1 2024-02-16T19:24:05.016 INFO:tasks.workunit.client.0.smithi040.stdout:6/111: chown d0/d17 205895 1 2024-02-16T19:24:05.016 INFO:tasks.workunit.client.0.smithi040.stdout:7/96: write fb [5069895,67258] 0 2024-02-16T19:24:05.016 INFO:tasks.workunit.client.0.smithi040.stdout:7/97: dread - f16 zero size 2024-02-16T19:24:05.016 INFO:tasks.workunit.client.0.smithi040.stdout:4/117: rmdir dd/de/d10/d13 39 2024-02-16T19:24:05.021 INFO:tasks.workunit.client.0.smithi040.stdout:3/133: mknod d1/dc/c27 0 2024-02-16T19:24:05.021 INFO:tasks.workunit.client.0.smithi040.stdout:3/134: creat d1/dc/f28 x:0 0 0 2024-02-16T19:24:05.022 INFO:tasks.workunit.client.0.smithi040.stdout:6/112: rename d0/d1/c3 to d0/d17/c26 0 2024-02-16T19:24:05.022 INFO:tasks.workunit.client.0.smithi040.stdout:6/113: chown d0/d1 21 1 2024-02-16T19:24:05.023 INFO:tasks.workunit.client.0.smithi040.stdout:4/118: unlink f7 0 2024-02-16T19:24:05.025 INFO:tasks.workunit.client.0.smithi040.stdout:7/98: read f9 [1943168,118038] 0 2024-02-16T19:24:05.025 INFO:tasks.workunit.client.0.smithi040.stdout:4/119: read dd/de/d10/f1c [445855,18270] 0 2024-02-16T19:24:05.025 INFO:tasks.workunit.client.0.smithi040.stdout:4/120: creat dd/f27 x:0 0 0 2024-02-16T19:24:05.025 INFO:tasks.workunit.client.0.smithi040.stdout:4/121: chown fb 236976659 1 2024-02-16T19:24:05.026 INFO:tasks.workunit.client.0.smithi040.stdout:6/114: symlink d0/d17/d24/l27 0 2024-02-16T19:24:05.027 INFO:tasks.workunit.client.0.smithi040.stdout:7/99: mknod c1c 0 2024-02-16T19:24:05.029 INFO:tasks.workunit.client.0.smithi040.stdout:4/122: dread dd/de/d10/d13/f15 [0,4194304] 0 2024-02-16T19:24:05.034 INFO:tasks.workunit.client.0.smithi040.stdout:0/124: dwrite db/ff [0,4194304] 0 2024-02-16T19:24:05.034 INFO:tasks.workunit.client.0.smithi040.stdout:0/125: fsync db/fd 0 2024-02-16T19:24:05.038 INFO:tasks.workunit.client.0.smithi040.stdout:0/126: unlink db/f11 0 2024-02-16T19:24:05.039 INFO:tasks.workunit.client.0.smithi040.stdout:0/127: chown db/cc 25630 1 2024-02-16T19:24:05.039 INFO:tasks.workunit.client.0.smithi040.stdout:0/128: readlink db/d16/d1d/l19 0 2024-02-16T19:24:05.041 INFO:tasks.workunit.client.0.smithi040.stdout:3/135: dwrite d1/d4/f19 [0,4194304] 0 2024-02-16T19:24:05.041 INFO:tasks.workunit.client.0.smithi040.stdout:3/136: creat d1/dc/f29 x:0 0 0 2024-02-16T19:24:05.042 INFO:tasks.workunit.client.0.smithi040.stdout:3/137: fdatasync d1/dc/f29 0 2024-02-16T19:24:05.042 INFO:tasks.workunit.client.0.smithi040.stdout:0/129: unlink db/ff 0 2024-02-16T19:24:05.042 INFO:tasks.workunit.client.0.smithi040.stdout:0/130: chown db/d16/c17 89996595 1 2024-02-16T19:24:05.042 INFO:tasks.workunit.client.0.smithi040.stdout:0/131: fdatasync f9 0 2024-02-16T19:24:05.043 INFO:tasks.workunit.client.0.smithi040.stdout:4/123: dread dd/f19 [0,4194304] 0 2024-02-16T19:24:05.043 INFO:tasks.workunit.client.0.smithi040.stdout:4/124: chown dd 1987 1 2024-02-16T19:24:05.043 INFO:tasks.workunit.client.0.smithi040.stdout:4/125: write fa [295571,121980] 0 2024-02-16T19:24:05.045 INFO:tasks.workunit.client.0.smithi040.stdout:5/99: dwrite d1/f17 [0,4194304] 0 2024-02-16T19:24:05.045 INFO:tasks.workunit.client.0.smithi040.stdout:5/100: stat d1/da 0 2024-02-16T19:24:05.046 INFO:tasks.workunit.client.0.smithi040.stdout:5/101: dread d1/f14 [0,4194304] 0 2024-02-16T19:24:05.049 INFO:tasks.workunit.client.0.smithi040.stdout:0/132: mkdir db/d16/d23 0 2024-02-16T19:24:05.049 INFO:tasks.workunit.client.0.smithi040.stdout:0/133: write f9 [1014817,95148] 0 2024-02-16T19:24:05.049 INFO:tasks.workunit.client.0.smithi040.stdout:0/134: truncate db/f20 612303 0 2024-02-16T19:24:05.050 INFO:tasks.workunit.client.0.smithi040.stdout:4/126: unlink dd/de/d10/d13/c17 0 2024-02-16T19:24:05.050 INFO:tasks.workunit.client.0.smithi040.stdout:4/127: write dd/f23 [656362,10321] 0 2024-02-16T19:24:05.055 INFO:tasks.workunit.client.0.smithi040.stdout:1/119: sync 2024-02-16T19:24:05.057 INFO:tasks.workunit.client.0.smithi040.stdout:4/128: mkdir dd/de/d10/d28 0 2024-02-16T19:24:05.058 INFO:tasks.workunit.client.0.smithi040.stdout:4/129: symlink dd/l29 0 2024-02-16T19:24:05.058 INFO:tasks.workunit.client.0.smithi040.stdout:4/130: stat dd/de/d10/d28 0 2024-02-16T19:24:05.059 INFO:tasks.workunit.client.0.smithi040.stdout:4/131: creat dd/de/f2a x:0 0 0 2024-02-16T19:24:05.060 INFO:tasks.workunit.client.0.smithi040.stdout:7/100: dwrite f10 [0,4194304] 0 2024-02-16T19:24:05.060 INFO:tasks.workunit.client.0.smithi040.stdout:7/101: read f17 [3178982,110622] 0 2024-02-16T19:24:05.067 INFO:tasks.workunit.client.0.smithi040.stdout:4/132: dread dd/f11 [0,4194304] 0 2024-02-16T19:24:05.067 INFO:tasks.workunit.client.0.smithi040.stdout:4/133: dread - dd/f27 zero size 2024-02-16T19:24:05.068 INFO:tasks.workunit.client.0.smithi040.stdout:4/134: mknod dd/de/d10/d13/c2b 0 2024-02-16T19:24:05.068 INFO:tasks.workunit.client.0.smithi040.stdout:4/135: fsync dd/de/f12 0 2024-02-16T19:24:05.068 INFO:tasks.workunit.client.0.smithi040.stdout:4/136: write dd/de/f12 [859401,18006] 0 2024-02-16T19:24:05.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/137: read - dd/de/d10/d13/f26 zero size 2024-02-16T19:24:05.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/138: rename dd/l29 to dd/de/d10/l2c 0 2024-02-16T19:24:05.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/139: chown dd 78932616 1 2024-02-16T19:24:05.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/140: dread - dd/de/f2a zero size 2024-02-16T19:24:05.074 INFO:tasks.workunit.client.0.smithi040.stdout:7/102: dread f10 [0,4194304] 0 2024-02-16T19:24:05.076 INFO:tasks.workunit.client.0.smithi040.stdout:6/115: dwrite d0/f4 [0,4194304] 0 2024-02-16T19:24:05.077 INFO:tasks.workunit.client.0.smithi040.stdout:3/138: truncate d1/d4/f19 861666 0 2024-02-16T19:24:05.077 INFO:tasks.workunit.client.0.smithi040.stdout:3/139: chown d1/dc/l24 304457312 1 2024-02-16T19:24:05.077 INFO:tasks.workunit.client.0.smithi040.stdout:3/140: dread - d1/f23 zero size 2024-02-16T19:24:05.077 INFO:tasks.workunit.client.0.smithi040.stdout:3/141: write d1/f6 [1130102,71624] 0 2024-02-16T19:24:05.077 INFO:tasks.workunit.client.0.smithi040.stdout:3/142: write d1/d7/f9 [101483,90761] 0 2024-02-16T19:24:05.077 INFO:tasks.workunit.client.0.smithi040.stdout:5/102: dwrite d1/da/f19 [0,4194304] 0 2024-02-16T19:24:05.077 INFO:tasks.workunit.client.0.smithi040.stdout:5/103: write d1/f4 [4236502,108277] 0 2024-02-16T19:24:05.077 INFO:tasks.workunit.client.0.smithi040.stdout:1/120: dwrite f16 [0,4194304] 0 2024-02-16T19:24:05.078 INFO:tasks.workunit.client.0.smithi040.stdout:4/141: dread dd/f23 [0,4194304] 0 2024-02-16T19:24:05.086 INFO:tasks.workunit.client.0.smithi040.stdout:4/142: mknod dd/de/d10/d28/c2d 0 2024-02-16T19:24:05.089 INFO:tasks.workunit.client.0.smithi040.stdout:4/143: rename dd/f11 to dd/de/d10/f2e 0 2024-02-16T19:24:05.091 INFO:tasks.workunit.client.0.smithi040.stdout:5/104: dread d1/f4 [0,4194304] 0 2024-02-16T19:24:05.091 INFO:tasks.workunit.client.0.smithi040.stdout:5/105: chown d1/f1d 26 1 2024-02-16T19:24:05.091 INFO:tasks.workunit.client.0.smithi040.stdout:5/106: write f0 [1043897,115887] 0 2024-02-16T19:24:05.093 INFO:tasks.workunit.client.0.smithi040.stdout:2/90: sync 2024-02-16T19:24:05.093 INFO:tasks.workunit.client.0.smithi040.stdout:9/150: sync 2024-02-16T19:24:05.094 INFO:tasks.workunit.client.0.smithi040.stdout:8/107: sync 2024-02-16T19:24:05.095 INFO:tasks.workunit.client.0.smithi040.stdout:2/91: mkdir d5/d7/d12/d1b 0 2024-02-16T19:24:05.100 INFO:tasks.workunit.client.0.smithi040.stdout:8/108: rename d6/l17 to d6/d7/l21 0 2024-02-16T19:24:05.101 INFO:tasks.workunit.client.0.smithi040.stdout:1/121: dwrite fb [0,4194304] 0 2024-02-16T19:24:05.101 INFO:tasks.workunit.client.0.smithi040.stdout:1/122: truncate d17/f28 888355 0 2024-02-16T19:24:05.104 INFO:tasks.workunit.client.0.smithi040.stdout:2/92: rename d5/d7/dd/lf to d5/d7/dd/l1c 0 2024-02-16T19:24:05.104 INFO:tasks.workunit.client.0.smithi040.stdout:2/93: chown d5/d7/d12/d1b 4 1 2024-02-16T19:24:05.104 INFO:tasks.workunit.client.0.smithi040.stdout:1/123: dread fd [0,4194304] 0 2024-02-16T19:24:05.105 INFO:tasks.workunit.client.0.smithi040.stdout:6/116: dwrite d0/d1/f25 [0,4194304] 0 2024-02-16T19:24:05.105 INFO:tasks.workunit.client.0.smithi040.stdout:8/109: mkdir d6/d12/d22 0 2024-02-16T19:24:05.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/151: rename d3/d4/d11/d15 to d3/d4/d11/d29 0 2024-02-16T19:24:05.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/152: getdents d3 0 2024-02-16T19:24:05.107 INFO:tasks.workunit.client.0.smithi040.stdout:8/110: truncate f4 6656207 0 2024-02-16T19:24:05.108 INFO:tasks.workunit.client.0.smithi040.stdout:8/111: chown d6/d7/c11 1687 1 2024-02-16T19:24:05.108 INFO:tasks.workunit.client.0.smithi040.stdout:6/117: read d0/f1d [691489,15692] 0 2024-02-16T19:24:05.108 INFO:tasks.workunit.client.0.smithi040.stdout:5/107: dread d1/f17 [0,4194304] 0 2024-02-16T19:24:05.108 INFO:tasks.workunit.client.0.smithi040.stdout:5/108: chown d1/l18 29 1 2024-02-16T19:24:05.108 INFO:tasks.workunit.client.0.smithi040.stdout:5/109: stat d1/da/ff 0 2024-02-16T19:24:05.109 INFO:tasks.workunit.client.0.smithi040.stdout:9/153: symlink d3/d4/d11/l2a 0 2024-02-16T19:24:05.109 INFO:tasks.workunit.client.0.smithi040.stdout:8/112: creat d6/d12/d22/f23 x:0 0 0 2024-02-16T19:24:05.109 INFO:tasks.workunit.client.0.smithi040.stdout:8/113: stat d6/d7/d15 0 2024-02-16T19:24:05.111 INFO:tasks.workunit.client.0.smithi040.stdout:9/154: link d3/f1f d3/d4/d11/d29/f2b 0 2024-02-16T19:24:05.111 INFO:tasks.workunit.client.0.smithi040.stdout:5/110: rmdir d1/da 39 2024-02-16T19:24:05.111 INFO:tasks.workunit.client.0.smithi040.stdout:5/111: readlink d1/l1b 0 2024-02-16T19:24:05.112 INFO:tasks.workunit.client.0.smithi040.stdout:5/112: chown d1/da/c1a 59400608 1 2024-02-16T19:24:05.112 INFO:tasks.workunit.client.0.smithi040.stdout:9/155: chown d3/d4/l1b 503013 1 2024-02-16T19:24:05.122 INFO:tasks.workunit.client.0.smithi040.stdout:2/94: write d5/f18 [3552236,22286] 0 2024-02-16T19:24:05.122 INFO:tasks.workunit.client.0.smithi040.stdout:2/95: chown d5/d7/ce 109411040 1 2024-02-16T19:24:05.122 INFO:tasks.workunit.client.0.smithi040.stdout:2/96: fdatasync d5/d7/d12/f15 0 2024-02-16T19:24:05.124 INFO:tasks.workunit.client.0.smithi040.stdout:2/97: mknod d5/d7/c1d 0 2024-02-16T19:24:05.126 INFO:tasks.workunit.client.0.smithi040.stdout:2/98: mkdir d5/d7/d1e 0 2024-02-16T19:24:05.127 INFO:tasks.workunit.client.0.smithi040.stdout:3/143: dwrite d1/d7/f18 [0,4194304] 0 2024-02-16T19:24:05.128 INFO:tasks.workunit.client.0.smithi040.stdout:4/144: dwrite dd/f19 [0,4194304] 0 2024-02-16T19:24:05.128 INFO:tasks.workunit.client.0.smithi040.stdout:4/145: fdatasync f0 0 2024-02-16T19:24:05.128 INFO:tasks.workunit.client.0.smithi040.stdout:8/114: write f4 [2091051,126928] 0 2024-02-16T19:24:05.130 INFO:tasks.workunit.client.0.smithi040.stdout:6/118: dwrite d0/d17/f1a [0,4194304] 0 2024-02-16T19:24:05.130 INFO:tasks.workunit.client.0.smithi040.stdout:5/113: dwrite d1/f13 [0,4194304] 0 2024-02-16T19:24:05.131 INFO:tasks.workunit.client.0.smithi040.stdout:4/146: link dd/de/f2a dd/de/f2f 0 2024-02-16T19:24:05.133 INFO:tasks.workunit.client.0.smithi040.stdout:9/156: rmdir d3/d4/d11 39 2024-02-16T19:24:05.133 INFO:tasks.workunit.client.0.smithi040.stdout:9/157: chown d3/d4/d11/l2a 3724 1 2024-02-16T19:24:05.138 INFO:tasks.workunit.client.0.smithi040.stdout:2/99: getdents d5/d7 0 2024-02-16T19:24:05.140 INFO:tasks.workunit.client.0.smithi040.stdout:5/114: rename d1/f7 to d1/da/d1c/f1f 0 2024-02-16T19:24:05.143 INFO:tasks.workunit.client.0.smithi040.stdout:2/100: creat d5/d8/f1f x:0 0 0 2024-02-16T19:24:05.144 INFO:tasks.workunit.client.0.smithi040.stdout:5/115: truncate d1/f1d 3904611 0 2024-02-16T19:24:05.145 INFO:tasks.workunit.client.0.smithi040.stdout:3/144: read d1/d4/f25 [3108636,108257] 0 2024-02-16T19:24:05.146 INFO:tasks.workunit.client.0.smithi040.stdout:9/158: link d3/f21 d3/d4/f2c 0 2024-02-16T19:24:05.146 INFO:tasks.workunit.client.0.smithi040.stdout:9/159: truncate d3/d4/f2c 296105 0 2024-02-16T19:24:05.147 INFO:tasks.workunit.client.0.smithi040.stdout:2/101: rename d5/f18 to d5/d7/d1e/f20 0 2024-02-16T19:24:05.147 INFO:tasks.workunit.client.0.smithi040.stdout:2/102: dread - d5/d7/f19 zero size 2024-02-16T19:24:05.148 INFO:tasks.workunit.client.0.smithi040.stdout:6/119: dread d0/d1/fa [0,4194304] 0 2024-02-16T19:24:05.149 INFO:tasks.workunit.client.0.smithi040.stdout:6/120: chown d0/d1/cc 1779538 1 2024-02-16T19:24:05.153 INFO:tasks.workunit.client.0.smithi040.stdout:8/115: dwrite f3 [4194304,4194304] 0 2024-02-16T19:24:05.153 INFO:tasks.workunit.client.0.smithi040.stdout:8/116: truncate d6/d12/f14 4275926 0 2024-02-16T19:24:05.153 INFO:tasks.workunit.client.0.smithi040.stdout:9/160: symlink d3/d4/d11/d29/l2d 0 2024-02-16T19:24:05.155 INFO:tasks.workunit.client.0.smithi040.stdout:3/145: mknod d1/d4/c2a 0 2024-02-16T19:24:05.157 INFO:tasks.workunit.client.0.smithi040.stdout:2/103: truncate d5/d7/d12/f13 2873394 0 2024-02-16T19:24:05.159 INFO:tasks.workunit.client.0.smithi040.stdout:6/121: rename d0/ff to d0/d1/d12/f28 0 2024-02-16T19:24:05.161 INFO:tasks.workunit.client.0.smithi040.stdout:8/117: creat d6/f24 x:0 0 0 2024-02-16T19:24:05.161 INFO:tasks.workunit.client.0.smithi040.stdout:9/161: creat d3/d4/d11/d29/d18/f2e x:0 0 0 2024-02-16T19:24:05.161 INFO:tasks.workunit.client.0.smithi040.stdout:9/162: write d3/d4/f2c [1052128,50363] 0 2024-02-16T19:24:05.163 INFO:tasks.workunit.client.0.smithi040.stdout:9/163: symlink d3/d4/d11/d29/d18/l2f 0 2024-02-16T19:24:05.164 INFO:tasks.workunit.client.0.smithi040.stdout:5/116: dwrite d1/da/d1c/f1f [4194304,4194304] 0 2024-02-16T19:24:05.164 INFO:tasks.workunit.client.0.smithi040.stdout:5/117: fdatasync d1/f4 0 2024-02-16T19:24:05.166 INFO:tasks.workunit.client.0.smithi040.stdout:9/164: dread d3/d4/d11/d29/d18/f26 [0,4194304] 0 2024-02-16T19:24:05.166 INFO:tasks.workunit.client.0.smithi040.stdout:9/165: dread - d3/d4/d11/d29/d18/f2e zero size 2024-02-16T19:24:05.166 INFO:tasks.workunit.client.0.smithi040.stdout:9/166: chown d3/fc 23986 1 2024-02-16T19:24:05.166 INFO:tasks.workunit.client.0.smithi040.stdout:9/167: chown d3/d4/d11/d29 1114510 1 2024-02-16T19:24:05.166 INFO:tasks.workunit.client.0.smithi040.stdout:9/168: write d3/d4/d11/f19 [4802628,108589] 0 2024-02-16T19:24:05.166 INFO:tasks.workunit.client.0.smithi040.stdout:5/118: unlink d1/l18 0 2024-02-16T19:24:05.167 INFO:tasks.workunit.client.0.smithi040.stdout:9/169: symlink d3/d4/d11/d29/l30 0 2024-02-16T19:24:05.167 INFO:tasks.workunit.client.0.smithi040.stdout:9/170: fdatasync d3/d4/d11/d29/d18/f23 0 2024-02-16T19:24:05.168 INFO:tasks.workunit.client.0.smithi040.stdout:5/119: unlink d1/f4 0 2024-02-16T19:24:05.170 INFO:tasks.workunit.client.0.smithi040.stdout:5/120: link f0 d1/f20 0 2024-02-16T19:24:05.172 INFO:tasks.workunit.client.0.smithi040.stdout:9/171: dread d3/f1f [0,4194304] 0 2024-02-16T19:24:05.172 INFO:tasks.workunit.client.0.smithi040.stdout:5/121: rmdir d1/da/d1e 0 2024-02-16T19:24:05.174 INFO:tasks.workunit.client.0.smithi040.stdout:5/122: mkdir d1/da/d1c/d21 0 2024-02-16T19:24:05.179 INFO:tasks.workunit.client.0.smithi040.stdout:3/146: dwrite d1/dc/fd [0,4194304] 0 2024-02-16T19:24:05.181 INFO:tasks.workunit.client.0.smithi040.stdout:3/147: truncate d1/d4/fa 40636 0 2024-02-16T19:24:05.181 INFO:tasks.workunit.client.0.smithi040.stdout:3/148: creat d1/d7/f2b x:0 0 0 2024-02-16T19:24:05.183 INFO:tasks.workunit.client.0.smithi040.stdout:3/149: rename d1/d4/db/c26 to d1/d4/db/c2c 0 2024-02-16T19:24:05.186 INFO:tasks.workunit.client.0.smithi040.stdout:9/172: dread d3/d4/d11/f19 [4194304,4194304] 0 2024-02-16T19:24:05.186 INFO:tasks.workunit.client.0.smithi040.stdout:8/118: dwrite d6/d7/f19 [0,4194304] 0 2024-02-16T19:24:05.187 INFO:tasks.workunit.client.0.smithi040.stdout:6/122: dwrite d0/d1/d12/f28 [0,4194304] 0 2024-02-16T19:24:05.187 INFO:tasks.workunit.client.0.smithi040.stdout:5/123: dread d1/fd [0,4194304] 0 2024-02-16T19:24:05.187 INFO:tasks.workunit.client.0.smithi040.stdout:5/124: truncate f0 1558286 0 2024-02-16T19:24:05.188 INFO:tasks.workunit.client.0.smithi040.stdout:5/125: rename d1/da/d1c to d1/da/d1c/d22 22 2024-02-16T19:24:05.188 INFO:tasks.workunit.client.0.smithi040.stdout:5/126: chown d1/f13 2 1 2024-02-16T19:24:05.190 INFO:tasks.workunit.client.0.smithi040.stdout:2/104: dread d5/d7/d12/f13 [0,4194304] 0 2024-02-16T19:24:05.193 INFO:tasks.workunit.client.0.smithi040.stdout:8/119: dread f2 [0,4194304] 0 2024-02-16T19:24:05.194 INFO:tasks.workunit.client.0.smithi040.stdout:8/120: fsync f2 0 2024-02-16T19:24:05.194 INFO:tasks.workunit.client.0.smithi040.stdout:8/121: write d6/f24 [440588,32517] 0 2024-02-16T19:24:05.203 INFO:tasks.workunit.client.0.smithi040.stdout:8/122: symlink d6/d7/d1d/l25 0 2024-02-16T19:24:05.205 INFO:tasks.workunit.client.0.smithi040.stdout:8/123: symlink d6/d7/l26 0 2024-02-16T19:24:05.205 INFO:tasks.workunit.client.0.smithi040.stdout:8/124: readlink d6/d7/l26 0 2024-02-16T19:24:05.205 INFO:tasks.workunit.client.0.smithi040.stdout:8/125: chown f2 19 1 2024-02-16T19:24:05.207 INFO:tasks.workunit.client.0.smithi040.stdout:8/126: stat d6/d7/l21 0 2024-02-16T19:24:05.207 INFO:tasks.workunit.client.0.smithi040.stdout:8/127: fdatasync d6/d7/d15/f1c 0 2024-02-16T19:24:05.209 INFO:tasks.workunit.client.0.smithi040.stdout:8/128: rename f2 to d6/d7/f27 0 2024-02-16T19:24:05.210 INFO:tasks.workunit.client.0.smithi040.stdout:8/129: write d6/d12/f14 [5272241,48855] 0 2024-02-16T19:24:05.210 INFO:tasks.workunit.client.0.smithi040.stdout:8/130: stat d6/d12 0 2024-02-16T19:24:05.213 INFO:tasks.workunit.client.0.smithi040.stdout:8/131: dread f5 [0,4194304] 0 2024-02-16T19:24:05.213 INFO:tasks.workunit.client.0.smithi040.stdout:8/132: readlink d6/lf 0 2024-02-16T19:24:05.230 INFO:tasks.workunit.client.0.smithi040.stdout:8/133: write d6/d7/f19 [4680523,35798] 0 2024-02-16T19:24:05.232 INFO:tasks.workunit.client.0.smithi040.stdout:8/134: getdents d6/d7/d15 0 2024-02-16T19:24:05.240 INFO:tasks.workunit.client.0.smithi040.stdout:3/150: dwrite d1/d7/f18 [0,4194304] 0 2024-02-16T19:24:05.240 INFO:tasks.workunit.client.0.smithi040.stdout:6/123: dwrite d0/d17/f21 [0,4194304] 0 2024-02-16T19:24:05.240 INFO:tasks.workunit.client.0.smithi040.stdout:9/173: dwrite d3/d4/f2c [0,4194304] 0 2024-02-16T19:24:05.240 INFO:tasks.workunit.client.0.smithi040.stdout:6/124: creat d0/d1/f29 x:0 0 0 2024-02-16T19:24:05.240 INFO:tasks.workunit.client.0.smithi040.stdout:9/174: stat d3/d4/d11 0 2024-02-16T19:24:05.242 INFO:tasks.workunit.client.0.smithi040.stdout:5/127: dwrite d1/f17 [0,4194304] 0 2024-02-16T19:24:05.243 INFO:tasks.workunit.client.0.smithi040.stdout:8/135: mknod d6/d7/d15/c28 0 2024-02-16T19:24:05.246 INFO:tasks.workunit.client.0.smithi040.stdout:0/135: sync 2024-02-16T19:24:05.246 INFO:tasks.workunit.client.0.smithi040.stdout:9/175: read d3/f21 [3283970,50160] 0 2024-02-16T19:24:05.250 INFO:tasks.workunit.client.0.smithi040.stdout:9/176: dread d3/d4/f2c [0,4194304] 0 2024-02-16T19:24:05.253 INFO:tasks.workunit.client.0.smithi040.stdout:8/136: dread f4 [0,4194304] 0 2024-02-16T19:24:05.253 INFO:tasks.workunit.client.0.smithi040.stdout:5/128: dread d1/f20 [0,4194304] 0 2024-02-16T19:24:05.254 INFO:tasks.workunit.client.0.smithi040.stdout:6/125: unlink d0/d1/ld 0 2024-02-16T19:24:05.254 INFO:tasks.workunit.client.0.smithi040.stdout:6/126: fsync d0/f1d 0 2024-02-16T19:24:05.256 INFO:tasks.workunit.client.0.smithi040.stdout:0/136: rmdir db/de 39 2024-02-16T19:24:05.258 INFO:tasks.workunit.client.0.smithi040.stdout:9/177: mkdir d3/d4/d11/d29/d18/d31 0 2024-02-16T19:24:05.260 INFO:tasks.workunit.client.0.smithi040.stdout:3/151: dwrite d1/dc/f29 [0,4194304] 0 2024-02-16T19:24:05.261 INFO:tasks.workunit.client.0.smithi040.stdout:8/137: rename d6/d7/f19 to d6/d7/d1d/f29 0 2024-02-16T19:24:05.261 INFO:tasks.workunit.client.0.smithi040.stdout:7/103: sync 2024-02-16T19:24:05.261 INFO:tasks.workunit.client.0.smithi040.stdout:4/147: sync 2024-02-16T19:24:05.261 INFO:tasks.workunit.client.0.smithi040.stdout:4/148: write dd/de/d10/f1b [3729548,53769] 0 2024-02-16T19:24:05.261 INFO:tasks.workunit.client.0.smithi040.stdout:4/149: stat dd/f27 0 2024-02-16T19:24:05.263 INFO:tasks.workunit.client.0.smithi040.stdout:0/137: link db/c15 db/d16/d23/c24 0 2024-02-16T19:24:05.263 INFO:tasks.workunit.client.0.smithi040.stdout:0/138: fdatasync db/f20 0 2024-02-16T19:24:05.264 INFO:tasks.workunit.client.0.smithi040.stdout:1/124: sync 2024-02-16T19:24:05.266 INFO:tasks.workunit.client.0.smithi040.stdout:9/178: mkdir d3/d4/d11/d32 0 2024-02-16T19:24:05.273 INFO:tasks.workunit.client.0.smithi040.stdout:4/150: link f0 dd/de/d10/f30 0 2024-02-16T19:24:05.273 INFO:tasks.workunit.client.0.smithi040.stdout:4/151: stat dd/de/d10/f1c 0 2024-02-16T19:24:05.273 INFO:tasks.workunit.client.0.smithi040.stdout:4/152: chown dd/f1e 29687 1 2024-02-16T19:24:05.273 INFO:tasks.workunit.client.0.smithi040.stdout:8/138: dread d6/f9 [0,4194304] 0 2024-02-16T19:24:05.276 INFO:tasks.workunit.client.0.smithi040.stdout:0/139: mkdir db/d25 0 2024-02-16T19:24:05.278 INFO:tasks.workunit.client.0.smithi040.stdout:1/125: rename d17/f1a to d17/d1d/f2a 0 2024-02-16T19:24:05.278 INFO:tasks.workunit.client.0.smithi040.stdout:1/126: readlink d17/l23 0 2024-02-16T19:24:05.280 INFO:tasks.workunit.client.0.smithi040.stdout:3/152: rmdir d1 39 2024-02-16T19:24:05.281 INFO:tasks.workunit.client.0.smithi040.stdout:4/153: mknod dd/de/d10/d28/c31 0 2024-02-16T19:24:05.281 INFO:tasks.workunit.client.0.smithi040.stdout:4/154: fdatasync dd/de/f1a 0 2024-02-16T19:24:05.283 INFO:tasks.workunit.client.0.smithi040.stdout:1/127: creat d17/d29/f2b x:0 0 0 2024-02-16T19:24:05.283 INFO:tasks.workunit.client.0.smithi040.stdout:1/128: getdents d17/d26 0 2024-02-16T19:24:05.283 INFO:tasks.workunit.client.0.smithi040.stdout:1/129: creat d17/f2c x:0 0 0 2024-02-16T19:24:05.283 INFO:tasks.workunit.client.0.smithi040.stdout:1/130: chown d17/d1d 18211558 1 2024-02-16T19:24:05.284 INFO:tasks.workunit.client.0.smithi040.stdout:3/153: getdents d1/dc 0 2024-02-16T19:24:05.288 INFO:tasks.workunit.client.0.smithi040.stdout:1/131: mkdir d17/d2d 0 2024-02-16T19:24:05.290 INFO:tasks.workunit.client.0.smithi040.stdout:1/132: creat d17/d2d/f2e x:0 0 0 2024-02-16T19:24:05.290 INFO:tasks.workunit.client.0.smithi040.stdout:1/133: chown d17/l18 9907861 1 2024-02-16T19:24:05.290 INFO:tasks.workunit.client.0.smithi040.stdout:1/134: creat d17/d2d/f2f x:0 0 0 2024-02-16T19:24:05.293 INFO:tasks.workunit.client.0.smithi040.stdout:1/135: truncate d17/f1f 4101330 0 2024-02-16T19:24:05.293 INFO:tasks.workunit.client.0.smithi040.stdout:1/136: chown d17/l1b 314 1 2024-02-16T19:24:05.293 INFO:tasks.workunit.client.0.smithi040.stdout:1/137: chown f16 74663109 1 2024-02-16T19:24:05.294 INFO:tasks.workunit.client.0.smithi040.stdout:1/138: dread - d17/d2d/f2e zero size 2024-02-16T19:24:05.299 INFO:tasks.workunit.client.0.smithi040.stdout:9/179: dwrite d3/fc [0,4194304] 0 2024-02-16T19:24:05.301 INFO:tasks.workunit.client.0.smithi040.stdout:3/154: dread d1/dc/f29 [0,4194304] 0 2024-02-16T19:24:05.301 INFO:tasks.workunit.client.0.smithi040.stdout:6/127: dwrite d0/d1/f29 [0,4194304] 0 2024-02-16T19:24:05.301 INFO:tasks.workunit.client.0.smithi040.stdout:3/155: chown d1/d4/c2a 5 1 2024-02-16T19:24:05.301 INFO:tasks.workunit.client.0.smithi040.stdout:9/180: mknod d3/d4/d11/c33 0 2024-02-16T19:24:05.301 INFO:tasks.workunit.client.0.smithi040.stdout:5/129: dwrite d1/da/ff [0,4194304] 0 2024-02-16T19:24:05.301 INFO:tasks.workunit.client.0.smithi040.stdout:5/130: rename d1/da/d1c/d21 to d1/da/d1c/d21/d23 22 2024-02-16T19:24:05.303 INFO:tasks.workunit.client.0.smithi040.stdout:3/156: unlink d1/f23 0 2024-02-16T19:24:05.303 INFO:tasks.workunit.client.0.smithi040.stdout:3/157: write d1/d4/db/f16 [840507,57301] 0 2024-02-16T19:24:05.303 INFO:tasks.workunit.client.0.smithi040.stdout:6/128: symlink d0/d17/d24/l2a 0 2024-02-16T19:24:05.304 INFO:tasks.workunit.client.0.smithi040.stdout:9/181: symlink d3/d4/d11/d29/d18/d31/l34 0 2024-02-16T19:24:05.304 INFO:tasks.workunit.client.0.smithi040.stdout:8/139: dwrite d6/f9 [0,4194304] 0 2024-02-16T19:24:05.304 INFO:tasks.workunit.client.0.smithi040.stdout:8/140: chown d6/d7/f27 817978 1 2024-02-16T19:24:05.305 INFO:tasks.workunit.client.0.smithi040.stdout:5/131: creat d1/f24 x:0 0 0 2024-02-16T19:24:05.306 INFO:tasks.workunit.client.0.smithi040.stdout:5/132: write d1/da/f19 [8761590,26765] 0 2024-02-16T19:24:05.306 INFO:tasks.workunit.client.0.smithi040.stdout:0/140: dwrite f7 [0,4194304] 0 2024-02-16T19:24:05.307 INFO:tasks.workunit.client.0.smithi040.stdout:3/158: rename d1/d4/db/c12 to d1/dc/c2d 0 2024-02-16T19:24:05.308 INFO:tasks.workunit.client.0.smithi040.stdout:6/129: mknod d0/d17/c2b 0 2024-02-16T19:24:05.309 INFO:tasks.workunit.client.0.smithi040.stdout:4/155: dwrite f0 [0,4194304] 0 2024-02-16T19:24:05.310 INFO:tasks.workunit.client.0.smithi040.stdout:5/133: unlink d1/da/c16 0 2024-02-16T19:24:05.312 INFO:tasks.workunit.client.0.smithi040.stdout:0/141: creat db/d25/f26 x:0 0 0 2024-02-16T19:24:05.313 INFO:tasks.workunit.client.0.smithi040.stdout:8/141: dread d6/f8 [0,4194304] 0 2024-02-16T19:24:05.313 INFO:tasks.workunit.client.0.smithi040.stdout:3/159: rename d1/d7/f18 to d1/d4/f2e 0 2024-02-16T19:24:05.315 INFO:tasks.workunit.client.0.smithi040.stdout:6/130: link d0/f5 d0/d1/d12/f2c 0 2024-02-16T19:24:05.315 INFO:tasks.workunit.client.0.smithi040.stdout:6/131: creat d0/d17/f2d x:0 0 0 2024-02-16T19:24:05.316 INFO:tasks.workunit.client.0.smithi040.stdout:1/139: fsync d17/d2d/f2f 0 2024-02-16T19:24:05.317 INFO:tasks.workunit.client.0.smithi040.stdout:4/156: mkdir dd/de/d10/d13/d32 0 2024-02-16T19:24:05.317 INFO:tasks.workunit.client.0.smithi040.stdout:4/157: truncate f8 366919 0 2024-02-16T19:24:05.318 INFO:tasks.workunit.client.0.smithi040.stdout:5/134: creat d1/da/d1c/f25 x:0 0 0 2024-02-16T19:24:05.319 INFO:tasks.workunit.client.0.smithi040.stdout:5/135: fdatasync d1/f17 0 2024-02-16T19:24:05.319 INFO:tasks.workunit.client.0.smithi040.stdout:0/142: getdents db/d16/d23 0 2024-02-16T19:24:05.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/142: symlink d6/d12/l2a 0 2024-02-16T19:24:05.320 INFO:tasks.workunit.client.0.smithi040.stdout:3/160: symlink d1/dc/l2f 0 2024-02-16T19:24:05.321 INFO:tasks.workunit.client.0.smithi040.stdout:0/143: dread f9 [0,4194304] 0 2024-02-16T19:24:05.321 INFO:tasks.workunit.client.0.smithi040.stdout:0/144: truncate db/d25/f26 67392 0 2024-02-16T19:24:05.322 INFO:tasks.workunit.client.0.smithi040.stdout:3/161: dread d1/d4/fa [0,4194304] 0 2024-02-16T19:24:05.329 INFO:tasks.workunit.client.0.smithi040.stdout:5/136: rename d1/f14 to d1/da/d1c/f26 0 2024-02-16T19:24:05.329 INFO:tasks.workunit.client.0.smithi040.stdout:5/137: truncate d1/da/d1c/f26 1877033 0 2024-02-16T19:24:05.329 INFO:tasks.workunit.client.0.smithi040.stdout:8/143: chown d6/d7/d15/l1b 7398841 1 2024-02-16T19:24:05.330 INFO:tasks.workunit.client.0.smithi040.stdout:0/145: rmdir db/de 39 2024-02-16T19:24:05.330 INFO:tasks.workunit.client.0.smithi040.stdout:3/162: dread d1/f6 [0,4194304] 0 2024-02-16T19:24:05.331 INFO:tasks.workunit.client.0.smithi040.stdout:1/140: truncate d17/f1f 3964248 0 2024-02-16T19:24:05.331 INFO:tasks.workunit.client.0.smithi040.stdout:1/141: chown d17/d2d/f2e 10 1 2024-02-16T19:24:05.332 INFO:tasks.workunit.client.0.smithi040.stdout:1/142: truncate d17/d2d/f2f 868810 0 2024-02-16T19:24:05.332 INFO:tasks.workunit.client.0.smithi040.stdout:1/143: chown d17/f2c 0 1 2024-02-16T19:24:05.332 INFO:tasks.workunit.client.0.smithi040.stdout:1/144: readlink d17/l20 0 2024-02-16T19:24:05.332 INFO:tasks.workunit.client.0.smithi040.stdout:1/145: chown d17/l23 2153 1 2024-02-16T19:24:05.332 INFO:tasks.workunit.client.0.smithi040.stdout:1/146: stat d17/d2d 0 2024-02-16T19:24:05.334 INFO:tasks.workunit.client.0.smithi040.stdout:6/132: fsync d0/d1/d12/f2c 0 2024-02-16T19:24:05.335 INFO:tasks.workunit.client.0.smithi040.stdout:8/144: mkdir d6/d2b 0 2024-02-16T19:24:05.336 INFO:tasks.workunit.client.0.smithi040.stdout:8/145: fdatasync d6/f24 0 2024-02-16T19:24:05.336 INFO:tasks.workunit.client.0.smithi040.stdout:4/158: rmdir dd/de 39 2024-02-16T19:24:05.337 INFO:tasks.workunit.client.0.smithi040.stdout:3/163: rmdir d1/dc 39 2024-02-16T19:24:05.338 INFO:tasks.workunit.client.0.smithi040.stdout:8/146: dread d6/d12/f14 [4194304,4194304] 0 2024-02-16T19:24:05.338 INFO:tasks.workunit.client.0.smithi040.stdout:1/147: mknod d17/d2d/c30 0 2024-02-16T19:24:05.339 INFO:tasks.workunit.client.0.smithi040.stdout:6/133: truncate d0/d1/fa 3389432 0 2024-02-16T19:24:05.340 INFO:tasks.workunit.client.0.smithi040.stdout:4/159: link dd/de/d10/l21 dd/de/l33 0 2024-02-16T19:24:05.342 INFO:tasks.workunit.client.0.smithi040.stdout:0/146: truncate f7 3551265 0 2024-02-16T19:24:05.342 INFO:tasks.workunit.client.0.smithi040.stdout:2/105: sync 2024-02-16T19:24:05.343 INFO:tasks.workunit.client.0.smithi040.stdout:8/147: mknod d6/d12/c2c 0 2024-02-16T19:24:05.343 INFO:tasks.workunit.client.0.smithi040.stdout:8/148: fsync f5 0 2024-02-16T19:24:05.344 INFO:tasks.workunit.client.0.smithi040.stdout:1/148: mknod d17/d2d/c31 0 2024-02-16T19:24:05.345 INFO:tasks.workunit.client.0.smithi040.stdout:4/160: dread dd/de/d10/f2e [0,4194304] 0 2024-02-16T19:24:05.346 INFO:tasks.workunit.client.0.smithi040.stdout:4/161: chown fa 1 1 2024-02-16T19:24:05.346 INFO:tasks.workunit.client.0.smithi040.stdout:4/162: creat dd/f34 x:0 0 0 2024-02-16T19:24:05.346 INFO:tasks.workunit.client.0.smithi040.stdout:4/163: stat dd/de/d10/f1b 0 2024-02-16T19:24:05.346 INFO:tasks.workunit.client.0.smithi040.stdout:0/147: link db/d16/c17 db/de/c27 0 2024-02-16T19:24:05.347 INFO:tasks.workunit.client.0.smithi040.stdout:2/106: symlink d5/d7/d10/l21 0 2024-02-16T19:24:05.347 INFO:tasks.workunit.client.0.smithi040.stdout:2/107: readlink d5/d7/d10/l21 0 2024-02-16T19:24:05.347 INFO:tasks.workunit.client.0.smithi040.stdout:5/138: fsync f0 0 2024-02-16T19:24:05.351 INFO:tasks.workunit.client.0.smithi040.stdout:4/164: mknod dd/de/d10/d13/d32/c35 0 2024-02-16T19:24:05.351 INFO:tasks.workunit.client.0.smithi040.stdout:4/165: dread - dd/de/d10/f20 zero size 2024-02-16T19:24:05.353 INFO:tasks.workunit.client.0.smithi040.stdout:0/148: mknod db/d16/d1d/c28 0 2024-02-16T19:24:05.355 INFO:tasks.workunit.client.0.smithi040.stdout:1/149: write f1 [3830809,130372] 0 2024-02-16T19:24:05.355 INFO:tasks.workunit.client.0.smithi040.stdout:1/150: chown l10 1324 1 2024-02-16T19:24:05.355 INFO:tasks.workunit.client.0.smithi040.stdout:1/151: write f14 [380791,82081] 0 2024-02-16T19:24:05.355 INFO:tasks.workunit.client.0.smithi040.stdout:1/152: read d17/f25 [2200908,14130] 0 2024-02-16T19:24:05.355 INFO:tasks.workunit.client.0.smithi040.stdout:2/108: link d5/d7/d10/c17 d5/c22 0 2024-02-16T19:24:05.356 INFO:tasks.workunit.client.0.smithi040.stdout:2/109: write d5/d7/d12/f15 [4520695,77317] 0 2024-02-16T19:24:05.359 INFO:tasks.workunit.client.0.smithi040.stdout:5/139: dread d1/da/d1c/f1f [4194304,4194304] 0 2024-02-16T19:24:05.359 INFO:tasks.workunit.client.0.smithi040.stdout:5/140: rename d1/da/d1c/d21 to d1/da/d1c/d21/d27 22 2024-02-16T19:24:05.359 INFO:tasks.workunit.client.0.smithi040.stdout:5/141: chown d1/da/f19 103 1 2024-02-16T19:24:05.359 INFO:tasks.workunit.client.0.smithi040.stdout:4/166: mkdir dd/de/d10/d36 0 2024-02-16T19:24:05.361 INFO:tasks.workunit.client.0.smithi040.stdout:0/149: symlink db/d16/d23/l29 0 2024-02-16T19:24:05.361 INFO:tasks.workunit.client.0.smithi040.stdout:3/164: dwrite d1/f3 [0,4194304] 0 2024-02-16T19:24:05.363 INFO:tasks.workunit.client.0.smithi040.stdout:2/110: creat d5/d7/dd/f23 x:0 0 0 2024-02-16T19:24:05.363 INFO:tasks.workunit.client.0.smithi040.stdout:2/111: fdatasync d5/d8/f1f 0 2024-02-16T19:24:05.364 INFO:tasks.workunit.client.0.smithi040.stdout:2/112: write d5/d7/f19 [562236,7286] 0 2024-02-16T19:24:05.364 INFO:tasks.workunit.client.0.smithi040.stdout:6/134: dwrite d0/d1/f20 [0,4194304] 0 2024-02-16T19:24:05.364 INFO:tasks.workunit.client.0.smithi040.stdout:6/135: chown d0/d17/c2b 0 1 2024-02-16T19:24:05.366 INFO:tasks.workunit.client.0.smithi040.stdout:4/167: mkdir dd/de/d10/d28/d37 0 2024-02-16T19:24:05.369 INFO:tasks.workunit.client.0.smithi040.stdout:3/165: getdents d1/dc 0 2024-02-16T19:24:05.370 INFO:tasks.workunit.client.0.smithi040.stdout:3/166: dread d1/d4/db/f16 [0,4194304] 0 2024-02-16T19:24:05.376 INFO:tasks.workunit.client.0.smithi040.stdout:5/142: rmdir d1 39 2024-02-16T19:24:05.376 INFO:tasks.workunit.client.0.smithi040.stdout:5/143: chown d1/da/c10 217515 1 2024-02-16T19:24:05.377 INFO:tasks.workunit.client.0.smithi040.stdout:0/150: truncate f3 4905442 0 2024-02-16T19:24:05.378 INFO:tasks.workunit.client.0.smithi040.stdout:3/167: dread d1/dc/fd [0,4194304] 0 2024-02-16T19:24:05.378 INFO:tasks.workunit.client.0.smithi040.stdout:6/136: dread d0/d1/f25 [0,4194304] 0 2024-02-16T19:24:05.378 INFO:tasks.workunit.client.0.smithi040.stdout:6/137: creat d0/f2e x:0 0 0 2024-02-16T19:24:05.378 INFO:tasks.workunit.client.0.smithi040.stdout:2/113: truncate d5/d7/d12/f15 433735 0 2024-02-16T19:24:05.378 INFO:tasks.workunit.client.0.smithi040.stdout:2/114: write d5/d8/f1f [530813,30371] 0 2024-02-16T19:24:05.379 INFO:tasks.workunit.client.0.smithi040.stdout:4/168: write dd/f19 [2729533,61837] 0 2024-02-16T19:24:05.379 INFO:tasks.workunit.client.0.smithi040.stdout:4/169: readlink dd/de/d10/l21 0 2024-02-16T19:24:05.379 INFO:tasks.workunit.client.0.smithi040.stdout:4/170: write dd/de/f1a [248226,43845] 0 2024-02-16T19:24:05.381 INFO:tasks.workunit.client.0.smithi040.stdout:0/151: link db/d16/d1d/l19 db/l2a 0 2024-02-16T19:24:05.381 INFO:tasks.workunit.client.0.smithi040.stdout:2/115: dread d5/d7/d12/f13 [0,4194304] 0 2024-02-16T19:24:05.382 INFO:tasks.workunit.client.0.smithi040.stdout:3/168: unlink d1/dc/f28 0 2024-02-16T19:24:05.382 INFO:tasks.workunit.client.0.smithi040.stdout:3/169: readlink d1/dc/l1c 0 2024-02-16T19:24:05.383 INFO:tasks.workunit.client.0.smithi040.stdout:6/138: symlink d0/d1/d12/l2f 0 2024-02-16T19:24:05.385 INFO:tasks.workunit.client.0.smithi040.stdout:4/171: rename dd/de/d10/f1c to dd/de/d10/d13/f38 0 2024-02-16T19:24:05.385 INFO:tasks.workunit.client.0.smithi040.stdout:4/172: fdatasync fb 0 2024-02-16T19:24:05.386 INFO:tasks.workunit.client.0.smithi040.stdout:4/173: chown dd/de/d10/f20 907690 1 2024-02-16T19:24:05.386 INFO:tasks.workunit.client.0.smithi040.stdout:4/174: chown dd/de 10 1 2024-02-16T19:24:05.389 INFO:tasks.workunit.client.0.smithi040.stdout:3/170: mkdir d1/d7/d30 0 2024-02-16T19:24:05.389 INFO:tasks.workunit.client.0.smithi040.stdout:0/152: read db/f1b [456395,39415] 0 2024-02-16T19:24:05.390 INFO:tasks.workunit.client.0.smithi040.stdout:6/139: mkdir d0/d30 0 2024-02-16T19:24:05.392 INFO:tasks.workunit.client.0.smithi040.stdout:0/153: unlink db/d16/d1d/c1a 0 2024-02-16T19:24:05.392 INFO:tasks.workunit.client.0.smithi040.stdout:0/154: creat db/f2b x:0 0 0 2024-02-16T19:24:05.393 INFO:tasks.workunit.client.0.smithi040.stdout:2/116: dread d5/d7/d12/f15 [0,4194304] 0 2024-02-16T19:24:05.394 INFO:tasks.workunit.client.0.smithi040.stdout:0/155: mkdir db/d25/d2c 0 2024-02-16T19:24:05.394 INFO:tasks.workunit.client.0.smithi040.stdout:0/156: write db/fd [915972,3199] 0 2024-02-16T19:24:05.397 INFO:tasks.workunit.client.0.smithi040.stdout:2/117: rename d5/d7/dd/f23 to d5/d7/f24 0 2024-02-16T19:24:05.399 INFO:tasks.workunit.client.0.smithi040.stdout:4/175: dread f0 [0,4194304] 0 2024-02-16T19:24:05.399 INFO:tasks.workunit.client.0.smithi040.stdout:4/176: chown dd/f34 1720 1 2024-02-16T19:24:05.399 INFO:tasks.workunit.client.0.smithi040.stdout:7/104: sync 2024-02-16T19:24:05.400 INFO:tasks.workunit.client.0.smithi040.stdout:7/105: write ff [2263505,66530] 0 2024-02-16T19:24:05.400 INFO:tasks.workunit.client.0.smithi040.stdout:7/106: fdatasync f10 0 2024-02-16T19:24:05.400 INFO:tasks.workunit.client.0.smithi040.stdout:7/107: rmdir - no directory 2024-02-16T19:24:05.400 INFO:tasks.workunit.client.0.smithi040.stdout:0/157: symlink db/l2d 0 2024-02-16T19:24:05.402 INFO:tasks.workunit.client.0.smithi040.stdout:2/118: link d5/d7/d12/f15 d5/d7/d1e/f25 0 2024-02-16T19:24:05.403 INFO:tasks.workunit.client.0.smithi040.stdout:2/119: write d5/d7/d1e/f25 [260186,58643] 0 2024-02-16T19:24:05.404 INFO:tasks.workunit.client.0.smithi040.stdout:4/177: mknod dd/de/d10/d28/c39 0 2024-02-16T19:24:05.405 INFO:tasks.workunit.client.0.smithi040.stdout:7/108: creat f1d x:0 0 0 2024-02-16T19:24:05.409 INFO:tasks.workunit.client.0.smithi040.stdout:3/171: dwrite d1/d4/fa [0,4194304] 0 2024-02-16T19:24:05.417 INFO:tasks.workunit.client.0.smithi040.stdout:3/172: write d1/f3 [2191930,98659] 0 2024-02-16T19:24:05.417 INFO:tasks.workunit.client.0.smithi040.stdout:3/173: write d1/d7/f9 [6161252,62844] 0 2024-02-16T19:24:05.421 INFO:tasks.workunit.client.0.smithi040.stdout:3/174: write d1/d4/f19 [584860,48002] 0 2024-02-16T19:24:05.421 INFO:tasks.workunit.client.0.smithi040.stdout:3/175: write d1/d7/f2b [151008,116946] 0 2024-02-16T19:24:05.421 INFO:tasks.workunit.client.0.smithi040.stdout:3/176: chown d1/c13 1404 1 2024-02-16T19:24:05.423 INFO:tasks.workunit.client.0.smithi040.stdout:2/120: dwrite d5/d7/d12/f13 [0,4194304] 0 2024-02-16T19:24:05.424 INFO:tasks.workunit.client.0.smithi040.stdout:0/158: dwrite db/d25/f26 [0,4194304] 0 2024-02-16T19:24:05.425 INFO:tasks.workunit.client.0.smithi040.stdout:2/121: getdents d5/d7/dd 0 2024-02-16T19:24:05.425 INFO:tasks.workunit.client.0.smithi040.stdout:2/122: fdatasync d5/d7/f19 0 2024-02-16T19:24:05.429 INFO:tasks.workunit.client.0.smithi040.stdout:7/109: getdents . 0 2024-02-16T19:24:05.430 INFO:tasks.workunit.client.0.smithi040.stdout:7/110: creat f1e x:0 0 0 2024-02-16T19:24:05.431 INFO:tasks.workunit.client.0.smithi040.stdout:7/111: chown f15 27210 1 2024-02-16T19:24:05.431 INFO:tasks.workunit.client.0.smithi040.stdout:7/112: truncate f1d 528182 0 2024-02-16T19:24:05.431 INFO:tasks.workunit.client.0.smithi040.stdout:2/123: dread d5/d7/d1e/f20 [0,4194304] 0 2024-02-16T19:24:05.432 INFO:tasks.workunit.client.0.smithi040.stdout:0/159: write db/d25/f26 [4975311,119124] 0 2024-02-16T19:24:05.433 INFO:tasks.workunit.client.0.smithi040.stdout:0/160: truncate db/f1b 3318111 0 2024-02-16T19:24:05.437 INFO:tasks.workunit.client.0.smithi040.stdout:2/124: dread d5/d7/d1e/f20 [0,4194304] 0 2024-02-16T19:24:05.440 INFO:tasks.workunit.client.0.smithi040.stdout:0/161: dread f7 [0,4194304] 0 2024-02-16T19:24:05.441 INFO:tasks.workunit.client.0.smithi040.stdout:0/162: mkdir db/d16/d1d/d21/d2e 0 2024-02-16T19:24:05.441 INFO:tasks.workunit.client.0.smithi040.stdout:0/163: truncate f3 5157394 0 2024-02-16T19:24:05.441 INFO:tasks.workunit.client.0.smithi040.stdout:0/164: fdatasync db/fd 0 2024-02-16T19:24:05.534 INFO:tasks.workunit.client.0.smithi040.stdout:3/177: dwrite d1/f6 [0,4194304] 0 2024-02-16T19:24:05.534 INFO:tasks.workunit.client.0.smithi040.stdout:7/113: dread ff [0,4194304] 0 2024-02-16T19:24:05.535 INFO:tasks.workunit.client.0.smithi040.stdout:3/178: symlink d1/d7/d30/l31 0 2024-02-16T19:24:05.535 INFO:tasks.workunit.client.0.smithi040.stdout:3/179: creat d1/f32 x:0 0 0 2024-02-16T19:24:05.536 INFO:tasks.workunit.client.0.smithi040.stdout:3/180: stat d1/d4/db/l22 0 2024-02-16T19:24:05.539 INFO:tasks.workunit.client.0.smithi040.stdout:8/149: sync 2024-02-16T19:24:05.539 INFO:tasks.workunit.client.0.smithi040.stdout:9/182: sync 2024-02-16T19:24:05.539 INFO:tasks.workunit.client.0.smithi040.stdout:1/153: sync 2024-02-16T19:24:05.540 INFO:tasks.workunit.client.0.smithi040.stdout:0/165: dread db/d25/f26 [0,4194304] 0 2024-02-16T19:24:05.540 INFO:tasks.workunit.client.0.smithi040.stdout:0/166: write f9 [1992467,93642] 0 2024-02-16T19:24:05.541 INFO:tasks.workunit.client.0.smithi040.stdout:0/167: stat db/d25/f26 0 2024-02-16T19:24:05.541 INFO:tasks.workunit.client.0.smithi040.stdout:8/150: truncate d6/d12/f14 1700036 0 2024-02-16T19:24:05.543 INFO:tasks.workunit.client.0.smithi040.stdout:0/168: rename db/d16 to db/d2f 0 2024-02-16T19:24:05.543 INFO:tasks.workunit.client.0.smithi040.stdout:0/169: stat db/f20 0 2024-02-16T19:24:05.543 INFO:tasks.workunit.client.0.smithi040.stdout:3/181: dread d1/d4/f19 [0,4194304] 0 2024-02-16T19:24:05.544 INFO:tasks.workunit.client.0.smithi040.stdout:0/170: mknod db/d2f/d1d/d21/c30 0 2024-02-16T19:24:05.546 INFO:tasks.workunit.client.0.smithi040.stdout:3/182: rename d1/d4/db to d1/d7/d33 0 2024-02-16T19:24:05.546 INFO:tasks.workunit.client.0.smithi040.stdout:3/183: write d1/f32 [216433,120972] 0 2024-02-16T19:24:05.549 INFO:tasks.workunit.client.0.smithi040.stdout:1/154: write d17/f1f [3523064,85889] 0 2024-02-16T19:24:05.549 INFO:tasks.workunit.client.0.smithi040.stdout:1/155: creat d17/d29/f32 x:0 0 0 2024-02-16T19:24:05.550 INFO:tasks.workunit.client.0.smithi040.stdout:7/114: dwrite f1d [0,4194304] 0 2024-02-16T19:24:05.550 INFO:tasks.workunit.client.0.smithi040.stdout:7/115: write f9 [4645724,117487] 0 2024-02-16T19:24:05.582 INFO:tasks.workunit.client.0.smithi040.stdout:9/183: dwrite d3/d4/d11/d29/f24 [0,4194304] 0 2024-02-16T19:24:05.584 INFO:tasks.workunit.client.0.smithi040.stdout:1/156: dwrite d17/f28 [0,4194304] 0 2024-02-16T19:24:05.584 INFO:tasks.workunit.client.0.smithi040.stdout:9/184: creat d3/d4/d11/d29/d18/d31/f35 x:0 0 0 2024-02-16T19:24:05.586 INFO:tasks.workunit.client.0.smithi040.stdout:9/185: mkdir d3/d4/d11/d29/d36 0 2024-02-16T19:24:05.587 INFO:tasks.workunit.client.0.smithi040.stdout:9/186: symlink d3/l37 0 2024-02-16T19:24:05.587 INFO:tasks.workunit.client.0.smithi040.stdout:9/187: readlink d3/d4/d11/d29/l28 0 2024-02-16T19:24:05.587 INFO:tasks.workunit.client.0.smithi040.stdout:9/188: mkdir d3/d4/d11/d29/d38 0 2024-02-16T19:24:05.587 INFO:tasks.workunit.client.0.smithi040.stdout:9/189: creat d3/d4/d11/d29/d18/f39 x:0 0 0 2024-02-16T19:24:05.592 INFO:tasks.workunit.client.0.smithi040.stdout:0/171: dwrite db/f2b [0,4194304] 0 2024-02-16T19:24:05.593 INFO:tasks.workunit.client.0.smithi040.stdout:8/151: dwrite f4 [0,4194304] 0 2024-02-16T19:24:05.593 INFO:tasks.workunit.client.0.smithi040.stdout:3/184: dwrite d1/d4/f2e [4194304,4194304] 0 2024-02-16T19:24:05.593 INFO:tasks.workunit.client.0.smithi040.stdout:3/185: creat d1/f34 x:0 0 0 2024-02-16T19:24:05.593 INFO:tasks.workunit.client.0.smithi040.stdout:7/116: dwrite f10 [0,4194304] 0 2024-02-16T19:24:05.595 INFO:tasks.workunit.client.0.smithi040.stdout:1/157: write fd [395957,2747] 0 2024-02-16T19:24:05.595 INFO:tasks.workunit.client.0.smithi040.stdout:1/158: chown c15 126953960 1 2024-02-16T19:24:05.595 INFO:tasks.workunit.client.0.smithi040.stdout:1/159: creat d17/d2d/f33 x:0 0 0 2024-02-16T19:24:05.597 INFO:tasks.workunit.client.0.smithi040.stdout:0/172: symlink db/d2f/d1d/l31 0 2024-02-16T19:24:05.598 INFO:tasks.workunit.client.0.smithi040.stdout:8/152: mknod d6/d7/d1d/c2d 0 2024-02-16T19:24:05.598 INFO:tasks.workunit.client.0.smithi040.stdout:8/153: fdatasync d6/d12/f1f 0 2024-02-16T19:24:05.609 INFO:tasks.workunit.client.0.smithi040.stdout:0/173: link db/f1b db/d25/f32 0 2024-02-16T19:24:05.609 INFO:tasks.workunit.client.0.smithi040.stdout:0/174: fsync f3 0 2024-02-16T19:24:05.612 INFO:tasks.workunit.client.0.smithi040.stdout:1/160: write f5 [987865,12750] 0 2024-02-16T19:24:05.612 INFO:tasks.workunit.client.0.smithi040.stdout:1/161: chown d17/d2d/c31 1715658033 1 2024-02-16T19:24:05.613 INFO:tasks.workunit.client.0.smithi040.stdout:0/175: dread db/d25/f26 [4194304,4194304] 0 2024-02-16T19:24:05.613 INFO:tasks.workunit.client.0.smithi040.stdout:1/162: dread d17/d2d/f2f [0,4194304] 0 2024-02-16T19:24:05.615 INFO:tasks.workunit.client.0.smithi040.stdout:0/176: creat db/d2f/d23/f33 x:0 0 0 2024-02-16T19:24:05.615 INFO:tasks.workunit.client.0.smithi040.stdout:0/177: stat f9 0 2024-02-16T19:24:05.616 INFO:tasks.workunit.client.0.smithi040.stdout:0/178: dread db/f20 [0,4194304] 0 2024-02-16T19:24:05.616 INFO:tasks.workunit.client.0.smithi040.stdout:0/179: write db/d25/f26 [5951004,128291] 0 2024-02-16T19:24:05.617 INFO:tasks.workunit.client.0.smithi040.stdout:9/190: dwrite d3/d4/d11/d29/f2b [0,4194304] 0 2024-02-16T19:24:05.620 INFO:tasks.workunit.client.0.smithi040.stdout:9/191: truncate d3/d4/f8 3654169 0 2024-02-16T19:24:05.620 INFO:tasks.workunit.client.0.smithi040.stdout:9/192: dread - d3/d4/d11/d29/d18/f2e zero size 2024-02-16T19:24:05.621 INFO:tasks.workunit.client.0.smithi040.stdout:9/193: creat d3/d4/d11/d29/d36/f3a x:0 0 0 2024-02-16T19:24:05.622 INFO:tasks.workunit.client.0.smithi040.stdout:9/194: rmdir d3/d4/d11/d29 39 2024-02-16T19:24:05.624 INFO:tasks.workunit.client.0.smithi040.stdout:3/186: dwrite d1/f6 [0,4194304] 0 2024-02-16T19:24:05.625 INFO:tasks.workunit.client.0.smithi040.stdout:3/187: unlink d1/f6 0 2024-02-16T19:24:05.626 INFO:tasks.workunit.client.0.smithi040.stdout:3/188: read d1/d7/f2b [228875,107474] 0 2024-02-16T19:24:05.626 INFO:tasks.workunit.client.0.smithi040.stdout:3/189: chown d1/d7/c21 798 1 2024-02-16T19:24:05.629 INFO:tasks.workunit.client.0.smithi040.stdout:3/190: dread d1/d7/f2b [0,4194304] 0 2024-02-16T19:24:05.630 INFO:tasks.workunit.client.0.smithi040.stdout:3/191: mkdir d1/dc/d35 0 2024-02-16T19:24:05.630 INFO:tasks.workunit.client.0.smithi040.stdout:3/192: truncate d1/d4/fa 402795 0 2024-02-16T19:24:05.631 INFO:tasks.workunit.client.0.smithi040.stdout:8/154: dwrite d6/f8 [0,4194304] 0 2024-02-16T19:24:05.632 INFO:tasks.workunit.client.0.smithi040.stdout:3/193: mkdir d1/d7/d36 0 2024-02-16T19:24:05.632 INFO:tasks.workunit.client.0.smithi040.stdout:3/194: creat d1/f37 x:0 0 0 2024-02-16T19:24:05.632 INFO:tasks.workunit.client.0.smithi040.stdout:7/117: dwrite f9 [0,4194304] 0 2024-02-16T19:24:05.632 INFO:tasks.workunit.client.0.smithi040.stdout:7/118: truncate f1e 343966 0 2024-02-16T19:24:05.632 INFO:tasks.workunit.client.0.smithi040.stdout:7/119: readlink l19 0 2024-02-16T19:24:05.633 INFO:tasks.workunit.client.0.smithi040.stdout:1/163: dwrite f16 [0,4194304] 0 2024-02-16T19:24:05.638 INFO:tasks.workunit.client.0.smithi040.stdout:8/155: getdents d6 0 2024-02-16T19:24:05.638 INFO:tasks.workunit.client.0.smithi040.stdout:8/156: chown c1 309276 1 2024-02-16T19:24:05.642 INFO:tasks.workunit.client.0.smithi040.stdout:6/140: sync 2024-02-16T19:24:05.642 INFO:tasks.workunit.client.0.smithi040.stdout:4/178: sync 2024-02-16T19:24:05.643 INFO:tasks.workunit.client.0.smithi040.stdout:5/144: sync 2024-02-16T19:24:05.643 INFO:tasks.workunit.client.0.smithi040.stdout:2/125: sync 2024-02-16T19:24:05.643 INFO:tasks.workunit.client.0.smithi040.stdout:4/179: write dd/f1e [1575694,21468] 0 2024-02-16T19:24:05.643 INFO:tasks.workunit.client.0.smithi040.stdout:4/180: dread - dd/f34 zero size 2024-02-16T19:24:05.646 INFO:tasks.workunit.client.0.smithi040.stdout:9/195: write d3/d4/d11/d29/f2b [4365036,127521] 0 2024-02-16T19:24:05.650 INFO:tasks.workunit.client.0.smithi040.stdout:8/157: creat d6/d7/f2e x:0 0 0 2024-02-16T19:24:05.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/141: rename d0/d1/f20 to d0/d1/d12/f31 0 2024-02-16T19:24:05.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/142: truncate d0/f2e 511825 0 2024-02-16T19:24:05.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/143: chown d0/d17/f1a 662453063 1 2024-02-16T19:24:05.654 INFO:tasks.workunit.client.0.smithi040.stdout:4/181: getdents dd/de/d10/d13/d32 0 2024-02-16T19:24:05.658 INFO:tasks.workunit.client.0.smithi040.stdout:0/180: dwrite db/d25/f26 [0,4194304] 0 2024-02-16T19:24:05.659 INFO:tasks.workunit.client.0.smithi040.stdout:8/158: creat d6/d7/d15/f2f x:0 0 0 2024-02-16T19:24:05.659 INFO:tasks.workunit.client.0.smithi040.stdout:8/159: chown d6/c16 675715 1 2024-02-16T19:24:05.660 INFO:tasks.workunit.client.0.smithi040.stdout:5/145: rename d1/c8 to d1/da/d1c/d21/c28 0 2024-02-16T19:24:05.661 INFO:tasks.workunit.client.0.smithi040.stdout:5/146: stat d1/f24 0 2024-02-16T19:24:05.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/182: mknod dd/de/d10/d28/d37/c3a 0 2024-02-16T19:24:05.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/183: truncate dd/de/d10/d13/f26 354619 0 2024-02-16T19:24:05.666 INFO:tasks.workunit.client.0.smithi040.stdout:0/181: mkdir db/d25/d34 0 2024-02-16T19:24:05.666 INFO:tasks.workunit.client.0.smithi040.stdout:0/182: creat db/f35 x:0 0 0 2024-02-16T19:24:05.666 INFO:tasks.workunit.client.0.smithi040.stdout:0/183: write db/f35 [918101,40283] 0 2024-02-16T19:24:05.666 INFO:tasks.workunit.client.0.smithi040.stdout:1/164: dwrite d17/d2d/f2e [0,4194304] 0 2024-02-16T19:24:05.667 INFO:tasks.workunit.client.0.smithi040.stdout:1/165: readlink d17/l20 0 2024-02-16T19:24:05.667 INFO:tasks.workunit.client.0.smithi040.stdout:1/166: write f9 [2211199,44472] 0 2024-02-16T19:24:05.667 INFO:tasks.workunit.client.0.smithi040.stdout:8/160: write d6/d7/d15/f1c [3333046,59093] 0 2024-02-16T19:24:05.668 INFO:tasks.workunit.client.0.smithi040.stdout:5/147: rmdir d1 39 2024-02-16T19:24:05.668 INFO:tasks.workunit.client.0.smithi040.stdout:5/148: truncate d1/da/f19 9644947 0 2024-02-16T19:24:05.669 INFO:tasks.workunit.client.0.smithi040.stdout:4/184: creat dd/de/d10/d28/d37/f3b x:0 0 0 2024-02-16T19:24:05.672 INFO:tasks.workunit.client.0.smithi040.stdout:1/167: link l10 d17/l34 0 2024-02-16T19:24:05.672 INFO:tasks.workunit.client.0.smithi040.stdout:1/168: readlink d17/l20 0 2024-02-16T19:24:05.672 INFO:tasks.workunit.client.0.smithi040.stdout:1/169: stat d17/l23 0 2024-02-16T19:24:05.672 INFO:tasks.workunit.client.0.smithi040.stdout:1/170: chown f16 285703 1 2024-02-16T19:24:05.673 INFO:tasks.workunit.client.0.smithi040.stdout:9/196: dwrite d3/fc [0,4194304] 0 2024-02-16T19:24:05.673 INFO:tasks.workunit.client.0.smithi040.stdout:9/197: creat d3/d4/d11/d29/d18/d31/f3b x:0 0 0 2024-02-16T19:24:05.673 INFO:tasks.workunit.client.0.smithi040.stdout:8/161: truncate d6/d12/f14 1016328 0 2024-02-16T19:24:05.673 INFO:tasks.workunit.client.0.smithi040.stdout:3/195: dwrite d1/d4/fa [0,4194304] 0 2024-02-16T19:24:05.678 INFO:tasks.workunit.client.0.smithi040.stdout:8/162: mkdir d6/d2b/d30 0 2024-02-16T19:24:05.678 INFO:tasks.workunit.client.0.smithi040.stdout:8/163: fsync d6/d7/d15/f1c 0 2024-02-16T19:24:05.679 INFO:tasks.workunit.client.0.smithi040.stdout:3/196: mkdir d1/d7/d33/d38 0 2024-02-16T19:24:05.679 INFO:tasks.workunit.client.0.smithi040.stdout:3/197: fsync d1/f3 0 2024-02-16T19:24:05.685 INFO:tasks.workunit.client.0.smithi040.stdout:6/144: rename d0/d17 to d0/d30/d32 0 2024-02-16T19:24:05.688 INFO:tasks.workunit.client.0.smithi040.stdout:3/198: unlink d1/d7/d33/f16 0 2024-02-16T19:24:05.689 INFO:tasks.workunit.client.0.smithi040.stdout:4/185: rename dd/f34 to dd/de/d10/f3c 0 2024-02-16T19:24:05.689 INFO:tasks.workunit.client.0.smithi040.stdout:4/186: write dd/de/d10/f3c [810766,51782] 0 2024-02-16T19:24:05.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/145: truncate d0/fe 1209795 0 2024-02-16T19:24:05.691 INFO:tasks.workunit.client.0.smithi040.stdout:3/199: rmdir d1/d7/d30 39 2024-02-16T19:24:05.694 INFO:tasks.workunit.client.0.smithi040.stdout:9/198: rename d3/d4/d11/d29/d18/f26 to d3/f3c 0 2024-02-16T19:24:05.695 INFO:tasks.workunit.client.0.smithi040.stdout:4/187: symlink dd/de/d10/l3d 0 2024-02-16T19:24:05.695 INFO:tasks.workunit.client.0.smithi040.stdout:4/188: stat dd/de 0 2024-02-16T19:24:05.697 INFO:tasks.workunit.client.0.smithi040.stdout:3/200: symlink d1/dc/d35/l39 0 2024-02-16T19:24:05.697 INFO:tasks.workunit.client.0.smithi040.stdout:3/201: stat d1/dc/c27 0 2024-02-16T19:24:05.697 INFO:tasks.workunit.client.0.smithi040.stdout:3/202: truncate d1/f32 797447 0 2024-02-16T19:24:05.697 INFO:tasks.workunit.client.0.smithi040.stdout:3/203: truncate d1/f34 1023473 0 2024-02-16T19:24:05.697 INFO:tasks.workunit.client.0.smithi040.stdout:3/204: fsync d1/dc/fd 0 2024-02-16T19:24:05.698 INFO:tasks.workunit.client.0.smithi040.stdout:3/205: mknod d1/d7/d36/c3a 0 2024-02-16T19:24:05.698 INFO:tasks.workunit.client.0.smithi040.stdout:3/206: stat d1/dc/d35 0 2024-02-16T19:24:05.701 INFO:tasks.workunit.client.0.smithi040.stdout:0/184: dwrite db/f35 [0,4194304] 0 2024-02-16T19:24:05.702 INFO:tasks.workunit.client.0.smithi040.stdout:1/171: dwrite f16 [0,4194304] 0 2024-02-16T19:24:05.702 INFO:tasks.workunit.client.0.smithi040.stdout:3/207: mknod d1/d7/d33/c3b 0 2024-02-16T19:24:05.704 INFO:tasks.workunit.client.0.smithi040.stdout:6/146: dread d0/d1/f16 [0,4194304] 0 2024-02-16T19:24:05.705 INFO:tasks.workunit.client.0.smithi040.stdout:0/185: mknod db/de/c36 0 2024-02-16T19:24:05.709 INFO:tasks.workunit.client.0.smithi040.stdout:8/164: dwrite d6/d7/f27 [4194304,4194304] 0 2024-02-16T19:24:05.709 INFO:tasks.workunit.client.0.smithi040.stdout:3/208: creat d1/dc/f3c x:0 0 0 2024-02-16T19:24:05.709 INFO:tasks.workunit.client.0.smithi040.stdout:9/199: dwrite d3/f3c [0,4194304] 0 2024-02-16T19:24:05.710 INFO:tasks.workunit.client.0.smithi040.stdout:9/200: readlink d3/d4/d11/d29/l30 0 2024-02-16T19:24:05.710 INFO:tasks.workunit.client.0.smithi040.stdout:9/201: write d3/d4/d11/d29/d18/d31/f3b [531373,45489] 0 2024-02-16T19:24:05.710 INFO:tasks.workunit.client.0.smithi040.stdout:1/172: dread d17/f25 [0,4194304] 0 2024-02-16T19:24:05.710 INFO:tasks.workunit.client.0.smithi040.stdout:1/173: truncate fd 4722264 0 2024-02-16T19:24:05.710 INFO:tasks.workunit.client.0.smithi040.stdout:1/174: read - d17/d29/f2b zero size 2024-02-16T19:24:05.711 INFO:tasks.workunit.client.0.smithi040.stdout:0/186: unlink db/d2f/d1d/d21/c30 0 2024-02-16T19:24:05.712 INFO:tasks.workunit.client.0.smithi040.stdout:3/209: mknod d1/c3d 0 2024-02-16T19:24:05.715 INFO:tasks.workunit.client.0.smithi040.stdout:1/175: symlink d17/d29/l35 0 2024-02-16T19:24:05.715 INFO:tasks.workunit.client.0.smithi040.stdout:1/176: creat d17/f36 x:0 0 0 2024-02-16T19:24:05.715 INFO:tasks.workunit.client.0.smithi040.stdout:1/177: fsync d17/f25 0 2024-02-16T19:24:05.716 INFO:tasks.workunit.client.0.smithi040.stdout:0/187: mknod db/d2f/d23/c37 0 2024-02-16T19:24:05.717 INFO:tasks.workunit.client.0.smithi040.stdout:3/210: mknod d1/d7/d33/c3e 0 2024-02-16T19:24:05.718 INFO:tasks.workunit.client.0.smithi040.stdout:3/211: chown d1/d7/c21 757 1 2024-02-16T19:24:05.718 INFO:tasks.workunit.client.0.smithi040.stdout:8/165: dread d6/d7/d1d/f29 [0,4194304] 0 2024-02-16T19:24:05.718 INFO:tasks.workunit.client.0.smithi040.stdout:8/166: read - d6/d7/f2e zero size 2024-02-16T19:24:05.720 INFO:tasks.workunit.client.0.smithi040.stdout:1/178: unlink cf 0 2024-02-16T19:24:05.720 INFO:tasks.workunit.client.0.smithi040.stdout:1/179: dread - d17/d29/f2b zero size 2024-02-16T19:24:05.721 INFO:tasks.workunit.client.0.smithi040.stdout:1/180: write d17/f25 [4272307,11383] 0 2024-02-16T19:24:05.721 INFO:tasks.workunit.client.0.smithi040.stdout:1/181: creat d17/f37 x:0 0 0 2024-02-16T19:24:05.725 INFO:tasks.workunit.client.0.smithi040.stdout:0/188: creat db/de/f38 x:0 0 0 2024-02-16T19:24:05.725 INFO:tasks.workunit.client.0.smithi040.stdout:6/147: dwrite d0/f4 [0,4194304] 0 2024-02-16T19:24:05.726 INFO:tasks.workunit.client.0.smithi040.stdout:8/167: write d6/d7/d15/f18 [1077065,41901] 0 2024-02-16T19:24:05.732 INFO:tasks.workunit.client.0.smithi040.stdout:8/168: truncate f5 3758032 0 2024-02-16T19:24:05.732 INFO:tasks.workunit.client.0.smithi040.stdout:8/169: creat d6/d12/f31 x:0 0 0 2024-02-16T19:24:05.732 INFO:tasks.workunit.client.0.smithi040.stdout:8/170: creat d6/d7/f32 x:0 0 0 2024-02-16T19:24:05.733 INFO:tasks.workunit.client.0.smithi040.stdout:0/189: getdents db/d25 0 2024-02-16T19:24:05.735 INFO:tasks.workunit.client.0.smithi040.stdout:8/171: creat d6/f33 x:0 0 0 2024-02-16T19:24:05.735 INFO:tasks.workunit.client.0.smithi040.stdout:8/172: getdents d6/d7/d1d 0 2024-02-16T19:24:05.736 INFO:tasks.workunit.client.0.smithi040.stdout:0/190: write f7 [2569352,13737] 0 2024-02-16T19:24:05.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:05 smithi163 ceph-mon[36452]: pgmap v241: 65 pgs: 65 active+clean; 340 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 7.5 MiB/s rd, 28 MiB/s wr, 1.05k op/s 2024-02-16T19:24:05.884 INFO:tasks.workunit.client.0.smithi040.stdout:0/191: symlink db/d25/d2c/l39 0 2024-02-16T19:24:05.884 INFO:tasks.workunit.client.0.smithi040.stdout:0/192: readlink db/l2d 0 2024-02-16T19:24:05.886 INFO:tasks.workunit.client.0.smithi040.stdout:0/193: creat db/d25/d34/f3a x:0 0 0 2024-02-16T19:24:05.889 INFO:tasks.workunit.client.0.smithi040.stdout:0/194: dread f9 [0,4194304] 0 2024-02-16T19:24:05.890 INFO:tasks.workunit.client.0.smithi040.stdout:0/195: write db/f20 [160621,9557] 0 2024-02-16T19:24:05.890 INFO:tasks.workunit.client.0.smithi040.stdout:0/196: chown db/d25/f26 110 1 2024-02-16T19:24:05.890 INFO:tasks.workunit.client.0.smithi040.stdout:3/212: dwrite d1/dc/f29 [0,4194304] 0 2024-02-16T19:24:05.893 INFO:tasks.workunit.client.0.smithi040.stdout:6/148: dread d0/d1/d12/f31 [0,4194304] 0 2024-02-16T19:24:05.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:05 smithi040 ceph-mon[29201]: pgmap v241: 65 pgs: 65 active+clean; 340 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 7.5 MiB/s rd, 28 MiB/s wr, 1.05k op/s 2024-02-16T19:24:05.897 INFO:tasks.workunit.client.0.smithi040.stdout:6/149: dread d0/d1/d12/f31 [0,4194304] 0 2024-02-16T19:24:05.903 INFO:tasks.workunit.client.0.smithi040.stdout:0/197: mkdir db/d2f/d23/d3b 0 2024-02-16T19:24:05.910 INFO:tasks.workunit.client.0.smithi040.stdout:1/182: getdents d17 0 2024-02-16T19:24:05.914 INFO:tasks.workunit.client.0.smithi040.stdout:8/173: rmdir d6/d7 39 2024-02-16T19:24:05.914 INFO:tasks.workunit.client.0.smithi040.stdout:8/174: chown d6/d7 107 1 2024-02-16T19:24:05.915 INFO:tasks.workunit.client.0.smithi040.stdout:6/150: dwrite d0/d30/d32/f21 [0,4194304] 0 2024-02-16T19:24:05.920 INFO:tasks.workunit.client.0.smithi040.stdout:3/213: creat d1/d7/d33/d38/f3f x:0 0 0 2024-02-16T19:24:05.921 INFO:tasks.workunit.client.0.smithi040.stdout:0/198: unlink f3 0 2024-02-16T19:24:05.922 INFO:tasks.workunit.client.0.smithi040.stdout:8/175: creat d6/d2b/f34 x:0 0 0 2024-02-16T19:24:05.922 INFO:tasks.workunit.client.0.smithi040.stdout:8/176: write d6/d12/d22/f23 [287796,57174] 0 2024-02-16T19:24:05.923 INFO:tasks.workunit.client.0.smithi040.stdout:8/177: chown d6/f33 125 1 2024-02-16T19:24:05.928 INFO:tasks.workunit.client.0.smithi040.stdout:3/214: getdents d1/d7/d33 0 2024-02-16T19:24:05.931 INFO:tasks.workunit.client.0.smithi040.stdout:3/215: creat d1/d7/d30/f40 x:0 0 0 2024-02-16T19:24:05.931 INFO:tasks.workunit.client.0.smithi040.stdout:3/216: readlink d1/dc/l1c 0 2024-02-16T19:24:05.933 INFO:tasks.workunit.client.0.smithi040.stdout:7/120: sync 2024-02-16T19:24:05.933 INFO:tasks.workunit.client.0.smithi040.stdout:7/121: chown l14 65327 1 2024-02-16T19:24:05.944 INFO:tasks.workunit.client.0.smithi040.stdout:5/149: sync 2024-02-16T19:24:05.944 INFO:tasks.workunit.client.0.smithi040.stdout:9/202: sync 2024-02-16T19:24:05.944 INFO:tasks.workunit.client.0.smithi040.stdout:2/126: sync 2024-02-16T19:24:05.944 INFO:tasks.workunit.client.0.smithi040.stdout:9/203: chown d3/fc 11964603 1 2024-02-16T19:24:05.944 INFO:tasks.workunit.client.0.smithi040.stdout:4/189: sync 2024-02-16T19:24:05.944 INFO:tasks.workunit.client.0.smithi040.stdout:9/204: dread - d3/d4/d11/d29/d18/f39 zero size 2024-02-16T19:24:05.944 INFO:tasks.workunit.client.0.smithi040.stdout:9/205: write d3/d4/d11/d29/d18/f23 [442157,8626] 0 2024-02-16T19:24:05.944 INFO:tasks.workunit.client.0.smithi040.stdout:9/206: dread - d3/d4/d11/d29/d18/d31/f35 zero size 2024-02-16T19:24:05.945 INFO:tasks.workunit.client.0.smithi040.stdout:5/150: creat d1/da/f29 x:0 0 0 2024-02-16T19:24:05.945 INFO:tasks.workunit.client.0.smithi040.stdout:7/122: write fa [2485280,82795] 0 2024-02-16T19:24:05.945 INFO:tasks.workunit.client.0.smithi040.stdout:5/151: fdatasync d1/da/d1c/f25 0 2024-02-16T19:24:05.945 INFO:tasks.workunit.client.0.smithi040.stdout:7/123: creat f1f x:0 0 0 2024-02-16T19:24:05.947 INFO:tasks.workunit.client.0.smithi040.stdout:4/190: rename dd/l22 to dd/de/d10/d28/l3e 0 2024-02-16T19:24:05.947 INFO:tasks.workunit.client.0.smithi040.stdout:4/191: truncate dd/de/f2a 734903 0 2024-02-16T19:24:05.949 INFO:tasks.workunit.client.0.smithi040.stdout:4/192: mknod dd/de/d10/d13/d32/c3f 0 2024-02-16T19:24:05.949 INFO:tasks.workunit.client.0.smithi040.stdout:2/127: dread d5/d8/f1f [0,4194304] 0 2024-02-16T19:24:05.949 INFO:tasks.workunit.client.0.smithi040.stdout:2/128: write d5/d8/f1f [649370,91417] 0 2024-02-16T19:24:05.949 INFO:tasks.workunit.client.0.smithi040.stdout:2/129: fdatasync d5/d7/d12/f15 0 2024-02-16T19:24:05.950 INFO:tasks.workunit.client.0.smithi040.stdout:1/183: dwrite d17/d2d/f2f [0,4194304] 0 2024-02-16T19:24:05.950 INFO:tasks.workunit.client.0.smithi040.stdout:4/193: link dd/de/d10/d28/d37/c3a dd/de/d10/d13/d32/c40 0 2024-02-16T19:24:05.952 INFO:tasks.workunit.client.0.smithi040.stdout:2/130: link c0 d5/d7/c26 0 2024-02-16T19:24:05.953 INFO:tasks.workunit.client.0.smithi040.stdout:1/184: mkdir d17/d1d/d38 0 2024-02-16T19:24:05.956 INFO:tasks.workunit.client.0.smithi040.stdout:4/194: creat dd/de/d10/d13/d32/f41 x:0 0 0 2024-02-16T19:24:05.957 INFO:tasks.workunit.client.0.smithi040.stdout:2/131: truncate d5/d7/d12/f13 685083 0 2024-02-16T19:24:05.959 INFO:tasks.workunit.client.0.smithi040.stdout:1/185: unlink f9 0 2024-02-16T19:24:05.960 INFO:tasks.workunit.client.0.smithi040.stdout:4/195: mknod dd/de/d10/d13/c42 0 2024-02-16T19:24:05.960 INFO:tasks.workunit.client.0.smithi040.stdout:4/196: fdatasync dd/de/f2a 0 2024-02-16T19:24:05.962 INFO:tasks.workunit.client.0.smithi040.stdout:2/132: rename c2 to d5/d7/dd/c27 0 2024-02-16T19:24:05.962 INFO:tasks.workunit.client.0.smithi040.stdout:1/186: creat d17/d1d/f39 x:0 0 0 2024-02-16T19:24:05.964 INFO:tasks.workunit.client.0.smithi040.stdout:2/133: rename d5/c1a to d5/d7/c28 0 2024-02-16T19:24:05.964 INFO:tasks.workunit.client.0.smithi040.stdout:2/134: chown d5/c22 24844 1 2024-02-16T19:24:05.964 INFO:tasks.workunit.client.0.smithi040.stdout:6/151: dwrite d0/d1/fa [0,4194304] 0 2024-02-16T19:24:05.964 INFO:tasks.workunit.client.0.smithi040.stdout:0/199: dwrite db/de/f38 [0,4194304] 0 2024-02-16T19:24:05.968 INFO:tasks.workunit.client.0.smithi040.stdout:6/152: rename d0/d1/f25 to d0/f33 0 2024-02-16T19:24:05.970 INFO:tasks.workunit.client.0.smithi040.stdout:0/200: dread db/d25/f26 [4194304,4194304] 0 2024-02-16T19:24:05.972 INFO:tasks.workunit.client.0.smithi040.stdout:6/153: rename d0/d30/d32/d24 to d0/d34 0 2024-02-16T19:24:05.972 INFO:tasks.workunit.client.0.smithi040.stdout:6/154: chown d0/d30/d32/f21 12195729 1 2024-02-16T19:24:05.974 INFO:tasks.workunit.client.0.smithi040.stdout:0/201: creat db/d2f/d1d/f3c x:0 0 0 2024-02-16T19:24:05.974 INFO:tasks.workunit.client.0.smithi040.stdout:0/202: write db/fd [3529098,102975] 0 2024-02-16T19:24:05.974 INFO:tasks.workunit.client.0.smithi040.stdout:8/178: dwrite d6/f9 [0,4194304] 0 2024-02-16T19:24:05.974 INFO:tasks.workunit.client.0.smithi040.stdout:8/179: truncate d6/d7/d15/f2f 278179 0 2024-02-16T19:24:05.974 INFO:tasks.workunit.client.0.smithi040.stdout:8/180: read - d6/d7/f32 zero size 2024-02-16T19:24:05.976 INFO:tasks.workunit.client.0.smithi040.stdout:0/203: link db/d2f/l22 db/d2f/l3d 0 2024-02-16T19:24:05.978 INFO:tasks.workunit.client.0.smithi040.stdout:6/155: dread d0/f5 [0,4194304] 0 2024-02-16T19:24:05.982 INFO:tasks.workunit.client.0.smithi040.stdout:1/187: dread f16 [0,4194304] 0 2024-02-16T19:24:05.984 INFO:tasks.workunit.client.0.smithi040.stdout:1/188: symlink d17/l3a 0 2024-02-16T19:24:05.987 INFO:tasks.workunit.client.0.smithi040.stdout:3/217: dwrite d1/f32 [0,4194304] 0 2024-02-16T19:24:05.993 INFO:tasks.workunit.client.0.smithi040.stdout:8/181: write d6/f8 [3198723,86398] 0 2024-02-16T19:24:05.996 INFO:tasks.workunit.client.0.smithi040.stdout:9/207: dwrite d3/f1f [0,4194304] 0 2024-02-16T19:24:05.996 INFO:tasks.workunit.client.0.smithi040.stdout:9/208: write d3/f1f [5321558,120051] 0 2024-02-16T19:24:06.000 INFO:tasks.workunit.client.0.smithi040.stdout:5/152: dwrite d1/f20 [0,4194304] 0 2024-02-16T19:24:06.001 INFO:tasks.workunit.client.0.smithi040.stdout:8/182: rename d6/d7/l21 to d6/l35 0 2024-02-16T19:24:06.003 INFO:tasks.workunit.client.0.smithi040.stdout:3/218: dread d1/dc/f29 [0,4194304] 0 2024-02-16T19:24:06.003 INFO:tasks.workunit.client.0.smithi040.stdout:9/209: creat d3/f3d x:0 0 0 2024-02-16T19:24:06.007 INFO:tasks.workunit.client.0.smithi040.stdout:8/183: mkdir d6/d2b/d36 0 2024-02-16T19:24:06.007 INFO:tasks.workunit.client.0.smithi040.stdout:9/210: truncate d3/f3c 1877749 0 2024-02-16T19:24:06.011 INFO:tasks.workunit.client.0.smithi040.stdout:5/153: dread d1/f20 [0,4194304] 0 2024-02-16T19:24:06.011 INFO:tasks.workunit.client.0.smithi040.stdout:5/154: creat d1/da/f2a x:0 0 0 2024-02-16T19:24:06.015 INFO:tasks.workunit.client.0.smithi040.stdout:3/219: dread d1/f32 [0,4194304] 0 2024-02-16T19:24:06.015 INFO:tasks.workunit.client.0.smithi040.stdout:3/220: read - d1/d7/d30/f40 zero size 2024-02-16T19:24:06.016 INFO:tasks.workunit.client.0.smithi040.stdout:3/221: write d1/d7/d30/f40 [336790,72157] 0 2024-02-16T19:24:06.016 INFO:tasks.workunit.client.0.smithi040.stdout:4/197: dwrite dd/de/d10/d28/d37/f3b [0,4194304] 0 2024-02-16T19:24:06.016 INFO:tasks.workunit.client.0.smithi040.stdout:4/198: creat dd/f43 x:0 0 0 2024-02-16T19:24:06.017 INFO:tasks.workunit.client.0.smithi040.stdout:2/135: dwrite d5/d7/d1e/f25 [0,4194304] 0 2024-02-16T19:24:06.017 INFO:tasks.workunit.client.0.smithi040.stdout:2/136: chown d5/d7/dd/l1c 802825469 1 2024-02-16T19:24:06.017 INFO:tasks.workunit.client.0.smithi040.stdout:2/137: chown d5/d7/d10/l21 156374966 1 2024-02-16T19:24:06.017 INFO:tasks.workunit.client.0.smithi040.stdout:2/138: readlink d5/d7/dd/l1c 0 2024-02-16T19:24:06.020 INFO:tasks.workunit.client.0.smithi040.stdout:6/156: dwrite d0/d1/f16 [0,4194304] 0 2024-02-16T19:24:06.021 INFO:tasks.workunit.client.0.smithi040.stdout:1/189: dwrite d17/f37 [0,4194304] 0 2024-02-16T19:24:06.022 INFO:tasks.workunit.client.0.smithi040.stdout:4/199: creat dd/de/f44 x:0 0 0 2024-02-16T19:24:06.022 INFO:tasks.workunit.client.0.smithi040.stdout:4/200: truncate dd/f43 838911 0 2024-02-16T19:24:06.023 INFO:tasks.workunit.client.0.smithi040.stdout:2/139: rename c0 to d5/c29 0 2024-02-16T19:24:06.026 INFO:tasks.workunit.client.0.smithi040.stdout:0/204: getdents db/d2f/d1d 0 2024-02-16T19:24:06.027 INFO:tasks.workunit.client.0.smithi040.stdout:0/205: chown db/d2f/d1d 8135604 1 2024-02-16T19:24:06.027 INFO:tasks.workunit.client.0.smithi040.stdout:8/184: dread d6/d7/f27 [4194304,4194304] 0 2024-02-16T19:24:06.029 INFO:tasks.workunit.client.0.smithi040.stdout:4/201: unlink dd/f27 0 2024-02-16T19:24:06.029 INFO:tasks.workunit.client.0.smithi040.stdout:4/202: readlink dd/de/l24 0 2024-02-16T19:24:06.031 INFO:tasks.workunit.client.0.smithi040.stdout:6/157: dread d0/d1/d12/f28 [0,4194304] 0 2024-02-16T19:24:06.032 INFO:tasks.workunit.client.0.smithi040.stdout:6/158: readlink d0/d34/l27 0 2024-02-16T19:24:06.032 INFO:tasks.workunit.client.0.smithi040.stdout:1/190: rename d17/l34 to d17/d2d/l3b 0 2024-02-16T19:24:06.033 INFO:tasks.workunit.client.0.smithi040.stdout:3/222: dwrite d1/d7/d33/d38/f3f [0,4194304] 0 2024-02-16T19:24:06.034 INFO:tasks.workunit.client.0.smithi040.stdout:2/140: unlink d5/d7/c28 0 2024-02-16T19:24:06.037 INFO:tasks.workunit.client.0.smithi040.stdout:6/159: unlink d0/l15 0 2024-02-16T19:24:06.037 INFO:tasks.workunit.client.0.smithi040.stdout:5/155: dwrite d1/fc [0,4194304] 0 2024-02-16T19:24:06.038 INFO:tasks.workunit.client.0.smithi040.stdout:8/185: rename d6/l1e to d6/d7/l37 0 2024-02-16T19:24:06.040 INFO:tasks.workunit.client.0.smithi040.stdout:1/191: mkdir d17/d3c 0 2024-02-16T19:24:06.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/141: symlink d5/d7/d12/d1b/l2a 0 2024-02-16T19:24:06.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/142: readlink d5/d7/dd/l1c 0 2024-02-16T19:24:06.045 INFO:tasks.workunit.client.0.smithi040.stdout:6/160: write d0/d1/d12/f28 [3156736,44684] 0 2024-02-16T19:24:06.048 INFO:tasks.workunit.client.0.smithi040.stdout:4/203: dwrite dd/de/f1a [0,4194304] 0 2024-02-16T19:24:06.048 INFO:tasks.workunit.client.0.smithi040.stdout:4/204: truncate dd/de/f44 700262 0 2024-02-16T19:24:06.049 INFO:tasks.workunit.client.0.smithi040.stdout:5/156: mknod d1/da/c2b 0 2024-02-16T19:24:06.050 INFO:tasks.workunit.client.0.smithi040.stdout:1/192: mkdir d17/d26/d3d 0 2024-02-16T19:24:06.052 INFO:tasks.workunit.client.0.smithi040.stdout:2/143: write d5/d7/d12/f15 [3295948,124483] 0 2024-02-16T19:24:06.053 INFO:tasks.workunit.client.0.smithi040.stdout:2/144: creat d5/d7/d12/f2b x:0 0 0 2024-02-16T19:24:06.055 INFO:tasks.workunit.client.0.smithi040.stdout:8/186: dwrite d6/d12/f1f [0,4194304] 0 2024-02-16T19:24:06.060 INFO:tasks.workunit.client.0.smithi040.stdout:1/193: symlink d17/d26/l3e 0 2024-02-16T19:24:06.062 INFO:tasks.workunit.client.0.smithi040.stdout:2/145: creat d5/d7/dd/f2c x:0 0 0 2024-02-16T19:24:06.062 INFO:tasks.workunit.client.0.smithi040.stdout:2/146: truncate d5/d7/dd/f2c 901565 0 2024-02-16T19:24:06.063 INFO:tasks.workunit.client.0.smithi040.stdout:5/157: read d1/da/ff [2962687,126502] 0 2024-02-16T19:24:06.066 INFO:tasks.workunit.client.0.smithi040.stdout:4/205: rename dd/de/d10/d28 to dd/de/d45 0 2024-02-16T19:24:06.067 INFO:tasks.workunit.client.0.smithi040.stdout:5/158: getdents d1/da/d1c/d21 0 2024-02-16T19:24:06.067 INFO:tasks.workunit.client.0.smithi040.stdout:1/194: dread f1 [0,4194304] 0 2024-02-16T19:24:06.068 INFO:tasks.workunit.client.0.smithi040.stdout:1/195: creat d17/d2d/f3f x:0 0 0 2024-02-16T19:24:06.071 INFO:tasks.workunit.client.0.smithi040.stdout:6/161: dwrite d0/d1/f1f [0,4194304] 0 2024-02-16T19:24:06.071 INFO:tasks.workunit.client.0.smithi040.stdout:6/162: fsync d0/d1/d12/f2c 0 2024-02-16T19:24:06.072 INFO:tasks.workunit.client.0.smithi040.stdout:1/196: rename l10 to d17/d26/l40 0 2024-02-16T19:24:06.072 INFO:tasks.workunit.client.0.smithi040.stdout:1/197: chown c15 37055 1 2024-02-16T19:24:06.073 INFO:tasks.workunit.client.0.smithi040.stdout:5/159: rename d1/fc to d1/f2c 0 2024-02-16T19:24:06.076 INFO:tasks.workunit.client.0.smithi040.stdout:6/163: link d0/d30/d32/f1a d0/f35 0 2024-02-16T19:24:06.076 INFO:tasks.workunit.client.0.smithi040.stdout:6/164: write d0/d30/d32/f2d [994485,31924] 0 2024-02-16T19:24:06.076 INFO:tasks.workunit.client.0.smithi040.stdout:6/165: chown d0/d30/d32/l1b 61250436 1 2024-02-16T19:24:06.077 INFO:tasks.workunit.client.0.smithi040.stdout:1/198: mknod d17/d3c/c41 0 2024-02-16T19:24:06.077 INFO:tasks.workunit.client.0.smithi040.stdout:6/166: read d0/f2e [403548,6398] 0 2024-02-16T19:24:06.077 INFO:tasks.workunit.client.0.smithi040.stdout:5/160: symlink d1/da/l2d 0 2024-02-16T19:24:06.079 INFO:tasks.workunit.client.0.smithi040.stdout:1/199: unlink d17/c22 0 2024-02-16T19:24:06.079 INFO:tasks.workunit.client.0.smithi040.stdout:8/187: dwrite d6/d7/f27 [0,4194304] 0 2024-02-16T19:24:06.079 INFO:tasks.workunit.client.0.smithi040.stdout:1/200: chown d17/f2c 1525277908 1 2024-02-16T19:24:06.079 INFO:tasks.workunit.client.0.smithi040.stdout:8/188: getdents d6/d7/d1d 0 2024-02-16T19:24:06.079 INFO:tasks.workunit.client.0.smithi040.stdout:8/189: write d6/d2b/f34 [262918,66375] 0 2024-02-16T19:24:06.079 INFO:tasks.workunit.client.0.smithi040.stdout:8/190: truncate d6/d12/f31 231394 0 2024-02-16T19:24:06.080 INFO:tasks.workunit.client.0.smithi040.stdout:1/201: write d17/d1d/f21 [792356,82759] 0 2024-02-16T19:24:06.080 INFO:tasks.workunit.client.0.smithi040.stdout:9/211: dwrite d3/f3c [0,4194304] 0 2024-02-16T19:24:06.080 INFO:tasks.workunit.client.0.smithi040.stdout:6/167: unlink d0/l1c 0 2024-02-16T19:24:06.080 INFO:tasks.workunit.client.0.smithi040.stdout:6/168: fsync d0/f2e 0 2024-02-16T19:24:06.080 INFO:tasks.workunit.client.0.smithi040.stdout:6/169: creat d0/f36 x:0 0 0 2024-02-16T19:24:06.081 INFO:tasks.workunit.client.0.smithi040.stdout:8/191: mkdir d6/d7/d1d/d38 0 2024-02-16T19:24:06.081 INFO:tasks.workunit.client.0.smithi040.stdout:8/192: getdents d6/d2b/d36 0 2024-02-16T19:24:06.082 INFO:tasks.workunit.client.0.smithi040.stdout:1/202: symlink d17/d26/d3d/l42 0 2024-02-16T19:24:06.082 INFO:tasks.workunit.client.0.smithi040.stdout:1/203: write d17/d1d/f21 [1561047,42066] 0 2024-02-16T19:24:06.082 INFO:tasks.workunit.client.0.smithi040.stdout:1/204: fsync d17/f1f 0 2024-02-16T19:24:06.082 INFO:tasks.workunit.client.0.smithi040.stdout:1/205: write fd [2147176,24671] 0 2024-02-16T19:24:06.082 INFO:tasks.workunit.client.0.smithi040.stdout:1/206: fdatasync d17/d29/f2b 0 2024-02-16T19:24:06.083 INFO:tasks.workunit.client.0.smithi040.stdout:7/124: sync 2024-02-16T19:24:06.083 INFO:tasks.workunit.client.0.smithi040.stdout:5/161: dread d1/da/d1c/f26 [0,4194304] 0 2024-02-16T19:24:06.084 INFO:tasks.workunit.client.0.smithi040.stdout:9/212: creat d3/d4/d11/d29/d38/f3e x:0 0 0 2024-02-16T19:24:06.085 INFO:tasks.workunit.client.0.smithi040.stdout:9/213: dread d3/d4/d11/d29/f1d [0,4194304] 0 2024-02-16T19:24:06.086 INFO:tasks.workunit.client.0.smithi040.stdout:6/170: mkdir d0/d37 0 2024-02-16T19:24:06.086 INFO:tasks.workunit.client.0.smithi040.stdout:6/171: fdatasync d0/f1d 0 2024-02-16T19:24:06.088 INFO:tasks.workunit.client.0.smithi040.stdout:9/214: truncate d3/f14 3754437 0 2024-02-16T19:24:06.091 INFO:tasks.workunit.client.0.smithi040.stdout:6/172: creat d0/d34/f38 x:0 0 0 2024-02-16T19:24:06.091 INFO:tasks.workunit.client.0.smithi040.stdout:9/215: write d3/d4/d11/d29/d18/d31/f3b [27186,108830] 0 2024-02-16T19:24:06.091 INFO:tasks.workunit.client.0.smithi040.stdout:9/216: truncate d3/d4/d11/d29/f1d 1423093 0 2024-02-16T19:24:06.093 INFO:tasks.workunit.client.0.smithi040.stdout:6/173: mkdir d0/d37/d39 0 2024-02-16T19:24:06.096 INFO:tasks.workunit.client.0.smithi040.stdout:9/217: truncate d3/d4/d11/f19 3310615 0 2024-02-16T19:24:06.100 INFO:tasks.workunit.client.0.smithi040.stdout:9/218: creat d3/d1c/f3f x:0 0 0 2024-02-16T19:24:06.100 INFO:tasks.workunit.client.0.smithi040.stdout:9/219: fsync d3/d4/d11/d29/f1d 0 2024-02-16T19:24:06.107 INFO:tasks.workunit.client.0.smithi040.stdout:5/162: getdents d1 0 2024-02-16T19:24:06.117 INFO:tasks.workunit.client.0.smithi040.stdout:7/125: dwrite f17 [4194304,4194304] 0 2024-02-16T19:24:06.118 INFO:tasks.workunit.client.0.smithi040.stdout:8/193: dwrite d6/d7/d15/f18 [0,4194304] 0 2024-02-16T19:24:06.119 INFO:tasks.workunit.client.0.smithi040.stdout:7/126: unlink c12 0 2024-02-16T19:24:06.119 INFO:tasks.workunit.client.0.smithi040.stdout:7/127: readlink l19 0 2024-02-16T19:24:06.120 INFO:tasks.workunit.client.0.smithi040.stdout:8/194: mknod d6/d12/d22/c39 0 2024-02-16T19:24:06.120 INFO:tasks.workunit.client.0.smithi040.stdout:8/195: fsync d6/f24 0 2024-02-16T19:24:06.120 INFO:tasks.workunit.client.0.smithi040.stdout:7/128: mknod c20 0 2024-02-16T19:24:06.122 INFO:tasks.workunit.client.0.smithi040.stdout:8/196: link d6/d7/d15/c28 d6/d7/d1d/c3a 0 2024-02-16T19:24:06.122 INFO:tasks.workunit.client.0.smithi040.stdout:8/197: getdents d6/d7/d1d/d38 0 2024-02-16T19:24:06.123 INFO:tasks.workunit.client.0.smithi040.stdout:6/174: dwrite d0/d1/d12/f31 [4194304,4194304] 0 2024-02-16T19:24:06.123 INFO:tasks.workunit.client.0.smithi040.stdout:7/129: mkdir d21 0 2024-02-16T19:24:06.123 INFO:tasks.workunit.client.0.smithi040.stdout:1/207: dwrite d17/f1c [0,4194304] 0 2024-02-16T19:24:06.124 INFO:tasks.workunit.client.0.smithi040.stdout:8/198: mknod d6/d2b/d36/c3b 0 2024-02-16T19:24:06.124 INFO:tasks.workunit.client.0.smithi040.stdout:8/199: stat d6/d12/f1f 0 2024-02-16T19:24:06.124 INFO:tasks.workunit.client.0.smithi040.stdout:8/200: creat d6/f3c x:0 0 0 2024-02-16T19:24:06.128 INFO:tasks.workunit.client.0.smithi040.stdout:6/175: rename d0/d37/d39 to d0/d1/d12/d3a 0 2024-02-16T19:24:06.132 INFO:tasks.workunit.client.0.smithi040.stdout:5/163: dwrite d1/f17 [4194304,4194304] 0 2024-02-16T19:24:06.132 INFO:tasks.workunit.client.0.smithi040.stdout:5/164: creat d1/da/f2e x:0 0 0 2024-02-16T19:24:06.132 INFO:tasks.workunit.client.0.smithi040.stdout:5/165: chown d1/f13 0 1 2024-02-16T19:24:06.133 INFO:tasks.workunit.client.0.smithi040.stdout:9/220: rmdir d3 39 2024-02-16T19:24:06.133 INFO:tasks.workunit.client.0.smithi040.stdout:9/221: write d3/d4/d11/d29/f1d [1727702,89726] 0 2024-02-16T19:24:06.133 INFO:tasks.workunit.client.0.smithi040.stdout:9/222: chown d3/d4/d11/c33 478863 1 2024-02-16T19:24:06.146 INFO:tasks.workunit.client.0.smithi040.stdout:7/130: dwrite f7 [0,4194304] 0 2024-02-16T19:24:06.146 INFO:tasks.workunit.client.0.smithi040.stdout:7/131: truncate f16 937204 0 2024-02-16T19:24:06.146 INFO:tasks.workunit.client.0.smithi040.stdout:7/132: fsync f9 0 2024-02-16T19:24:06.149 INFO:tasks.workunit.client.0.smithi040.stdout:9/223: unlink d3/fc 0 2024-02-16T19:24:06.150 INFO:tasks.workunit.client.0.smithi040.stdout:6/176: getdents d0/d1 0 2024-02-16T19:24:06.150 INFO:tasks.workunit.client.0.smithi040.stdout:6/177: truncate d0/d34/f38 37728 0 2024-02-16T19:24:06.151 INFO:tasks.workunit.client.0.smithi040.stdout:1/208: rename d17/d1d to d17/d43 0 2024-02-16T19:24:06.151 INFO:tasks.workunit.client.0.smithi040.stdout:8/201: dwrite d6/f33 [0,4194304] 0 2024-02-16T19:24:06.152 INFO:tasks.workunit.client.0.smithi040.stdout:5/166: dwrite d1/da/f29 [0,4194304] 0 2024-02-16T19:24:06.153 INFO:tasks.workunit.client.0.smithi040.stdout:7/133: truncate fa 2011563 0 2024-02-16T19:24:06.153 INFO:tasks.workunit.client.0.smithi040.stdout:7/134: stat ff 0 2024-02-16T19:24:06.153 INFO:tasks.workunit.client.0.smithi040.stdout:9/224: stat d3/d4/d11/d29/l28 0 2024-02-16T19:24:06.154 INFO:tasks.workunit.client.0.smithi040.stdout:9/225: write d3/d4/d11/d29/d18/d31/f3b [1545270,19486] 0 2024-02-16T19:24:06.155 INFO:tasks.workunit.client.0.smithi040.stdout:6/178: unlink d0/d1/d12/f31 0 2024-02-16T19:24:06.155 INFO:tasks.workunit.client.0.smithi040.stdout:1/209: creat d17/f44 x:0 0 0 2024-02-16T19:24:06.157 INFO:tasks.workunit.client.0.smithi040.stdout:8/202: creat d6/d7/d1d/d38/f3d x:0 0 0 2024-02-16T19:24:06.160 INFO:tasks.workunit.client.0.smithi040.stdout:5/167: creat d1/da/d1c/d21/f2f x:0 0 0 2024-02-16T19:24:06.161 INFO:tasks.workunit.client.0.smithi040.stdout:7/135: creat d21/f22 x:0 0 0 2024-02-16T19:24:06.161 INFO:tasks.workunit.client.0.smithi040.stdout:7/136: stat d21 0 2024-02-16T19:24:06.161 INFO:tasks.workunit.client.0.smithi040.stdout:6/179: write d0/d30/d32/f1a [1986095,58182] 0 2024-02-16T19:24:06.164 INFO:tasks.workunit.client.0.smithi040.stdout:9/226: unlink d3/d4/d11/d29/f1d 0 2024-02-16T19:24:06.166 INFO:tasks.workunit.client.0.smithi040.stdout:1/210: symlink d17/d26/d3d/l45 0 2024-02-16T19:24:06.166 INFO:tasks.workunit.client.0.smithi040.stdout:1/211: write d17/d29/f2b [375547,14182] 0 2024-02-16T19:24:06.166 INFO:tasks.workunit.client.0.smithi040.stdout:1/212: write f1 [784088,35758] 0 2024-02-16T19:24:06.166 INFO:tasks.workunit.client.0.smithi040.stdout:1/213: fsync d17/d43/f21 0 2024-02-16T19:24:06.167 INFO:tasks.workunit.client.0.smithi040.stdout:8/203: dread f3 [4194304,4194304] 0 2024-02-16T19:24:06.167 INFO:tasks.workunit.client.0.smithi040.stdout:5/168: truncate d1/f1d 3140958 0 2024-02-16T19:24:06.167 INFO:tasks.workunit.client.0.smithi040.stdout:5/169: creat d1/da/f30 x:0 0 0 2024-02-16T19:24:06.171 INFO:tasks.workunit.client.0.smithi040.stdout:6/180: mkdir d0/d1/d12/d3b 0 2024-02-16T19:24:06.172 INFO:tasks.workunit.client.0.smithi040.stdout:6/181: creat d0/d34/f3c x:0 0 0 2024-02-16T19:24:06.172 INFO:tasks.workunit.client.0.smithi040.stdout:6/182: chown d0/f35 0 1 2024-02-16T19:24:06.173 INFO:tasks.workunit.client.0.smithi040.stdout:1/214: mknod d17/d3c/c46 0 2024-02-16T19:24:06.175 INFO:tasks.workunit.client.0.smithi040.stdout:9/227: write d3/f14 [2290485,9316] 0 2024-02-16T19:24:06.175 INFO:tasks.workunit.client.0.smithi040.stdout:5/170: mknod d1/da/c31 0 2024-02-16T19:24:06.176 INFO:tasks.workunit.client.0.smithi040.stdout:0/206: sync 2024-02-16T19:24:06.176 INFO:tasks.workunit.client.0.smithi040.stdout:3/223: sync 2024-02-16T19:24:06.176 INFO:tasks.workunit.client.0.smithi040.stdout:0/207: stat db/d25 0 2024-02-16T19:24:06.177 INFO:tasks.workunit.client.0.smithi040.stdout:0/208: truncate db/d25/d34/f3a 626359 0 2024-02-16T19:24:06.177 INFO:tasks.workunit.client.0.smithi040.stdout:0/209: chown db/cc 0 1 2024-02-16T19:24:06.180 INFO:tasks.workunit.client.0.smithi040.stdout:7/137: dwrite f16 [0,4194304] 0 2024-02-16T19:24:06.180 INFO:tasks.workunit.client.0.smithi040.stdout:7/138: creat d21/f23 x:0 0 0 2024-02-16T19:24:06.181 INFO:tasks.workunit.client.0.smithi040.stdout:7/139: truncate d21/f23 569340 0 2024-02-16T19:24:06.181 INFO:tasks.workunit.client.0.smithi040.stdout:7/140: dread - d21/f22 zero size 2024-02-16T19:24:06.181 INFO:tasks.workunit.client.0.smithi040.stdout:6/183: rmdir d0/d30 39 2024-02-16T19:24:06.181 INFO:tasks.workunit.client.0.smithi040.stdout:6/184: write d0/d30/d32/f1a [716480,80601] 0 2024-02-16T19:24:06.183 INFO:tasks.workunit.client.0.smithi040.stdout:9/228: symlink d3/d1c/l40 0 2024-02-16T19:24:06.187 INFO:tasks.workunit.client.0.smithi040.stdout:6/185: write d0/fe [7561,39305] 0 2024-02-16T19:24:06.187 INFO:tasks.workunit.client.0.smithi040.stdout:6/186: write d0/f5 [4403745,92212] 0 2024-02-16T19:24:06.188 INFO:tasks.workunit.client.0.smithi040.stdout:0/210: creat db/d2f/f3e x:0 0 0 2024-02-16T19:24:06.198 INFO:tasks.workunit.client.0.smithi040.stdout:3/224: dread d1/d7/f9 [4194304,4194304] 0 2024-02-16T19:24:06.198 INFO:tasks.workunit.client.0.smithi040.stdout:3/225: chown d1/l1b 3046 1 2024-02-16T19:24:06.204 INFO:tasks.workunit.client.0.smithi040.stdout:7/141: rename fc to d21/f24 0 2024-02-16T19:24:06.205 INFO:tasks.workunit.client.0.smithi040.stdout:7/142: creat d21/f25 x:0 0 0 2024-02-16T19:24:06.205 INFO:tasks.workunit.client.0.smithi040.stdout:7/143: write d21/f23 [1573201,117508] 0 2024-02-16T19:24:06.205 INFO:tasks.workunit.client.0.smithi040.stdout:7/144: creat d21/f26 x:0 0 0 2024-02-16T19:24:06.205 INFO:tasks.workunit.client.0.smithi040.stdout:7/145: chown f17 2053020 1 2024-02-16T19:24:06.207 INFO:tasks.workunit.client.0.smithi040.stdout:3/226: read d1/f32 [3829694,88940] 0 2024-02-16T19:24:06.207 INFO:tasks.workunit.client.0.smithi040.stdout:5/171: rmdir d1/da 39 2024-02-16T19:24:06.210 INFO:tasks.workunit.client.0.smithi040.stdout:6/187: creat d0/d1/d12/d3a/f3d x:0 0 0 2024-02-16T19:24:06.212 INFO:tasks.workunit.client.0.smithi040.stdout:7/146: mknod d21/c27 0 2024-02-16T19:24:06.212 INFO:tasks.workunit.client.0.smithi040.stdout:7/147: chown l1a 1441554 1 2024-02-16T19:24:06.212 INFO:tasks.workunit.client.0.smithi040.stdout:5/172: write d1/da/d1c/f26 [7980,91385] 0 2024-02-16T19:24:06.215 INFO:tasks.workunit.client.0.smithi040.stdout:6/188: mknod d0/c3e 0 2024-02-16T19:24:06.215 INFO:tasks.workunit.client.0.smithi040.stdout:0/211: dwrite db/d25/d34/f3a [0,4194304] 0 2024-02-16T19:24:06.215 INFO:tasks.workunit.client.0.smithi040.stdout:0/212: write db/d2f/d1d/f3c [748871,43170] 0 2024-02-16T19:24:06.216 INFO:tasks.workunit.client.0.smithi040.stdout:9/229: dwrite d3/d4/d11/d29/f24 [0,4194304] 0 2024-02-16T19:24:06.216 INFO:tasks.workunit.client.0.smithi040.stdout:9/230: read d3/f21 [740407,88831] 0 2024-02-16T19:24:06.216 INFO:tasks.workunit.client.0.smithi040.stdout:2/147: sync 2024-02-16T19:24:06.216 INFO:tasks.workunit.client.0.smithi040.stdout:4/206: sync 2024-02-16T19:24:06.216 INFO:tasks.workunit.client.0.smithi040.stdout:2/148: fdatasync d5/d7/d1e/f20 0 2024-02-16T19:24:06.220 INFO:tasks.workunit.client.0.smithi040.stdout:0/213: unlink db/l2a 0 2024-02-16T19:24:06.221 INFO:tasks.workunit.client.0.smithi040.stdout:9/231: symlink d3/d4/d11/d29/d18/l41 0 2024-02-16T19:24:06.223 INFO:tasks.workunit.client.0.smithi040.stdout:4/207: rename dd/f1e to dd/de/d10/d13/f46 0 2024-02-16T19:24:06.224 INFO:tasks.workunit.client.0.smithi040.stdout:2/149: mknod d5/d7/d12/d1b/c2d 0 2024-02-16T19:24:06.224 INFO:tasks.workunit.client.0.smithi040.stdout:2/150: write d5/d7/f19 [585846,44992] 0 2024-02-16T19:24:06.225 INFO:tasks.workunit.client.0.smithi040.stdout:3/227: dread d1/d4/fa [0,4194304] 0 2024-02-16T19:24:06.228 INFO:tasks.workunit.client.0.smithi040.stdout:2/151: symlink d5/d7/l2e 0 2024-02-16T19:24:06.228 INFO:tasks.workunit.client.0.smithi040.stdout:2/152: chown d5/d8/cc 29174 1 2024-02-16T19:24:06.230 INFO:tasks.workunit.client.0.smithi040.stdout:3/228: creat d1/d4/f41 x:0 0 0 2024-02-16T19:24:06.233 INFO:tasks.workunit.client.0.smithi040.stdout:2/153: creat d5/d7/d12/d1b/f2f x:0 0 0 2024-02-16T19:24:06.233 INFO:tasks.workunit.client.0.smithi040.stdout:2/154: write d5/d7/d12/f15 [1732322,90686] 0 2024-02-16T19:24:06.233 INFO:tasks.workunit.client.0.smithi040.stdout:0/214: write db/f2b [3481294,65827] 0 2024-02-16T19:24:06.233 INFO:tasks.workunit.client.0.smithi040.stdout:0/215: read db/f1b [3254675,3949] 0 2024-02-16T19:24:06.235 INFO:tasks.workunit.client.0.smithi040.stdout:3/229: unlink d1/f32 0 2024-02-16T19:24:06.235 INFO:tasks.workunit.client.0.smithi040.stdout:3/230: dread - d1/dc/f3c zero size 2024-02-16T19:24:06.235 INFO:tasks.workunit.client.0.smithi040.stdout:4/208: write fb [3800801,63496] 0 2024-02-16T19:24:06.237 INFO:tasks.workunit.client.0.smithi040.stdout:0/216: symlink db/d2f/l3f 0 2024-02-16T19:24:06.240 INFO:tasks.workunit.client.0.smithi040.stdout:4/209: creat dd/de/d10/f47 x:0 0 0 2024-02-16T19:24:06.243 INFO:tasks.workunit.client.0.smithi040.stdout:0/217: link db/fd db/d2f/d23/f40 0 2024-02-16T19:24:06.244 INFO:tasks.workunit.client.0.smithi040.stdout:6/189: dwrite d0/f4 [0,4194304] 0 2024-02-16T19:24:06.244 INFO:tasks.workunit.client.0.smithi040.stdout:6/190: chown d0/d1/d12/f18 1709 1 2024-02-16T19:24:06.244 INFO:tasks.workunit.client.0.smithi040.stdout:7/148: dwrite f9 [0,4194304] 0 2024-02-16T19:24:06.244 INFO:tasks.workunit.client.0.smithi040.stdout:7/149: chown l13 53 1 2024-02-16T19:24:06.244 INFO:tasks.workunit.client.0.smithi040.stdout:7/150: creat d21/f28 x:0 0 0 2024-02-16T19:24:06.245 INFO:tasks.workunit.client.0.smithi040.stdout:4/210: unlink dd/de/d45/c2d 0 2024-02-16T19:24:06.246 INFO:tasks.workunit.client.0.smithi040.stdout:0/218: link db/d2f/l3d db/d2f/d1d/d21/d2e/l41 0 2024-02-16T19:24:06.249 INFO:tasks.workunit.client.0.smithi040.stdout:7/151: mkdir d21/d29 0 2024-02-16T19:24:06.250 INFO:tasks.workunit.client.0.smithi040.stdout:7/152: write fb [2289236,41331] 0 2024-02-16T19:24:06.256 INFO:tasks.workunit.client.0.smithi040.stdout:4/211: mknod dd/de/d45/c48 0 2024-02-16T19:24:06.256 INFO:tasks.workunit.client.0.smithi040.stdout:3/231: rmdir d1/d4 39 2024-02-16T19:24:06.258 INFO:tasks.workunit.client.0.smithi040.stdout:0/219: mknod db/d2f/d1d/d21/c42 0 2024-02-16T19:24:06.258 INFO:tasks.workunit.client.0.smithi040.stdout:0/220: chown db/c15 337 1 2024-02-16T19:24:06.259 INFO:tasks.workunit.client.0.smithi040.stdout:6/191: dwrite d0/f2e [0,4194304] 0 2024-02-16T19:24:06.259 INFO:tasks.workunit.client.0.smithi040.stdout:6/192: chown d0 2194598 1 2024-02-16T19:24:06.260 INFO:tasks.workunit.client.0.smithi040.stdout:7/153: mknod d21/d29/c2a 0 2024-02-16T19:24:06.260 INFO:tasks.workunit.client.0.smithi040.stdout:7/154: fsync f17 0 2024-02-16T19:24:06.261 INFO:tasks.workunit.client.0.smithi040.stdout:3/232: mkdir d1/dc/d42 0 2024-02-16T19:24:06.264 INFO:tasks.workunit.client.0.smithi040.stdout:7/155: unlink c1c 0 2024-02-16T19:24:06.265 INFO:tasks.workunit.client.0.smithi040.stdout:3/233: rmdir d1/d7 39 2024-02-16T19:24:06.265 INFO:tasks.workunit.client.0.smithi040.stdout:3/234: dread - d1/d4/f41 zero size 2024-02-16T19:24:06.267 INFO:tasks.workunit.client.0.smithi040.stdout:3/235: creat d1/d4/f43 x:0 0 0 2024-02-16T19:24:06.269 INFO:tasks.workunit.client.0.smithi040.stdout:3/236: rename d1/dc/c2d to d1/dc/d35/c44 0 2024-02-16T19:24:06.272 INFO:tasks.workunit.client.0.smithi040.stdout:0/221: dwrite f7 [0,4194304] 0 2024-02-16T19:24:06.273 INFO:tasks.workunit.client.0.smithi040.stdout:0/222: mknod db/d25/d34/c43 0 2024-02-16T19:24:06.273 INFO:tasks.workunit.client.0.smithi040.stdout:0/223: creat db/de/f44 x:0 0 0 2024-02-16T19:24:06.274 INFO:tasks.workunit.client.0.smithi040.stdout:0/224: write f9 [853498,105200] 0 2024-02-16T19:24:06.274 INFO:tasks.workunit.client.0.smithi040.stdout:0/225: fdatasync db/f20 0 2024-02-16T19:24:06.276 INFO:tasks.workunit.client.0.smithi040.stdout:0/226: dread db/d2f/d1d/f3c [0,4194304] 0 2024-02-16T19:24:06.277 INFO:tasks.workunit.client.0.smithi040.stdout:0/227: creat db/d2f/d1d/d21/f45 x:0 0 0 2024-02-16T19:24:06.277 INFO:tasks.workunit.client.0.smithi040.stdout:0/228: chown db/de/c36 465648617 1 2024-02-16T19:24:06.284 INFO:tasks.workunit.client.0.smithi040.stdout:3/237: dread d1/d4/f2e [0,4194304] 0 2024-02-16T19:24:06.285 INFO:tasks.workunit.client.0.smithi040.stdout:3/238: write d1/f34 [588486,119057] 0 2024-02-16T19:24:06.296 INFO:tasks.workunit.client.0.smithi040.stdout:7/156: dwrite fb [4194304,4194304] 0 2024-02-16T19:24:06.296 INFO:tasks.workunit.client.0.smithi040.stdout:7/157: readlink l13 0 2024-02-16T19:24:06.301 INFO:tasks.workunit.client.0.smithi040.stdout:0/229: dwrite db/de/f44 [0,4194304] 0 2024-02-16T19:24:06.303 INFO:tasks.workunit.client.0.smithi040.stdout:3/239: dwrite d1/dc/fd [0,4194304] 0 2024-02-16T19:24:06.304 INFO:tasks.workunit.client.0.smithi040.stdout:0/230: creat db/d2f/d1d/f46 x:0 0 0 2024-02-16T19:24:06.311 INFO:tasks.workunit.client.0.smithi040.stdout:7/158: write f7 [2298519,11236] 0 2024-02-16T19:24:06.315 INFO:tasks.workunit.client.0.smithi040.stdout:7/159: rename f10 to d21/d29/f2b 0 2024-02-16T19:24:06.315 INFO:tasks.workunit.client.0.smithi040.stdout:7/160: truncate d21/f26 709873 0 2024-02-16T19:24:06.315 INFO:tasks.workunit.client.0.smithi040.stdout:7/161: chown d21/f26 421074 1 2024-02-16T19:24:06.317 INFO:tasks.workunit.client.0.smithi040.stdout:8/204: sync 2024-02-16T19:24:06.317 INFO:tasks.workunit.client.0.smithi040.stdout:8/205: chown d6/f9 64872 1 2024-02-16T19:24:06.317 INFO:tasks.workunit.client.0.smithi040.stdout:8/206: write d6/d12/d22/f23 [1001123,81873] 0 2024-02-16T19:24:06.318 INFO:tasks.workunit.client.0.smithi040.stdout:8/207: creat d6/d2b/d36/f3e x:0 0 0 2024-02-16T19:24:06.319 INFO:tasks.workunit.client.0.smithi040.stdout:8/208: stat d6/d7/d1d/c3a 0 2024-02-16T19:24:06.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/209: rmdir d6/d2b/d30 0 2024-02-16T19:24:06.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/210: creat d6/f3f x:0 0 0 2024-02-16T19:24:06.321 INFO:tasks.workunit.client.0.smithi040.stdout:8/211: fdatasync d6/d7/d1d/f29 0 2024-02-16T19:24:06.321 INFO:tasks.workunit.client.0.smithi040.stdout:8/212: readlink d6/d7/l26 0 2024-02-16T19:24:06.321 INFO:tasks.workunit.client.0.smithi040.stdout:8/213: stat d6/d7/d1d 0 2024-02-16T19:24:06.321 INFO:tasks.workunit.client.0.smithi040.stdout:8/214: creat d6/d2b/f40 x:0 0 0 2024-02-16T19:24:06.321 INFO:tasks.workunit.client.0.smithi040.stdout:8/215: chown d6/d7/d1d/l25 762 1 2024-02-16T19:24:06.322 INFO:tasks.workunit.client.0.smithi040.stdout:8/216: mkdir d6/d12/d41 0 2024-02-16T19:24:06.322 INFO:tasks.workunit.client.0.smithi040.stdout:8/217: creat d6/d12/d22/f42 x:0 0 0 2024-02-16T19:24:06.322 INFO:tasks.workunit.client.0.smithi040.stdout:8/218: read d6/d7/d15/f1c [1321258,54267] 0 2024-02-16T19:24:06.322 INFO:tasks.workunit.client.0.smithi040.stdout:8/219: creat d6/f43 x:0 0 0 2024-02-16T19:24:06.325 INFO:tasks.workunit.client.0.smithi040.stdout:7/162: dread d21/f24 [0,4194304] 0 2024-02-16T19:24:06.325 INFO:tasks.workunit.client.0.smithi040.stdout:0/231: dwrite f9 [0,4194304] 0 2024-02-16T19:24:06.325 INFO:tasks.workunit.client.0.smithi040.stdout:0/232: fdatasync db/f35 0 2024-02-16T19:24:06.326 INFO:tasks.workunit.client.0.smithi040.stdout:0/233: fsync db/f35 0 2024-02-16T19:24:06.326 INFO:tasks.workunit.client.0.smithi040.stdout:0/234: chown db/d25/f26 5482 1 2024-02-16T19:24:06.326 INFO:tasks.workunit.client.0.smithi040.stdout:0/235: chown db/f1b 97 1 2024-02-16T19:24:06.326 INFO:tasks.workunit.client.0.smithi040.stdout:0/236: write db/d2f/d1d/f3c [1656765,123019] 0 2024-02-16T19:24:06.326 INFO:tasks.workunit.client.0.smithi040.stdout:3/240: dwrite d1/f34 [0,4194304] 0 2024-02-16T19:24:06.328 INFO:tasks.workunit.client.0.smithi040.stdout:7/163: getdents d21/d29 0 2024-02-16T19:24:06.328 INFO:tasks.workunit.client.0.smithi040.stdout:7/164: write d21/f26 [1383146,37914] 0 2024-02-16T19:24:06.333 INFO:tasks.workunit.client.0.smithi040.stdout:0/237: creat db/d2f/d23/d3b/f47 x:0 0 0 2024-02-16T19:24:06.333 INFO:tasks.workunit.client.0.smithi040.stdout:0/238: read db/d25/f32 [707492,42494] 0 2024-02-16T19:24:06.333 INFO:tasks.workunit.client.0.smithi040.stdout:8/220: rmdir d6/d12 39 2024-02-16T19:24:06.334 INFO:tasks.workunit.client.0.smithi040.stdout:3/241: truncate d1/d7/f9 3526232 0 2024-02-16T19:24:06.336 INFO:tasks.workunit.client.0.smithi040.stdout:0/239: rename db/l2d to db/d2f/d23/l48 0 2024-02-16T19:24:06.336 INFO:tasks.workunit.client.0.smithi040.stdout:7/165: write d21/d29/f2b [2311753,53120] 0 2024-02-16T19:24:06.337 INFO:tasks.workunit.client.0.smithi040.stdout:8/221: mkdir d6/d2b/d44 0 2024-02-16T19:24:06.338 INFO:tasks.workunit.client.0.smithi040.stdout:3/242: creat d1/d7/d36/f45 x:0 0 0 2024-02-16T19:24:06.340 INFO:tasks.workunit.client.0.smithi040.stdout:0/240: mkdir db/d2f/d1d/d21/d2e/d49 0 2024-02-16T19:24:06.340 INFO:tasks.workunit.client.0.smithi040.stdout:0/241: chown db/d2f/d1d/d21/d2e 6371 1 2024-02-16T19:24:06.341 INFO:tasks.workunit.client.0.smithi040.stdout:7/166: truncate f17 2439880 0 2024-02-16T19:24:06.341 INFO:tasks.workunit.client.0.smithi040.stdout:7/167: fsync d21/f22 0 2024-02-16T19:24:06.342 INFO:tasks.workunit.client.0.smithi040.stdout:3/243: rename d1/dc/d35/l39 to d1/dc/d35/l46 0 2024-02-16T19:24:06.342 INFO:tasks.workunit.client.0.smithi040.stdout:3/244: truncate d1/d4/f43 12927 0 2024-02-16T19:24:06.347 INFO:tasks.workunit.client.0.smithi040.stdout:0/242: unlink db/d2f/d1d/d21/c42 0 2024-02-16T19:24:06.348 INFO:tasks.workunit.client.0.smithi040.stdout:7/168: symlink d21/d29/l2c 0 2024-02-16T19:24:06.349 INFO:tasks.workunit.client.0.smithi040.stdout:3/245: mknod d1/c47 0 2024-02-16T19:24:06.351 INFO:tasks.workunit.client.0.smithi040.stdout:0/243: unlink db/d25/f32 0 2024-02-16T19:24:06.351 INFO:tasks.workunit.client.0.smithi040.stdout:0/244: chown db/f1b 3632017 1 2024-02-16T19:24:06.351 INFO:tasks.workunit.client.0.smithi040.stdout:7/169: symlink d21/l2d 0 2024-02-16T19:24:06.352 INFO:tasks.workunit.client.0.smithi040.stdout:3/246: mkdir d1/dc/d35/d48 0 2024-02-16T19:24:06.354 INFO:tasks.workunit.client.0.smithi040.stdout:0/245: link db/cc db/c4a 0 2024-02-16T19:24:06.355 INFO:tasks.workunit.client.0.smithi040.stdout:0/246: unlink f9 0 2024-02-16T19:24:06.356 INFO:tasks.workunit.client.0.smithi040.stdout:0/247: unlink db/f35 0 2024-02-16T19:24:06.357 INFO:tasks.workunit.client.0.smithi040.stdout:0/248: symlink db/de/l4b 0 2024-02-16T19:24:06.357 INFO:tasks.workunit.client.0.smithi040.stdout:0/249: chown db/d2f/d1d/d21/d2e/d49 54316 1 2024-02-16T19:24:06.358 INFO:tasks.workunit.client.0.smithi040.stdout:0/250: rmdir db/d2f/d23 39 2024-02-16T19:24:06.359 INFO:tasks.workunit.client.0.smithi040.stdout:0/251: stat db/c15 0 2024-02-16T19:24:06.359 INFO:tasks.workunit.client.0.smithi040.stdout:0/252: chown db/de/l1f 130 1 2024-02-16T19:24:06.360 INFO:tasks.workunit.client.0.smithi040.stdout:0/253: rename db/d25/d2c/l39 to db/d2f/d23/d3b/l4c 0 2024-02-16T19:24:06.373 INFO:tasks.workunit.client.0.smithi040.stdout:9/232: sync 2024-02-16T19:24:06.373 INFO:tasks.workunit.client.0.smithi040.stdout:1/215: sync 2024-02-16T19:24:06.373 INFO:tasks.workunit.client.0.smithi040.stdout:5/173: sync 2024-02-16T19:24:06.373 INFO:tasks.workunit.client.0.smithi040.stdout:1/216: dread - d17/d2d/f33 zero size 2024-02-16T19:24:06.373 INFO:tasks.workunit.client.0.smithi040.stdout:9/233: read d3/d4/f2c [3191872,107824] 0 2024-02-16T19:24:06.373 INFO:tasks.workunit.client.0.smithi040.stdout:9/234: chown d3/cb 1323 1 2024-02-16T19:24:06.374 INFO:tasks.workunit.client.0.smithi040.stdout:1/217: rename d17/d26/l3e to d17/d29/l47 0 2024-02-16T19:24:06.375 INFO:tasks.workunit.client.0.smithi040.stdout:3/247: dread d1/d4/f19 [0,4194304] 0 2024-02-16T19:24:06.375 INFO:tasks.workunit.client.0.smithi040.stdout:3/248: read d1/d4/f19 [373733,54357] 0 2024-02-16T19:24:06.375 INFO:tasks.workunit.client.0.smithi040.stdout:9/235: truncate d3/f1f 3019289 0 2024-02-16T19:24:06.375 INFO:tasks.workunit.client.0.smithi040.stdout:0/254: dread db/d2f/d1d/f3c [0,4194304] 0 2024-02-16T19:24:06.376 INFO:tasks.workunit.client.0.smithi040.stdout:1/218: mknod d17/d3c/c48 0 2024-02-16T19:24:06.377 INFO:tasks.workunit.client.0.smithi040.stdout:3/249: symlink d1/dc/d35/l49 0 2024-02-16T19:24:06.379 INFO:tasks.workunit.client.0.smithi040.stdout:0/255: rename db/c4a to db/d2f/d1d/d21/d2e/d49/c4d 0 2024-02-16T19:24:06.379 INFO:tasks.workunit.client.0.smithi040.stdout:0/256: readlink db/d2f/d23/l48 0 2024-02-16T19:24:06.380 INFO:tasks.workunit.client.0.smithi040.stdout:1/219: mkdir d17/d3c/d49 0 2024-02-16T19:24:06.380 INFO:tasks.workunit.client.0.smithi040.stdout:5/174: read d1/f1d [95787,79456] 0 2024-02-16T19:24:06.382 INFO:tasks.workunit.client.0.smithi040.stdout:3/250: unlink d1/d7/d33/d38/f3f 0 2024-02-16T19:24:06.382 INFO:tasks.workunit.client.0.smithi040.stdout:3/251: fdatasync d1/f3 0 2024-02-16T19:24:06.386 INFO:tasks.workunit.client.0.smithi040.stdout:0/257: mknod db/d2f/d23/c4e 0 2024-02-16T19:24:06.387 INFO:tasks.workunit.client.0.smithi040.stdout:5/175: symlink d1/l32 0 2024-02-16T19:24:06.389 INFO:tasks.workunit.client.0.smithi040.stdout:7/170: dwrite f9 [0,4194304] 0 2024-02-16T19:24:06.389 INFO:tasks.workunit.client.0.smithi040.stdout:7/171: fdatasync f1d 0 2024-02-16T19:24:06.389 INFO:tasks.workunit.client.0.smithi040.stdout:0/258: creat db/d25/f4f x:0 0 0 2024-02-16T19:24:06.390 INFO:tasks.workunit.client.0.smithi040.stdout:5/176: getdents d1/da/d1c/d21 0 2024-02-16T19:24:06.391 INFO:tasks.workunit.client.0.smithi040.stdout:5/177: stat d1/da/c2b 0 2024-02-16T19:24:06.391 INFO:tasks.workunit.client.0.smithi040.stdout:1/220: write d17/d2d/f2e [1761171,53806] 0 2024-02-16T19:24:06.394 INFO:tasks.workunit.client.0.smithi040.stdout:7/172: link d21/d29/f2b d21/d29/f2e 0 2024-02-16T19:24:06.394 INFO:tasks.workunit.client.0.smithi040.stdout:7/173: write fa [186350,16994] 0 2024-02-16T19:24:06.394 INFO:tasks.workunit.client.0.smithi040.stdout:7/174: creat d21/f2f x:0 0 0 2024-02-16T19:24:06.395 INFO:tasks.workunit.client.0.smithi040.stdout:0/259: rename db/d25/d34/c43 to db/d2f/d23/c50 0 2024-02-16T19:24:06.395 INFO:tasks.workunit.client.0.smithi040.stdout:0/260: write db/d2f/d1d/f46 [23958,23210] 0 2024-02-16T19:24:06.395 INFO:tasks.workunit.client.0.smithi040.stdout:0/261: creat db/d2f/d1d/d21/f51 x:0 0 0 2024-02-16T19:24:06.397 INFO:tasks.workunit.client.0.smithi040.stdout:5/178: symlink d1/da/l33 0 2024-02-16T19:24:06.397 INFO:tasks.workunit.client.0.smithi040.stdout:5/179: truncate d1/da/f30 335608 0 2024-02-16T19:24:06.398 INFO:tasks.workunit.client.0.smithi040.stdout:9/236: dwrite d3/f3d [0,4194304] 0 2024-02-16T19:24:06.404 INFO:tasks.workunit.client.0.smithi040.stdout:1/221: rename d17/d3c/d49 to d17/d43/d38/d4a 0 2024-02-16T19:24:06.404 INFO:tasks.workunit.client.0.smithi040.stdout:1/222: fsync d17/d2d/f33 0 2024-02-16T19:24:06.404 INFO:tasks.workunit.client.0.smithi040.stdout:1/223: chown d17/d3c/c41 9 1 2024-02-16T19:24:06.406 INFO:tasks.workunit.client.0.smithi040.stdout:0/262: mkdir db/d25/d2c/d52 0 2024-02-16T19:24:06.407 INFO:tasks.workunit.client.0.smithi040.stdout:0/263: write db/d2f/d1d/d21/f51 [488450,1689] 0 2024-02-16T19:24:06.407 INFO:tasks.workunit.client.0.smithi040.stdout:0/264: creat db/f53 x:0 0 0 2024-02-16T19:24:06.407 INFO:tasks.workunit.client.0.smithi040.stdout:5/180: mknod d1/c34 0 2024-02-16T19:24:06.411 INFO:tasks.workunit.client.0.smithi040.stdout:9/237: creat d3/d4/d11/d32/f42 x:0 0 0 2024-02-16T19:24:06.413 INFO:tasks.workunit.client.0.smithi040.stdout:1/224: creat d17/d26/f4b x:0 0 0 2024-02-16T19:24:06.414 INFO:tasks.workunit.client.0.smithi040.stdout:7/175: getdents d21/d29 0 2024-02-16T19:24:06.416 INFO:tasks.workunit.client.0.smithi040.stdout:0/265: mknod db/de/c54 0 2024-02-16T19:24:06.416 INFO:tasks.workunit.client.0.smithi040.stdout:5/181: dread d1/da/f29 [0,4194304] 0 2024-02-16T19:24:06.417 INFO:tasks.workunit.client.0.smithi040.stdout:7/176: fsync fa 0 2024-02-16T19:24:06.417 INFO:tasks.workunit.client.0.smithi040.stdout:7/177: truncate d21/f2f 140553 0 2024-02-16T19:24:06.421 INFO:tasks.workunit.client.0.smithi040.stdout:1/225: write fb [3274374,90922] 0 2024-02-16T19:24:06.424 INFO:tasks.workunit.client.0.smithi040.stdout:0/266: write db/de/f38 [1545315,2176] 0 2024-02-16T19:24:06.425 INFO:tasks.workunit.client.0.smithi040.stdout:5/182: creat d1/da/f35 x:0 0 0 2024-02-16T19:24:06.425 INFO:tasks.workunit.client.0.smithi040.stdout:5/183: write d1/da/ff [4937015,21694] 0 2024-02-16T19:24:06.426 INFO:tasks.workunit.client.0.smithi040.stdout:9/238: truncate d3/f3d 1548450 0 2024-02-16T19:24:06.426 INFO:tasks.workunit.client.0.smithi040.stdout:9/239: creat d3/d4/d11/d32/f43 x:0 0 0 2024-02-16T19:24:06.431 INFO:tasks.workunit.client.0.smithi040.stdout:5/184: creat d1/da/d1c/d21/f36 x:0 0 0 2024-02-16T19:24:06.436 INFO:tasks.workunit.client.0.smithi040.stdout:7/178: dwrite f16 [0,4194304] 0 2024-02-16T19:24:06.440 INFO:tasks.workunit.client.0.smithi040.stdout:7/179: rename f15 to d21/d29/f30 0 2024-02-16T19:24:06.440 INFO:tasks.workunit.client.0.smithi040.stdout:9/240: dread d3/d4/d11/f19 [0,4194304] 0 2024-02-16T19:24:06.440 INFO:tasks.workunit.client.0.smithi040.stdout:9/241: chown d3 18985326 1 2024-02-16T19:24:06.440 INFO:tasks.workunit.client.0.smithi040.stdout:4/212: sync 2024-02-16T19:24:06.440 INFO:tasks.workunit.client.0.smithi040.stdout:2/155: sync 2024-02-16T19:24:06.440 INFO:tasks.workunit.client.0.smithi040.stdout:6/193: sync 2024-02-16T19:24:06.440 INFO:tasks.workunit.client.0.smithi040.stdout:6/194: chown d0/d1/c10 6770 1 2024-02-16T19:24:06.443 INFO:tasks.workunit.client.0.smithi040.stdout:4/213: unlink dd/de/l24 0 2024-02-16T19:24:06.451 INFO:tasks.workunit.client.0.smithi040.stdout:0/267: dwrite db/f1b [0,4194304] 0 2024-02-16T19:24:06.451 INFO:tasks.workunit.client.0.smithi040.stdout:0/268: write db/d2f/d23/f33 [967555,18864] 0 2024-02-16T19:24:06.456 INFO:tasks.workunit.client.0.smithi040.stdout:2/156: dread d5/d7/d1e/f20 [0,4194304] 0 2024-02-16T19:24:06.456 INFO:tasks.workunit.client.0.smithi040.stdout:2/157: readlink d5/d7/d12/d1b/l2a 0 2024-02-16T19:24:06.456 INFO:tasks.workunit.client.0.smithi040.stdout:2/158: write d5/d7/d12/f2b [186651,18777] 0 2024-02-16T19:24:06.456 INFO:tasks.workunit.client.0.smithi040.stdout:0/269: dread db/f20 [0,4194304] 0 2024-02-16T19:24:06.459 INFO:tasks.workunit.client.0.smithi040.stdout:0/270: dread db/d2f/d1d/f3c [0,4194304] 0 2024-02-16T19:24:06.459 INFO:tasks.workunit.client.0.smithi040.stdout:0/271: fdatasync f7 0 2024-02-16T19:24:06.460 INFO:tasks.workunit.client.0.smithi040.stdout:0/272: rename db/f2b to db/d25/d2c/d52/f55 0 2024-02-16T19:24:06.460 INFO:tasks.workunit.client.0.smithi040.stdout:0/273: stat db/c1e 0 2024-02-16T19:24:06.520 INFO:tasks.workunit.client.0.smithi040.stdout:7/180: rmdir d21 39 2024-02-16T19:24:06.524 INFO:tasks.workunit.client.0.smithi040.stdout:4/214: dread dd/de/d45/d37/f3b [0,4194304] 0 2024-02-16T19:24:06.524 INFO:tasks.workunit.client.0.smithi040.stdout:4/215: read dd/de/d10/d13/f46 [2643123,94776] 0 2024-02-16T19:24:06.526 INFO:tasks.workunit.client.0.smithi040.stdout:7/181: write f1d [1136917,98507] 0 2024-02-16T19:24:06.528 INFO:tasks.workunit.client.0.smithi040.stdout:7/182: creat d21/f31 x:0 0 0 2024-02-16T19:24:06.528 INFO:tasks.workunit.client.0.smithi040.stdout:7/183: stat f1f 0 2024-02-16T19:24:06.538 INFO:tasks.workunit.client.0.smithi040.stdout:6/195: dwrite d0/d34/f38 [0,4194304] 0 2024-02-16T19:24:06.538 INFO:tasks.workunit.client.0.smithi040.stdout:9/242: dwrite d3/d4/d11/f19 [0,4194304] 0 2024-02-16T19:24:06.538 INFO:tasks.workunit.client.0.smithi040.stdout:9/243: write d3/d4/d11/d32/f42 [628846,58714] 0 2024-02-16T19:24:06.538 INFO:tasks.workunit.client.0.smithi040.stdout:0/274: dwrite db/d25/d2c/d52/f55 [4194304,4194304] 0 2024-02-16T19:24:06.544 INFO:tasks.workunit.client.0.smithi040.stdout:7/184: dread fb [0,4194304] 0 2024-02-16T19:24:06.544 INFO:tasks.workunit.client.0.smithi040.stdout:7/185: fsync f1e 0 2024-02-16T19:24:06.549 INFO:tasks.workunit.client.0.smithi040.stdout:3/252: sync 2024-02-16T19:24:06.549 INFO:tasks.workunit.client.0.smithi040.stdout:1/226: sync 2024-02-16T19:24:06.550 INFO:tasks.workunit.client.0.smithi040.stdout:8/222: sync 2024-02-16T19:24:06.550 INFO:tasks.workunit.client.0.smithi040.stdout:8/223: truncate d6/f3c 865577 0 2024-02-16T19:24:06.550 INFO:tasks.workunit.client.0.smithi040.stdout:8/224: fdatasync d6/f9 0 2024-02-16T19:24:06.550 INFO:tasks.workunit.client.0.smithi040.stdout:8/225: write d6/d7/d15/f18 [5221282,896] 0 2024-02-16T19:24:06.550 INFO:tasks.workunit.client.0.smithi040.stdout:3/253: mkdir d1/dc/d42/d4a 0 2024-02-16T19:24:06.551 INFO:tasks.workunit.client.0.smithi040.stdout:3/254: chown d1/dc/l24 133 1 2024-02-16T19:24:06.553 INFO:tasks.workunit.client.0.smithi040.stdout:3/255: symlink d1/d7/d33/d38/l4b 0 2024-02-16T19:24:06.554 INFO:tasks.workunit.client.0.smithi040.stdout:8/226: dread d6/d7/d1d/f29 [0,4194304] 0 2024-02-16T19:24:06.557 INFO:tasks.workunit.client.0.smithi040.stdout:8/227: mkdir d6/d2b/d45 0 2024-02-16T19:24:06.557 INFO:tasks.workunit.client.0.smithi040.stdout:8/228: chown d6/d2b 29753247 1 2024-02-16T19:24:06.557 INFO:tasks.workunit.client.0.smithi040.stdout:8/229: fdatasync d6/d2b/d36/f3e 0 2024-02-16T19:24:06.562 INFO:tasks.workunit.client.0.smithi040.stdout:8/230: dread d6/d12/f14 [0,4194304] 0 2024-02-16T19:24:06.562 INFO:tasks.workunit.client.0.smithi040.stdout:8/231: readlink d6/lf 0 2024-02-16T19:24:06.600 INFO:tasks.workunit.client.0.smithi040.stdout:6/196: dwrite d0/d1/d12/d3a/f3d [0,4194304] 0 2024-02-16T19:24:06.601 INFO:tasks.workunit.client.0.smithi040.stdout:9/244: dwrite d3/d4/d11/d29/d18/f39 [0,4194304] 0 2024-02-16T19:24:06.601 INFO:tasks.workunit.client.0.smithi040.stdout:9/245: truncate d3/d1c/f3f 789585 0 2024-02-16T19:24:06.601 INFO:tasks.workunit.client.0.smithi040.stdout:9/246: fsync d3/f1f 0 2024-02-16T19:24:06.601 INFO:tasks.workunit.client.0.smithi040.stdout:3/256: dwrite d1/f34 [0,4194304] 0 2024-02-16T19:24:06.601 INFO:tasks.workunit.client.0.smithi040.stdout:8/232: dwrite f3 [4194304,4194304] 0 2024-02-16T19:24:06.601 INFO:tasks.workunit.client.0.smithi040.stdout:8/233: chown d6/d12/d22/c39 2189204 1 2024-02-16T19:24:06.603 INFO:tasks.workunit.client.0.smithi040.stdout:7/186: dwrite d21/d29/f30 [0,4194304] 0 2024-02-16T19:24:06.603 INFO:tasks.workunit.client.0.smithi040.stdout:0/275: dwrite db/d25/f26 [4194304,4194304] 0 2024-02-16T19:24:06.603 INFO:tasks.workunit.client.0.smithi040.stdout:7/187: creat d21/f32 x:0 0 0 2024-02-16T19:24:06.603 INFO:tasks.workunit.client.0.smithi040.stdout:7/188: chown d21/f25 1107 1 2024-02-16T19:24:06.609 INFO:tasks.workunit.client.0.smithi040.stdout:7/189: unlink ff 0 2024-02-16T19:24:06.612 INFO:tasks.workunit.client.0.smithi040.stdout:7/190: creat d21/d29/f33 x:0 0 0 2024-02-16T19:24:06.612 INFO:tasks.workunit.client.0.smithi040.stdout:7/191: creat d21/d29/f34 x:0 0 0 2024-02-16T19:24:06.612 INFO:tasks.workunit.client.0.smithi040.stdout:7/192: chown d21/f2f 158963 1 2024-02-16T19:24:06.612 INFO:tasks.workunit.client.0.smithi040.stdout:9/247: dread d3/f3c [0,4194304] 0 2024-02-16T19:24:06.612 INFO:tasks.workunit.client.0.smithi040.stdout:7/193: creat d21/f35 x:0 0 0 2024-02-16T19:24:06.613 INFO:tasks.workunit.client.0.smithi040.stdout:8/234: dread d6/d7/d15/f1c [0,4194304] 0 2024-02-16T19:24:06.617 INFO:tasks.workunit.client.0.smithi040.stdout:9/248: symlink d3/l44 0 2024-02-16T19:24:06.618 INFO:tasks.workunit.client.0.smithi040.stdout:9/249: rmdir d3/d4/d11 39 2024-02-16T19:24:06.624 INFO:tasks.workunit.client.0.smithi040.stdout:0/276: write db/d25/f26 [7643935,50064] 0 2024-02-16T19:24:06.633 INFO:tasks.workunit.client.0.smithi040.stdout:9/250: read d3/d4/d11/d29/f24 [2271123,81051] 0 2024-02-16T19:24:06.634 INFO:tasks.workunit.client.0.smithi040.stdout:9/251: symlink d3/d1c/l45 0 2024-02-16T19:24:06.635 INFO:tasks.workunit.client.0.smithi040.stdout:9/252: mkdir d3/d4/d11/d29/d38/d46 0 2024-02-16T19:24:06.635 INFO:tasks.workunit.client.0.smithi040.stdout:9/253: chown d3/l16 14822934 1 2024-02-16T19:24:06.635 INFO:tasks.workunit.client.0.smithi040.stdout:6/197: dwrite d0/d34/f38 [0,4194304] 0 2024-02-16T19:24:06.635 INFO:tasks.workunit.client.0.smithi040.stdout:9/254: write d3/d4/d11/d29/d38/f3e [382829,130798] 0 2024-02-16T19:24:06.639 INFO:tasks.workunit.client.0.smithi040.stdout:0/277: dread db/d25/d34/f3a [0,4194304] 0 2024-02-16T19:24:06.640 INFO:tasks.workunit.client.0.smithi040.stdout:3/257: dwrite d1/dc/f29 [0,4194304] 0 2024-02-16T19:24:06.640 INFO:tasks.workunit.client.0.smithi040.stdout:3/258: dread - d1/dc/f3c zero size 2024-02-16T19:24:06.642 INFO:tasks.workunit.client.0.smithi040.stdout:3/259: mknod d1/c4c 0 2024-02-16T19:24:06.642 INFO:tasks.workunit.client.0.smithi040.stdout:3/260: chown d1/le 1653 1 2024-02-16T19:24:06.644 INFO:tasks.workunit.client.0.smithi040.stdout:8/235: dwrite f5 [0,4194304] 0 2024-02-16T19:24:06.644 INFO:tasks.workunit.client.0.smithi040.stdout:7/194: dwrite d21/f23 [0,4194304] 0 2024-02-16T19:24:06.644 INFO:tasks.workunit.client.0.smithi040.stdout:7/195: dread - d21/f35 zero size 2024-02-16T19:24:06.644 INFO:tasks.workunit.client.0.smithi040.stdout:7/196: dread - d21/f31 zero size 2024-02-16T19:24:06.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/198: write d0/d1/d12/d3a/f3d [2951679,100484] 0 2024-02-16T19:24:06.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/199: dread - d0/f36 zero size 2024-02-16T19:24:06.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/200: fdatasync d0/d1/f1f 0 2024-02-16T19:24:06.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/201: stat d0/d1/d12/d3b 0 2024-02-16T19:24:06.653 INFO:tasks.workunit.client.0.smithi040.stdout:8/236: write f5 [2856792,89936] 0 2024-02-16T19:24:06.653 INFO:tasks.workunit.client.0.smithi040.stdout:8/237: readlink d6/d7/l37 0 2024-02-16T19:24:06.654 INFO:tasks.workunit.client.0.smithi040.stdout:7/197: unlink l19 0 2024-02-16T19:24:06.654 INFO:tasks.workunit.client.0.smithi040.stdout:9/255: write d3/d4/d11/f19 [3698196,7337] 0 2024-02-16T19:24:06.656 INFO:tasks.workunit.client.0.smithi040.stdout:6/202: unlink d0/d34/l27 0 2024-02-16T19:24:06.658 INFO:tasks.workunit.client.0.smithi040.stdout:8/238: rename d6/d2b/f40 to d6/f46 0 2024-02-16T19:24:06.659 INFO:tasks.workunit.client.0.smithi040.stdout:8/239: write d6/d12/f31 [660865,79874] 0 2024-02-16T19:24:06.659 INFO:tasks.workunit.client.0.smithi040.stdout:8/240: write d6/f3f [26417,58547] 0 2024-02-16T19:24:06.659 INFO:tasks.workunit.client.0.smithi040.stdout:8/241: readlink d6/d7/le 0 2024-02-16T19:24:06.659 INFO:tasks.workunit.client.0.smithi040.stdout:8/242: truncate d6/d7/f32 363009 0 2024-02-16T19:24:06.659 INFO:tasks.workunit.client.0.smithi040.stdout:7/198: mkdir d21/d29/d36 0 2024-02-16T19:24:06.659 INFO:tasks.workunit.client.0.smithi040.stdout:7/199: truncate fa 2711282 0 2024-02-16T19:24:06.660 INFO:tasks.workunit.client.0.smithi040.stdout:9/256: symlink d3/d4/d11/d29/d18/d31/l47 0 2024-02-16T19:24:06.661 INFO:tasks.workunit.client.0.smithi040.stdout:2/159: sync 2024-02-16T19:24:06.661 INFO:tasks.workunit.client.0.smithi040.stdout:5/185: sync 2024-02-16T19:24:06.661 INFO:tasks.workunit.client.0.smithi040.stdout:4/216: sync 2024-02-16T19:24:06.662 INFO:tasks.workunit.client.0.smithi040.stdout:2/160: chown d5/d7/d10/l11 123 1 2024-02-16T19:24:06.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/217: dread - dd/de/d10/f20 zero size 2024-02-16T19:24:06.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/218: creat dd/f49 x:0 0 0 2024-02-16T19:24:06.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/219: creat dd/de/d10/f4a x:0 0 0 2024-02-16T19:24:06.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/220: fdatasync f0 0 2024-02-16T19:24:06.662 INFO:tasks.workunit.client.0.smithi040.stdout:9/257: creat d3/d4/d11/d29/f48 x:0 0 0 2024-02-16T19:24:06.666 INFO:tasks.workunit.client.0.smithi040.stdout:8/243: link d6/d12/d22/f42 d6/d7/d15/f47 0 2024-02-16T19:24:06.666 INFO:tasks.workunit.client.0.smithi040.stdout:8/244: chown d6/d7/l37 1 1 2024-02-16T19:24:06.666 INFO:tasks.workunit.client.0.smithi040.stdout:8/245: readlink d6/d12/l2a 0 2024-02-16T19:24:06.666 INFO:tasks.workunit.client.0.smithi040.stdout:8/246: creat d6/d2b/f48 x:0 0 0 2024-02-16T19:24:06.670 INFO:tasks.workunit.client.0.smithi040.stdout:4/221: mknod dd/de/d10/d13/c4b 0 2024-02-16T19:24:06.670 INFO:tasks.workunit.client.0.smithi040.stdout:4/222: chown dd/de 2125 1 2024-02-16T19:24:06.670 INFO:tasks.workunit.client.0.smithi040.stdout:4/223: write dd/de/d10/f20 [528015,101250] 0 2024-02-16T19:24:06.671 INFO:tasks.workunit.client.0.smithi040.stdout:1/227: sync 2024-02-16T19:24:06.672 INFO:tasks.workunit.client.0.smithi040.stdout:9/258: mknod d3/c49 0 2024-02-16T19:24:06.672 INFO:tasks.workunit.client.0.smithi040.stdout:0/278: dwrite db/f53 [0,4194304] 0 2024-02-16T19:24:06.674 INFO:tasks.workunit.client.0.smithi040.stdout:8/247: creat d6/d2b/d44/f49 x:0 0 0 2024-02-16T19:24:06.675 INFO:tasks.workunit.client.0.smithi040.stdout:4/224: rename fa to dd/de/d45/f4c 0 2024-02-16T19:24:06.675 INFO:tasks.workunit.client.0.smithi040.stdout:4/225: write dd/f23 [1278722,44095] 0 2024-02-16T19:24:06.676 INFO:tasks.workunit.client.0.smithi040.stdout:1/228: mkdir d17/d26/d4c 0 2024-02-16T19:24:06.678 INFO:tasks.workunit.client.0.smithi040.stdout:0/279: mkdir db/d2f/d23/d56 0 2024-02-16T19:24:06.679 INFO:tasks.workunit.client.0.smithi040.stdout:4/226: link dd/de/d10/l25 dd/de/d45/l4d 0 2024-02-16T19:24:06.679 INFO:tasks.workunit.client.0.smithi040.stdout:4/227: write dd/f1d [1302381,118233] 0 2024-02-16T19:24:06.679 INFO:tasks.workunit.client.0.smithi040.stdout:4/228: truncate dd/de/f2a 1437695 0 2024-02-16T19:24:06.679 INFO:tasks.workunit.client.0.smithi040.stdout:4/229: chown dd/de/f2a 851708 1 2024-02-16T19:24:06.679 INFO:tasks.workunit.client.0.smithi040.stdout:1/229: symlink d17/d43/d38/d4a/l4d 0 2024-02-16T19:24:06.680 INFO:tasks.workunit.client.0.smithi040.stdout:0/280: creat db/d25/d2c/d52/f57 x:0 0 0 2024-02-16T19:24:06.682 INFO:tasks.workunit.client.0.smithi040.stdout:1/230: creat d17/d26/d4c/f4e x:0 0 0 2024-02-16T19:24:06.684 INFO:tasks.workunit.client.0.smithi040.stdout:1/231: unlink f16 0 2024-02-16T19:24:06.684 INFO:tasks.workunit.client.0.smithi040.stdout:1/232: readlink d17/l18 0 2024-02-16T19:24:06.687 INFO:tasks.workunit.client.0.smithi040.stdout:4/230: getdents dd/de/d10 0 2024-02-16T19:24:06.687 INFO:tasks.workunit.client.0.smithi040.stdout:1/233: dread f8 [0,4194304] 0 2024-02-16T19:24:06.690 INFO:tasks.workunit.client.0.smithi040.stdout:0/281: dread db/fd [0,4194304] 0 2024-02-16T19:24:06.690 INFO:tasks.workunit.client.0.smithi040.stdout:0/282: truncate db/d25/d34/f3a 4250483 0 2024-02-16T19:24:06.692 INFO:tasks.workunit.client.0.smithi040.stdout:0/283: link db/f20 db/d2f/d1d/d21/d2e/d49/f58 0 2024-02-16T19:24:06.699 INFO:tasks.workunit.client.0.smithi040.stdout:0/284: getdents db/d2f/d1d/d21/d2e/d49 0 2024-02-16T19:24:06.699 INFO:tasks.workunit.client.0.smithi040.stdout:0/285: creat db/de/f59 x:0 0 0 2024-02-16T19:24:06.701 INFO:tasks.workunit.client.0.smithi040.stdout:0/286: creat db/d25/d2c/f5a x:0 0 0 2024-02-16T19:24:06.705 INFO:tasks.workunit.client.0.smithi040.stdout:0/287: getdents db/d25/d2c 0 2024-02-16T19:24:06.705 INFO:tasks.workunit.client.0.smithi040.stdout:0/288: chown db/d2f/d1d/f3c 10 1 2024-02-16T19:24:06.716 INFO:tasks.workunit.client.0.smithi040.stdout:3/261: dwrite d1/d4/f43 [0,4194304] 0 2024-02-16T19:24:06.716 INFO:tasks.workunit.client.0.smithi040.stdout:3/262: chown d1/dc/f3c 488986 1 2024-02-16T19:24:06.717 INFO:tasks.workunit.client.0.smithi040.stdout:0/289: truncate db/f1b 3873500 0 2024-02-16T19:24:06.717 INFO:tasks.workunit.client.0.smithi040.stdout:0/290: truncate db/de/f59 193128 0 2024-02-16T19:24:06.717 INFO:tasks.workunit.client.0.smithi040.stdout:0/291: readlink db/d2f/d23/l48 0 2024-02-16T19:24:06.721 INFO:tasks.workunit.client.0.smithi040.stdout:3/263: mknod d1/d7/d33/d38/c4d 0 2024-02-16T19:24:06.722 INFO:tasks.workunit.client.0.smithi040.stdout:0/292: mkdir db/d2f/d1d/d21/d2e/d49/d5b 0 2024-02-16T19:24:06.724 INFO:tasks.workunit.client.0.smithi040.stdout:3/264: creat d1/d7/f4e x:0 0 0 2024-02-16T19:24:06.724 INFO:tasks.workunit.client.0.smithi040.stdout:3/265: stat d1/d7/d33/c3b 0 2024-02-16T19:24:06.725 INFO:tasks.workunit.client.0.smithi040.stdout:0/293: symlink db/d25/d2c/d52/l5c 0 2024-02-16T19:24:06.725 INFO:tasks.workunit.client.0.smithi040.stdout:3/266: dread d1/d4/f19 [0,4194304] 0 2024-02-16T19:24:06.726 INFO:tasks.workunit.client.0.smithi040.stdout:0/294: truncate db/d2f/d1d/d21/f51 212283 0 2024-02-16T19:24:06.726 INFO:tasks.workunit.client.0.smithi040.stdout:0/295: chown db/f53 28 1 2024-02-16T19:24:06.728 INFO:tasks.workunit.client.0.smithi040.stdout:0/296: rename db/d2f/d1d/f3c to db/d2f/d1d/d21/d2e/d49/d5b/f5d 0 2024-02-16T19:24:06.728 INFO:tasks.workunit.client.0.smithi040.stdout:0/297: write db/d25/f4f [714555,20331] 0 2024-02-16T19:24:06.728 INFO:tasks.workunit.client.0.smithi040.stdout:0/298: truncate db/d25/f4f 1714022 0 2024-02-16T19:24:06.729 INFO:tasks.workunit.client.0.smithi040.stdout:0/299: mkdir db/d25/d34/d5e 0 2024-02-16T19:24:06.729 INFO:tasks.workunit.client.0.smithi040.stdout:0/300: dread - db/d25/d2c/d52/f57 zero size 2024-02-16T19:24:06.731 INFO:tasks.workunit.client.0.smithi040.stdout:0/301: rename db/de/l1f to db/d25/l5f 0 2024-02-16T19:24:06.732 INFO:tasks.workunit.client.0.smithi040.stdout:9/259: dread d3/f1f [0,4194304] 0 2024-02-16T19:24:06.732 INFO:tasks.workunit.client.0.smithi040.stdout:0/302: symlink db/d2f/d23/d3b/l60 0 2024-02-16T19:24:06.732 INFO:tasks.workunit.client.0.smithi040.stdout:0/303: creat db/d25/f61 x:0 0 0 2024-02-16T19:24:06.732 INFO:tasks.workunit.client.0.smithi040.stdout:0/304: stat db/d2f/d1d/d21/d2e/d49 0 2024-02-16T19:24:06.744 INFO:tasks.workunit.client.0.smithi040.stdout:3/267: dread d1/d4/fa [0,4194304] 0 2024-02-16T19:24:06.745 INFO:tasks.workunit.client.0.smithi040.stdout:1/234: dwrite d17/d29/f2b [0,4194304] 0 2024-02-16T19:24:06.745 INFO:tasks.workunit.client.0.smithi040.stdout:1/235: write d17/f2c [853237,69833] 0 2024-02-16T19:24:06.752 INFO:tasks.workunit.client.0.smithi040.stdout:7/200: dwrite d21/f35 [0,4194304] 0 2024-02-16T19:24:06.752 INFO:tasks.workunit.client.0.smithi040.stdout:6/203: dwrite d0/d30/d32/f2d [0,4194304] 0 2024-02-16T19:24:06.752 INFO:tasks.workunit.client.0.smithi040.stdout:7/201: creat d21/d29/f37 x:0 0 0 2024-02-16T19:24:06.752 INFO:tasks.workunit.client.0.smithi040.stdout:6/204: creat d0/f3f x:0 0 0 2024-02-16T19:24:06.752 INFO:tasks.workunit.client.0.smithi040.stdout:2/161: dwrite d5/d7/d1e/f20 [0,4194304] 0 2024-02-16T19:24:06.752 INFO:tasks.workunit.client.0.smithi040.stdout:2/162: chown d5/d7/d12/d1b/f2f 292863026 1 2024-02-16T19:24:06.753 INFO:tasks.workunit.client.0.smithi040.stdout:8/248: dwrite d6/d12/f1f [0,4194304] 0 2024-02-16T19:24:06.753 INFO:tasks.workunit.client.0.smithi040.stdout:4/231: dwrite dd/de/d10/d13/f46 [0,4194304] 0 2024-02-16T19:24:06.753 INFO:tasks.workunit.client.0.smithi040.stdout:9/260: dwrite d3/d4/d11/d32/f43 [0,4194304] 0 2024-02-16T19:24:06.760 INFO:tasks.workunit.client.0.smithi040.stdout:2/163: dread d5/d7/d12/f13 [0,4194304] 0 2024-02-16T19:24:06.760 INFO:tasks.workunit.client.0.smithi040.stdout:2/164: fdatasync d5/d7/dd/f2c 0 2024-02-16T19:24:06.760 INFO:tasks.workunit.client.0.smithi040.stdout:2/165: write d5/d7/f24 [351474,57828] 0 2024-02-16T19:24:06.760 INFO:tasks.workunit.client.0.smithi040.stdout:3/268: dwrite d1/d7/f4e [0,4194304] 0 2024-02-16T19:24:06.762 INFO:tasks.workunit.client.0.smithi040.stdout:6/205: creat d0/d1/d12/d3b/f40 x:0 0 0 2024-02-16T19:24:06.762 INFO:tasks.workunit.client.0.smithi040.stdout:6/206: write d0/fe [1562232,125333] 0 2024-02-16T19:24:06.766 INFO:tasks.workunit.client.0.smithi040.stdout:8/249: creat d6/d12/d41/f4a x:0 0 0 2024-02-16T19:24:06.770 INFO:tasks.workunit.client.0.smithi040.stdout:8/250: dread d6/d7/f32 [0,4194304] 0 2024-02-16T19:24:06.770 INFO:tasks.workunit.client.0.smithi040.stdout:8/251: read f5 [544464,121003] 0 2024-02-16T19:24:06.778 INFO:tasks.workunit.client.0.smithi040.stdout:9/261: mkdir d3/d4/d4a 0 2024-02-16T19:24:06.780 INFO:tasks.workunit.client.0.smithi040.stdout:3/269: unlink d1/d4/c2a 0 2024-02-16T19:24:06.781 INFO:tasks.workunit.client.0.smithi040.stdout:9/262: truncate d3/d4/f2c 3495840 0 2024-02-16T19:24:06.781 INFO:tasks.workunit.client.0.smithi040.stdout:9/263: readlink d3/d4/d11/d29/d18/d31/l34 0 2024-02-16T19:24:06.782 INFO:tasks.workunit.client.0.smithi040.stdout:2/166: dwrite d5/d7/dd/f2c [0,4194304] 0 2024-02-16T19:24:06.782 INFO:tasks.workunit.client.0.smithi040.stdout:3/270: creat d1/d4/f4f x:0 0 0 2024-02-16T19:24:06.783 INFO:tasks.workunit.client.0.smithi040.stdout:9/264: symlink d3/d4/d11/d29/l4b 0 2024-02-16T19:24:06.785 INFO:tasks.workunit.client.0.smithi040.stdout:3/271: rmdir d1/d7/d33 39 2024-02-16T19:24:06.786 INFO:tasks.workunit.client.0.smithi040.stdout:9/265: creat d3/d4/d11/d29/d18/d31/f4c x:0 0 0 2024-02-16T19:24:06.787 INFO:tasks.workunit.client.0.smithi040.stdout:3/272: creat d1/dc/d42/d4a/f50 x:0 0 0 2024-02-16T19:24:06.788 INFO:tasks.workunit.client.0.smithi040.stdout:9/266: dread d3/d4/d11/d32/f42 [0,4194304] 0 2024-02-16T19:24:06.789 INFO:tasks.workunit.client.0.smithi040.stdout:7/202: rmdir d21 39 2024-02-16T19:24:06.789 INFO:tasks.workunit.client.0.smithi040.stdout:7/203: dread - d21/f31 zero size 2024-02-16T19:24:06.789 INFO:tasks.workunit.client.0.smithi040.stdout:7/204: stat d21/f32 0 2024-02-16T19:24:06.791 INFO:tasks.workunit.client.0.smithi040.stdout:9/267: dread d3/d1c/f3f [0,4194304] 0 2024-02-16T19:24:06.791 INFO:tasks.workunit.client.0.smithi040.stdout:8/252: dwrite d6/d2b/d44/f49 [0,4194304] 0 2024-02-16T19:24:06.791 INFO:tasks.workunit.client.0.smithi040.stdout:6/207: dwrite d0/d1/d12/d3a/f3d [4194304,4194304] 0 2024-02-16T19:24:06.791 INFO:tasks.workunit.client.0.smithi040.stdout:6/208: readlink d0/l13 0 2024-02-16T19:24:06.791 INFO:tasks.workunit.client.0.smithi040.stdout:6/209: dread - d0/f3f zero size 2024-02-16T19:24:06.791 INFO:tasks.workunit.client.0.smithi040.stdout:7/205: creat d21/d29/f38 x:0 0 0 2024-02-16T19:24:06.792 INFO:tasks.workunit.client.0.smithi040.stdout:9/268: creat d3/d4/f4d x:0 0 0 2024-02-16T19:24:06.793 INFO:tasks.workunit.client.0.smithi040.stdout:3/273: dread d1/d4/f2e [0,4194304] 0 2024-02-16T19:24:06.793 INFO:tasks.workunit.client.0.smithi040.stdout:3/274: fdatasync d1/ff 0 2024-02-16T19:24:06.796 INFO:tasks.workunit.client.0.smithi040.stdout:6/210: mknod d0/d1/d12/d3b/c41 0 2024-02-16T19:24:06.796 INFO:tasks.workunit.client.0.smithi040.stdout:6/211: dread - d0/d1/d12/f18 zero size 2024-02-16T19:24:06.796 INFO:tasks.workunit.client.0.smithi040.stdout:6/212: fsync d0/d30/d32/f1a 0 2024-02-16T19:24:06.798 INFO:tasks.workunit.client.0.smithi040.stdout:3/275: rmdir d1/d7/d30 39 2024-02-16T19:24:06.800 INFO:tasks.workunit.client.0.smithi040.stdout:2/167: write d5/d7/d1e/f20 [3988470,13374] 0 2024-02-16T19:24:06.800 INFO:tasks.workunit.client.0.smithi040.stdout:3/276: mknod d1/dc/d42/c51 0 2024-02-16T19:24:06.800 INFO:tasks.workunit.client.0.smithi040.stdout:2/168: stat d5/d7/d1e 0 2024-02-16T19:24:06.800 INFO:tasks.workunit.client.0.smithi040.stdout:3/277: truncate d1/d4/f19 1772138 0 2024-02-16T19:24:06.800 INFO:tasks.workunit.client.0.smithi040.stdout:3/278: readlink d1/dc/l24 0 2024-02-16T19:24:06.807 INFO:tasks.workunit.client.0.smithi040.stdout:2/169: rmdir d5/d7/d10/d16 0 2024-02-16T19:24:06.807 INFO:tasks.workunit.client.0.smithi040.stdout:2/170: write d5/d7/f24 [1448894,103148] 0 2024-02-16T19:24:06.807 INFO:tasks.workunit.client.0.smithi040.stdout:2/171: fsync d5/d7/d12/d1b/f2f 0 2024-02-16T19:24:06.809 INFO:tasks.workunit.client.0.smithi040.stdout:3/279: fdatasync d1/d4/f25 0 2024-02-16T19:24:06.809 INFO:tasks.workunit.client.0.smithi040.stdout:3/280: write d1/d4/f4f [1034805,100270] 0 2024-02-16T19:24:06.809 INFO:tasks.workunit.client.0.smithi040.stdout:3/281: chown d1/dc/l2f 9687 1 2024-02-16T19:24:06.811 INFO:tasks.workunit.client.0.smithi040.stdout:2/172: rename d5/d8/f1f to d5/d7/d12/d1b/f30 0 2024-02-16T19:24:06.811 INFO:tasks.workunit.client.0.smithi040.stdout:2/173: chown d5/d7/d12/d1b/c2d 7019176 1 2024-02-16T19:24:06.812 INFO:tasks.workunit.client.0.smithi040.stdout:2/174: stat d5/d7/dd/l1c 0 2024-02-16T19:24:06.823 INFO:tasks.workunit.client.0.smithi040.stdout:8/253: dwrite d6/f3f [0,4194304] 0 2024-02-16T19:24:06.823 INFO:tasks.workunit.client.0.smithi040.stdout:8/254: write d6/d7/d1d/f29 [5081606,23901] 0 2024-02-16T19:24:06.823 INFO:tasks.workunit.client.0.smithi040.stdout:8/255: write d6/d12/d22/f23 [1792504,18108] 0 2024-02-16T19:24:06.825 INFO:tasks.workunit.client.0.smithi040.stdout:3/282: dwrite d1/d7/f2b [0,4194304] 0 2024-02-16T19:24:06.825 INFO:tasks.workunit.client.0.smithi040.stdout:3/283: fsync d1/f34 0 2024-02-16T19:24:06.827 INFO:tasks.workunit.client.0.smithi040.stdout:8/256: unlink d6/d7/d1d/c3a 0 2024-02-16T19:24:06.827 INFO:tasks.workunit.client.0.smithi040.stdout:7/206: dwrite d21/d29/f30 [0,4194304] 0 2024-02-16T19:24:06.827 INFO:tasks.workunit.client.0.smithi040.stdout:6/213: dwrite d0/d30/d32/f2d [0,4194304] 0 2024-02-16T19:24:06.827 INFO:tasks.workunit.client.0.smithi040.stdout:6/214: write d0/f2e [4655156,99649] 0 2024-02-16T19:24:06.827 INFO:tasks.workunit.client.0.smithi040.stdout:6/215: chown d0/d30/d32/l1b 1011655510 1 2024-02-16T19:24:06.827 INFO:tasks.workunit.client.0.smithi040.stdout:7/207: read d21/f2f [111383,83397] 0 2024-02-16T19:24:06.830 INFO:tasks.workunit.client.0.smithi040.stdout:3/284: dread d1/d7/d30/f40 [0,4194304] 0 2024-02-16T19:24:06.832 INFO:tasks.workunit.client.0.smithi040.stdout:8/257: read d6/d7/d1d/f29 [1206859,99856] 0 2024-02-16T19:24:06.835 INFO:tasks.workunit.client.0.smithi040.stdout:2/175: dread d5/d7/d1e/f20 [0,4194304] 0 2024-02-16T19:24:06.835 INFO:tasks.workunit.client.0.smithi040.stdout:2/176: write d5/d7/d12/d1b/f30 [1013172,120906] 0 2024-02-16T19:24:06.835 INFO:tasks.workunit.client.0.smithi040.stdout:2/177: stat d5/d7/c1d 0 2024-02-16T19:24:06.836 INFO:tasks.workunit.client.0.smithi040.stdout:6/216: creat d0/d1/d12/d3a/f42 x:0 0 0 2024-02-16T19:24:06.838 INFO:tasks.workunit.client.0.smithi040.stdout:7/208: rename l14 to d21/d29/l39 0 2024-02-16T19:24:06.838 INFO:tasks.workunit.client.0.smithi040.stdout:7/209: creat d21/d29/f3a x:0 0 0 2024-02-16T19:24:06.839 INFO:tasks.workunit.client.0.smithi040.stdout:8/258: creat d6/d12/f4b x:0 0 0 2024-02-16T19:24:06.839 INFO:tasks.workunit.client.0.smithi040.stdout:8/259: dread - d6/d7/d15/f47 zero size 2024-02-16T19:24:06.839 INFO:tasks.workunit.client.0.smithi040.stdout:3/285: dread d1/d7/f4e [0,4194304] 0 2024-02-16T19:24:06.840 INFO:tasks.workunit.client.0.smithi040.stdout:3/286: chown d1/d4/fa 325175 1 2024-02-16T19:24:06.841 INFO:tasks.workunit.client.0.smithi040.stdout:6/217: mknod d0/d1/d12/c43 0 2024-02-16T19:24:06.842 INFO:tasks.workunit.client.0.smithi040.stdout:7/210: mknod d21/c3b 0 2024-02-16T19:24:06.843 INFO:tasks.workunit.client.0.smithi040.stdout:5/186: sync 2024-02-16T19:24:06.843 INFO:tasks.workunit.client.0.smithi040.stdout:5/187: chown d1/da/f35 3389 1 2024-02-16T19:24:06.844 INFO:tasks.workunit.client.0.smithi040.stdout:5/188: truncate d1/f13 2236074 0 2024-02-16T19:24:06.844 INFO:tasks.workunit.client.0.smithi040.stdout:5/189: chown d1/c34 2 1 2024-02-16T19:24:06.844 INFO:tasks.workunit.client.0.smithi040.stdout:5/190: write d1/da/d1c/d21/f2f [491304,53629] 0 2024-02-16T19:24:06.844 INFO:tasks.workunit.client.0.smithi040.stdout:5/191: dread - d1/da/f35 zero size 2024-02-16T19:24:06.846 INFO:tasks.workunit.client.0.smithi040.stdout:5/192: unlink d1/da/ff 0 2024-02-16T19:24:06.847 INFO:tasks.workunit.client.0.smithi040.stdout:5/193: creat d1/da/d1c/f37 x:0 0 0 2024-02-16T19:24:06.851 INFO:tasks.workunit.client.0.smithi040.stdout:7/211: write f9 [431373,17367] 0 2024-02-16T19:24:06.853 INFO:tasks.workunit.client.0.smithi040.stdout:5/194: dread d1/f20 [0,4194304] 0 2024-02-16T19:24:06.853 INFO:tasks.workunit.client.0.smithi040.stdout:5/195: truncate d1/da/f35 169823 0 2024-02-16T19:24:06.853 INFO:tasks.workunit.client.0.smithi040.stdout:5/196: stat d1/c34 0 2024-02-16T19:24:06.853 INFO:tasks.workunit.client.0.smithi040.stdout:5/197: dread - d1/da/f2a zero size 2024-02-16T19:24:06.853 INFO:tasks.workunit.client.0.smithi040.stdout:7/212: rename d21/f31 to d21/f3c 0 2024-02-16T19:24:06.854 INFO:tasks.workunit.client.0.smithi040.stdout:5/198: mknod d1/c38 0 2024-02-16T19:24:06.854 INFO:tasks.workunit.client.0.smithi040.stdout:5/199: write d1/f1d [304541,625] 0 2024-02-16T19:24:06.854 INFO:tasks.workunit.client.0.smithi040.stdout:0/305: sync 2024-02-16T19:24:06.854 INFO:tasks.workunit.client.0.smithi040.stdout:7/213: mknod d21/c3d 0 2024-02-16T19:24:06.854 INFO:tasks.workunit.client.0.smithi040.stdout:5/200: mknod d1/da/d1c/d21/c39 0 2024-02-16T19:24:06.855 INFO:tasks.workunit.client.0.smithi040.stdout:0/306: symlink db/d25/d34/l62 0 2024-02-16T19:24:06.855 INFO:tasks.workunit.client.0.smithi040.stdout:0/307: creat db/f63 x:0 0 0 2024-02-16T19:24:06.856 INFO:tasks.workunit.client.0.smithi040.stdout:5/201: rename d1/cb to d1/c3a 0 2024-02-16T19:24:06.857 INFO:tasks.workunit.client.0.smithi040.stdout:0/308: mknod db/d2f/c64 0 2024-02-16T19:24:06.858 INFO:tasks.workunit.client.0.smithi040.stdout:7/214: getdents d21/d29 0 2024-02-16T19:24:06.860 INFO:tasks.workunit.client.0.smithi040.stdout:5/202: truncate d1/f20 2783907 0 2024-02-16T19:24:06.861 INFO:tasks.workunit.client.0.smithi040.stdout:0/309: symlink db/d2f/d1d/d21/d2e/d49/l65 0 2024-02-16T19:24:06.862 INFO:tasks.workunit.client.0.smithi040.stdout:7/215: link l1a d21/l3e 0 2024-02-16T19:24:06.863 INFO:tasks.workunit.client.0.smithi040.stdout:5/203: mknod d1/da/d1c/d21/c3b 0 2024-02-16T19:24:06.863 INFO:tasks.workunit.client.0.smithi040.stdout:5/204: write d1/f1d [1400654,127321] 0 2024-02-16T19:24:06.864 INFO:tasks.workunit.client.0.smithi040.stdout:7/216: creat d21/d29/f3f x:0 0 0 2024-02-16T19:24:06.865 INFO:tasks.workunit.client.0.smithi040.stdout:5/205: mkdir d1/d3c 0 2024-02-16T19:24:06.865 INFO:tasks.workunit.client.0.smithi040.stdout:5/206: readlink d1/l32 0 2024-02-16T19:24:06.866 INFO:tasks.workunit.client.0.smithi040.stdout:7/217: symlink d21/l40 0 2024-02-16T19:24:06.867 INFO:tasks.workunit.client.0.smithi040.stdout:5/207: mknod d1/da/c3d 0 2024-02-16T19:24:06.868 INFO:tasks.workunit.client.0.smithi040.stdout:7/218: mknod d21/d29/d36/c41 0 2024-02-16T19:24:06.871 INFO:tasks.workunit.client.0.smithi040.stdout:7/219: unlink fb 0 2024-02-16T19:24:06.871 INFO:tasks.workunit.client.0.smithi040.stdout:7/220: dread - d21/d29/f38 zero size 2024-02-16T19:24:06.872 INFO:tasks.workunit.client.0.smithi040.stdout:0/310: dread db/f1b [0,4194304] 0 2024-02-16T19:24:06.872 INFO:tasks.workunit.client.0.smithi040.stdout:0/311: truncate db/f20 1335469 0 2024-02-16T19:24:06.872 INFO:tasks.workunit.client.0.smithi040.stdout:0/312: creat db/d2f/f66 x:0 0 0 2024-02-16T19:24:06.872 INFO:tasks.workunit.client.0.smithi040.stdout:0/313: readlink db/de/l4b 0 2024-02-16T19:24:06.872 INFO:tasks.workunit.client.0.smithi040.stdout:0/314: truncate db/d2f/d1d/f46 774393 0 2024-02-16T19:24:06.872 INFO:tasks.workunit.client.0.smithi040.stdout:0/315: dread - db/d2f/f3e zero size 2024-02-16T19:24:06.873 INFO:tasks.workunit.client.0.smithi040.stdout:6/218: dwrite d0/d1/d12/d3a/f42 [0,4194304] 0 2024-02-16T19:24:06.873 INFO:tasks.workunit.client.0.smithi040.stdout:6/219: chown d0/f5 142679 1 2024-02-16T19:24:06.876 INFO:tasks.workunit.client.0.smithi040.stdout:5/208: rmdir d1/da/d1c/d21 39 2024-02-16T19:24:06.876 INFO:tasks.workunit.client.0.smithi040.stdout:5/209: creat d1/da/f3e x:0 0 0 2024-02-16T19:24:06.879 INFO:tasks.workunit.client.0.smithi040.stdout:7/221: getdents d21 0 2024-02-16T19:24:06.884 INFO:tasks.workunit.client.0.smithi040.stdout:5/210: truncate d1/da/d1c/f26 817555 0 2024-02-16T19:24:06.885 INFO:tasks.workunit.client.0.smithi040.stdout:7/222: mknod d21/c42 0 2024-02-16T19:24:06.885 INFO:tasks.workunit.client.0.smithi040.stdout:6/220: write d0/f2e [1566928,41717] 0 2024-02-16T19:24:06.886 INFO:tasks.workunit.client.0.smithi040.stdout:6/221: write d0/f2e [4588566,4211] 0 2024-02-16T19:24:06.886 INFO:tasks.workunit.client.0.smithi040.stdout:6/222: dread - d0/d34/f3c zero size 2024-02-16T19:24:06.886 INFO:tasks.workunit.client.0.smithi040.stdout:6/223: readlink d0/d1/l7 0 2024-02-16T19:24:06.886 INFO:tasks.workunit.client.0.smithi040.stdout:7/223: dread f1e [0,4194304] 0 2024-02-16T19:24:06.887 INFO:tasks.workunit.client.0.smithi040.stdout:6/224: link d0/d1/l7 d0/d1/d12/d3a/l44 0 2024-02-16T19:24:06.888 INFO:tasks.workunit.client.0.smithi040.stdout:6/225: rmdir d0/d30 39 2024-02-16T19:24:06.889 INFO:tasks.workunit.client.0.smithi040.stdout:1/236: sync 2024-02-16T19:24:06.889 INFO:tasks.workunit.client.0.smithi040.stdout:9/269: sync 2024-02-16T19:24:06.889 INFO:tasks.workunit.client.0.smithi040.stdout:4/232: sync 2024-02-16T19:24:06.889 INFO:tasks.workunit.client.0.smithi040.stdout:4/233: write dd/de/d10/d13/f15 [1620870,18895] 0 2024-02-16T19:24:06.889 INFO:tasks.workunit.client.0.smithi040.stdout:7/224: mknod d21/d29/c43 0 2024-02-16T19:24:06.890 INFO:tasks.workunit.client.0.smithi040.stdout:9/270: rmdir d3/d4/d11/d29/d36 39 2024-02-16T19:24:06.891 INFO:tasks.workunit.client.0.smithi040.stdout:9/271: chown d3/d4/d11/d29/d18/l1e 6825859 1 2024-02-16T19:24:06.891 INFO:tasks.workunit.client.0.smithi040.stdout:9/272: truncate d3/d4/d11/f19 4572882 0 2024-02-16T19:24:06.891 INFO:tasks.workunit.client.0.smithi040.stdout:4/234: symlink dd/de/d45/l4e 0 2024-02-16T19:24:06.893 INFO:tasks.workunit.client.0.smithi040.stdout:7/225: mknod d21/d29/c44 0 2024-02-16T19:24:06.894 INFO:tasks.workunit.client.0.smithi040.stdout:5/211: truncate d1/f20 844905 0 2024-02-16T19:24:06.895 INFO:tasks.workunit.client.0.smithi040.stdout:1/237: dread d17/f1f [0,4194304] 0 2024-02-16T19:24:06.895 INFO:tasks.workunit.client.0.smithi040.stdout:1/238: dread - d17/d43/f39 zero size 2024-02-16T19:24:06.898 INFO:tasks.workunit.client.0.smithi040.stdout:9/273: write d3/f3c [2121809,79829] 0 2024-02-16T19:24:06.899 INFO:tasks.workunit.client.0.smithi040.stdout:5/212: creat d1/d3c/f3f x:0 0 0 2024-02-16T19:24:06.902 INFO:tasks.workunit.client.0.smithi040.stdout:1/239: mknod d17/c4f 0 2024-02-16T19:24:06.903 INFO:tasks.workunit.client.0.smithi040.stdout:9/274: mknod d3/d4/d4a/c4e 0 2024-02-16T19:24:06.905 INFO:tasks.workunit.client.0.smithi040.stdout:1/240: getdents d17/d43/d38 0 2024-02-16T19:24:06.909 INFO:tasks.workunit.client.0.smithi040.stdout:1/241: rename d17/l3a to d17/d3c/l50 0 2024-02-16T19:24:06.910 INFO:tasks.workunit.client.0.smithi040.stdout:1/242: write d17/d29/f32 [996241,38732] 0 2024-02-16T19:24:06.910 INFO:tasks.workunit.client.0.smithi040.stdout:1/243: chown d17/d3c 5892841 1 2024-02-16T19:24:06.911 INFO:tasks.workunit.client.0.smithi040.stdout:1/244: mknod d17/d26/c51 0 2024-02-16T19:24:06.911 INFO:tasks.workunit.client.0.smithi040.stdout:1/245: write fd [4116059,60726] 0 2024-02-16T19:24:06.911 INFO:tasks.workunit.client.0.smithi040.stdout:1/246: chown f8 7670792 1 2024-02-16T19:24:06.912 INFO:tasks.workunit.client.0.smithi040.stdout:1/247: mknod d17/c52 0 2024-02-16T19:24:06.913 INFO:tasks.workunit.client.0.smithi040.stdout:1/248: getdents d17/d26/d3d 0 2024-02-16T19:24:06.930 INFO:tasks.workunit.client.0.smithi040.stdout:6/226: dwrite d0/d1/f29 [0,4194304] 0 2024-02-16T19:24:06.930 INFO:tasks.workunit.client.0.smithi040.stdout:4/235: dwrite dd/f43 [0,4194304] 0 2024-02-16T19:24:06.930 INFO:tasks.workunit.client.0.smithi040.stdout:4/236: stat dd/f19 0 2024-02-16T19:24:06.931 INFO:tasks.workunit.client.0.smithi040.stdout:7/226: dwrite d21/f28 [0,4194304] 0 2024-02-16T19:24:06.931 INFO:tasks.workunit.client.0.smithi040.stdout:9/275: dwrite d3/d4/d11/d29/f24 [4194304,4194304] 0 2024-02-16T19:24:06.931 INFO:tasks.workunit.client.0.smithi040.stdout:6/227: mkdir d0/d30/d32/d45 0 2024-02-16T19:24:06.933 INFO:tasks.workunit.client.0.smithi040.stdout:1/249: dread d17/d29/f32 [0,4194304] 0 2024-02-16T19:24:06.933 INFO:tasks.workunit.client.0.smithi040.stdout:1/250: creat d17/d26/f53 x:0 0 0 2024-02-16T19:24:06.933 INFO:tasks.workunit.client.0.smithi040.stdout:4/237: truncate dd/de/d10/f30 2416231 0 2024-02-16T19:24:06.967 INFO:tasks.workunit.client.0.smithi040.stdout:7/227: truncate f16 1012270 0 2024-02-16T19:24:06.985 INFO:tasks.workunit.client.0.smithi040.stdout:7/228: dwrite d21/d29/f37 [0,4194304] 0 2024-02-16T19:24:06.985 INFO:tasks.workunit.client.0.smithi040.stdout:7/229: write d21/d29/f2e [2241578,21799] 0 2024-02-16T19:24:06.985 INFO:tasks.workunit.client.0.smithi040.stdout:7/230: fdatasync d21/d29/f3a 0 2024-02-16T19:24:06.994 INFO:tasks.workunit.client.0.smithi040.stdout:7/231: dread d21/f28 [0,4194304] 0 2024-02-16T19:24:07.001 INFO:tasks.workunit.client.0.smithi040.stdout:7/232: dread d21/d29/f2b [0,4194304] 0 2024-02-16T19:24:07.016 INFO:tasks.workunit.client.0.smithi040.stdout:6/228: rmdir d0/d30/d32/d45 0 2024-02-16T19:24:07.020 INFO:tasks.workunit.client.0.smithi040.stdout:2/178: sync 2024-02-16T19:24:07.022 INFO:tasks.workunit.client.0.smithi040.stdout:7/233: rename d21/d29/f2e to d21/d29/f45 0 2024-02-16T19:24:07.025 INFO:tasks.workunit.client.0.smithi040.stdout:9/276: mknod d3/d4/c4f 0 2024-02-16T19:24:07.025 INFO:tasks.workunit.client.0.smithi040.stdout:0/316: sync 2024-02-16T19:24:07.025 INFO:tasks.workunit.client.0.smithi040.stdout:3/287: sync 2024-02-16T19:24:07.025 INFO:tasks.workunit.client.0.smithi040.stdout:8/260: sync 2024-02-16T19:24:07.025 INFO:tasks.workunit.client.0.smithi040.stdout:5/213: sync 2024-02-16T19:24:07.025 INFO:tasks.workunit.client.0.smithi040.stdout:5/214: write d1/da/d1c/f25 [240384,18900] 0 2024-02-16T19:24:07.026 INFO:tasks.workunit.client.0.smithi040.stdout:6/229: symlink d0/d30/l46 0 2024-02-16T19:24:07.027 INFO:tasks.workunit.client.0.smithi040.stdout:2/179: mkdir d5/d7/d1e/d31 0 2024-02-16T19:24:07.028 INFO:tasks.workunit.client.0.smithi040.stdout:3/288: symlink d1/dc/l52 0 2024-02-16T19:24:07.029 INFO:tasks.workunit.client.0.smithi040.stdout:0/317: mkdir db/d2f/d1d/d21/d2e/d49/d5b/d67 0 2024-02-16T19:24:07.030 INFO:tasks.workunit.client.0.smithi040.stdout:0/318: readlink db/d2f/d1d/d21/d2e/l41 0 2024-02-16T19:24:07.030 INFO:tasks.workunit.client.0.smithi040.stdout:1/251: sync 2024-02-16T19:24:07.030 INFO:tasks.workunit.client.0.smithi040.stdout:0/319: chown db/d2f/d1d/d21/d2e/d49/d5b/d67 944203 1 2024-02-16T19:24:07.030 INFO:tasks.workunit.client.0.smithi040.stdout:9/277: dread d3/f3c [0,4194304] 0 2024-02-16T19:24:07.030 INFO:tasks.workunit.client.0.smithi040.stdout:9/278: write d3/d4/d11/d29/d18/d31/f35 [622259,34437] 0 2024-02-16T19:24:07.031 INFO:tasks.workunit.client.0.smithi040.stdout:9/279: chown d3/d4/d11/f19 52556411 1 2024-02-16T19:24:07.031 INFO:tasks.workunit.client.0.smithi040.stdout:5/215: link d1/da/d1c/f37 d1/da/d1c/f40 0 2024-02-16T19:24:07.033 INFO:tasks.workunit.client.0.smithi040.stdout:6/230: link d0/d1/fa d0/d1/d12/f47 0 2024-02-16T19:24:07.033 INFO:tasks.workunit.client.0.smithi040.stdout:6/231: write d0/fe [1017599,33127] 0 2024-02-16T19:24:07.034 INFO:tasks.workunit.client.0.smithi040.stdout:4/238: sync 2024-02-16T19:24:07.034 INFO:tasks.workunit.client.0.smithi040.stdout:2/180: unlink d5/d7/f19 0 2024-02-16T19:24:07.035 INFO:tasks.workunit.client.0.smithi040.stdout:4/239: dread dd/f23 [0,4194304] 0 2024-02-16T19:24:07.036 INFO:tasks.workunit.client.0.smithi040.stdout:1/252: mkdir d17/d26/d54 0 2024-02-16T19:24:07.036 INFO:tasks.workunit.client.0.smithi040.stdout:1/253: write d17/f44 [702052,45372] 0 2024-02-16T19:24:07.037 INFO:tasks.workunit.client.0.smithi040.stdout:1/254: stat f5 0 2024-02-16T19:24:07.037 INFO:tasks.workunit.client.0.smithi040.stdout:1/255: creat d17/d26/f55 x:0 0 0 2024-02-16T19:24:07.037 INFO:tasks.workunit.client.0.smithi040.stdout:9/280: symlink d3/l50 0 2024-02-16T19:24:07.039 INFO:tasks.workunit.client.0.smithi040.stdout:5/216: unlink d1/da/f19 0 2024-02-16T19:24:07.040 INFO:tasks.workunit.client.0.smithi040.stdout:3/289: dread d1/f34 [0,4194304] 0 2024-02-16T19:24:07.042 INFO:tasks.workunit.client.0.smithi040.stdout:2/181: symlink d5/d7/l32 0 2024-02-16T19:24:07.043 INFO:tasks.workunit.client.0.smithi040.stdout:4/240: dread dd/de/d45/d37/f3b [0,4194304] 0 2024-02-16T19:24:07.043 INFO:tasks.workunit.client.0.smithi040.stdout:0/320: getdents db/d2f 0 2024-02-16T19:24:07.043 INFO:tasks.workunit.client.0.smithi040.stdout:0/321: write db/d2f/d23/d3b/f47 [145827,38439] 0 2024-02-16T19:24:07.044 INFO:tasks.workunit.client.0.smithi040.stdout:7/234: dwrite d21/f24 [0,4194304] 0 2024-02-16T19:24:07.047 INFO:tasks.workunit.client.0.smithi040.stdout:5/217: mkdir d1/da/d1c/d21/d41 0 2024-02-16T19:24:07.047 INFO:tasks.workunit.client.0.smithi040.stdout:3/290: mkdir d1/dc/d35/d53 0 2024-02-16T19:24:07.048 INFO:tasks.workunit.client.0.smithi040.stdout:3/291: chown d1/dc 707043404 1 2024-02-16T19:24:07.048 INFO:tasks.workunit.client.0.smithi040.stdout:3/292: getdents d1/dc/d35/d53 0 2024-02-16T19:24:07.048 INFO:tasks.workunit.client.0.smithi040.stdout:3/293: chown d1/dc/f3c 0 1 2024-02-16T19:24:07.048 INFO:tasks.workunit.client.0.smithi040.stdout:3/294: truncate d1/ff 1267598 0 2024-02-16T19:24:07.048 INFO:tasks.workunit.client.0.smithi040.stdout:3/295: creat d1/dc/d42/d4a/f54 x:0 0 0 2024-02-16T19:24:07.049 INFO:tasks.workunit.client.0.smithi040.stdout:1/256: dread d17/f1c [0,4194304] 0 2024-02-16T19:24:07.049 INFO:tasks.workunit.client.0.smithi040.stdout:2/182: dread d5/d7/d12/d1b/f30 [0,4194304] 0 2024-02-16T19:24:07.050 INFO:tasks.workunit.client.0.smithi040.stdout:4/241: mknod dd/de/c4f 0 2024-02-16T19:24:07.051 INFO:tasks.workunit.client.0.smithi040.stdout:0/322: symlink db/de/l68 0 2024-02-16T19:24:07.053 INFO:tasks.workunit.client.0.smithi040.stdout:7/235: creat d21/f46 x:0 0 0 2024-02-16T19:24:07.053 INFO:tasks.workunit.client.0.smithi040.stdout:7/236: dread - d21/d29/f3f zero size 2024-02-16T19:24:07.054 INFO:tasks.workunit.client.0.smithi040.stdout:5/218: creat d1/da/d1c/d21/f42 x:0 0 0 2024-02-16T19:24:07.056 INFO:tasks.workunit.client.0.smithi040.stdout:3/296: mkdir d1/dc/d35/d55 0 2024-02-16T19:24:07.056 INFO:tasks.workunit.client.0.smithi040.stdout:3/297: creat d1/d7/d33/d38/f56 x:0 0 0 2024-02-16T19:24:07.057 INFO:tasks.workunit.client.0.smithi040.stdout:6/232: rename d0/d34 to d0/d1/d12/d3b/d48 0 2024-02-16T19:24:07.058 INFO:tasks.workunit.client.0.smithi040.stdout:1/257: unlink d17/d26/d4c/f4e 0 2024-02-16T19:24:07.058 INFO:tasks.workunit.client.0.smithi040.stdout:9/281: dwrite d3/d4/d11/d29/f24 [0,4194304] 0 2024-02-16T19:24:07.059 INFO:tasks.workunit.client.0.smithi040.stdout:4/242: getdents dd/de/d10/d13 0 2024-02-16T19:24:07.059 INFO:tasks.workunit.client.0.smithi040.stdout:4/243: fsync dd/de/f12 0 2024-02-16T19:24:07.059 INFO:tasks.workunit.client.0.smithi040.stdout:4/244: readlink dd/de/d10/l3d 0 2024-02-16T19:24:07.060 INFO:tasks.workunit.client.0.smithi040.stdout:0/323: mknod db/de/c69 0 2024-02-16T19:24:07.060 INFO:tasks.workunit.client.0.smithi040.stdout:5/219: truncate d1/f17 7224805 0 2024-02-16T19:24:07.063 INFO:tasks.workunit.client.0.smithi040.stdout:3/298: mkdir d1/dc/d42/d4a/d57 0 2024-02-16T19:24:07.063 INFO:tasks.workunit.client.0.smithi040.stdout:5/220: dread f0 [0,4194304] 0 2024-02-16T19:24:07.064 INFO:tasks.workunit.client.0.smithi040.stdout:6/233: creat d0/d1/d12/d3a/f49 x:0 0 0 2024-02-16T19:24:07.064 INFO:tasks.workunit.client.0.smithi040.stdout:6/234: stat d0/d1/d12/d3b/d48/f38 0 2024-02-16T19:24:07.065 INFO:tasks.workunit.client.0.smithi040.stdout:1/258: creat d17/d26/d3d/f56 x:0 0 0 2024-02-16T19:24:07.066 INFO:tasks.workunit.client.0.smithi040.stdout:9/282: unlink d3/d4/d11/d29/l30 0 2024-02-16T19:24:07.067 INFO:tasks.workunit.client.0.smithi040.stdout:2/183: getdents d5/d7/dd 0 2024-02-16T19:24:07.067 INFO:tasks.workunit.client.0.smithi040.stdout:2/184: write d5/d7/d1e/f20 [5078423,42308] 0 2024-02-16T19:24:07.067 INFO:tasks.workunit.client.0.smithi040.stdout:2/185: readlink d5/d7/l32 0 2024-02-16T19:24:07.069 INFO:tasks.workunit.client.0.smithi040.stdout:2/186: dread d5/d7/d12/f13 [0,4194304] 0 2024-02-16T19:24:07.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/245: dread dd/de/f1a [0,4194304] 0 2024-02-16T19:24:07.069 INFO:tasks.workunit.client.0.smithi040.stdout:3/299: mkdir d1/dc/d42/d58 0 2024-02-16T19:24:07.071 INFO:tasks.workunit.client.0.smithi040.stdout:4/246: dread dd/de/f2f [0,4194304] 0 2024-02-16T19:24:07.071 INFO:tasks.workunit.client.0.smithi040.stdout:7/237: rename d21/d29/c43 to d21/c47 0 2024-02-16T19:24:07.074 INFO:tasks.workunit.client.0.smithi040.stdout:7/238: fsync d21/f46 0 2024-02-16T19:24:07.074 INFO:tasks.workunit.client.0.smithi040.stdout:7/239: write f17 [1985777,73800] 0 2024-02-16T19:24:07.077 INFO:tasks.workunit.client.0.smithi040.stdout:6/235: stat d0/d30/d32/l1b 0 2024-02-16T19:24:07.077 INFO:tasks.workunit.client.0.smithi040.stdout:2/187: dread d5/d7/d12/f15 [0,4194304] 0 2024-02-16T19:24:07.078 INFO:tasks.workunit.client.0.smithi040.stdout:3/300: symlink d1/dc/d42/d4a/l59 0 2024-02-16T19:24:07.080 INFO:tasks.workunit.client.0.smithi040.stdout:2/188: dread d5/d7/d12/d1b/f30 [0,4194304] 0 2024-02-16T19:24:07.080 INFO:tasks.workunit.client.0.smithi040.stdout:4/247: dread dd/de/d10/d13/f46 [0,4194304] 0 2024-02-16T19:24:07.080 INFO:tasks.workunit.client.0.smithi040.stdout:4/248: fsync dd/de/d45/f4c 0 2024-02-16T19:24:07.080 INFO:tasks.workunit.client.0.smithi040.stdout:2/189: chown d5/d7/f24 0 1 2024-02-16T19:24:07.080 INFO:tasks.workunit.client.0.smithi040.stdout:5/221: rmdir d1/d3c 39 2024-02-16T19:24:07.080 INFO:tasks.workunit.client.0.smithi040.stdout:2/190: write d5/d7/d12/d1b/f2f [829131,29676] 0 2024-02-16T19:24:07.083 INFO:tasks.workunit.client.0.smithi040.stdout:1/259: rename d17/d2d/c30 to d17/d3c/c57 0 2024-02-16T19:24:07.084 INFO:tasks.workunit.client.0.smithi040.stdout:3/301: dread d1/f34 [0,4194304] 0 2024-02-16T19:24:07.085 INFO:tasks.workunit.client.0.smithi040.stdout:7/240: creat d21/f48 x:0 0 0 2024-02-16T19:24:07.086 INFO:tasks.workunit.client.0.smithi040.stdout:9/283: dwrite d3/d4/d11/d32/f43 [0,4194304] 0 2024-02-16T19:24:07.086 INFO:tasks.workunit.client.0.smithi040.stdout:9/284: fdatasync d3/d4/d11/d32/f43 0 2024-02-16T19:24:07.086 INFO:tasks.workunit.client.0.smithi040.stdout:9/285: write d3/d4/f8 [4232717,52533] 0 2024-02-16T19:24:07.087 INFO:tasks.workunit.client.0.smithi040.stdout:4/249: unlink dd/de/d10/l3d 0 2024-02-16T19:24:07.087 INFO:tasks.workunit.client.0.smithi040.stdout:4/250: rename dd to dd/de/d45/d37/d50 22 2024-02-16T19:24:07.088 INFO:tasks.workunit.client.0.smithi040.stdout:5/222: creat d1/d3c/f43 x:0 0 0 2024-02-16T19:24:07.088 INFO:tasks.workunit.client.0.smithi040.stdout:5/223: write d1/da/d1c/f25 [548703,13318] 0 2024-02-16T19:24:07.090 INFO:tasks.workunit.client.0.smithi040.stdout:1/260: creat d17/d26/d54/f58 x:0 0 0 2024-02-16T19:24:07.092 INFO:tasks.workunit.client.0.smithi040.stdout:3/302: link d1/dc/l2f d1/d7/l5a 0 2024-02-16T19:24:07.093 INFO:tasks.workunit.client.0.smithi040.stdout:1/261: dread d17/d43/f21 [0,4194304] 0 2024-02-16T19:24:07.093 INFO:tasks.workunit.client.0.smithi040.stdout:1/262: fsync d17/d29/f32 0 2024-02-16T19:24:07.093 INFO:tasks.workunit.client.0.smithi040.stdout:1/263: write d17/d26/f4b [44790,6385] 0 2024-02-16T19:24:07.093 INFO:tasks.workunit.client.0.smithi040.stdout:1/264: write f14 [882146,120675] 0 2024-02-16T19:24:07.093 INFO:tasks.workunit.client.0.smithi040.stdout:1/265: getdents d17/d43/d38 0 2024-02-16T19:24:07.098 INFO:tasks.workunit.client.0.smithi040.stdout:6/236: dwrite d0/d1/f16 [0,4194304] 0 2024-02-16T19:24:07.101 INFO:tasks.workunit.client.0.smithi040.stdout:3/303: write d1/dc/fd [2834255,52388] 0 2024-02-16T19:24:07.101 INFO:tasks.workunit.client.0.smithi040.stdout:1/266: link d17/l1b d17/d26/d3d/l59 0 2024-02-16T19:24:07.104 INFO:tasks.workunit.client.0.smithi040.stdout:7/241: unlink d21/f2f 0 2024-02-16T19:24:07.107 INFO:tasks.workunit.client.0.smithi040.stdout:6/237: mkdir d0/d37/d4a 0 2024-02-16T19:24:07.111 INFO:tasks.workunit.client.0.smithi040.stdout:3/304: symlink d1/l5b 0 2024-02-16T19:24:07.112 INFO:tasks.workunit.client.0.smithi040.stdout:6/238: mkdir d0/d30/d4b 0 2024-02-16T19:24:07.115 INFO:tasks.workunit.client.0.smithi040.stdout:4/251: dwrite dd/de/d10/f30 [0,4194304] 0 2024-02-16T19:24:07.116 INFO:tasks.workunit.client.0.smithi040.stdout:5/224: rmdir d1/d3c 39 2024-02-16T19:24:07.116 INFO:tasks.workunit.client.0.smithi040.stdout:3/305: rmdir d1/dc/d42/d4a 39 2024-02-16T19:24:07.119 INFO:tasks.workunit.client.0.smithi040.stdout:3/306: mkdir d1/dc/d42/d5c 0 2024-02-16T19:24:07.119 INFO:tasks.workunit.client.0.smithi040.stdout:3/307: write d1/dc/f3c [964560,24758] 0 2024-02-16T19:24:07.119 INFO:tasks.workunit.client.0.smithi040.stdout:3/308: chown d1/dc/d35/l49 118 1 2024-02-16T19:24:07.120 INFO:tasks.workunit.client.0.smithi040.stdout:7/242: dread d21/d29/f37 [0,4194304] 0 2024-02-16T19:24:07.120 INFO:tasks.workunit.client.0.smithi040.stdout:3/309: symlink d1/dc/d42/d4a/d57/l5d 0 2024-02-16T19:24:07.120 INFO:tasks.workunit.client.0.smithi040.stdout:3/310: chown d1/le 239000 1 2024-02-16T19:24:07.123 INFO:tasks.workunit.client.0.smithi040.stdout:3/311: creat d1/d7/d33/f5e x:0 0 0 2024-02-16T19:24:07.123 INFO:tasks.workunit.client.0.smithi040.stdout:3/312: write d1/d7/d36/f45 [702461,109583] 0 2024-02-16T19:24:07.123 INFO:tasks.workunit.client.0.smithi040.stdout:3/313: rmdir d1/d7/d30 39 2024-02-16T19:24:07.124 INFO:tasks.workunit.client.0.smithi040.stdout:9/286: dwrite d3/d4/f8 [0,4194304] 0 2024-02-16T19:24:07.124 INFO:tasks.workunit.client.0.smithi040.stdout:9/287: chown d3/d4/f4d 7962 1 2024-02-16T19:24:07.124 INFO:tasks.workunit.client.0.smithi040.stdout:1/267: dwrite d17/f36 [0,4194304] 0 2024-02-16T19:24:07.126 INFO:tasks.workunit.client.0.smithi040.stdout:3/314: rename d1/c47 to d1/dc/d42/d58/c5f 0 2024-02-16T19:24:07.126 INFO:tasks.workunit.client.0.smithi040.stdout:3/315: creat d1/d7/d33/d38/f60 x:0 0 0 2024-02-16T19:24:07.126 INFO:tasks.workunit.client.0.smithi040.stdout:3/316: stat d1/d4/f25 0 2024-02-16T19:24:07.126 INFO:tasks.workunit.client.0.smithi040.stdout:3/317: write d1/f3 [372877,45613] 0 2024-02-16T19:24:07.136 INFO:tasks.workunit.client.0.smithi040.stdout:5/225: dread d1/f1d [0,4194304] 0 2024-02-16T19:24:07.136 INFO:tasks.workunit.client.0.smithi040.stdout:1/268: creat d17/d3c/f5a x:0 0 0 2024-02-16T19:24:07.136 INFO:tasks.workunit.client.0.smithi040.stdout:1/269: write d17/d2d/f2e [4581149,94797] 0 2024-02-16T19:24:07.136 INFO:tasks.workunit.client.0.smithi040.stdout:1/270: creat d17/d26/d4c/f5b x:0 0 0 2024-02-16T19:24:07.136 INFO:tasks.workunit.client.0.smithi040.stdout:1/271: write d17/f37 [5204969,26564] 0 2024-02-16T19:24:07.139 INFO:tasks.workunit.client.0.smithi040.stdout:5/226: symlink d1/da/d1c/l44 0 2024-02-16T19:24:07.142 INFO:tasks.workunit.client.0.smithi040.stdout:1/272: rename d17/d43/l27 to d17/d26/d3d/l5c 0 2024-02-16T19:24:07.143 INFO:tasks.workunit.client.0.smithi040.stdout:1/273: write d17/d26/f4b [968476,117847] 0 2024-02-16T19:24:07.143 INFO:tasks.workunit.client.0.smithi040.stdout:1/274: write d17/d43/f39 [1023861,13628] 0 2024-02-16T19:24:07.143 INFO:tasks.workunit.client.0.smithi040.stdout:1/275: chown d17 191459136 1 2024-02-16T19:24:07.143 INFO:tasks.workunit.client.0.smithi040.stdout:9/288: write d3/d4/f8 [2065162,79594] 0 2024-02-16T19:24:07.150 INFO:tasks.workunit.client.0.smithi040.stdout:6/239: dwrite d0/fe [0,4194304] 0 2024-02-16T19:24:07.157 INFO:tasks.workunit.client.0.smithi040.stdout:6/240: dread d0/f35 [0,4194304] 0 2024-02-16T19:24:07.158 INFO:tasks.workunit.client.0.smithi040.stdout:6/241: truncate d0/f35 1887218 0 2024-02-16T19:24:07.159 INFO:tasks.workunit.client.0.smithi040.stdout:5/227: write d1/f2c [1459154,70350] 0 2024-02-16T19:24:07.161 INFO:tasks.workunit.client.0.smithi040.stdout:5/228: mkdir d1/d45 0 2024-02-16T19:24:07.163 INFO:tasks.workunit.client.0.smithi040.stdout:9/289: dread d3/d4/d11/d29/f24 [0,4194304] 0 2024-02-16T19:24:07.165 INFO:tasks.workunit.client.0.smithi040.stdout:9/290: creat d3/d4/d11/d29/d18/f51 x:0 0 0 2024-02-16T19:24:07.165 INFO:tasks.workunit.client.0.smithi040.stdout:9/291: creat d3/d4/d11/d29/f52 x:0 0 0 2024-02-16T19:24:07.165 INFO:tasks.workunit.client.0.smithi040.stdout:9/292: read d3/f14 [1480297,100361] 0 2024-02-16T19:24:07.165 INFO:tasks.workunit.client.0.smithi040.stdout:9/293: fdatasync d3/d4/d11/d29/f48 0 2024-02-16T19:24:07.167 INFO:tasks.workunit.client.0.smithi040.stdout:9/294: rename d3/f3c to d3/d4/d11/f53 0 2024-02-16T19:24:07.167 INFO:tasks.workunit.client.0.smithi040.stdout:9/295: write d3/d4/d11/d32/f42 [794970,45837] 0 2024-02-16T19:24:07.170 INFO:tasks.workunit.client.0.smithi040.stdout:9/296: creat d3/d4/d11/d29/d38/f54 x:0 0 0 2024-02-16T19:24:07.170 INFO:tasks.workunit.client.0.smithi040.stdout:9/297: write d3/d4/d11/f19 [2919685,43179] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:6/242: write d0/d1/f16 [1813303,53073] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:6/243: write d0/d1/f16 [3013393,26527] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:9/298: write d3/d1c/f3f [339574,107778] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:1/276: dwrite f5 [0,4194304] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:6/244: write d0/f1d [1085662,121987] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:7/243: dwrite f16 [0,4194304] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:7/244: write f9 [5398999,100108] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:4/252: dwrite dd/de/d10/d13/d32/f41 [0,4194304] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:3/318: dwrite d1/ff [0,4194304] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:3/319: write d1/dc/d42/d4a/f50 [209686,113884] 0 2024-02-16T19:24:07.175 INFO:tasks.workunit.client.0.smithi040.stdout:3/320: write d1/d7/d33/f5e [401361,25866] 0 2024-02-16T19:24:07.186 INFO:tasks.workunit.client.0.smithi040.stdout:5/229: dwrite d1/da/d1c/d21/f2f [0,4194304] 0 2024-02-16T19:24:07.187 INFO:tasks.workunit.client.0.smithi040.stdout:6/245: symlink d0/d30/d32/l4c 0 2024-02-16T19:24:07.194 INFO:tasks.workunit.client.0.smithi040.stdout:7/245: write d21/f26 [261097,71198] 0 2024-02-16T19:24:07.197 INFO:tasks.workunit.client.0.smithi040.stdout:5/230: mknod d1/d45/c46 0 2024-02-16T19:24:07.198 INFO:tasks.workunit.client.0.smithi040.stdout:6/246: mkdir d0/d1/d12/d3b/d48/d4d 0 2024-02-16T19:24:07.198 INFO:tasks.workunit.client.0.smithi040.stdout:7/246: symlink d21/d29/l49 0 2024-02-16T19:24:07.199 INFO:tasks.workunit.client.0.smithi040.stdout:5/231: symlink d1/da/d1c/d21/l47 0 2024-02-16T19:24:07.199 INFO:tasks.workunit.client.0.smithi040.stdout:5/232: chown d1/da/c1a 16276711 1 2024-02-16T19:24:07.199 INFO:tasks.workunit.client.0.smithi040.stdout:7/247: truncate d21/f25 690125 0 2024-02-16T19:24:07.200 INFO:tasks.workunit.client.0.smithi040.stdout:5/233: symlink d1/d45/l48 0 2024-02-16T19:24:07.200 INFO:tasks.workunit.client.0.smithi040.stdout:7/248: chown d21/c27 584825112 1 2024-02-16T19:24:07.202 INFO:tasks.workunit.client.0.smithi040.stdout:7/249: rename d21/d29/f34 to d21/f4a 0 2024-02-16T19:24:07.203 INFO:tasks.workunit.client.0.smithi040.stdout:7/250: creat d21/d29/d36/f4b x:0 0 0 2024-02-16T19:24:07.203 INFO:tasks.workunit.client.0.smithi040.stdout:7/251: dread - d21/d29/f38 zero size 2024-02-16T19:24:07.204 INFO:tasks.workunit.client.0.smithi040.stdout:7/252: rename d21/d29/f3a to d21/d29/f4c 0 2024-02-16T19:24:07.211 INFO:tasks.workunit.client.0.smithi040.stdout:6/247: write d0/d1/f1f [1888923,18060] 0 2024-02-16T19:24:07.211 INFO:tasks.workunit.client.0.smithi040.stdout:6/248: chown d0/d30/d32/l1b 1831309 1 2024-02-16T19:24:07.211 INFO:tasks.workunit.client.0.smithi040.stdout:6/249: fsync d0/f2e 0 2024-02-16T19:24:07.212 INFO:tasks.workunit.client.0.smithi040.stdout:6/250: mkdir d0/d1/d12/d3a/d4e 0 2024-02-16T19:24:07.212 INFO:tasks.workunit.client.0.smithi040.stdout:4/253: dwrite f8 [0,4194304] 0 2024-02-16T19:24:07.213 INFO:tasks.workunit.client.0.smithi040.stdout:3/321: dwrite d1/d7/d36/f45 [0,4194304] 0 2024-02-16T19:24:07.213 INFO:tasks.workunit.client.0.smithi040.stdout:9/299: dwrite d3/d4/d11/d29/d18/d31/f3b [0,4194304] 0 2024-02-16T19:24:07.213 INFO:tasks.workunit.client.0.smithi040.stdout:1/277: dwrite f1 [0,4194304] 0 2024-02-16T19:24:07.213 INFO:tasks.workunit.client.0.smithi040.stdout:6/251: mkdir d0/d1/d12/d3b/d48/d4f 0 2024-02-16T19:24:07.214 INFO:tasks.workunit.client.0.smithi040.stdout:6/252: creat d0/d1/d12/d3b/f50 x:0 0 0 2024-02-16T19:24:07.214 INFO:tasks.workunit.client.0.smithi040.stdout:6/253: chown d0/d1/d12/d3a 102774 1 2024-02-16T19:24:07.217 INFO:tasks.workunit.client.0.smithi040.stdout:4/254: dread dd/de/d10/f3c [0,4194304] 0 2024-02-16T19:24:07.217 INFO:tasks.workunit.client.0.smithi040.stdout:3/322: creat d1/dc/d42/d4a/f61 x:0 0 0 2024-02-16T19:24:07.226 INFO:tasks.workunit.client.0.smithi040.stdout:1/278: rename d17/d43/f2a to d17/d43/f5d 0 2024-02-16T19:24:07.226 INFO:tasks.workunit.client.0.smithi040.stdout:1/279: chown d17/d26/f55 4035 1 2024-02-16T19:24:07.227 INFO:tasks.workunit.client.0.smithi040.stdout:6/254: symlink d0/d30/l51 0 2024-02-16T19:24:07.229 INFO:tasks.workunit.client.0.smithi040.stdout:3/323: mknod d1/d7/c62 0 2024-02-16T19:24:07.229 INFO:tasks.workunit.client.0.smithi040.stdout:3/324: truncate d1/dc/d42/d4a/f61 580970 0 2024-02-16T19:24:07.230 INFO:tasks.workunit.client.0.smithi040.stdout:3/325: getdents d1/dc/d42/d4a/d57 0 2024-02-16T19:24:07.230 INFO:tasks.workunit.client.0.smithi040.stdout:4/255: dread fb [0,4194304] 0 2024-02-16T19:24:07.230 INFO:tasks.workunit.client.0.smithi040.stdout:1/280: rmdir d17/d26/d54 39 2024-02-16T19:24:07.230 INFO:tasks.workunit.client.0.smithi040.stdout:1/281: getdents d17/d43/d38 0 2024-02-16T19:24:07.231 INFO:tasks.workunit.client.0.smithi040.stdout:3/326: creat d1/d7/d30/f63 x:0 0 0 2024-02-16T19:24:07.232 INFO:tasks.workunit.client.0.smithi040.stdout:4/256: link cc dd/de/c51 0 2024-02-16T19:24:07.233 INFO:tasks.workunit.client.0.smithi040.stdout:1/282: creat d17/d26/d54/f5e x:0 0 0 2024-02-16T19:24:07.234 INFO:tasks.workunit.client.0.smithi040.stdout:1/283: truncate d17/d26/d4c/f5b 264241 0 2024-02-16T19:24:07.236 INFO:tasks.workunit.client.0.smithi040.stdout:3/327: write d1/d4/fa [1137821,8297] 0 2024-02-16T19:24:07.236 INFO:tasks.workunit.client.0.smithi040.stdout:3/328: chown d1/d7 4990 1 2024-02-16T19:24:07.237 INFO:tasks.workunit.client.0.smithi040.stdout:8/261: sync 2024-02-16T19:24:07.237 INFO:tasks.workunit.client.0.smithi040.stdout:8/262: fdatasync d6/d2b/f34 0 2024-02-16T19:24:07.237 INFO:tasks.workunit.client.0.smithi040.stdout:8/263: fdatasync d6/d7/f32 0 2024-02-16T19:24:07.237 INFO:tasks.workunit.client.0.smithi040.stdout:8/264: mknod d6/d7/c4c 0 2024-02-16T19:24:07.238 INFO:tasks.workunit.client.0.smithi040.stdout:8/265: mknod d6/d7/d1d/d38/c4d 0 2024-02-16T19:24:07.239 INFO:tasks.workunit.client.0.smithi040.stdout:8/266: creat d6/d12/d22/f4e x:0 0 0 2024-02-16T19:24:07.240 INFO:tasks.workunit.client.0.smithi040.stdout:8/267: creat d6/d2b/d36/f4f x:0 0 0 2024-02-16T19:24:07.243 INFO:tasks.workunit.client.0.smithi040.stdout:7/253: rename d21/d29/d36 to d21/d4d 0 2024-02-16T19:24:07.244 INFO:tasks.workunit.client.0.smithi040.stdout:1/284: read f1 [2204198,62315] 0 2024-02-16T19:24:07.244 INFO:tasks.workunit.client.0.smithi040.stdout:5/234: dwrite d1/f2c [0,4194304] 0 2024-02-16T19:24:07.245 INFO:tasks.workunit.client.0.smithi040.stdout:5/235: chown d1/f17 0 1 2024-02-16T19:24:07.245 INFO:tasks.workunit.client.0.smithi040.stdout:4/257: truncate f8 2609123 0 2024-02-16T19:24:07.245 INFO:tasks.workunit.client.0.smithi040.stdout:3/329: dread d1/f3 [0,4194304] 0 2024-02-16T19:24:07.246 INFO:tasks.workunit.client.0.smithi040.stdout:1/285: mkdir d17/d26/d5f 0 2024-02-16T19:24:07.246 INFO:tasks.workunit.client.0.smithi040.stdout:1/286: chown d17/f1f 2 1 2024-02-16T19:24:07.246 INFO:tasks.workunit.client.0.smithi040.stdout:1/287: write d17/d26/f53 [843422,3663] 0 2024-02-16T19:24:07.246 INFO:tasks.workunit.client.0.smithi040.stdout:1/288: readlink d17/d2d/l3b 0 2024-02-16T19:24:07.246 INFO:tasks.workunit.client.0.smithi040.stdout:4/258: write dd/de/d10/f3c [3996,33042] 0 2024-02-16T19:24:07.248 INFO:tasks.workunit.client.0.smithi040.stdout:5/236: creat d1/d45/f49 x:0 0 0 2024-02-16T19:24:07.249 INFO:tasks.workunit.client.0.smithi040.stdout:4/259: fsync dd/de/d10/f3c 0 2024-02-16T19:24:07.250 INFO:tasks.workunit.client.0.smithi040.stdout:1/289: dread d17/f25 [0,4194304] 0 2024-02-16T19:24:07.250 INFO:tasks.workunit.client.0.smithi040.stdout:1/290: write fb [4153098,103160] 0 2024-02-16T19:24:07.250 INFO:tasks.workunit.client.0.smithi040.stdout:1/291: readlink d17/d26/d3d/l5c 0 2024-02-16T19:24:07.250 INFO:tasks.workunit.client.0.smithi040.stdout:1/292: dread - d17/d26/d54/f58 zero size 2024-02-16T19:24:07.250 INFO:tasks.workunit.client.0.smithi040.stdout:3/330: mkdir d1/dc/d35/d53/d64 0 2024-02-16T19:24:07.250 INFO:tasks.workunit.client.0.smithi040.stdout:3/331: getdents d1/dc/d35/d55 0 2024-02-16T19:24:07.253 INFO:tasks.workunit.client.0.smithi040.stdout:3/332: dread d1/d4/f4f [0,4194304] 0 2024-02-16T19:24:07.253 INFO:tasks.workunit.client.0.smithi040.stdout:3/333: dread - d1/d7/d30/f63 zero size 2024-02-16T19:24:07.253 INFO:tasks.workunit.client.0.smithi040.stdout:3/334: chown d1/d7/d33/c3b 9295996 1 2024-02-16T19:24:07.253 INFO:tasks.workunit.client.0.smithi040.stdout:3/335: readlink d1/l5b 0 2024-02-16T19:24:07.253 INFO:tasks.workunit.client.0.smithi040.stdout:7/254: dread d21/d29/f37 [0,4194304] 0 2024-02-16T19:24:07.253 INFO:tasks.workunit.client.0.smithi040.stdout:7/255: write d21/d29/f4c [296850,121114] 0 2024-02-16T19:24:07.254 INFO:tasks.workunit.client.0.smithi040.stdout:4/260: rmdir dd/de/d45/d37 39 2024-02-16T19:24:07.256 INFO:tasks.workunit.client.0.smithi040.stdout:1/293: symlink d17/d43/l60 0 2024-02-16T19:24:07.256 INFO:tasks.workunit.client.0.smithi040.stdout:1/294: fsync d17/f36 0 2024-02-16T19:24:07.257 INFO:tasks.workunit.client.0.smithi040.stdout:9/300: rename d3/d4/d11/d29/d18 to d3/d4/d55 0 2024-02-16T19:24:07.258 INFO:tasks.workunit.client.0.smithi040.stdout:7/256: dread - d21/f32 zero size 2024-02-16T19:24:07.258 INFO:tasks.workunit.client.0.smithi040.stdout:3/336: dread d1/f34 [0,4194304] 0 2024-02-16T19:24:07.258 INFO:tasks.workunit.client.0.smithi040.stdout:3/337: chown d1/d7/d33 475 1 2024-02-16T19:24:07.260 INFO:tasks.workunit.client.0.smithi040.stdout:4/261: write dd/f1d [253160,40729] 0 2024-02-16T19:24:07.260 INFO:tasks.workunit.client.0.smithi040.stdout:4/262: write dd/de/f12 [2647744,129366] 0 2024-02-16T19:24:07.260 INFO:tasks.workunit.client.0.smithi040.stdout:4/263: write dd/de/d45/f4c [986006,54779] 0 2024-02-16T19:24:07.263 INFO:tasks.workunit.client.0.smithi040.stdout:6/255: rename d0/d30/d32/f1a to d0/d1/f52 0 2024-02-16T19:24:07.263 INFO:tasks.workunit.client.0.smithi040.stdout:6/256: chown d0/d37 9 1 2024-02-16T19:24:07.266 INFO:tasks.workunit.client.0.smithi040.stdout:8/268: dwrite d6/d7/d15/f18 [0,4194304] 0 2024-02-16T19:24:07.266 INFO:tasks.workunit.client.0.smithi040.stdout:9/301: link d3/d1c/f3f d3/d4/d11/d29/f56 0 2024-02-16T19:24:07.267 INFO:tasks.workunit.client.0.smithi040.stdout:5/237: read d1/f17 [276759,47316] 0 2024-02-16T19:24:07.267 INFO:tasks.workunit.client.0.smithi040.stdout:5/238: chown d1/l1b 92 1 2024-02-16T19:24:07.267 INFO:tasks.workunit.client.0.smithi040.stdout:5/239: fdatasync d1/da/f2e 0 2024-02-16T19:24:07.268 INFO:tasks.workunit.client.0.smithi040.stdout:7/257: symlink d21/d4d/l4e 0 2024-02-16T19:24:07.269 INFO:tasks.workunit.client.0.smithi040.stdout:3/338: mkdir d1/dc/d35/d53/d64/d65 0 2024-02-16T19:24:07.270 INFO:tasks.workunit.client.0.smithi040.stdout:5/240: dread d1/da/d1c/f25 [0,4194304] 0 2024-02-16T19:24:07.270 INFO:tasks.workunit.client.0.smithi040.stdout:5/241: fsync d1/da/d1c/f26 0 2024-02-16T19:24:07.270 INFO:tasks.workunit.client.0.smithi040.stdout:6/257: link d0/d1/d12/d3a/l44 d0/d30/d32/l53 0 2024-02-16T19:24:07.271 INFO:tasks.workunit.client.0.smithi040.stdout:5/242: dread d1/da/f35 [0,4194304] 0 2024-02-16T19:24:07.271 INFO:tasks.workunit.client.0.smithi040.stdout:3/339: fsync d1/d7/d33/f5e 0 2024-02-16T19:24:07.274 INFO:tasks.workunit.client.0.smithi040.stdout:8/269: rmdir d6/d7 39 2024-02-16T19:24:07.275 INFO:tasks.workunit.client.0.smithi040.stdout:3/340: dread d1/f3 [0,4194304] 0 2024-02-16T19:24:07.276 INFO:tasks.workunit.client.0.smithi040.stdout:9/302: creat d3/d4/f57 x:0 0 0 2024-02-16T19:24:07.276 INFO:tasks.workunit.client.0.smithi040.stdout:4/264: dwrite dd/de/f2f [0,4194304] 0 2024-02-16T19:24:07.276 INFO:tasks.workunit.client.0.smithi040.stdout:4/265: chown dd/de/d10 904091224 1 2024-02-16T19:24:07.276 INFO:tasks.workunit.client.0.smithi040.stdout:4/266: creat dd/f52 x:0 0 0 2024-02-16T19:24:07.281 INFO:tasks.workunit.client.0.smithi040.stdout:0/324: sync 2024-02-16T19:24:07.281 INFO:tasks.workunit.client.0.smithi040.stdout:2/191: sync 2024-02-16T19:24:07.288 INFO:tasks.workunit.client.0.smithi040.stdout:7/258: write d21/f23 [894182,109868] 0 2024-02-16T19:24:07.289 INFO:tasks.workunit.client.0.smithi040.stdout:7/259: creat d21/d29/f4f x:0 0 0 2024-02-16T19:24:07.289 INFO:tasks.workunit.client.0.smithi040.stdout:7/260: creat d21/d29/f50 x:0 0 0 2024-02-16T19:24:07.289 INFO:tasks.workunit.client.0.smithi040.stdout:7/261: stat f1e 0 2024-02-16T19:24:07.291 INFO:tasks.workunit.client.0.smithi040.stdout:3/341: truncate d1/ff 987834 0 2024-02-16T19:24:07.291 INFO:tasks.workunit.client.0.smithi040.stdout:6/258: dread d0/d1/d12/d3a/f3d [4194304,4194304] 0 2024-02-16T19:24:07.291 INFO:tasks.workunit.client.0.smithi040.stdout:6/259: stat d0/d37/d4a 0 2024-02-16T19:24:07.294 INFO:tasks.workunit.client.0.smithi040.stdout:7/262: write d21/f3c [516146,47446] 0 2024-02-16T19:24:07.295 INFO:tasks.workunit.client.0.smithi040.stdout:3/342: mknod d1/dc/d42/d58/c66 0 2024-02-16T19:24:07.296 INFO:tasks.workunit.client.0.smithi040.stdout:6/260: unlink d0/d1/fa 0 2024-02-16T19:24:07.297 INFO:tasks.workunit.client.0.smithi040.stdout:3/343: unlink d1/d7/d33/d38/f60 0 2024-02-16T19:24:07.297 INFO:tasks.workunit.client.0.smithi040.stdout:3/344: chown d1/d7/d36/f45 1528 1 2024-02-16T19:24:07.297 INFO:tasks.workunit.client.0.smithi040.stdout:6/261: symlink d0/d1/d12/d3a/l54 0 2024-02-16T19:24:07.302 INFO:tasks.workunit.client.0.smithi040.stdout:6/262: write d0/d1/d12/d3a/f3d [5461948,17739] 0 2024-02-16T19:24:07.374 INFO:tasks.workunit.client.0.smithi040.stdout:8/270: dwrite d6/d7/d15/f47 [0,4194304] 0 2024-02-16T19:24:07.403 INFO:tasks.workunit.client.0.smithi040.stdout:5/243: dwrite d1/f20 [0,4194304] 0 2024-02-16T19:24:07.404 INFO:tasks.workunit.client.0.smithi040.stdout:5/244: write d1/da/f2a [217300,83283] 0 2024-02-16T19:24:07.404 INFO:tasks.workunit.client.0.smithi040.stdout:5/245: write d1/f24 [423257,122669] 0 2024-02-16T19:24:07.404 INFO:tasks.workunit.client.0.smithi040.stdout:5/246: fsync d1/d3c/f3f 0 2024-02-16T19:24:07.425 INFO:tasks.workunit.client.0.smithi040.stdout:0/325: dwrite db/d2f/f3e [0,4194304] 0 2024-02-16T19:24:07.425 INFO:tasks.workunit.client.0.smithi040.stdout:0/326: write db/d2f/d23/f40 [4127918,124281] 0 2024-02-16T19:24:07.425 INFO:tasks.workunit.client.0.smithi040.stdout:0/327: write db/f63 [331651,107148] 0 2024-02-16T19:24:07.428 INFO:tasks.workunit.client.0.smithi040.stdout:0/328: mkdir db/de/d6a 0 2024-02-16T19:24:07.428 INFO:tasks.workunit.client.0.smithi040.stdout:0/329: write db/d2f/d23/f40 [219056,129061] 0 2024-02-16T19:24:07.428 INFO:tasks.workunit.client.0.smithi040.stdout:9/303: dwrite d3/d4/d11/f53 [0,4194304] 0 2024-02-16T19:24:07.428 INFO:tasks.workunit.client.0.smithi040.stdout:9/304: creat d3/d4/d11/d32/f58 x:0 0 0 2024-02-16T19:24:07.428 INFO:tasks.workunit.client.0.smithi040.stdout:0/330: creat db/d25/d34/d5e/f6b x:0 0 0 2024-02-16T19:24:07.429 INFO:tasks.workunit.client.0.smithi040.stdout:9/305: mknod d3/d4/d55/d31/c59 0 2024-02-16T19:24:07.430 INFO:tasks.workunit.client.0.smithi040.stdout:0/331: symlink db/d2f/d1d/d21/l6c 0 2024-02-16T19:24:07.430 INFO:tasks.workunit.client.0.smithi040.stdout:0/332: chown db/d2f/d1d/c28 6411 1 2024-02-16T19:24:07.430 INFO:tasks.workunit.client.0.smithi040.stdout:9/306: creat d3/d4/d55/f5a x:0 0 0 2024-02-16T19:24:07.432 INFO:tasks.workunit.client.0.smithi040.stdout:6/263: dwrite d0/d1/d12/f47 [4194304,4194304] 0 2024-02-16T19:24:07.432 INFO:tasks.workunit.client.0.smithi040.stdout:6/264: read d0/f1d [187229,62043] 0 2024-02-16T19:24:07.432 INFO:tasks.workunit.client.0.smithi040.stdout:2/192: dwrite d5/d7/d1e/f20 [0,4194304] 0 2024-02-16T19:24:07.432 INFO:tasks.workunit.client.0.smithi040.stdout:2/193: write d5/d7/d1e/f25 [3383924,130208] 0 2024-02-16T19:24:07.432 INFO:tasks.workunit.client.0.smithi040.stdout:0/333: creat db/d25/d2c/f6d x:0 0 0 2024-02-16T19:24:07.432 INFO:tasks.workunit.client.0.smithi040.stdout:7/263: dwrite d21/d4d/f4b [0,4194304] 0 2024-02-16T19:24:07.433 INFO:tasks.workunit.client.0.smithi040.stdout:9/307: creat d3/d4/d11/d29/d38/d46/f5b x:0 0 0 2024-02-16T19:24:07.433 INFO:tasks.workunit.client.0.smithi040.stdout:9/308: creat d3/d4/d11/d32/f5c x:0 0 0 2024-02-16T19:24:07.433 INFO:tasks.workunit.client.0.smithi040.stdout:3/345: dwrite d1/dc/d42/d4a/f50 [0,4194304] 0 2024-02-16T19:24:07.434 INFO:tasks.workunit.client.0.smithi040.stdout:8/271: dwrite d6/d7/d15/f18 [0,4194304] 0 2024-02-16T19:24:07.435 INFO:tasks.workunit.client.0.smithi040.stdout:5/247: dwrite d1/d3c/f43 [0,4194304] 0 2024-02-16T19:24:07.435 INFO:tasks.workunit.client.0.smithi040.stdout:6/265: rename d0/d1/f29 to d0/f55 0 2024-02-16T19:24:07.436 INFO:tasks.workunit.client.0.smithi040.stdout:2/194: creat d5/d7/d1e/f33 x:0 0 0 2024-02-16T19:24:07.441 INFO:tasks.workunit.client.0.smithi040.stdout:7/264: mknod d21/d4d/c51 0 2024-02-16T19:24:07.441 INFO:tasks.workunit.client.0.smithi040.stdout:7/265: write fa [2772501,52916] 0 2024-02-16T19:24:07.441 INFO:tasks.workunit.client.0.smithi040.stdout:7/266: write f1d [4370256,42775] 0 2024-02-16T19:24:07.442 INFO:tasks.workunit.client.0.smithi040.stdout:9/309: mknod d3/d4/d11/d29/d36/c5d 0 2024-02-16T19:24:07.443 INFO:tasks.workunit.client.0.smithi040.stdout:9/310: truncate d3/f3d 2370388 0 2024-02-16T19:24:07.443 INFO:tasks.workunit.client.0.smithi040.stdout:9/311: fdatasync d3/f3d 0 2024-02-16T19:24:07.451 INFO:tasks.workunit.client.0.smithi040.stdout:3/346: unlink d1/d7/d33/d38/c4d 0 2024-02-16T19:24:07.455 INFO:tasks.workunit.client.0.smithi040.stdout:8/272: mkdir d6/d12/d41/d50 0 2024-02-16T19:24:07.455 INFO:tasks.workunit.client.0.smithi040.stdout:8/273: truncate d6/f46 121076 0 2024-02-16T19:24:07.455 INFO:tasks.workunit.client.0.smithi040.stdout:8/274: write d6/d2b/f34 [233386,74912] 0 2024-02-16T19:24:07.455 INFO:tasks.workunit.client.0.smithi040.stdout:5/248: dwrite d1/da/d1c/f25 [0,4194304] 0 2024-02-16T19:24:07.455 INFO:tasks.workunit.client.0.smithi040.stdout:5/249: dread - d1/d3c/f3f zero size 2024-02-16T19:24:07.455 INFO:tasks.workunit.client.0.smithi040.stdout:5/250: readlink d1/da/d1c/d21/l47 0 2024-02-16T19:24:07.457 INFO:tasks.workunit.client.0.smithi040.stdout:0/334: rename db/d25/d34/f3a to db/de/d6a/f6e 0 2024-02-16T19:24:07.458 INFO:tasks.workunit.client.0.smithi040.stdout:6/266: getdents d0/d30/d32 0 2024-02-16T19:24:07.463 INFO:tasks.workunit.client.0.smithi040.stdout:3/347: getdents d1/dc/d42 0 2024-02-16T19:24:07.468 INFO:tasks.workunit.client.0.smithi040.stdout:5/251: link d1/c3a d1/d45/c4a 0 2024-02-16T19:24:07.469 INFO:tasks.workunit.client.0.smithi040.stdout:6/267: dread d0/f4 [0,4194304] 0 2024-02-16T19:24:07.470 INFO:tasks.workunit.client.0.smithi040.stdout:7/267: rename c1b to d21/d29/c52 0 2024-02-16T19:24:07.471 INFO:tasks.workunit.client.0.smithi040.stdout:2/195: truncate d5/d7/d12/f15 508312 0 2024-02-16T19:24:07.473 INFO:tasks.workunit.client.0.smithi040.stdout:3/348: chown d1/d4/c8 90 1 2024-02-16T19:24:07.473 INFO:tasks.workunit.client.0.smithi040.stdout:3/349: write d1/d4/f25 [4098456,84994] 0 2024-02-16T19:24:07.475 INFO:tasks.workunit.client.0.smithi040.stdout:1/295: sync 2024-02-16T19:24:07.475 INFO:tasks.workunit.client.0.smithi040.stdout:6/268: creat d0/d30/d32/f56 x:0 0 0 2024-02-16T19:24:07.478 INFO:tasks.workunit.client.0.smithi040.stdout:9/312: rename d3/f3d to d3/d4/d55/d31/f5e 0 2024-02-16T19:24:07.478 INFO:tasks.workunit.client.0.smithi040.stdout:9/313: truncate d3/d4/d55/f5a 606149 0 2024-02-16T19:24:07.478 INFO:tasks.workunit.client.0.smithi040.stdout:9/314: chown d3/f21 1879564353 1 2024-02-16T19:24:07.478 INFO:tasks.workunit.client.0.smithi040.stdout:4/267: sync 2024-02-16T19:24:07.482 INFO:tasks.workunit.client.0.smithi040.stdout:9/315: mknod d3/c5f 0 2024-02-16T19:24:07.482 INFO:tasks.workunit.client.0.smithi040.stdout:7/268: symlink d21/l53 0 2024-02-16T19:24:07.482 INFO:tasks.workunit.client.0.smithi040.stdout:8/275: dwrite d6/d12/d22/f23 [0,4194304] 0 2024-02-16T19:24:07.482 INFO:tasks.workunit.client.0.smithi040.stdout:7/269: truncate d21/d29/f33 180886 0 2024-02-16T19:24:07.482 INFO:tasks.workunit.client.0.smithi040.stdout:7/270: chown d21/d29/f4f 1025155785 1 2024-02-16T19:24:07.487 INFO:tasks.workunit.client.0.smithi040.stdout:8/276: read d6/f24 [353671,82340] 0 2024-02-16T19:24:07.488 INFO:tasks.workunit.client.0.smithi040.stdout:9/316: creat d3/d4/d11/f60 x:0 0 0 2024-02-16T19:24:07.488 INFO:tasks.workunit.client.0.smithi040.stdout:9/317: write d3/d4/d55/d31/f35 [1532384,103786] 0 2024-02-16T19:24:07.488 INFO:tasks.workunit.client.0.smithi040.stdout:7/271: link d21/d4d/c51 d21/d4d/c54 0 2024-02-16T19:24:07.488 INFO:tasks.workunit.client.0.smithi040.stdout:0/335: dwrite db/d25/f61 [0,4194304] 0 2024-02-16T19:24:07.488 INFO:tasks.workunit.client.0.smithi040.stdout:0/336: write db/f63 [719680,87896] 0 2024-02-16T19:24:07.488 INFO:tasks.workunit.client.0.smithi040.stdout:0/337: fsync db/d25/d34/d5e/f6b 0 2024-02-16T19:24:07.489 INFO:tasks.workunit.client.0.smithi040.stdout:4/268: rmdir dd/de/d45 39 2024-02-16T19:24:07.490 INFO:tasks.workunit.client.0.smithi040.stdout:8/277: dread d6/d12/f14 [0,4194304] 0 2024-02-16T19:24:07.490 INFO:tasks.workunit.client.0.smithi040.stdout:8/278: write d6/f13 [533870,67346] 0 2024-02-16T19:24:07.490 INFO:tasks.workunit.client.0.smithi040.stdout:8/279: write d6/f46 [337387,46075] 0 2024-02-16T19:24:07.491 INFO:tasks.workunit.client.0.smithi040.stdout:8/280: write d6/d7/f32 [285469,130643] 0 2024-02-16T19:24:07.497 INFO:tasks.workunit.client.0.smithi040.stdout:0/338: truncate db/de/f38 3992003 0 2024-02-16T19:24:07.498 INFO:tasks.workunit.client.0.smithi040.stdout:5/252: dwrite d1/da/f3e [0,4194304] 0 2024-02-16T19:24:07.498 INFO:tasks.workunit.client.0.smithi040.stdout:5/253: write d1/f24 [803136,6904] 0 2024-02-16T19:24:07.500 INFO:tasks.workunit.client.0.smithi040.stdout:0/339: mkdir db/d2f/d23/d6f 0 2024-02-16T19:24:07.500 INFO:tasks.workunit.client.0.smithi040.stdout:0/340: write db/d2f/f3e [5129093,126747] 0 2024-02-16T19:24:07.509 INFO:tasks.workunit.client.0.smithi040.stdout:7/272: dread d21/d29/f30 [0,4194304] 0 2024-02-16T19:24:07.510 INFO:tasks.workunit.client.0.smithi040.stdout:7/273: mknod d21/d29/c55 0 2024-02-16T19:24:07.510 INFO:tasks.workunit.client.0.smithi040.stdout:7/274: truncate d21/d29/f37 4995404 0 2024-02-16T19:24:07.522 INFO:tasks.workunit.client.0.smithi040.stdout:5/254: write d1/da/d1c/f1f [4565127,84632] 0 2024-02-16T19:24:07.522 INFO:tasks.workunit.client.0.smithi040.stdout:5/255: truncate d1/da/d1c/d21/f42 652596 0 2024-02-16T19:24:07.538 INFO:tasks.workunit.client.0.smithi040.stdout:3/350: dwrite d1/d7/f2b [0,4194304] 0 2024-02-16T19:24:07.539 INFO:tasks.workunit.client.0.smithi040.stdout:3/351: truncate d1/f37 478429 0 2024-02-16T19:24:07.539 INFO:tasks.workunit.client.0.smithi040.stdout:0/341: fdatasync db/f63 0 2024-02-16T19:24:07.539 INFO:tasks.workunit.client.0.smithi040.stdout:0/342: write db/d2f/d1d/f46 [1230612,123695] 0 2024-02-16T19:24:07.540 INFO:tasks.workunit.client.0.smithi040.stdout:0/343: creat db/d2f/f70 x:0 0 0 2024-02-16T19:24:07.541 INFO:tasks.workunit.client.0.smithi040.stdout:0/344: symlink db/d2f/d1d/d21/d2e/d49/l71 0 2024-02-16T19:24:07.542 INFO:tasks.workunit.client.0.smithi040.stdout:0/345: rename db/f53 to db/d2f/d23/d6f/f72 0 2024-02-16T19:24:07.543 INFO:tasks.workunit.client.0.smithi040.stdout:0/346: mknod db/de/c73 0 2024-02-16T19:24:07.546 INFO:tasks.workunit.client.0.smithi040.stdout:1/296: dwrite d17/f2c [0,4194304] 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:1/297: creat d17/f61 x:0 0 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:2/196: dwrite d5/d7/d12/d1b/f30 [0,4194304] 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:5/256: dwrite d1/da/d1c/d21/f42 [0,4194304] 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:6/269: dwrite d0/d1/d12/f28 [8388608,4194304] 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:7/275: dwrite f1f [0,4194304] 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:7/276: truncate d21/f4a 287466 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:7/277: chown d21/f32 5695 1 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:7/278: fsync d21/d29/f2b 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:7/279: chown c20 3093336 1 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:4/269: dwrite dd/f19 [4194304,4194304] 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:9/318: dwrite d3/d4/d55/d31/f35 [0,4194304] 0 2024-02-16T19:24:07.548 INFO:tasks.workunit.client.0.smithi040.stdout:9/319: fsync d3/d4/d55/f5a 0 2024-02-16T19:24:07.549 INFO:tasks.workunit.client.0.smithi040.stdout:3/352: dread d1/d7/d36/f45 [0,4194304] 0 2024-02-16T19:24:07.549 INFO:tasks.workunit.client.0.smithi040.stdout:3/353: write d1/d4/fa [345698,102506] 0 2024-02-16T19:24:07.549 INFO:tasks.workunit.client.0.smithi040.stdout:8/281: dread d6/d2b/f34 [0,4194304] 0 2024-02-16T19:24:07.549 INFO:tasks.workunit.client.0.smithi040.stdout:8/282: fsync f4 0 2024-02-16T19:24:07.553 INFO:tasks.workunit.client.0.smithi040.stdout:1/298: dread fb [0,4194304] 0 2024-02-16T19:24:07.555 INFO:tasks.workunit.client.0.smithi040.stdout:5/257: write d1/da/f3e [1876765,52228] 0 2024-02-16T19:24:07.556 INFO:tasks.workunit.client.0.smithi040.stdout:1/299: dread d17/d26/f53 [0,4194304] 0 2024-02-16T19:24:07.562 INFO:tasks.workunit.client.0.smithi040.stdout:6/270: truncate d0/d1/f1f 3969999 0 2024-02-16T19:24:07.564 INFO:tasks.workunit.client.0.smithi040.stdout:2/197: read d5/d7/d12/d1b/f30 [3601970,59440] 0 2024-02-16T19:24:07.565 INFO:tasks.workunit.client.0.smithi040.stdout:4/270: rename dd/de/d10/d13/f26 to dd/de/d45/f53 0 2024-02-16T19:24:07.571 INFO:tasks.workunit.client.0.smithi040.stdout:3/354: creat d1/dc/d42/f67 x:0 0 0 2024-02-16T19:24:07.574 INFO:tasks.workunit.client.0.smithi040.stdout:8/283: mkdir d6/d12/d51 0 2024-02-16T19:24:07.577 INFO:tasks.workunit.client.0.smithi040.stdout:6/271: creat d0/d30/d4b/f57 x:0 0 0 2024-02-16T19:24:07.581 INFO:tasks.workunit.client.0.smithi040.stdout:9/320: rename d3/d4/d11/d29/d36/c5d to d3/d1c/c61 0 2024-02-16T19:24:07.584 INFO:tasks.workunit.client.0.smithi040.stdout:5/258: getdents d1/d45 0 2024-02-16T19:24:07.593 INFO:tasks.workunit.client.0.smithi040.stdout:3/355: write d1/d4/f43 [1113945,102396] 0 2024-02-16T19:24:07.607 INFO:tasks.workunit.client.0.smithi040.stdout:5/259: dread f0 [0,4194304] 0 2024-02-16T19:24:07.610 INFO:tasks.workunit.client.0.smithi040.stdout:0/347: dwrite db/d2f/d23/f40 [4194304,4194304] 0 2024-02-16T19:24:07.610 INFO:tasks.workunit.client.0.smithi040.stdout:7/280: dwrite d21/d29/f50 [0,4194304] 0 2024-02-16T19:24:07.611 INFO:tasks.workunit.client.0.smithi040.stdout:0/348: creat db/d2f/d1d/d21/d2e/d49/f74 x:0 0 0 2024-02-16T19:24:07.614 INFO:tasks.workunit.client.0.smithi040.stdout:7/281: creat d21/d4d/f56 x:0 0 0 2024-02-16T19:24:07.615 INFO:tasks.workunit.client.0.smithi040.stdout:2/198: dwrite d5/d7/dd/f2c [0,4194304] 0 2024-02-16T19:24:07.615 INFO:tasks.workunit.client.0.smithi040.stdout:2/199: chown d5/d7/d12/f15 6 1 2024-02-16T19:24:07.617 INFO:tasks.workunit.client.0.smithi040.stdout:4/271: dwrite dd/de/d10/d13/f46 [0,4194304] 0 2024-02-16T19:24:07.617 INFO:tasks.workunit.client.0.smithi040.stdout:1/300: dwrite d17/d29/f32 [0,4194304] 0 2024-02-16T19:24:07.618 INFO:tasks.workunit.client.0.smithi040.stdout:8/284: dwrite d6/d2b/f48 [0,4194304] 0 2024-02-16T19:24:07.619 INFO:tasks.workunit.client.0.smithi040.stdout:5/260: read d1/d3c/f43 [2304443,2032] 0 2024-02-16T19:24:07.619 INFO:tasks.workunit.client.0.smithi040.stdout:2/200: creat d5/d7/d1e/d31/f34 x:0 0 0 2024-02-16T19:24:07.619 INFO:tasks.workunit.client.0.smithi040.stdout:2/201: write d5/d7/d1e/f33 [432748,26453] 0 2024-02-16T19:24:07.619 INFO:tasks.workunit.client.0.smithi040.stdout:9/321: dwrite d3/d4/d55/f39 [0,4194304] 0 2024-02-16T19:24:07.619 INFO:tasks.workunit.client.0.smithi040.stdout:3/356: dwrite d1/d4/f41 [0,4194304] 0 2024-02-16T19:24:07.619 INFO:tasks.workunit.client.0.smithi040.stdout:7/282: write d21/d29/f37 [526095,125345] 0 2024-02-16T19:24:07.620 INFO:tasks.workunit.client.0.smithi040.stdout:7/283: write d21/f23 [4161771,124025] 0 2024-02-16T19:24:07.622 INFO:tasks.workunit.client.0.smithi040.stdout:0/349: write f7 [1144861,102617] 0 2024-02-16T19:24:07.622 INFO:tasks.workunit.client.0.smithi040.stdout:1/301: creat d17/d26/d5f/f62 x:0 0 0 2024-02-16T19:24:07.628 INFO:tasks.workunit.client.0.smithi040.stdout:0/350: dread db/de/d6a/f6e [0,4194304] 0 2024-02-16T19:24:07.629 INFO:tasks.workunit.client.0.smithi040.stdout:4/272: write dd/de/f2a [1898059,14409] 0 2024-02-16T19:24:07.630 INFO:tasks.workunit.client.0.smithi040.stdout:8/285: creat d6/d7/d1d/f52 x:0 0 0 2024-02-16T19:24:07.630 INFO:tasks.workunit.client.0.smithi040.stdout:8/286: chown d6/d2b/f48 1768 1 2024-02-16T19:24:07.631 INFO:tasks.workunit.client.0.smithi040.stdout:8/287: chown d6/d12/d22/f42 3019 1 2024-02-16T19:24:07.631 INFO:tasks.workunit.client.0.smithi040.stdout:8/288: creat d6/d7/d1d/d38/f53 x:0 0 0 2024-02-16T19:24:07.631 INFO:tasks.workunit.client.0.smithi040.stdout:8/289: fdatasync d6/d12/f4b 0 2024-02-16T19:24:07.631 INFO:tasks.workunit.client.0.smithi040.stdout:8/290: readlink d6/l20 0 2024-02-16T19:24:07.633 INFO:tasks.workunit.client.0.smithi040.stdout:9/322: creat d3/d4/d4a/f62 x:0 0 0 2024-02-16T19:24:07.636 INFO:tasks.workunit.client.0.smithi040.stdout:3/357: mknod d1/d7/d30/c68 0 2024-02-16T19:24:07.643 INFO:tasks.workunit.client.0.smithi040.stdout:0/351: dread db/d25/f61 [0,4194304] 0 2024-02-16T19:24:07.643 INFO:tasks.workunit.client.0.smithi040.stdout:2/202: dwrite d5/d7/dd/f2c [0,4194304] 0 2024-02-16T19:24:07.643 INFO:tasks.workunit.client.0.smithi040.stdout:0/352: chown db/d2f/d23/c4e 845 1 2024-02-16T19:24:07.643 INFO:tasks.workunit.client.0.smithi040.stdout:0/353: creat db/de/f75 x:0 0 0 2024-02-16T19:24:07.643 INFO:tasks.workunit.client.0.smithi040.stdout:5/261: dwrite d1/da/f3e [0,4194304] 0 2024-02-16T19:24:07.652 INFO:tasks.workunit.client.0.smithi040.stdout:7/284: symlink d21/d4d/l57 0 2024-02-16T19:24:07.652 INFO:tasks.workunit.client.0.smithi040.stdout:9/323: dwrite d3/d4/d11/d29/d38/f3e [0,4194304] 0 2024-02-16T19:24:07.652 INFO:tasks.workunit.client.0.smithi040.stdout:1/302: mkdir d17/d26/d5f/d63 0 2024-02-16T19:24:07.655 INFO:tasks.workunit.client.0.smithi040.stdout:4/273: rename dd/de/d10/f3c to dd/de/f54 0 2024-02-16T19:24:07.659 INFO:tasks.workunit.client.0.smithi040.stdout:8/291: symlink d6/d2b/l54 0 2024-02-16T19:24:07.659 INFO:tasks.workunit.client.0.smithi040.stdout:8/292: fdatasync d6/f3c 0 2024-02-16T19:24:07.660 INFO:tasks.workunit.client.0.smithi040.stdout:3/358: mknod d1/dc/d35/d53/d64/c69 0 2024-02-16T19:24:07.660 INFO:tasks.workunit.client.0.smithi040.stdout:3/359: stat d1/d7/d36 0 2024-02-16T19:24:07.660 INFO:tasks.workunit.client.0.smithi040.stdout:2/203: symlink d5/d7/d12/d1b/l35 0 2024-02-16T19:24:07.663 INFO:tasks.workunit.client.0.smithi040.stdout:8/293: write f4 [5860471,54843] 0 2024-02-16T19:24:07.663 INFO:tasks.workunit.client.0.smithi040.stdout:0/354: link db/d2f/d23/c50 db/d2f/d23/d6f/c76 0 2024-02-16T19:24:07.665 INFO:tasks.workunit.client.0.smithi040.stdout:7/285: symlink d21/d29/l58 0 2024-02-16T19:24:07.665 INFO:tasks.workunit.client.0.smithi040.stdout:7/286: write d21/d29/f30 [4458497,65928] 0 2024-02-16T19:24:07.669 INFO:tasks.workunit.client.0.smithi040.stdout:9/324: mkdir d3/d4/d55/d63 0 2024-02-16T19:24:07.669 INFO:tasks.workunit.client.0.smithi040.stdout:9/325: dread - d3/d4/d11/d29/f48 zero size 2024-02-16T19:24:07.669 INFO:tasks.workunit.client.0.smithi040.stdout:9/326: truncate d3/d4/d11/f60 618800 0 2024-02-16T19:24:07.669 INFO:tasks.workunit.client.0.smithi040.stdout:1/303: mknod d17/d26/d54/c64 0 2024-02-16T19:24:07.672 INFO:tasks.workunit.client.0.smithi040.stdout:4/274: rename dd/de/f2f to dd/de/d10/f55 0 2024-02-16T19:24:07.674 INFO:tasks.workunit.client.0.smithi040.stdout:3/360: symlink d1/dc/d35/d53/d64/d65/l6a 0 2024-02-16T19:24:07.674 INFO:tasks.workunit.client.0.smithi040.stdout:3/361: stat d1/ff 0 2024-02-16T19:24:07.674 INFO:tasks.workunit.client.0.smithi040.stdout:3/362: stat d1/dc/d35/d53 0 2024-02-16T19:24:07.675 INFO:tasks.workunit.client.0.smithi040.stdout:2/204: symlink d5/d7/d1e/d31/l36 0 2024-02-16T19:24:07.679 INFO:tasks.workunit.client.0.smithi040.stdout:5/262: dwrite d1/da/d1c/d21/f36 [0,4194304] 0 2024-02-16T19:24:07.679 INFO:tasks.workunit.client.0.smithi040.stdout:8/294: symlink d6/d2b/l55 0 2024-02-16T19:24:07.679 INFO:tasks.workunit.client.0.smithi040.stdout:8/295: stat c1 0 2024-02-16T19:24:07.679 INFO:tasks.workunit.client.0.smithi040.stdout:8/296: fdatasync d6/d7/d1d/d38/f3d 0 2024-02-16T19:24:07.679 INFO:tasks.workunit.client.0.smithi040.stdout:8/297: dread - d6/d7/f2e zero size 2024-02-16T19:24:07.682 INFO:tasks.workunit.client.0.smithi040.stdout:7/287: rename d21/d29/f33 to d21/d29/f59 0 2024-02-16T19:24:07.683 INFO:tasks.workunit.client.0.smithi040.stdout:1/304: mknod d17/c65 0 2024-02-16T19:24:07.684 INFO:tasks.workunit.client.0.smithi040.stdout:4/275: symlink dd/de/d10/d13/d32/l56 0 2024-02-16T19:24:07.686 INFO:tasks.workunit.client.0.smithi040.stdout:2/205: rename d5/d7/d1e/f20 to d5/d7/d1e/f37 0 2024-02-16T19:24:07.686 INFO:tasks.workunit.client.0.smithi040.stdout:2/206: truncate d5/d7/d12/f13 1236394 0 2024-02-16T19:24:07.686 INFO:tasks.workunit.client.0.smithi040.stdout:2/207: creat d5/d8/f38 x:0 0 0 2024-02-16T19:24:07.690 INFO:tasks.workunit.client.0.smithi040.stdout:4/276: dread dd/de/d10/f2e [0,4194304] 0 2024-02-16T19:24:07.692 INFO:tasks.workunit.client.0.smithi040.stdout:3/363: dwrite d1/d4/f2e [0,4194304] 0 2024-02-16T19:24:07.695 INFO:tasks.workunit.client.0.smithi040.stdout:4/277: dread dd/de/f2a [0,4194304] 0 2024-02-16T19:24:07.695 INFO:tasks.workunit.client.0.smithi040.stdout:4/278: fdatasync dd/de/d10/d13/d32/f41 0 2024-02-16T19:24:07.695 INFO:tasks.workunit.client.0.smithi040.stdout:5/263: mknod d1/da/d1c/d21/c4b 0 2024-02-16T19:24:07.697 INFO:tasks.workunit.client.0.smithi040.stdout:8/298: truncate d6/f9 3817827 0 2024-02-16T19:24:07.700 INFO:tasks.workunit.client.0.smithi040.stdout:7/288: symlink d21/d29/l5a 0 2024-02-16T19:24:07.700 INFO:tasks.workunit.client.0.smithi040.stdout:3/364: dread d1/d7/f4e [0,4194304] 0 2024-02-16T19:24:07.700 INFO:tasks.workunit.client.0.smithi040.stdout:3/365: write d1/dc/fd [920191,86734] 0 2024-02-16T19:24:07.701 INFO:tasks.workunit.client.0.smithi040.stdout:7/289: write f17 [2488566,62357] 0 2024-02-16T19:24:07.701 INFO:tasks.workunit.client.0.smithi040.stdout:3/366: truncate d1/dc/d42/d4a/f50 4248232 0 2024-02-16T19:24:07.701 INFO:tasks.workunit.client.0.smithi040.stdout:3/367: fsync d1/d7/d33/d38/f56 0 2024-02-16T19:24:07.701 INFO:tasks.workunit.client.0.smithi040.stdout:1/305: creat d17/d26/d5f/d63/f66 x:0 0 0 2024-02-16T19:24:07.701 INFO:tasks.workunit.client.0.smithi040.stdout:1/306: chown d17/d43/f5d 0 1 2024-02-16T19:24:07.702 INFO:tasks.workunit.client.0.smithi040.stdout:4/279: truncate dd/de/f2a 3272837 0 2024-02-16T19:24:07.704 INFO:tasks.workunit.client.0.smithi040.stdout:8/299: mknod d6/d12/d41/c56 0 2024-02-16T19:24:07.704 INFO:tasks.workunit.client.0.smithi040.stdout:8/300: chown d6/d2b 366 1 2024-02-16T19:24:07.704 INFO:tasks.workunit.client.0.smithi040.stdout:8/301: fsync d6/d7/d1d/d38/f3d 0 2024-02-16T19:24:07.706 INFO:tasks.workunit.client.0.smithi040.stdout:4/280: rename dd/de/d10/d13/d32/l56 to dd/de/d10/d13/d32/l57 0 2024-02-16T19:24:07.706 INFO:tasks.workunit.client.0.smithi040.stdout:3/368: read d1/ff [524185,72097] 0 2024-02-16T19:24:07.707 INFO:tasks.workunit.client.0.smithi040.stdout:3/369: chown d1/dc/d42/c51 19 1 2024-02-16T19:24:07.708 INFO:tasks.workunit.client.0.smithi040.stdout:2/208: dwrite d5/d7/d1e/d31/f34 [0,4194304] 0 2024-02-16T19:24:07.709 INFO:tasks.workunit.client.0.smithi040.stdout:8/302: mknod d6/d7/c57 0 2024-02-16T19:24:07.710 INFO:tasks.workunit.client.0.smithi040.stdout:3/370: mknod d1/dc/d35/d55/c6b 0 2024-02-16T19:24:07.710 INFO:tasks.workunit.client.0.smithi040.stdout:3/371: write d1/dc/fd [3443002,54007] 0 2024-02-16T19:24:07.710 INFO:tasks.workunit.client.0.smithi040.stdout:3/372: readlink d1/dc/d35/l46 0 2024-02-16T19:24:07.711 INFO:tasks.workunit.client.0.smithi040.stdout:5/264: dwrite d1/f13 [0,4194304] 0 2024-02-16T19:24:07.711 INFO:tasks.workunit.client.0.smithi040.stdout:5/265: write d1/da/d1c/f40 [679523,121289] 0 2024-02-16T19:24:07.714 INFO:tasks.workunit.client.0.smithi040.stdout:2/209: mknod d5/d7/d1e/d31/c39 0 2024-02-16T19:24:07.715 INFO:tasks.workunit.client.0.smithi040.stdout:2/210: creat d5/f3a x:0 0 0 2024-02-16T19:24:07.715 INFO:tasks.workunit.client.0.smithi040.stdout:2/211: chown d5/d8/f38 15 1 2024-02-16T19:24:07.717 INFO:tasks.workunit.client.0.smithi040.stdout:3/373: rmdir d1/dc/d35/d53/d64/d65 39 2024-02-16T19:24:07.719 INFO:tasks.workunit.client.0.smithi040.stdout:8/303: dread d6/d7/d15/f1c [0,4194304] 0 2024-02-16T19:24:07.720 INFO:tasks.workunit.client.0.smithi040.stdout:5/266: creat d1/f4c x:0 0 0 2024-02-16T19:24:07.721 INFO:tasks.workunit.client.0.smithi040.stdout:1/307: dwrite d17/d2d/f2e [0,4194304] 0 2024-02-16T19:24:07.721 INFO:tasks.workunit.client.0.smithi040.stdout:1/308: read d17/f1f [1810284,15980] 0 2024-02-16T19:24:07.724 INFO:tasks.workunit.client.0.smithi040.stdout:2/212: unlink d5/d7/l14 0 2024-02-16T19:24:07.724 INFO:tasks.workunit.client.0.smithi040.stdout:2/213: creat d5/d7/d1e/f3b x:0 0 0 2024-02-16T19:24:07.725 INFO:tasks.workunit.client.0.smithi040.stdout:8/304: symlink d6/d2b/d36/l58 0 2024-02-16T19:24:07.726 INFO:tasks.workunit.client.0.smithi040.stdout:6/272: sync 2024-02-16T19:24:07.727 INFO:tasks.workunit.client.0.smithi040.stdout:8/305: dread d6/f46 [0,4194304] 0 2024-02-16T19:24:07.729 INFO:tasks.workunit.client.0.smithi040.stdout:2/214: rmdir d5/d7/d10 39 2024-02-16T19:24:07.729 INFO:tasks.workunit.client.0.smithi040.stdout:2/215: creat d5/f3c x:0 0 0 2024-02-16T19:24:07.730 INFO:tasks.workunit.client.0.smithi040.stdout:1/309: getdents d17/d26/d3d 0 2024-02-16T19:24:07.731 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:07 smithi163 ceph-mon[36452]: pgmap v242: 65 pgs: 65 active+clean; 483 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 22 MiB/s rd, 47 MiB/s wr, 1.06k op/s 2024-02-16T19:24:07.733 INFO:tasks.workunit.client.0.smithi040.stdout:5/267: read d1/f2c [4036325,5575] 0 2024-02-16T19:24:07.733 INFO:tasks.workunit.client.0.smithi040.stdout:8/306: mknod d6/d2b/d36/c59 0 2024-02-16T19:24:07.733 INFO:tasks.workunit.client.0.smithi040.stdout:8/307: write d6/d7/f32 [473839,14604] 0 2024-02-16T19:24:07.733 INFO:tasks.workunit.client.0.smithi040.stdout:8/308: write d6/f43 [937359,115143] 0 2024-02-16T19:24:07.733 INFO:tasks.workunit.client.0.smithi040.stdout:8/309: dread - d6/d12/f4b zero size 2024-02-16T19:24:07.736 INFO:tasks.workunit.client.0.smithi040.stdout:3/374: dwrite d1/d4/f25 [0,4194304] 0 2024-02-16T19:24:07.736 INFO:tasks.workunit.client.0.smithi040.stdout:3/375: fsync d1/d4/f2e 0 2024-02-16T19:24:07.737 INFO:tasks.workunit.client.0.smithi040.stdout:3/376: creat d1/d7/d33/f6c x:0 0 0 2024-02-16T19:24:07.738 INFO:tasks.workunit.client.0.smithi040.stdout:2/216: rename d5/d7/dd/c27 to d5/c3d 0 2024-02-16T19:24:07.738 INFO:tasks.workunit.client.0.smithi040.stdout:2/217: chown d5/d8/cc 2959 1 2024-02-16T19:24:07.739 INFO:tasks.workunit.client.0.smithi040.stdout:1/310: mknod d17/d26/d3d/c67 0 2024-02-16T19:24:07.741 INFO:tasks.workunit.client.0.smithi040.stdout:5/268: mknod d1/d3c/c4d 0 2024-02-16T19:24:07.746 INFO:tasks.workunit.client.0.smithi040.stdout:8/310: rename d6/d12/d41/f4a to d6/d12/f5a 0 2024-02-16T19:24:07.746 INFO:tasks.workunit.client.0.smithi040.stdout:8/311: fdatasync d6/d7/d1d/f52 0 2024-02-16T19:24:07.746 INFO:tasks.workunit.client.0.smithi040.stdout:8/312: stat d6/d7/d1d/f29 0 2024-02-16T19:24:07.747 INFO:tasks.workunit.client.0.smithi040.stdout:8/313: chown d6/d2b/l54 62 1 2024-02-16T19:24:07.747 INFO:tasks.workunit.client.0.smithi040.stdout:6/273: dwrite d0/d1/d12/f47 [4194304,4194304] 0 2024-02-16T19:24:07.749 INFO:tasks.workunit.client.0.smithi040.stdout:5/269: symlink d1/da/d1c/d21/d41/l4e 0 2024-02-16T19:24:07.749 INFO:tasks.workunit.client.0.smithi040.stdout:5/270: chown d1/c3a 2615 1 2024-02-16T19:24:07.752 INFO:tasks.workunit.client.0.smithi040.stdout:3/377: rename d1/d7/d33/d38/f56 to d1/dc/d42/d5c/f6d 0 2024-02-16T19:24:07.753 INFO:tasks.workunit.client.0.smithi040.stdout:2/218: rmdir d5/d7/d12 39 2024-02-16T19:24:07.756 INFO:tasks.workunit.client.0.smithi040.stdout:3/378: mkdir d1/d4/d6e 0 2024-02-16T19:24:07.757 INFO:tasks.workunit.client.0.smithi040.stdout:6/274: dread d0/f33 [0,4194304] 0 2024-02-16T19:24:07.759 INFO:tasks.workunit.client.0.smithi040.stdout:3/379: unlink d1/dc/d35/d53/d64/d65/l6a 0 2024-02-16T19:24:07.759 INFO:tasks.workunit.client.0.smithi040.stdout:3/380: chown d1/d7/l5a 15 1 2024-02-16T19:24:07.759 INFO:tasks.workunit.client.0.smithi040.stdout:3/381: write d1/d7/d33/f5e [1430346,64273] 0 2024-02-16T19:24:07.762 INFO:tasks.workunit.client.0.smithi040.stdout:3/382: mknod d1/d7/c6f 0 2024-02-16T19:24:07.762 INFO:tasks.workunit.client.0.smithi040.stdout:3/383: write d1/f3 [4161478,6646] 0 2024-02-16T19:24:07.762 INFO:tasks.workunit.client.0.smithi040.stdout:3/384: chown d1/f37 357264149 1 2024-02-16T19:24:07.762 INFO:tasks.workunit.client.0.smithi040.stdout:1/311: dwrite d17/d3c/f5a [0,4194304] 0 2024-02-16T19:24:07.762 INFO:tasks.workunit.client.0.smithi040.stdout:1/312: write f1 [4359500,120022] 0 2024-02-16T19:24:07.764 INFO:tasks.workunit.client.0.smithi040.stdout:6/275: rename d0/f5 to d0/d1/d12/f58 0 2024-02-16T19:24:07.766 INFO:tasks.workunit.client.0.smithi040.stdout:3/385: mknod d1/d7/d36/c70 0 2024-02-16T19:24:07.770 INFO:tasks.workunit.client.0.smithi040.stdout:8/314: dwrite d6/d12/f5a [0,4194304] 0 2024-02-16T19:24:07.770 INFO:tasks.workunit.client.0.smithi040.stdout:2/219: dwrite d5/d7/d1e/f33 [0,4194304] 0 2024-02-16T19:24:07.770 INFO:tasks.workunit.client.0.smithi040.stdout:2/220: fsync d5/d7/d1e/f37 0 2024-02-16T19:24:07.772 INFO:tasks.workunit.client.0.smithi040.stdout:3/386: rename d1/d7/f9 to d1/dc/d42/d58/f71 0 2024-02-16T19:24:07.773 INFO:tasks.workunit.client.0.smithi040.stdout:6/276: mknod d0/d1/d12/d3b/d48/d4f/c59 0 2024-02-16T19:24:07.775 INFO:tasks.workunit.client.0.smithi040.stdout:8/315: dread d6/d7/f27 [4194304,4194304] 0 2024-02-16T19:24:07.775 INFO:tasks.workunit.client.0.smithi040.stdout:8/316: chown d6/d7/d1d/d38/f53 1307681 1 2024-02-16T19:24:07.776 INFO:tasks.workunit.client.0.smithi040.stdout:8/317: dread - d6/d2b/d36/f3e zero size 2024-02-16T19:24:07.776 INFO:tasks.workunit.client.0.smithi040.stdout:8/318: readlink d6/d2b/d36/l58 0 2024-02-16T19:24:07.782 INFO:tasks.workunit.client.0.smithi040.stdout:6/277: rename d0/d30/l51 to d0/d1/d12/l5a 0 2024-02-16T19:24:07.782 INFO:tasks.workunit.client.0.smithi040.stdout:3/387: dread d1/d7/f4e [0,4194304] 0 2024-02-16T19:24:07.783 INFO:tasks.workunit.client.0.smithi040.stdout:1/313: dwrite d17/f1c [0,4194304] 0 2024-02-16T19:24:07.783 INFO:tasks.workunit.client.0.smithi040.stdout:2/221: write d5/d7/d1e/d31/f34 [1903771,29939] 0 2024-02-16T19:24:07.783 INFO:tasks.workunit.client.0.smithi040.stdout:1/314: fsync d17/d26/d4c/f5b 0 2024-02-16T19:24:07.783 INFO:tasks.workunit.client.0.smithi040.stdout:2/222: chown d5/d7/d1e/d31/l36 18 1 2024-02-16T19:24:07.783 INFO:tasks.workunit.client.0.smithi040.stdout:1/315: write d17/d2d/f3f [425964,85460] 0 2024-02-16T19:24:07.783 INFO:tasks.workunit.client.0.smithi040.stdout:2/223: write d5/d7/d12/d1b/f2f [999691,88184] 0 2024-02-16T19:24:07.784 INFO:tasks.workunit.client.0.smithi040.stdout:6/278: chown d0/d1/d12/d3a/l44 79043 1 2024-02-16T19:24:07.786 INFO:tasks.workunit.client.0.smithi040.stdout:3/388: creat d1/dc/d42/d58/f72 x:0 0 0 2024-02-16T19:24:07.786 INFO:tasks.workunit.client.0.smithi040.stdout:1/316: dread d17/d26/f4b [0,4194304] 0 2024-02-16T19:24:07.794 INFO:tasks.workunit.client.0.smithi040.stdout:8/319: rename d6/d12/c2c to d6/d7/d15/c5b 0 2024-02-16T19:24:07.795 INFO:tasks.workunit.client.0.smithi040.stdout:6/279: mkdir d0/d30/d5b 0 2024-02-16T19:24:07.797 INFO:tasks.workunit.client.0.smithi040.stdout:2/224: dwrite d5/d7/d1e/f25 [0,4194304] 0 2024-02-16T19:24:07.799 INFO:tasks.workunit.client.0.smithi040.stdout:3/389: unlink d1/dc/d42/f67 0 2024-02-16T19:24:07.800 INFO:tasks.workunit.client.0.smithi040.stdout:1/317: mkdir d17/d26/d54/d68 0 2024-02-16T19:24:07.801 INFO:tasks.workunit.client.0.smithi040.stdout:8/320: mknod d6/d7/d1d/d38/c5c 0 2024-02-16T19:24:07.803 INFO:tasks.workunit.client.0.smithi040.stdout:6/280: rmdir d0/d1/d12/d3a/d4e 0 2024-02-16T19:24:07.805 INFO:tasks.workunit.client.0.smithi040.stdout:2/225: mknod d5/d7/d1e/d31/c3e 0 2024-02-16T19:24:07.806 INFO:tasks.workunit.client.0.smithi040.stdout:3/390: symlink d1/dc/d35/d53/d64/l73 0 2024-02-16T19:24:07.807 INFO:tasks.workunit.client.0.smithi040.stdout:1/318: symlink d17/l69 0 2024-02-16T19:24:07.808 INFO:tasks.workunit.client.0.smithi040.stdout:1/319: write f8 [956972,45812] 0 2024-02-16T19:24:07.808 INFO:tasks.workunit.client.0.smithi040.stdout:8/321: mknod d6/d12/d51/c5d 0 2024-02-16T19:24:07.810 INFO:tasks.workunit.client.0.smithi040.stdout:6/281: link d0/f55 d0/d30/d32/f5c 0 2024-02-16T19:24:07.810 INFO:tasks.workunit.client.0.smithi040.stdout:6/282: chown d0/d1/cc 1482 1 2024-02-16T19:24:07.810 INFO:tasks.workunit.client.0.smithi040.stdout:6/283: chown d0/f55 9 1 2024-02-16T19:24:07.811 INFO:tasks.workunit.client.0.smithi040.stdout:2/226: mknod d5/d7/c3f 0 2024-02-16T19:24:07.812 INFO:tasks.workunit.client.0.smithi040.stdout:9/327: sync 2024-02-16T19:24:07.812 INFO:tasks.workunit.client.0.smithi040.stdout:0/355: sync 2024-02-16T19:24:07.812 INFO:tasks.workunit.client.0.smithi040.stdout:0/356: stat db/d2f/d1d/d21/l6c 0 2024-02-16T19:24:07.812 INFO:tasks.workunit.client.0.smithi040.stdout:0/357: creat db/d2f/f77 x:0 0 0 2024-02-16T19:24:07.813 INFO:tasks.workunit.client.0.smithi040.stdout:3/391: unlink d1/dc/d42/d58/f71 0 2024-02-16T19:24:07.813 INFO:tasks.workunit.client.0.smithi040.stdout:3/392: dread - d1/dc/d42/d4a/f54 zero size 2024-02-16T19:24:07.813 INFO:tasks.workunit.client.0.smithi040.stdout:3/393: write d1/dc/d42/d4a/f54 [681135,118729] 0 2024-02-16T19:24:07.818 INFO:tasks.workunit.client.0.smithi040.stdout:2/227: mkdir d5/d7/d10/d40 0 2024-02-16T19:24:07.818 INFO:tasks.workunit.client.0.smithi040.stdout:2/228: write d5/d7/d12/d1b/f30 [432434,74386] 0 2024-02-16T19:24:07.820 INFO:tasks.workunit.client.0.smithi040.stdout:3/394: dread d1/d4/f41 [0,4194304] 0 2024-02-16T19:24:07.822 INFO:tasks.workunit.client.0.smithi040.stdout:0/358: creat db/d2f/d23/d6f/f78 x:0 0 0 2024-02-16T19:24:07.825 INFO:tasks.workunit.client.0.smithi040.stdout:1/320: rmdir d17/d26/d5f 39 2024-02-16T19:24:07.825 INFO:tasks.workunit.client.0.smithi040.stdout:1/321: readlink d17/d43/l24 0 2024-02-16T19:24:07.828 INFO:tasks.workunit.client.0.smithi040.stdout:2/229: truncate d5/d7/f24 847128 0 2024-02-16T19:24:07.830 INFO:tasks.workunit.client.0.smithi040.stdout:3/395: creat d1/dc/d35/d55/f74 x:0 0 0 2024-02-16T19:24:07.830 INFO:tasks.workunit.client.0.smithi040.stdout:3/396: getdents d1/dc/d42/d4a/d57 0 2024-02-16T19:24:07.831 INFO:tasks.workunit.client.0.smithi040.stdout:0/359: creat db/d2f/d1d/d21/d2e/d49/d5b/f79 x:0 0 0 2024-02-16T19:24:07.832 INFO:tasks.workunit.client.0.smithi040.stdout:1/322: write d17/d26/d3d/f56 [525838,67708] 0 2024-02-16T19:24:07.834 INFO:tasks.workunit.client.0.smithi040.stdout:0/360: symlink db/l7a 0 2024-02-16T19:24:07.836 INFO:tasks.workunit.client.0.smithi040.stdout:1/323: rename d17/d43/f39 to d17/d43/f6a 0 2024-02-16T19:24:07.836 INFO:tasks.workunit.client.0.smithi040.stdout:1/324: readlink d17/d29/l47 0 2024-02-16T19:24:07.836 INFO:tasks.workunit.client.0.smithi040.stdout:1/325: chown d17/d29/l35 30 1 2024-02-16T19:24:07.837 INFO:tasks.workunit.client.0.smithi040.stdout:3/397: dread d1/d4/f43 [0,4194304] 0 2024-02-16T19:24:07.837 INFO:tasks.workunit.client.0.smithi040.stdout:3/398: fdatasync d1/d4/f41 0 2024-02-16T19:24:07.841 INFO:tasks.workunit.client.0.smithi040.stdout:0/361: rmdir db/d25 39 2024-02-16T19:24:07.841 INFO:tasks.workunit.client.0.smithi040.stdout:0/362: chown db/d25/d2c/f6d 3888 1 2024-02-16T19:24:07.842 INFO:tasks.workunit.client.0.smithi040.stdout:0/363: symlink db/de/d6a/l7b 0 2024-02-16T19:24:07.842 INFO:tasks.workunit.client.0.smithi040.stdout:9/328: dwrite d3/d4/f4d [0,4194304] 0 2024-02-16T19:24:07.842 INFO:tasks.workunit.client.0.smithi040.stdout:9/329: stat d3/d4/d11/d29 0 2024-02-16T19:24:07.843 INFO:tasks.workunit.client.0.smithi040.stdout:0/364: mknod db/d2f/d1d/d21/d2e/d49/d5b/d67/c7c 0 2024-02-16T19:24:07.844 INFO:tasks.workunit.client.0.smithi040.stdout:8/322: dwrite d6/d12/d22/f23 [0,4194304] 0 2024-02-16T19:24:07.844 INFO:tasks.workunit.client.0.smithi040.stdout:8/323: dread - d6/d7/f2e zero size 2024-02-16T19:24:07.844 INFO:tasks.workunit.client.0.smithi040.stdout:6/284: dwrite d0/d1/d12/f2c [0,4194304] 0 2024-02-16T19:24:07.844 INFO:tasks.workunit.client.0.smithi040.stdout:6/285: chown d0/d30/d32 224082 1 2024-02-16T19:24:07.844 INFO:tasks.workunit.client.0.smithi040.stdout:6/286: fdatasync d0/f35 0 2024-02-16T19:24:07.844 INFO:tasks.workunit.client.0.smithi040.stdout:6/287: stat d0/d1/d12/d3a/f42 0 2024-02-16T19:24:07.846 INFO:tasks.workunit.client.0.smithi040.stdout:0/365: link db/d25/f61 db/de/f7d 0 2024-02-16T19:24:07.846 INFO:tasks.workunit.client.0.smithi040.stdout:1/326: dread d17/d2d/f2f [0,4194304] 0 2024-02-16T19:24:07.847 INFO:tasks.workunit.client.0.smithi040.stdout:2/230: truncate d5/d7/d12/d1b/f30 3224970 0 2024-02-16T19:24:07.850 INFO:tasks.workunit.client.0.smithi040.stdout:9/330: write d3/d4/d11/f53 [1569048,83626] 0 2024-02-16T19:24:07.850 INFO:tasks.workunit.client.0.smithi040.stdout:9/331: chown d3/d4/d4a/f62 5312017 1 2024-02-16T19:24:07.850 INFO:tasks.workunit.client.0.smithi040.stdout:8/324: symlink d6/d7/d1d/d38/l5e 0 2024-02-16T19:24:07.853 INFO:tasks.workunit.client.0.smithi040.stdout:6/288: creat d0/d37/f5d x:0 0 0 2024-02-16T19:24:07.853 INFO:tasks.workunit.client.0.smithi040.stdout:6/289: chown d0/d30/d4b 704823 1 2024-02-16T19:24:07.856 INFO:tasks.workunit.client.0.smithi040.stdout:0/366: mknod db/d25/d2c/c7e 0 2024-02-16T19:24:07.856 INFO:tasks.workunit.client.0.smithi040.stdout:0/367: read - db/d25/d2c/d52/f57 zero size 2024-02-16T19:24:07.856 INFO:tasks.workunit.client.0.smithi040.stdout:0/368: creat db/f7f x:0 0 0 2024-02-16T19:24:07.858 INFO:tasks.workunit.client.0.smithi040.stdout:1/327: unlink d17/d3c/f5a 0 2024-02-16T19:24:07.859 INFO:tasks.workunit.client.0.smithi040.stdout:2/231: mknod d5/d7/c41 0 2024-02-16T19:24:07.863 INFO:tasks.workunit.client.0.smithi040.stdout:8/325: truncate d6/f9 2504029 0 2024-02-16T19:24:07.863 INFO:tasks.workunit.client.0.smithi040.stdout:8/326: fsync d6/d7/d15/f2f 0 2024-02-16T19:24:07.864 INFO:tasks.workunit.client.0.smithi040.stdout:6/290: creat d0/d37/d4a/f5e x:0 0 0 2024-02-16T19:24:07.864 INFO:tasks.workunit.client.0.smithi040.stdout:6/291: chown d0/d30/l46 30 1 2024-02-16T19:24:07.864 INFO:tasks.workunit.client.0.smithi040.stdout:6/292: readlink d0/d1/l7 0 2024-02-16T19:24:07.866 INFO:tasks.workunit.client.0.smithi040.stdout:9/332: dwrite d3/d4/d55/f23 [0,4194304] 0 2024-02-16T19:24:07.867 INFO:tasks.workunit.client.0.smithi040.stdout:0/369: unlink db/d2f/d1d/d21/d2e/d49/f74 0 2024-02-16T19:24:07.869 INFO:tasks.workunit.client.0.smithi040.stdout:2/232: mkdir d5/d8/d42 0 2024-02-16T19:24:07.872 INFO:tasks.workunit.client.0.smithi040.stdout:8/327: rename d6/d7/d15/c5b to d6/d12/d51/c5f 0 2024-02-16T19:24:07.872 INFO:tasks.workunit.client.0.smithi040.stdout:4/281: sync 2024-02-16T19:24:07.872 INFO:tasks.workunit.client.0.smithi040.stdout:5/271: sync 2024-02-16T19:24:07.872 INFO:tasks.workunit.client.0.smithi040.stdout:7/290: sync 2024-02-16T19:24:07.872 INFO:tasks.workunit.client.0.smithi040.stdout:7/291: chown d21/l53 98187 1 2024-02-16T19:24:07.875 INFO:tasks.workunit.client.0.smithi040.stdout:9/333: creat d3/d1c/f64 x:0 0 0 2024-02-16T19:24:07.876 INFO:tasks.workunit.client.0.smithi040.stdout:4/282: dread dd/de/f54 [0,4194304] 0 2024-02-16T19:24:07.878 INFO:tasks.workunit.client.0.smithi040.stdout:5/272: symlink d1/da/d1c/l4f 0 2024-02-16T19:24:07.881 INFO:tasks.workunit.client.0.smithi040.stdout:9/334: rename d3/c49 to d3/d4/d55/d63/c65 0 2024-02-16T19:24:07.883 INFO:tasks.workunit.client.0.smithi040.stdout:4/283: mkdir dd/d58 0 2024-02-16T19:24:07.884 INFO:tasks.workunit.client.0.smithi040.stdout:2/233: getdents d5/d7/d1e 0 2024-02-16T19:24:07.884 INFO:tasks.workunit.client.0.smithi040.stdout:2/234: creat d5/d7/d1e/d31/f43 x:0 0 0 2024-02-16T19:24:07.886 INFO:tasks.workunit.client.0.smithi040.stdout:7/292: dread d21/d29/f50 [0,4194304] 0 2024-02-16T19:24:07.886 INFO:tasks.workunit.client.0.smithi040.stdout:7/293: creat d21/d4d/f5b x:0 0 0 2024-02-16T19:24:07.887 INFO:tasks.workunit.client.0.smithi040.stdout:5/273: truncate d1/f17 289950 0 2024-02-16T19:24:07.889 INFO:tasks.workunit.client.0.smithi040.stdout:9/335: link d3/d4/d11/d29/f52 d3/d4/d55/f66 0 2024-02-16T19:24:07.889 INFO:tasks.workunit.client.0.smithi040.stdout:9/336: chown d3/d4/d55/d31/f4c 0 1 2024-02-16T19:24:07.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:07 smithi040 ceph-mon[29201]: pgmap v242: 65 pgs: 65 active+clean; 483 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 22 MiB/s rd, 47 MiB/s wr, 1.06k op/s 2024-02-16T19:24:07.896 INFO:tasks.workunit.client.0.smithi040.stdout:1/328: dwrite fb [4194304,4194304] 0 2024-02-16T19:24:07.898 INFO:tasks.workunit.client.0.smithi040.stdout:1/329: mkdir d17/d3c/d6b 0 2024-02-16T19:24:07.901 INFO:tasks.workunit.client.0.smithi040.stdout:1/330: mknod d17/d26/d54/c6c 0 2024-02-16T19:24:07.901 INFO:tasks.workunit.client.0.smithi040.stdout:1/331: fdatasync d17/f1c 0 2024-02-16T19:24:07.901 INFO:tasks.workunit.client.0.smithi040.stdout:1/332: fsync d17/d26/d3d/f56 0 2024-02-16T19:24:07.901 INFO:tasks.workunit.client.0.smithi040.stdout:4/284: dread dd/de/d10/f30 [0,4194304] 0 2024-02-16T19:24:07.904 INFO:tasks.workunit.client.0.smithi040.stdout:2/235: dread d5/d7/d1e/f37 [0,4194304] 0 2024-02-16T19:24:07.905 INFO:tasks.workunit.client.0.smithi040.stdout:1/333: dread d17/f25 [0,4194304] 0 2024-02-16T19:24:07.906 INFO:tasks.workunit.client.0.smithi040.stdout:2/236: rename d5/d7/d12/d1b/c2d to d5/c44 0 2024-02-16T19:24:07.907 INFO:tasks.workunit.client.0.smithi040.stdout:9/337: read d3/d4/d55/f23 [4188996,40335] 0 2024-02-16T19:24:07.907 INFO:tasks.workunit.client.0.smithi040.stdout:9/338: dread - d3/d4/d11/d29/d36/f3a zero size 2024-02-16T19:24:07.907 INFO:tasks.workunit.client.0.smithi040.stdout:9/339: creat d3/d4/d11/f67 x:0 0 0 2024-02-16T19:24:07.987 INFO:tasks.workunit.client.0.smithi040.stdout:6/293: dwrite d0/d37/d4a/f5e [0,4194304] 0 2024-02-16T19:24:07.987 INFO:tasks.workunit.client.0.smithi040.stdout:1/334: rename d17/d3c/c41 to d17/d29/c6d 0 2024-02-16T19:24:07.987 INFO:tasks.workunit.client.0.smithi040.stdout:2/237: creat d5/d7/dd/f45 x:0 0 0 2024-02-16T19:24:07.987 INFO:tasks.workunit.client.0.smithi040.stdout:8/328: dwrite d6/f3c [0,4194304] 0 2024-02-16T19:24:07.988 INFO:tasks.workunit.client.0.smithi040.stdout:8/329: creat d6/d7/d1d/f60 x:0 0 0 2024-02-16T19:24:07.992 INFO:tasks.workunit.client.0.smithi040.stdout:6/294: rename d0/l13 to d0/d1/d12/d3b/d48/d4f/l5f 0 2024-02-16T19:24:07.992 INFO:tasks.workunit.client.0.smithi040.stdout:7/294: dwrite d21/d4d/f4b [0,4194304] 0 2024-02-16T19:24:07.993 INFO:tasks.workunit.client.0.smithi040.stdout:7/295: stat d21/d29/l39 0 2024-02-16T19:24:07.993 INFO:tasks.workunit.client.0.smithi040.stdout:2/238: mkdir d5/d46 0 2024-02-16T19:24:07.997 INFO:tasks.workunit.client.0.smithi040.stdout:8/330: creat d6/d7/d15/f61 x:0 0 0 2024-02-16T19:24:07.997 INFO:tasks.workunit.client.0.smithi040.stdout:8/331: fdatasync d6/f8 0 2024-02-16T19:24:07.998 INFO:tasks.workunit.client.0.smithi040.stdout:6/295: symlink d0/d1/d12/d3a/l60 0 2024-02-16T19:24:07.999 INFO:tasks.workunit.client.0.smithi040.stdout:2/239: creat d5/d7/f47 x:0 0 0 2024-02-16T19:24:07.999 INFO:tasks.workunit.client.0.smithi040.stdout:2/240: stat d5/d7/d10/l11 0 2024-02-16T19:24:07.999 INFO:tasks.workunit.client.0.smithi040.stdout:2/241: write d5/d7/d12/d1b/f2f [1945431,129812] 0 2024-02-16T19:24:08.000 INFO:tasks.workunit.client.0.smithi040.stdout:8/332: symlink d6/l62 0 2024-02-16T19:24:08.001 INFO:tasks.workunit.client.0.smithi040.stdout:6/296: unlink d0/d1/d12/d3a/f49 0 2024-02-16T19:24:08.002 INFO:tasks.workunit.client.0.smithi040.stdout:2/242: mknod d5/d7/c48 0 2024-02-16T19:24:08.004 INFO:tasks.workunit.client.0.smithi040.stdout:8/333: truncate d6/f3f 3899300 0 2024-02-16T19:24:08.004 INFO:tasks.workunit.client.0.smithi040.stdout:4/285: dwrite dd/de/d45/d37/f3b [0,4194304] 0 2024-02-16T19:24:08.004 INFO:tasks.workunit.client.0.smithi040.stdout:4/286: fdatasync dd/de/d10/d13/f38 0 2024-02-16T19:24:08.006 INFO:tasks.workunit.client.0.smithi040.stdout:5/274: dwrite d1/da/d1c/d21/f2f [4194304,4194304] 0 2024-02-16T19:24:08.006 INFO:tasks.workunit.client.0.smithi040.stdout:7/296: dread d21/d29/f45 [0,4194304] 0 2024-02-16T19:24:08.006 INFO:tasks.workunit.client.0.smithi040.stdout:5/275: readlink d1/da/l33 0 2024-02-16T19:24:08.006 INFO:tasks.workunit.client.0.smithi040.stdout:6/297: mkdir d0/d1/d12/d3b/d48/d61 0 2024-02-16T19:24:08.007 INFO:tasks.workunit.client.0.smithi040.stdout:5/276: write d1/f24 [44453,93831] 0 2024-02-16T19:24:08.007 INFO:tasks.workunit.client.0.smithi040.stdout:2/243: mkdir d5/d8/d49 0 2024-02-16T19:24:08.007 INFO:tasks.workunit.client.0.smithi040.stdout:2/244: write d5/d7/d12/f2b [853821,50260] 0 2024-02-16T19:24:08.008 INFO:tasks.workunit.client.0.smithi040.stdout:2/245: chown d5/d7/d1e/f37 32561294 1 2024-02-16T19:24:08.009 INFO:tasks.workunit.client.0.smithi040.stdout:4/287: creat dd/de/d45/d37/f59 x:0 0 0 2024-02-16T19:24:08.009 INFO:tasks.workunit.client.0.smithi040.stdout:4/288: stat dd/de 0 2024-02-16T19:24:08.009 INFO:tasks.workunit.client.0.smithi040.stdout:4/289: stat dd/de/d10/d13/d32/c3f 0 2024-02-16T19:24:08.015 INFO:tasks.workunit.client.0.smithi040.stdout:6/298: truncate d0/d1/d12/d3a/f3d 6809594 0 2024-02-16T19:24:08.015 INFO:tasks.workunit.client.0.smithi040.stdout:6/299: write d0/d1/d12/f2c [4579685,52189] 0 2024-02-16T19:24:08.016 INFO:tasks.workunit.client.0.smithi040.stdout:5/277: symlink d1/da/d1c/d21/l50 0 2024-02-16T19:24:08.018 INFO:tasks.workunit.client.0.smithi040.stdout:2/246: rmdir d5/d8/d49 0 2024-02-16T19:24:08.018 INFO:tasks.workunit.client.0.smithi040.stdout:2/247: write d5/d7/d1e/d31/f34 [1211207,77538] 0 2024-02-16T19:24:08.019 INFO:tasks.workunit.client.0.smithi040.stdout:4/290: rename dd/de/d45/l4e to dd/de/l5a 0 2024-02-16T19:24:08.021 INFO:tasks.workunit.client.0.smithi040.stdout:6/300: symlink d0/d37/l62 0 2024-02-16T19:24:08.023 INFO:tasks.workunit.client.0.smithi040.stdout:5/278: mkdir d1/da/d51 0 2024-02-16T19:24:08.025 INFO:tasks.workunit.client.0.smithi040.stdout:6/301: link d0/d37/l62 d0/d1/d12/d3b/d48/d4f/l63 0 2024-02-16T19:24:08.025 INFO:tasks.workunit.client.0.smithi040.stdout:6/302: chown d0/d30/l46 125682190 1 2024-02-16T19:24:08.027 INFO:tasks.workunit.client.0.smithi040.stdout:4/291: dread dd/de/d10/d13/f46 [0,4194304] 0 2024-02-16T19:24:08.030 INFO:tasks.workunit.client.0.smithi040.stdout:4/292: rename dd/de/d45/f4c to dd/de/d10/d36/f5b 0 2024-02-16T19:24:08.031 INFO:tasks.workunit.client.0.smithi040.stdout:5/279: dread d1/da/d1c/d21/f42 [0,4194304] 0 2024-02-16T19:24:08.035 INFO:tasks.workunit.client.0.smithi040.stdout:4/293: creat dd/de/d45/f5c x:0 0 0 2024-02-16T19:24:08.035 INFO:tasks.workunit.client.0.smithi040.stdout:4/294: creat dd/de/f5d x:0 0 0 2024-02-16T19:24:08.036 INFO:tasks.workunit.client.0.smithi040.stdout:5/280: write d1/da/d1c/f26 [1345905,69667] 0 2024-02-16T19:24:08.038 INFO:tasks.workunit.client.0.smithi040.stdout:4/295: mkdir dd/d58/d5e 0 2024-02-16T19:24:08.038 INFO:tasks.workunit.client.0.smithi040.stdout:3/399: sync 2024-02-16T19:24:08.038 INFO:tasks.workunit.client.0.smithi040.stdout:9/340: sync 2024-02-16T19:24:08.038 INFO:tasks.workunit.client.0.smithi040.stdout:9/341: write d3/d4/d11/d32/f5c [106807,38602] 0 2024-02-16T19:24:08.038 INFO:tasks.workunit.client.0.smithi040.stdout:3/400: write d1/dc/d42/d58/f72 [975412,32298] 0 2024-02-16T19:24:08.038 INFO:tasks.workunit.client.0.smithi040.stdout:0/370: sync 2024-02-16T19:24:08.038 INFO:tasks.workunit.client.0.smithi040.stdout:0/371: readlink db/d2f/d23/d3b/l60 0 2024-02-16T19:24:08.041 INFO:tasks.workunit.client.0.smithi040.stdout:4/296: unlink dd/de/d45/l3e 0 2024-02-16T19:24:08.042 INFO:tasks.workunit.client.0.smithi040.stdout:6/303: write d0/d30/d32/f2d [425692,126563] 0 2024-02-16T19:24:08.042 INFO:tasks.workunit.client.0.smithi040.stdout:6/304: chown d0/f3f 13753044 1 2024-02-16T19:24:08.042 INFO:tasks.workunit.client.0.smithi040.stdout:9/342: symlink d3/d4/d11/d29/l68 0 2024-02-16T19:24:08.044 INFO:tasks.workunit.client.0.smithi040.stdout:6/305: unlink d0/d1/d12/f58 0 2024-02-16T19:24:08.045 INFO:tasks.workunit.client.0.smithi040.stdout:9/343: mknod d3/d4/d11/d29/d36/c69 0 2024-02-16T19:24:08.045 INFO:tasks.workunit.client.0.smithi040.stdout:9/344: creat d3/d4/d11/d29/d38/d46/f6a x:0 0 0 2024-02-16T19:24:08.047 INFO:tasks.workunit.client.0.smithi040.stdout:2/248: dwrite d5/d7/d1e/d31/f34 [0,4194304] 0 2024-02-16T19:24:08.047 INFO:tasks.workunit.client.0.smithi040.stdout:9/345: mkdir d3/d4/d11/d29/d38/d46/d6b 0 2024-02-16T19:24:08.048 INFO:tasks.workunit.client.0.smithi040.stdout:9/346: write d3/d4/d11/d29/d38/f54 [762030,108446] 0 2024-02-16T19:24:08.048 INFO:tasks.workunit.client.0.smithi040.stdout:9/347: stat d3/d1c/f64 0 2024-02-16T19:24:08.050 INFO:tasks.workunit.client.0.smithi040.stdout:0/372: dread db/de/f38 [0,4194304] 0 2024-02-16T19:24:08.052 INFO:tasks.workunit.client.0.smithi040.stdout:9/348: rename d3/d4/d55/l1e to d3/d4/d11/l6c 0 2024-02-16T19:24:08.053 INFO:tasks.workunit.client.0.smithi040.stdout:5/281: dwrite d1/da/f35 [0,4194304] 0 2024-02-16T19:24:08.056 INFO:tasks.workunit.client.0.smithi040.stdout:3/401: dread d1/d4/fa [0,4194304] 0 2024-02-16T19:24:08.067 INFO:tasks.workunit.client.0.smithi040.stdout:3/402: dread d1/f34 [0,4194304] 0 2024-02-16T19:24:08.068 INFO:tasks.workunit.client.0.smithi040.stdout:3/403: mknod d1/d7/c75 0 2024-02-16T19:24:08.068 INFO:tasks.workunit.client.0.smithi040.stdout:3/404: creat d1/dc/d42/d58/f76 x:0 0 0 2024-02-16T19:24:08.068 INFO:tasks.workunit.client.0.smithi040.stdout:3/405: readlink d1/d7/d33/l22 0 2024-02-16T19:24:08.068 INFO:tasks.workunit.client.0.smithi040.stdout:3/406: fsync d1/dc/d35/d55/f74 0 2024-02-16T19:24:08.069 INFO:tasks.workunit.client.0.smithi040.stdout:3/407: read - d1/dc/d42/d5c/f6d zero size 2024-02-16T19:24:08.069 INFO:tasks.workunit.client.0.smithi040.stdout:3/408: chown d1/d4/d6e 6637 1 2024-02-16T19:24:08.069 INFO:tasks.workunit.client.0.smithi040.stdout:5/282: dread d1/da/f29 [0,4194304] 0 2024-02-16T19:24:08.069 INFO:tasks.workunit.client.0.smithi040.stdout:5/283: fdatasync d1/da/f30 0 2024-02-16T19:24:08.069 INFO:tasks.workunit.client.0.smithi040.stdout:5/284: dread - d1/da/f2e zero size 2024-02-16T19:24:08.071 INFO:tasks.workunit.client.0.smithi040.stdout:3/409: rename d1/d7/c6f to d1/dc/d35/d48/c77 0 2024-02-16T19:24:08.071 INFO:tasks.workunit.client.0.smithi040.stdout:3/410: write d1/dc/d42/d4a/f61 [272385,73849] 0 2024-02-16T19:24:08.071 INFO:tasks.workunit.client.0.smithi040.stdout:3/411: write d1/d7/d30/f63 [281666,45855] 0 2024-02-16T19:24:08.073 INFO:tasks.workunit.client.0.smithi040.stdout:5/285: link d1/d45/f49 d1/d45/f52 0 2024-02-16T19:24:08.076 INFO:tasks.workunit.client.0.smithi040.stdout:3/412: link d1/dc/d42/d5c/f6d d1/dc/d35/d48/f78 0 2024-02-16T19:24:08.077 INFO:tasks.workunit.client.0.smithi040.stdout:5/286: symlink d1/da/d1c/l53 0 2024-02-16T19:24:08.079 INFO:tasks.workunit.client.0.smithi040.stdout:3/413: mkdir d1/dc/d42/d4a/d57/d79 0 2024-02-16T19:24:08.080 INFO:tasks.workunit.client.0.smithi040.stdout:5/287: stat d1/l1b 0 2024-02-16T19:24:08.081 INFO:tasks.workunit.client.0.smithi040.stdout:5/288: read d1/da/d1c/f40 [28015,120636] 0 2024-02-16T19:24:08.082 INFO:tasks.workunit.client.0.smithi040.stdout:6/306: dwrite d0/d37/d4a/f5e [0,4194304] 0 2024-02-16T19:24:08.082 INFO:tasks.workunit.client.0.smithi040.stdout:9/349: dwrite d3/d4/d55/d31/f35 [0,4194304] 0 2024-02-16T19:24:08.082 INFO:tasks.workunit.client.0.smithi040.stdout:2/249: dwrite d5/f3c [0,4194304] 0 2024-02-16T19:24:08.082 INFO:tasks.workunit.client.0.smithi040.stdout:5/289: read d1/f24 [755188,13530] 0 2024-02-16T19:24:08.083 INFO:tasks.workunit.client.0.smithi040.stdout:5/290: chown d1/f4c 1153 1 2024-02-16T19:24:08.083 INFO:tasks.workunit.client.0.smithi040.stdout:5/291: chown d1/da/f29 28257 1 2024-02-16T19:24:08.085 INFO:tasks.workunit.client.0.smithi040.stdout:9/350: mkdir d3/d4/d55/d6d 0 2024-02-16T19:24:08.085 INFO:tasks.workunit.client.0.smithi040.stdout:9/351: chown d3/d4/f8 219 1 2024-02-16T19:24:08.085 INFO:tasks.workunit.client.0.smithi040.stdout:9/352: stat d3/l37 0 2024-02-16T19:24:08.085 INFO:tasks.workunit.client.0.smithi040.stdout:9/353: stat d3/d4/f8 0 2024-02-16T19:24:08.090 INFO:tasks.workunit.client.0.smithi040.stdout:2/250: rename d5/d7/d1e/f37 to d5/d7/f4a 0 2024-02-16T19:24:08.090 INFO:tasks.workunit.client.0.smithi040.stdout:2/251: truncate d5/f3a 517799 0 2024-02-16T19:24:08.090 INFO:tasks.workunit.client.0.smithi040.stdout:3/414: dread d1/dc/d42/d4a/f50 [0,4194304] 0 2024-02-16T19:24:08.090 INFO:tasks.workunit.client.0.smithi040.stdout:5/292: symlink d1/da/d1c/d21/l54 0 2024-02-16T19:24:08.090 INFO:tasks.workunit.client.0.smithi040.stdout:5/293: stat d1/da 0 2024-02-16T19:24:08.093 INFO:tasks.workunit.client.0.smithi040.stdout:2/252: symlink d5/d7/d1e/d31/l4b 0 2024-02-16T19:24:08.094 INFO:tasks.workunit.client.0.smithi040.stdout:3/415: mknod d1/dc/c7a 0 2024-02-16T19:24:08.096 INFO:tasks.workunit.client.0.smithi040.stdout:5/294: unlink d1/f20 0 2024-02-16T19:24:08.097 INFO:tasks.workunit.client.0.smithi040.stdout:9/354: getdents d3/d4/d11/d29/d38 0 2024-02-16T19:24:08.097 INFO:tasks.workunit.client.0.smithi040.stdout:9/355: chown d3/d4/d4a/f62 54 1 2024-02-16T19:24:08.097 INFO:tasks.workunit.client.0.smithi040.stdout:9/356: creat d3/d4/d11/d29/d38/d46/f6e x:0 0 0 2024-02-16T19:24:08.099 INFO:tasks.workunit.client.0.smithi040.stdout:2/253: mkdir d5/d7/d1e/d4c 0 2024-02-16T19:24:08.099 INFO:tasks.workunit.client.0.smithi040.stdout:1/335: sync 2024-02-16T19:24:08.099 INFO:tasks.workunit.client.0.smithi040.stdout:1/336: readlink d17/l69 0 2024-02-16T19:24:08.100 INFO:tasks.workunit.client.0.smithi040.stdout:6/307: dwrite d0/d30/d4b/f57 [0,4194304] 0 2024-02-16T19:24:08.101 INFO:tasks.workunit.client.0.smithi040.stdout:3/416: link d1/d7/d36/f45 d1/d7/d36/f7b 0 2024-02-16T19:24:08.106 INFO:tasks.workunit.client.0.smithi040.stdout:5/295: dread d1/da/d1c/f1f [0,4194304] 0 2024-02-16T19:24:08.106 INFO:tasks.workunit.client.0.smithi040.stdout:5/296: chown d1/d3c 390730 1 2024-02-16T19:24:08.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/357: rename d3/d4/f4d to d3/d4/d11/d29/d38/d46/d6b/f6f 0 2024-02-16T19:24:08.108 INFO:tasks.workunit.client.0.smithi040.stdout:9/358: fsync d3/d4/d11/d32/f5c 0 2024-02-16T19:24:08.120 INFO:tasks.workunit.client.0.smithi040.stdout:5/297: getdents d1/d45 0 2024-02-16T19:24:08.120 INFO:tasks.workunit.client.0.smithi040.stdout:5/298: chown d1/da 51 1 2024-02-16T19:24:08.122 INFO:tasks.workunit.client.0.smithi040.stdout:2/254: rename d5/d7/d12/f2b to d5/d7/d10/f4d 0 2024-02-16T19:24:08.123 INFO:tasks.workunit.client.0.smithi040.stdout:1/337: rmdir d17/d26/d54 39 2024-02-16T19:24:08.124 INFO:tasks.workunit.client.0.smithi040.stdout:2/255: write d5/d7/d12/f13 [270247,106583] 0 2024-02-16T19:24:08.124 INFO:tasks.workunit.client.0.smithi040.stdout:2/256: truncate d5/d7/d1e/d31/f43 83513 0 2024-02-16T19:24:08.127 INFO:tasks.workunit.client.0.smithi040.stdout:2/257: symlink d5/d46/l4e 0 2024-02-16T19:24:08.130 INFO:tasks.workunit.client.0.smithi040.stdout:6/308: rename d0/d1/d12/l2f to d0/d1/d12/d3a/l64 0 2024-02-16T19:24:08.130 INFO:tasks.workunit.client.0.smithi040.stdout:6/309: truncate d0/d1/d12/d3b/f50 586026 0 2024-02-16T19:24:08.131 INFO:tasks.workunit.client.0.smithi040.stdout:6/310: getdents d0/d30/d5b 0 2024-02-16T19:24:08.131 INFO:tasks.workunit.client.0.smithi040.stdout:6/311: mknod d0/c65 0 2024-02-16T19:24:08.139 INFO:tasks.workunit.client.0.smithi040.stdout:1/338: dread d17/f2c [0,4194304] 0 2024-02-16T19:24:08.140 INFO:tasks.workunit.client.0.smithi040.stdout:1/339: creat d17/d43/d38/d4a/f6e x:0 0 0 2024-02-16T19:24:08.142 INFO:tasks.workunit.client.0.smithi040.stdout:3/417: dwrite d1/d4/f43 [0,4194304] 0 2024-02-16T19:24:08.142 INFO:tasks.workunit.client.0.smithi040.stdout:3/418: creat d1/d7/d36/f7c x:0 0 0 2024-02-16T19:24:08.142 INFO:tasks.workunit.client.0.smithi040.stdout:3/419: creat d1/dc/d42/d4a/f7d x:0 0 0 2024-02-16T19:24:08.142 INFO:tasks.workunit.client.0.smithi040.stdout:3/420: dread - d1/dc/d35/d55/f74 zero size 2024-02-16T19:24:08.142 INFO:tasks.workunit.client.0.smithi040.stdout:5/299: dwrite d1/d45/f52 [0,4194304] 0 2024-02-16T19:24:08.142 INFO:tasks.workunit.client.0.smithi040.stdout:9/359: dwrite d3/d4/d11/f60 [0,4194304] 0 2024-02-16T19:24:08.142 INFO:tasks.workunit.client.0.smithi040.stdout:2/258: fdatasync d5/d7/d12/f13 0 2024-02-16T19:24:08.145 INFO:tasks.workunit.client.0.smithi040.stdout:2/259: dread d5/d7/d12/f13 [0,4194304] 0 2024-02-16T19:24:08.146 INFO:tasks.workunit.client.0.smithi040.stdout:2/260: truncate d5/d7/f47 295416 0 2024-02-16T19:24:08.147 INFO:tasks.workunit.client.0.smithi040.stdout:2/261: dread d5/d7/d1e/d31/f43 [0,4194304] 0 2024-02-16T19:24:08.149 INFO:tasks.workunit.client.0.smithi040.stdout:9/360: mkdir d3/d4/d55/d70 0 2024-02-16T19:24:08.150 INFO:tasks.workunit.client.0.smithi040.stdout:9/361: mkdir d3/d4/d11/d71 0 2024-02-16T19:24:08.162 INFO:tasks.workunit.client.0.smithi040.stdout:8/334: sync 2024-02-16T19:24:08.162 INFO:tasks.workunit.client.0.smithi040.stdout:0/373: sync 2024-02-16T19:24:08.162 INFO:tasks.workunit.client.0.smithi040.stdout:7/297: sync 2024-02-16T19:24:08.162 INFO:tasks.workunit.client.0.smithi040.stdout:0/374: chown db/d2f/f77 77 1 2024-02-16T19:24:08.162 INFO:tasks.workunit.client.0.smithi040.stdout:0/375: creat db/d2f/d1d/f80 x:0 0 0 2024-02-16T19:24:08.163 INFO:tasks.workunit.client.0.smithi040.stdout:1/340: dwrite d17/d2d/f2f [4194304,4194304] 0 2024-02-16T19:24:08.163 INFO:tasks.workunit.client.0.smithi040.stdout:1/341: chown d17/d2d/f2f 77224441 1 2024-02-16T19:24:08.163 INFO:tasks.workunit.client.0.smithi040.stdout:4/297: sync 2024-02-16T19:24:08.164 INFO:tasks.workunit.client.0.smithi040.stdout:4/298: stat dd/de/d10/d13/d32/f41 0 2024-02-16T19:24:08.164 INFO:tasks.workunit.client.0.smithi040.stdout:4/299: fdatasync dd/de/f5d 0 2024-02-16T19:24:08.164 INFO:tasks.workunit.client.0.smithi040.stdout:8/335: link d6/d7/d15/f61 d6/d2b/f63 0 2024-02-16T19:24:08.164 INFO:tasks.workunit.client.0.smithi040.stdout:8/336: chown d6/f9 6704 1 2024-02-16T19:24:08.164 INFO:tasks.workunit.client.0.smithi040.stdout:8/337: dread - d6/d7/d15/f61 zero size 2024-02-16T19:24:08.168 INFO:tasks.workunit.client.0.smithi040.stdout:1/342: rmdir d17/d26/d54/d68 0 2024-02-16T19:24:08.169 INFO:tasks.workunit.client.0.smithi040.stdout:4/300: dread dd/de/d10/f20 [0,4194304] 0 2024-02-16T19:24:08.169 INFO:tasks.workunit.client.0.smithi040.stdout:4/301: fdatasync fb 0 2024-02-16T19:24:08.170 INFO:tasks.workunit.client.0.smithi040.stdout:1/343: mknod d17/d26/d54/c6f 0 2024-02-16T19:24:08.170 INFO:tasks.workunit.client.0.smithi040.stdout:1/344: chown d17/d26/f55 87 1 2024-02-16T19:24:08.170 INFO:tasks.workunit.client.0.smithi040.stdout:8/338: read d6/d7/f27 [4608510,51027] 0 2024-02-16T19:24:08.170 INFO:tasks.workunit.client.0.smithi040.stdout:8/339: write d6/d7/d1d/f52 [166917,113912] 0 2024-02-16T19:24:08.170 INFO:tasks.workunit.client.0.smithi040.stdout:8/340: write d6/d12/d22/f4e [500327,6324] 0 2024-02-16T19:24:08.171 INFO:tasks.workunit.client.0.smithi040.stdout:4/302: symlink dd/d58/l5f 0 2024-02-16T19:24:08.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/303: rename dd/de/d10/d13/d32/l57 to dd/de/d10/d13/d32/l60 0 2024-02-16T19:24:08.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/304: stat dd/d58 0 2024-02-16T19:24:08.174 INFO:tasks.workunit.client.0.smithi040.stdout:1/345: write d17/d2d/f2f [3299527,108531] 0 2024-02-16T19:24:08.174 INFO:tasks.workunit.client.0.smithi040.stdout:4/305: creat dd/d58/d5e/f61 x:0 0 0 2024-02-16T19:24:08.175 INFO:tasks.workunit.client.0.smithi040.stdout:7/298: dread d21/d29/f45 [0,4194304] 0 2024-02-16T19:24:08.175 INFO:tasks.workunit.client.0.smithi040.stdout:7/299: creat d21/d29/f5c x:0 0 0 2024-02-16T19:24:08.175 INFO:tasks.workunit.client.0.smithi040.stdout:8/341: rename d6/d12/d41 to d6/d64 0 2024-02-16T19:24:08.176 INFO:tasks.workunit.client.0.smithi040.stdout:8/342: chown d6/d7/f27 821413212 1 2024-02-16T19:24:08.178 INFO:tasks.workunit.client.0.smithi040.stdout:8/343: symlink d6/d12/d51/l65 0 2024-02-16T19:24:08.180 INFO:tasks.workunit.client.0.smithi040.stdout:7/300: rename d21/d29/f4f to d21/f5d 0 2024-02-16T19:24:08.181 INFO:tasks.workunit.client.0.smithi040.stdout:8/344: mknod d6/d2b/d36/c66 0 2024-02-16T19:24:08.184 INFO:tasks.workunit.client.0.smithi040.stdout:7/301: unlink d21/l3e 0 2024-02-16T19:24:08.187 INFO:tasks.workunit.client.0.smithi040.stdout:7/302: link l13 d21/l5e 0 2024-02-16T19:24:08.187 INFO:tasks.workunit.client.0.smithi040.stdout:8/345: write d6/f33 [2629063,56070] 0 2024-02-16T19:24:08.197 INFO:tasks.workunit.client.0.smithi040.stdout:2/262: dwrite d5/d7/f47 [0,4194304] 0 2024-02-16T19:24:08.201 INFO:tasks.workunit.client.0.smithi040.stdout:2/263: mkdir d5/d7/d1e/d31/d4f 0 2024-02-16T19:24:08.201 INFO:tasks.workunit.client.0.smithi040.stdout:2/264: chown d5/d7/d12/f13 21098526 1 2024-02-16T19:24:08.201 INFO:tasks.workunit.client.0.smithi040.stdout:2/265: write d5/d7/f24 [1566857,50642] 0 2024-02-16T19:24:08.202 INFO:tasks.workunit.client.0.smithi040.stdout:9/362: dwrite d3/d4/d55/d31/f3b [0,4194304] 0 2024-02-16T19:24:08.204 INFO:tasks.workunit.client.0.smithi040.stdout:9/363: creat d3/d4/d55/d31/f72 x:0 0 0 2024-02-16T19:24:08.209 INFO:tasks.workunit.client.0.smithi040.stdout:5/300: dwrite d1/da/d1c/f37 [0,4194304] 0 2024-02-16T19:24:08.209 INFO:tasks.workunit.client.0.smithi040.stdout:5/301: creat d1/f55 x:0 0 0 2024-02-16T19:24:08.209 INFO:tasks.workunit.client.0.smithi040.stdout:5/302: fdatasync d1/da/f29 0 2024-02-16T19:24:08.209 INFO:tasks.workunit.client.0.smithi040.stdout:5/303: truncate d1/da/f30 398711 0 2024-02-16T19:24:08.209 INFO:tasks.workunit.client.0.smithi040.stdout:0/376: dwrite db/d2f/d1d/d21/f45 [0,4194304] 0 2024-02-16T19:24:08.214 INFO:tasks.workunit.client.0.smithi040.stdout:2/266: dread d5/d7/d1e/d31/f34 [0,4194304] 0 2024-02-16T19:24:08.215 INFO:tasks.workunit.client.0.smithi040.stdout:5/304: rename d1/da/d1c/d21/c28 to d1/da/d1c/d21/d41/c56 0 2024-02-16T19:24:08.215 INFO:tasks.workunit.client.0.smithi040.stdout:5/305: stat d1/d45/c46 0 2024-02-16T19:24:08.215 INFO:tasks.workunit.client.0.smithi040.stdout:5/306: fdatasync d1/f17 0 2024-02-16T19:24:08.216 INFO:tasks.workunit.client.0.smithi040.stdout:0/377: mkdir db/d2f/d1d/d81 0 2024-02-16T19:24:08.218 INFO:tasks.workunit.client.0.smithi040.stdout:8/346: dwrite d6/d2b/d36/f4f [0,4194304] 0 2024-02-16T19:24:08.218 INFO:tasks.workunit.client.0.smithi040.stdout:8/347: readlink d6/d12/l2a 0 2024-02-16T19:24:08.218 INFO:tasks.workunit.client.0.smithi040.stdout:8/348: dread - d6/d2b/d36/f3e zero size 2024-02-16T19:24:08.218 INFO:tasks.workunit.client.0.smithi040.stdout:8/349: stat d6/d7/d15/f2f 0 2024-02-16T19:24:08.218 INFO:tasks.workunit.client.0.smithi040.stdout:8/350: readlink d6/d2b/d36/l58 0 2024-02-16T19:24:08.218 INFO:tasks.workunit.client.0.smithi040.stdout:9/364: read d3/d4/d11/d29/d38/f3e [2875537,103864] 0 2024-02-16T19:24:08.218 INFO:tasks.workunit.client.0.smithi040.stdout:7/303: dwrite d21/f26 [0,4194304] 0 2024-02-16T19:24:08.218 INFO:tasks.workunit.client.0.smithi040.stdout:1/346: dwrite d17/d26/d5f/d63/f66 [0,4194304] 0 2024-02-16T19:24:08.220 INFO:tasks.workunit.client.0.smithi040.stdout:2/267: creat d5/d8/d42/f50 x:0 0 0 2024-02-16T19:24:08.227 INFO:tasks.workunit.client.0.smithi040.stdout:8/351: creat d6/d7/d15/f67 x:0 0 0 2024-02-16T19:24:08.227 INFO:tasks.workunit.client.0.smithi040.stdout:8/352: fsync d6/f9 0 2024-02-16T19:24:08.227 INFO:tasks.workunit.client.0.smithi040.stdout:8/353: stat d6/d2b/f63 0 2024-02-16T19:24:08.227 INFO:tasks.workunit.client.0.smithi040.stdout:8/354: chown d6/d7/d15/f67 497320471 1 2024-02-16T19:24:08.227 INFO:tasks.workunit.client.0.smithi040.stdout:8/355: write d6/f24 [542778,82185] 0 2024-02-16T19:24:08.228 INFO:tasks.workunit.client.0.smithi040.stdout:8/356: creat d6/d12/d22/f68 x:0 0 0 2024-02-16T19:24:08.231 INFO:tasks.workunit.client.0.smithi040.stdout:9/365: mkdir d3/d1c/d73 0 2024-02-16T19:24:08.232 INFO:tasks.workunit.client.0.smithi040.stdout:0/378: dread db/de/f7d [0,4194304] 0 2024-02-16T19:24:08.232 INFO:tasks.workunit.client.0.smithi040.stdout:2/268: symlink d5/d7/d1e/d31/l51 0 2024-02-16T19:24:08.232 INFO:tasks.workunit.client.0.smithi040.stdout:2/269: write d5/d8/f38 [259858,12212] 0 2024-02-16T19:24:08.234 INFO:tasks.workunit.client.0.smithi040.stdout:8/357: mkdir d6/d64/d50/d69 0 2024-02-16T19:24:08.234 INFO:tasks.workunit.client.0.smithi040.stdout:8/358: write d6/d2b/f34 [468933,45346] 0 2024-02-16T19:24:08.235 INFO:tasks.workunit.client.0.smithi040.stdout:9/366: creat d3/d4/d11/d29/d36/f74 x:0 0 0 2024-02-16T19:24:08.235 INFO:tasks.workunit.client.0.smithi040.stdout:9/367: read - d3/d4/d11/d29/d38/d46/f5b zero size 2024-02-16T19:24:08.235 INFO:tasks.workunit.client.0.smithi040.stdout:9/368: fsync d3/d1c/f64 0 2024-02-16T19:24:08.236 INFO:tasks.workunit.client.0.smithi040.stdout:0/379: truncate db/de/f38 2067391 0 2024-02-16T19:24:08.236 INFO:tasks.workunit.client.0.smithi040.stdout:2/270: mkdir d5/d7/d1e/d4c/d52 0 2024-02-16T19:24:08.237 INFO:tasks.workunit.client.0.smithi040.stdout:8/359: getdents d6/d64 0 2024-02-16T19:24:08.237 INFO:tasks.workunit.client.0.smithi040.stdout:8/360: write d6/f46 [1382358,113154] 0 2024-02-16T19:24:08.238 INFO:tasks.workunit.client.0.smithi040.stdout:0/380: creat db/de/d6a/f82 x:0 0 0 2024-02-16T19:24:08.239 INFO:tasks.workunit.client.0.smithi040.stdout:0/381: dread - db/d2f/f70 zero size 2024-02-16T19:24:08.239 INFO:tasks.workunit.client.0.smithi040.stdout:0/382: chown db/de/d6a/f6e 151 1 2024-02-16T19:24:08.239 INFO:tasks.workunit.client.0.smithi040.stdout:2/271: unlink d5/d7/f4a 0 2024-02-16T19:24:08.241 INFO:tasks.workunit.client.0.smithi040.stdout:8/361: rename d6/d2b/d36/c3b to d6/d12/d51/c6a 0 2024-02-16T19:24:08.241 INFO:tasks.workunit.client.0.smithi040.stdout:0/383: mkdir db/d2f/d1d/d21/d2e/d83 0 2024-02-16T19:24:08.242 INFO:tasks.workunit.client.0.smithi040.stdout:8/362: creat d6/d7/d15/f6b x:0 0 0 2024-02-16T19:24:08.244 INFO:tasks.workunit.client.0.smithi040.stdout:5/307: dwrite d1/f2c [0,4194304] 0 2024-02-16T19:24:08.245 INFO:tasks.workunit.client.0.smithi040.stdout:1/347: dwrite f8 [0,4194304] 0 2024-02-16T19:24:08.248 INFO:tasks.workunit.client.0.smithi040.stdout:9/369: write d3/d4/d55/f39 [3206367,94934] 0 2024-02-16T19:24:08.253 INFO:tasks.workunit.client.0.smithi040.stdout:1/348: mknod d17/d26/d5f/d63/c70 0 2024-02-16T19:24:08.254 INFO:tasks.workunit.client.0.smithi040.stdout:0/384: write db/fd [4856483,106715] 0 2024-02-16T19:24:08.255 INFO:tasks.workunit.client.0.smithi040.stdout:9/370: unlink d3/d4/d11/d29/d38/f3e 0 2024-02-16T19:24:08.256 INFO:tasks.workunit.client.0.smithi040.stdout:1/349: mkdir d17/d3c/d71 0 2024-02-16T19:24:08.257 INFO:tasks.workunit.client.0.smithi040.stdout:5/308: read d1/da/d1c/f25 [1702817,104915] 0 2024-02-16T19:24:08.257 INFO:tasks.workunit.client.0.smithi040.stdout:0/385: mkdir db/d2f/d1d/d21/d2e/d49/d5b/d67/d84 0 2024-02-16T19:24:08.259 INFO:tasks.workunit.client.0.smithi040.stdout:9/371: rename d3/f1f to d3/d4/d11/d29/d36/f75 0 2024-02-16T19:24:08.259 INFO:tasks.workunit.client.0.smithi040.stdout:9/372: fdatasync d3/d4/d55/d31/f35 0 2024-02-16T19:24:08.259 INFO:tasks.workunit.client.0.smithi040.stdout:1/350: creat d17/d43/d38/f72 x:0 0 0 2024-02-16T19:24:08.259 INFO:tasks.workunit.client.0.smithi040.stdout:1/351: creat d17/d2d/f73 x:0 0 0 2024-02-16T19:24:08.260 INFO:tasks.workunit.client.0.smithi040.stdout:5/309: rename d1/da/d1c/d21/d41 to d1/da/d1c/d21/d57 0 2024-02-16T19:24:08.261 INFO:tasks.workunit.client.0.smithi040.stdout:0/386: mkdir db/d2f/d1d/d21/d2e/d85 0 2024-02-16T19:24:08.262 INFO:tasks.workunit.client.0.smithi040.stdout:1/352: mkdir d17/d3c/d74 0 2024-02-16T19:24:08.263 INFO:tasks.workunit.client.0.smithi040.stdout:5/310: creat d1/da/d51/f58 x:0 0 0 2024-02-16T19:24:08.264 INFO:tasks.workunit.client.0.smithi040.stdout:0/387: creat db/d2f/d1d/d21/d2e/d85/f86 x:0 0 0 2024-02-16T19:24:08.265 INFO:tasks.workunit.client.0.smithi040.stdout:9/373: getdents d3/d4/d55 0 2024-02-16T19:24:08.265 INFO:tasks.workunit.client.0.smithi040.stdout:9/374: write d3/d4/d11/d29/d36/f3a [695783,34740] 0 2024-02-16T19:24:08.265 INFO:tasks.workunit.client.0.smithi040.stdout:9/375: stat d3/cb 0 2024-02-16T19:24:08.265 INFO:tasks.workunit.client.0.smithi040.stdout:1/353: truncate d17/d43/f21 418675 0 2024-02-16T19:24:08.266 INFO:tasks.workunit.client.0.smithi040.stdout:5/311: mknod d1/d45/c59 0 2024-02-16T19:24:08.266 INFO:tasks.workunit.client.0.smithi040.stdout:5/312: stat d1/d3c/f43 0 2024-02-16T19:24:08.266 INFO:tasks.workunit.client.0.smithi040.stdout:8/363: dwrite d6/d12/d22/f68 [0,4194304] 0 2024-02-16T19:24:08.266 INFO:tasks.workunit.client.0.smithi040.stdout:2/272: dwrite d5/d7/d12/d1b/f2f [0,4194304] 0 2024-02-16T19:24:08.268 INFO:tasks.workunit.client.0.smithi040.stdout:0/388: truncate db/d25/f26 857982 0 2024-02-16T19:24:08.268 INFO:tasks.workunit.client.0.smithi040.stdout:0/389: truncate db/d25/f4f 2296887 0 2024-02-16T19:24:08.268 INFO:tasks.workunit.client.0.smithi040.stdout:9/376: truncate d3/d4/f2c 753483 0 2024-02-16T19:24:08.269 INFO:tasks.workunit.client.0.smithi040.stdout:1/354: truncate d17/f36 1301284 0 2024-02-16T19:24:08.269 INFO:tasks.workunit.client.0.smithi040.stdout:1/355: truncate d17/d26/f55 363288 0 2024-02-16T19:24:08.271 INFO:tasks.workunit.client.0.smithi040.stdout:5/313: mknod d1/da/d51/c5a 0 2024-02-16T19:24:08.371 INFO:tasks.workunit.client.0.smithi040.stdout:2/273: dread d5/d7/d1e/d31/f43 [0,4194304] 0 2024-02-16T19:24:08.371 INFO:tasks.workunit.client.0.smithi040.stdout:2/274: write d5/d7/dd/f45 [359230,8669] 0 2024-02-16T19:24:08.375 INFO:tasks.workunit.client.0.smithi040.stdout:6/312: sync 2024-02-16T19:24:08.375 INFO:tasks.workunit.client.0.smithi040.stdout:8/364: dread d6/f24 [0,4194304] 0 2024-02-16T19:24:08.377 INFO:tasks.workunit.client.0.smithi040.stdout:1/356: creat d17/d26/d5f/d63/f75 x:0 0 0 2024-02-16T19:24:08.378 INFO:tasks.workunit.client.0.smithi040.stdout:2/275: mknod d5/c53 0 2024-02-16T19:24:08.378 INFO:tasks.workunit.client.0.smithi040.stdout:2/276: chown d5/d7/d12/d1b/l35 9528 1 2024-02-16T19:24:08.378 INFO:tasks.workunit.client.0.smithi040.stdout:2/277: truncate d5/d7/d1e/f3b 658310 0 2024-02-16T19:24:08.378 INFO:tasks.workunit.client.0.smithi040.stdout:2/278: chown d5/d7/d10/f4d 58293 1 2024-02-16T19:24:08.379 INFO:tasks.workunit.client.0.smithi040.stdout:2/279: dread d5/d8/f38 [0,4194304] 0 2024-02-16T19:24:08.379 INFO:tasks.workunit.client.0.smithi040.stdout:6/313: creat d0/d1/d12/f66 x:0 0 0 2024-02-16T19:24:08.380 INFO:tasks.workunit.client.0.smithi040.stdout:8/365: mknod d6/d2b/d36/c6c 0 2024-02-16T19:24:08.385 INFO:tasks.workunit.client.0.smithi040.stdout:9/377: rmdir d3/d4/d11/d29 39 2024-02-16T19:24:08.390 INFO:tasks.workunit.client.0.smithi040.stdout:1/357: mknod d17/d43/d38/c76 0 2024-02-16T19:24:08.394 INFO:tasks.workunit.client.0.smithi040.stdout:8/366: link d6/f46 d6/d64/f6d 0 2024-02-16T19:24:08.394 INFO:tasks.workunit.client.0.smithi040.stdout:8/367: getdents d6/d2b/d45 0 2024-02-16T19:24:08.394 INFO:tasks.workunit.client.0.smithi040.stdout:8/368: fdatasync d6/d7/d15/f47 0 2024-02-16T19:24:08.395 INFO:tasks.workunit.client.0.smithi040.stdout:2/280: getdents d5/d7/dd 0 2024-02-16T19:24:08.397 INFO:tasks.workunit.client.0.smithi040.stdout:2/281: dread d5/d8/f38 [0,4194304] 0 2024-02-16T19:24:08.398 INFO:tasks.workunit.client.0.smithi040.stdout:2/282: dread d5/f3a [0,4194304] 0 2024-02-16T19:24:08.399 INFO:tasks.workunit.client.0.smithi040.stdout:9/378: rename d3/d4/ld to d3/d4/d55/d70/l76 0 2024-02-16T19:24:08.399 INFO:tasks.workunit.client.0.smithi040.stdout:9/379: readlink d3/d4/d11/d29/l2d 0 2024-02-16T19:24:08.399 INFO:tasks.workunit.client.0.smithi040.stdout:9/380: dread - d3/d4/d55/d31/f72 zero size 2024-02-16T19:24:08.402 INFO:tasks.workunit.client.0.smithi040.stdout:9/381: symlink d3/d4/d11/d29/l77 0 2024-02-16T19:24:08.402 INFO:tasks.workunit.client.0.smithi040.stdout:6/314: write d0/d1/d12/f47 [7549495,29459] 0 2024-02-16T19:24:08.406 INFO:tasks.workunit.client.0.smithi040.stdout:4/306: sync 2024-02-16T19:24:08.406 INFO:tasks.workunit.client.0.smithi040.stdout:3/421: sync 2024-02-16T19:24:08.406 INFO:tasks.workunit.client.0.smithi040.stdout:4/307: fsync dd/f52 0 2024-02-16T19:24:08.407 INFO:tasks.workunit.client.0.smithi040.stdout:0/390: read db/d25/f26 [562463,4914] 0 2024-02-16T19:24:08.407 INFO:tasks.workunit.client.0.smithi040.stdout:6/315: symlink d0/d30/d32/l67 0 2024-02-16T19:24:08.410 INFO:tasks.workunit.client.0.smithi040.stdout:2/283: write d5/d7/d12/f15 [450833,20242] 0 2024-02-16T19:24:08.410 INFO:tasks.workunit.client.0.smithi040.stdout:6/316: creat d0/d1/d12/d3b/d48/d4f/f68 x:0 0 0 2024-02-16T19:24:08.412 INFO:tasks.workunit.client.0.smithi040.stdout:0/391: rmdir db/de 39 2024-02-16T19:24:08.412 INFO:tasks.workunit.client.0.smithi040.stdout:0/392: creat db/d2f/d1d/d21/d2e/d49/f87 x:0 0 0 2024-02-16T19:24:08.415 INFO:tasks.workunit.client.0.smithi040.stdout:2/284: rename d5/d7/d12/f13 to d5/d7/f54 0 2024-02-16T19:24:08.417 INFO:tasks.workunit.client.0.smithi040.stdout:7/304: sync 2024-02-16T19:24:08.418 INFO:tasks.workunit.client.0.smithi040.stdout:7/305: read - d21/f5d zero size 2024-02-16T19:24:08.418 INFO:tasks.workunit.client.0.smithi040.stdout:1/358: dwrite f5 [0,4194304] 0 2024-02-16T19:24:08.422 INFO:tasks.workunit.client.0.smithi040.stdout:7/306: dread f7 [0,4194304] 0 2024-02-16T19:24:08.423 INFO:tasks.workunit.client.0.smithi040.stdout:7/307: readlink d21/l53 0 2024-02-16T19:24:08.425 INFO:tasks.workunit.client.0.smithi040.stdout:7/308: symlink d21/l5f 0 2024-02-16T19:24:08.425 INFO:tasks.workunit.client.0.smithi040.stdout:7/309: chown d21/d4d/f4b 40 1 2024-02-16T19:24:08.426 INFO:tasks.workunit.client.0.smithi040.stdout:7/310: symlink d21/d4d/l60 0 2024-02-16T19:24:08.430 INFO:tasks.workunit.client.0.smithi040.stdout:6/317: dread d0/d30/d32/f5c [0,4194304] 0 2024-02-16T19:24:08.430 INFO:tasks.workunit.client.0.smithi040.stdout:6/318: write d0/d30/d32/f2d [4395551,79639] 0 2024-02-16T19:24:08.430 INFO:tasks.workunit.client.0.smithi040.stdout:6/319: creat d0/d1/d12/d3b/d48/d4f/f69 x:0 0 0 2024-02-16T19:24:08.430 INFO:tasks.workunit.client.0.smithi040.stdout:6/320: chown d0/d1/d12/d3b/d48/d4f/f68 24 1 2024-02-16T19:24:08.431 INFO:tasks.workunit.client.0.smithi040.stdout:8/369: dwrite d6/d12/d22/f42 [0,4194304] 0 2024-02-16T19:24:08.431 INFO:tasks.workunit.client.0.smithi040.stdout:8/370: write f5 [4021460,117219] 0 2024-02-16T19:24:08.434 INFO:tasks.workunit.client.0.smithi040.stdout:1/359: read fb [7099276,65438] 0 2024-02-16T19:24:08.434 INFO:tasks.workunit.client.0.smithi040.stdout:1/360: stat d17/d43/f21 0 2024-02-16T19:24:08.434 INFO:tasks.workunit.client.0.smithi040.stdout:6/321: rename d0/d1/d12/d3b/d48/d61 to d0/d1/d6a 0 2024-02-16T19:24:08.438 INFO:tasks.workunit.client.0.smithi040.stdout:9/382: getdents d3/d4/d11/d29 0 2024-02-16T19:24:08.440 INFO:tasks.workunit.client.0.smithi040.stdout:6/322: link d0/d1/d12/d3b/d48/l2a d0/d30/d32/l6b 0 2024-02-16T19:24:08.440 INFO:tasks.workunit.client.0.smithi040.stdout:4/308: dwrite dd/de/d45/d37/f59 [0,4194304] 0 2024-02-16T19:24:08.442 INFO:tasks.workunit.client.0.smithi040.stdout:9/383: symlink d3/d4/d11/d71/l78 0 2024-02-16T19:24:08.442 INFO:tasks.workunit.client.0.smithi040.stdout:9/384: chown d3/d4/d11/d29/f2b 7345121 1 2024-02-16T19:24:08.443 INFO:tasks.workunit.client.0.smithi040.stdout:6/323: symlink d0/d30/d4b/l6c 0 2024-02-16T19:24:08.445 INFO:tasks.workunit.client.0.smithi040.stdout:4/309: unlink dd/f49 0 2024-02-16T19:24:08.446 INFO:tasks.workunit.client.0.smithi040.stdout:8/371: rmdir d6/d2b/d45 0 2024-02-16T19:24:08.446 INFO:tasks.workunit.client.0.smithi040.stdout:8/372: truncate d6/d12/f4b 62661 0 2024-02-16T19:24:08.447 INFO:tasks.workunit.client.0.smithi040.stdout:1/361: dread d17/f37 [0,4194304] 0 2024-02-16T19:24:08.447 INFO:tasks.workunit.client.0.smithi040.stdout:1/362: chown d17/d2d/f33 198 1 2024-02-16T19:24:08.447 INFO:tasks.workunit.client.0.smithi040.stdout:9/385: creat d3/d4/f79 x:0 0 0 2024-02-16T19:24:08.448 INFO:tasks.workunit.client.0.smithi040.stdout:6/324: rmdir d0/d1/d12/d3a 39 2024-02-16T19:24:08.448 INFO:tasks.workunit.client.0.smithi040.stdout:6/325: read - d0/d30/d32/f56 zero size 2024-02-16T19:24:08.448 INFO:tasks.workunit.client.0.smithi040.stdout:6/326: read - d0/f3f zero size 2024-02-16T19:24:08.448 INFO:tasks.workunit.client.0.smithi040.stdout:6/327: chown d0/d30/d4b/l6c 2 1 2024-02-16T19:24:08.449 INFO:tasks.workunit.client.0.smithi040.stdout:1/363: symlink d17/d26/d54/l77 0 2024-02-16T19:24:08.449 INFO:tasks.workunit.client.0.smithi040.stdout:1/364: truncate d17/d2d/f3f 618773 0 2024-02-16T19:24:08.449 INFO:tasks.workunit.client.0.smithi040.stdout:9/386: rmdir d3/d4/d55/d70 39 2024-02-16T19:24:08.449 INFO:tasks.workunit.client.0.smithi040.stdout:9/387: read d3/f14 [1497761,12686] 0 2024-02-16T19:24:08.449 INFO:tasks.workunit.client.0.smithi040.stdout:9/388: chown d3/d4/d55/f51 33178580 1 2024-02-16T19:24:08.450 INFO:tasks.workunit.client.0.smithi040.stdout:2/285: dwrite d5/d7/d1e/d31/f43 [0,4194304] 0 2024-02-16T19:24:08.452 INFO:tasks.workunit.client.0.smithi040.stdout:7/311: dwrite d21/d29/f2b [0,4194304] 0 2024-02-16T19:24:08.452 INFO:tasks.workunit.client.0.smithi040.stdout:7/312: chown d21/c47 28645459 1 2024-02-16T19:24:08.452 INFO:tasks.workunit.client.0.smithi040.stdout:3/422: dwrite d1/d7/d33/f6c [0,4194304] 0 2024-02-16T19:24:08.452 INFO:tasks.workunit.client.0.smithi040.stdout:3/423: chown d1/d7/d33/f5e 811886779 1 2024-02-16T19:24:08.452 INFO:tasks.workunit.client.0.smithi040.stdout:3/424: readlink d1/le 0 2024-02-16T19:24:08.452 INFO:tasks.workunit.client.0.smithi040.stdout:6/328: unlink d0/d30/d32/f5c 0 2024-02-16T19:24:08.452 INFO:tasks.workunit.client.0.smithi040.stdout:4/310: getdents dd/d58/d5e 0 2024-02-16T19:24:08.452 INFO:tasks.workunit.client.0.smithi040.stdout:4/311: write dd/f52 [630947,20359] 0 2024-02-16T19:24:08.454 INFO:tasks.workunit.client.0.smithi040.stdout:1/365: dread d17/d26/d4c/f5b [0,4194304] 0 2024-02-16T19:24:08.454 INFO:tasks.workunit.client.0.smithi040.stdout:9/389: creat d3/d4/d55/d70/f7a x:0 0 0 2024-02-16T19:24:08.454 INFO:tasks.workunit.client.0.smithi040.stdout:9/390: chown d3/d4/f2c 1802579 1 2024-02-16T19:24:08.456 INFO:tasks.workunit.client.0.smithi040.stdout:4/312: dread dd/f23 [0,4194304] 0 2024-02-16T19:24:08.456 INFO:tasks.workunit.client.0.smithi040.stdout:4/313: stat dd/de/d45/c39 0 2024-02-16T19:24:08.456 INFO:tasks.workunit.client.0.smithi040.stdout:4/314: fdatasync dd/de/d10/f4a 0 2024-02-16T19:24:08.460 INFO:tasks.workunit.client.0.smithi040.stdout:3/425: mknod d1/d7/d36/c7e 0 2024-02-16T19:24:08.460 INFO:tasks.workunit.client.0.smithi040.stdout:3/426: chown d1/d4/f4f 3403595 1 2024-02-16T19:24:08.460 INFO:tasks.workunit.client.0.smithi040.stdout:3/427: creat d1/d7/d30/f7f x:0 0 0 2024-02-16T19:24:08.460 INFO:tasks.workunit.client.0.smithi040.stdout:2/286: dread d5/d7/d1e/f33 [0,4194304] 0 2024-02-16T19:24:08.460 INFO:tasks.workunit.client.0.smithi040.stdout:3/428: write d1/d4/fa [1386245,23597] 0 2024-02-16T19:24:08.461 INFO:tasks.workunit.client.0.smithi040.stdout:6/329: truncate d0/d1/d12/f18 706223 0 2024-02-16T19:24:08.464 INFO:tasks.workunit.client.0.smithi040.stdout:7/313: dread d21/d4d/f4b [0,4194304] 0 2024-02-16T19:24:08.464 INFO:tasks.workunit.client.0.smithi040.stdout:7/314: write d21/d29/f37 [4084450,73414] 0 2024-02-16T19:24:08.469 INFO:tasks.workunit.client.0.smithi040.stdout:1/366: mkdir d17/d26/d5f/d78 0 2024-02-16T19:24:08.469 INFO:tasks.workunit.client.0.smithi040.stdout:6/330: dread d0/f35 [0,4194304] 0 2024-02-16T19:24:08.469 INFO:tasks.workunit.client.0.smithi040.stdout:8/373: dwrite d6/f46 [0,4194304] 0 2024-02-16T19:24:08.470 INFO:tasks.workunit.client.0.smithi040.stdout:9/391: mknod d3/d4/d55/c7b 0 2024-02-16T19:24:08.471 INFO:tasks.workunit.client.0.smithi040.stdout:4/315: creat dd/de/d10/f62 x:0 0 0 2024-02-16T19:24:08.471 INFO:tasks.workunit.client.0.smithi040.stdout:2/287: write d5/d7/d12/d1b/f2f [413295,4514] 0 2024-02-16T19:24:08.474 INFO:tasks.workunit.client.0.smithi040.stdout:7/315: rename d21/f25 to d21/d4d/f61 0 2024-02-16T19:24:08.474 INFO:tasks.workunit.client.0.smithi040.stdout:7/316: write d21/d4d/f5b [305493,7428] 0 2024-02-16T19:24:08.475 INFO:tasks.workunit.client.0.smithi040.stdout:1/367: creat d17/d43/d38/d4a/f79 x:0 0 0 2024-02-16T19:24:08.475 INFO:tasks.workunit.client.0.smithi040.stdout:1/368: creat d17/d26/f7a x:0 0 0 2024-02-16T19:24:08.477 INFO:tasks.workunit.client.0.smithi040.stdout:9/392: creat d3/d4/d55/d6d/f7c x:0 0 0 2024-02-16T19:24:08.478 INFO:tasks.workunit.client.0.smithi040.stdout:9/393: stat d3/d4/d55/d31/f5e 0 2024-02-16T19:24:08.480 INFO:tasks.workunit.client.0.smithi040.stdout:2/288: mkdir d5/d7/d1e/d55 0 2024-02-16T19:24:08.480 INFO:tasks.workunit.client.0.smithi040.stdout:2/289: write d5/d7/d12/f15 [2395698,91465] 0 2024-02-16T19:24:08.486 INFO:tasks.workunit.client.0.smithi040.stdout:6/331: write d0/d1/d12/d3b/d48/f38 [196967,22000] 0 2024-02-16T19:24:08.487 INFO:tasks.workunit.client.0.smithi040.stdout:6/332: write d0/d1/d12/f66 [50760,100670] 0 2024-02-16T19:24:08.488 INFO:tasks.workunit.client.0.smithi040.stdout:6/333: mkdir d0/d1/d12/d3b/d48/d4d/d6d 0 2024-02-16T19:24:08.488 INFO:tasks.workunit.client.0.smithi040.stdout:3/429: dwrite d1/d7/d36/f7b [0,4194304] 0 2024-02-16T19:24:08.489 INFO:tasks.workunit.client.0.smithi040.stdout:5/314: sync 2024-02-16T19:24:08.490 INFO:tasks.workunit.client.0.smithi040.stdout:6/334: unlink d0/d1/d12/d3b/d48/d4f/l5f 0 2024-02-16T19:24:08.491 INFO:tasks.workunit.client.0.smithi040.stdout:3/430: link d1/d4/c8 d1/dc/d35/d53/d64/c80 0 2024-02-16T19:24:08.493 INFO:tasks.workunit.client.0.smithi040.stdout:3/431: getdents d1/dc/d35/d48 0 2024-02-16T19:24:08.493 INFO:tasks.workunit.client.0.smithi040.stdout:3/432: chown d1/dc/d42/d58/f72 55212657 1 2024-02-16T19:24:08.493 INFO:tasks.workunit.client.0.smithi040.stdout:3/433: write d1/d4/f25 [4333569,127463] 0 2024-02-16T19:24:08.494 INFO:tasks.workunit.client.0.smithi040.stdout:5/315: dread d1/da/f2a [0,4194304] 0 2024-02-16T19:24:08.494 INFO:tasks.workunit.client.0.smithi040.stdout:5/316: fsync d1/f2c 0 2024-02-16T19:24:08.494 INFO:tasks.workunit.client.0.smithi040.stdout:5/317: chown d1/da/d1c/d21/c4b 328 1 2024-02-16T19:24:08.495 INFO:tasks.workunit.client.0.smithi040.stdout:3/434: unlink d1/d7/l5a 0 2024-02-16T19:24:08.495 INFO:tasks.workunit.client.0.smithi040.stdout:3/435: write d1/dc/d42/d58/f72 [1240062,119213] 0 2024-02-16T19:24:08.505 INFO:tasks.workunit.client.0.smithi040.stdout:6/335: read d0/d1/d12/d3a/f42 [2768018,121346] 0 2024-02-16T19:24:08.511 INFO:tasks.workunit.client.0.smithi040.stdout:2/290: fsync d5/d7/d12/f15 0 2024-02-16T19:24:08.513 INFO:tasks.workunit.client.0.smithi040.stdout:8/374: dwrite d6/d7/d15/f2f [0,4194304] 0 2024-02-16T19:24:08.514 INFO:tasks.workunit.client.0.smithi040.stdout:7/317: dwrite d21/d4d/f4b [0,4194304] 0 2024-02-16T19:24:08.514 INFO:tasks.workunit.client.0.smithi040.stdout:3/436: write d1/dc/f29 [2136342,126450] 0 2024-02-16T19:24:08.514 INFO:tasks.workunit.client.0.smithi040.stdout:3/437: creat d1/d7/d30/f81 x:0 0 0 2024-02-16T19:24:08.514 INFO:tasks.workunit.client.0.smithi040.stdout:3/438: stat d1/d4/f4f 0 2024-02-16T19:24:08.515 INFO:tasks.workunit.client.0.smithi040.stdout:1/369: dwrite d17/d26/d5f/d63/f66 [4194304,4194304] 0 2024-02-16T19:24:08.516 INFO:tasks.workunit.client.0.smithi040.stdout:2/291: rename d5/d7/c3f to d5/d8/c56 0 2024-02-16T19:24:08.516 INFO:tasks.workunit.client.0.smithi040.stdout:9/394: dwrite d3/d4/d55/f5a [0,4194304] 0 2024-02-16T19:24:08.516 INFO:tasks.workunit.client.0.smithi040.stdout:9/395: creat d3/d4/d55/d6d/f7d x:0 0 0 2024-02-16T19:24:08.517 INFO:tasks.workunit.client.0.smithi040.stdout:3/439: dread d1/dc/f3c [0,4194304] 0 2024-02-16T19:24:08.518 INFO:tasks.workunit.client.0.smithi040.stdout:4/316: rmdir dd/de 39 2024-02-16T19:24:08.520 INFO:tasks.workunit.client.0.smithi040.stdout:8/375: mkdir d6/d12/d51/d6e 0 2024-02-16T19:24:08.520 INFO:tasks.workunit.client.0.smithi040.stdout:8/376: write d6/d7/d15/f67 [357394,30050] 0 2024-02-16T19:24:08.523 INFO:tasks.workunit.client.0.smithi040.stdout:9/396: read d3/d4/d55/f5a [86721,91326] 0 2024-02-16T19:24:08.523 INFO:tasks.workunit.client.0.smithi040.stdout:9/397: chown d3/c17 4560458 1 2024-02-16T19:24:08.529 INFO:tasks.workunit.client.0.smithi040.stdout:6/336: dwrite d0/d1/d12/d3a/f3d [0,4194304] 0 2024-02-16T19:24:08.538 INFO:tasks.workunit.client.0.smithi040.stdout:3/440: mknod d1/dc/d42/c82 0 2024-02-16T19:24:08.541 INFO:tasks.workunit.client.0.smithi040.stdout:7/318: link d21/d29/f45 d21/f62 0 2024-02-16T19:24:08.541 INFO:tasks.workunit.client.0.smithi040.stdout:7/319: chown d21/d29/l5a 13 1 2024-02-16T19:24:08.541 INFO:tasks.workunit.client.0.smithi040.stdout:1/370: getdents d17/d43/d38 0 2024-02-16T19:24:08.541 INFO:tasks.workunit.client.0.smithi040.stdout:1/371: fsync d17/f1f 0 2024-02-16T19:24:08.543 INFO:tasks.workunit.client.0.smithi040.stdout:6/337: symlink d0/d1/d12/d3b/d48/d4d/l6e 0 2024-02-16T19:24:08.544 INFO:tasks.workunit.client.0.smithi040.stdout:8/377: truncate d6/d64/f6d 1710275 0 2024-02-16T19:24:08.545 INFO:tasks.workunit.client.0.smithi040.stdout:8/378: getdents d6/d7 0 2024-02-16T19:24:08.549 INFO:tasks.workunit.client.0.smithi040.stdout:8/379: mknod d6/d7/d15/c6f 0 2024-02-16T19:24:08.551 INFO:tasks.workunit.client.0.smithi040.stdout:2/292: dwrite d5/d7/d1e/f25 [4194304,4194304] 0 2024-02-16T19:24:08.552 INFO:tasks.workunit.client.0.smithi040.stdout:3/441: write d1/f34 [2261078,102473] 0 2024-02-16T19:24:08.552 INFO:tasks.workunit.client.0.smithi040.stdout:9/398: dwrite d3/d4/d11/d29/d38/d46/f5b [0,4194304] 0 2024-02-16T19:24:08.552 INFO:tasks.workunit.client.0.smithi040.stdout:3/442: read d1/d4/fa [629128,79976] 0 2024-02-16T19:24:08.552 INFO:tasks.workunit.client.0.smithi040.stdout:8/380: creat d6/d7/f70 x:0 0 0 2024-02-16T19:24:08.554 INFO:tasks.workunit.client.0.smithi040.stdout:2/293: truncate d5/d7/d1e/d31/f34 3322035 0 2024-02-16T19:24:08.556 INFO:tasks.workunit.client.0.smithi040.stdout:7/320: read d21/d29/f45 [2835623,61356] 0 2024-02-16T19:24:08.557 INFO:tasks.workunit.client.0.smithi040.stdout:9/399: mknod d3/d4/d55/d70/c7e 0 2024-02-16T19:24:08.557 INFO:tasks.workunit.client.0.smithi040.stdout:9/400: write d3/d4/d11/d29/f48 [643310,25348] 0 2024-02-16T19:24:08.558 INFO:tasks.workunit.client.0.smithi040.stdout:8/381: symlink d6/d7/d1d/l71 0 2024-02-16T19:24:08.558 INFO:tasks.workunit.client.0.smithi040.stdout:8/382: read - d6/d2b/f63 zero size 2024-02-16T19:24:08.559 INFO:tasks.workunit.client.0.smithi040.stdout:2/294: mknod d5/d7/d1e/d4c/d52/c57 0 2024-02-16T19:24:08.561 INFO:tasks.workunit.client.0.smithi040.stdout:6/338: write d0/d30/d32/f21 [1312089,51553] 0 2024-02-16T19:24:08.562 INFO:tasks.workunit.client.0.smithi040.stdout:7/321: mknod d21/d4d/c63 0 2024-02-16T19:24:08.562 INFO:tasks.workunit.client.0.smithi040.stdout:1/372: dwrite d17/d29/f32 [0,4194304] 0 2024-02-16T19:24:08.563 INFO:tasks.workunit.client.0.smithi040.stdout:1/373: write d17/d26/f7a [267052,122889] 0 2024-02-16T19:24:08.563 INFO:tasks.workunit.client.0.smithi040.stdout:1/374: readlink d17/d26/l40 0 2024-02-16T19:24:08.563 INFO:tasks.workunit.client.0.smithi040.stdout:1/375: fdatasync f14 0 2024-02-16T19:24:08.564 INFO:tasks.workunit.client.0.smithi040.stdout:9/401: symlink d3/d4/d11/d29/d38/d46/l7f 0 2024-02-16T19:24:08.566 INFO:tasks.workunit.client.0.smithi040.stdout:2/295: symlink d5/d7/d1e/d31/l58 0 2024-02-16T19:24:08.566 INFO:tasks.workunit.client.0.smithi040.stdout:0/393: sync 2024-02-16T19:24:08.566 INFO:tasks.workunit.client.0.smithi040.stdout:0/394: chown db/d2f/f70 21837 1 2024-02-16T19:24:08.567 INFO:tasks.workunit.client.0.smithi040.stdout:0/395: dread - db/d2f/d1d/d21/d2e/d85/f86 zero size 2024-02-16T19:24:08.567 INFO:tasks.workunit.client.0.smithi040.stdout:0/396: fsync db/d2f/d1d/f46 0 2024-02-16T19:24:08.570 INFO:tasks.workunit.client.0.smithi040.stdout:6/339: rmdir d0/d1/d6a 0 2024-02-16T19:24:08.570 INFO:tasks.workunit.client.0.smithi040.stdout:6/340: dread - d0/d1/d12/d3b/d48/f3c zero size 2024-02-16T19:24:08.578 INFO:tasks.workunit.client.0.smithi040.stdout:0/397: link db/d25/d34/l62 db/l88 0 2024-02-16T19:24:08.579 INFO:tasks.workunit.client.0.smithi040.stdout:6/341: getdents d0/d1/d12/d3a 0 2024-02-16T19:24:08.580 INFO:tasks.workunit.client.0.smithi040.stdout:7/322: link c6 d21/c64 0 2024-02-16T19:24:08.583 INFO:tasks.workunit.client.0.smithi040.stdout:0/398: mkdir db/d2f/d1d/d21/d2e/d85/d89 0 2024-02-16T19:24:08.583 INFO:tasks.workunit.client.0.smithi040.stdout:0/399: chown db/d2f/f66 2 1 2024-02-16T19:24:08.585 INFO:tasks.workunit.client.0.smithi040.stdout:6/342: mkdir d0/d1/d6f 0 2024-02-16T19:24:08.585 INFO:tasks.workunit.client.0.smithi040.stdout:6/343: chown d0/d1/d12/d3b/d48/d4d/d6d 894539 1 2024-02-16T19:24:08.585 INFO:tasks.workunit.client.0.smithi040.stdout:3/443: dwrite d1/d4/f25 [0,4194304] 0 2024-02-16T19:24:08.585 INFO:tasks.workunit.client.0.smithi040.stdout:8/383: dwrite d6/d2b/f34 [0,4194304] 0 2024-02-16T19:24:08.585 INFO:tasks.workunit.client.0.smithi040.stdout:1/376: dwrite f8 [0,4194304] 0 2024-02-16T19:24:08.587 INFO:tasks.workunit.client.0.smithi040.stdout:9/402: truncate d3/d4/d11/d29/d38/d46/f5b 3817576 0 2024-02-16T19:24:08.587 INFO:tasks.workunit.client.0.smithi040.stdout:9/403: fdatasync d3/d4/d11/d32/f43 0 2024-02-16T19:24:08.587 INFO:tasks.workunit.client.0.smithi040.stdout:9/404: chown d3/d4/d55/d70 0 1 2024-02-16T19:24:08.591 INFO:tasks.workunit.client.0.smithi040.stdout:3/444: symlink d1/l83 0 2024-02-16T19:24:08.593 INFO:tasks.workunit.client.0.smithi040.stdout:8/384: mkdir d6/d64/d50/d69/d72 0 2024-02-16T19:24:08.597 INFO:tasks.workunit.client.0.smithi040.stdout:1/377: truncate d17/d26/d5f/d63/f66 7839392 0 2024-02-16T19:24:08.597 INFO:tasks.workunit.client.0.smithi040.stdout:1/378: read - d17/d26/d54/f5e zero size 2024-02-16T19:24:08.597 INFO:tasks.workunit.client.0.smithi040.stdout:2/296: rmdir d5 39 2024-02-16T19:24:08.600 INFO:tasks.workunit.client.0.smithi040.stdout:3/445: truncate d1/dc/f1d 1862563 0 2024-02-16T19:24:08.603 INFO:tasks.workunit.client.0.smithi040.stdout:8/385: mknod d6/d2b/c73 0 2024-02-16T19:24:08.605 INFO:tasks.workunit.client.0.smithi040.stdout:1/379: creat d17/d43/d38/f7b x:0 0 0 2024-02-16T19:24:08.605 INFO:tasks.workunit.client.0.smithi040.stdout:1/380: write d17/d43/d38/f72 [61564,101021] 0 2024-02-16T19:24:08.605 INFO:tasks.workunit.client.0.smithi040.stdout:1/381: chown d17/d3c/c46 313 1 2024-02-16T19:24:08.605 INFO:tasks.workunit.client.0.smithi040.stdout:1/382: write fb [1305621,63242] 0 2024-02-16T19:24:08.605 INFO:tasks.workunit.client.0.smithi040.stdout:2/297: mknod d5/d46/c59 0 2024-02-16T19:24:08.607 INFO:tasks.workunit.client.0.smithi040.stdout:8/386: rmdir d6/d64/d50/d69/d72 0 2024-02-16T19:24:08.608 INFO:tasks.workunit.client.0.smithi040.stdout:1/383: link d17/d26/c51 d17/d29/c7c 0 2024-02-16T19:24:08.610 INFO:tasks.workunit.client.0.smithi040.stdout:2/298: rename d5/d7/c41 to d5/d7/d1e/d4c/d52/c5a 0 2024-02-16T19:24:08.611 INFO:tasks.workunit.client.0.smithi040.stdout:2/299: mknod d5/d7/d1e/d31/c5b 0 2024-02-16T19:24:08.612 INFO:tasks.workunit.client.0.smithi040.stdout:7/323: getdents d21 0 2024-02-16T19:24:08.618 INFO:tasks.workunit.client.0.smithi040.stdout:8/387: dread d6/d12/f1f [0,4194304] 0 2024-02-16T19:24:08.618 INFO:tasks.workunit.client.0.smithi040.stdout:8/388: chown d6/f43 5895827 1 2024-02-16T19:24:08.618 INFO:tasks.workunit.client.0.smithi040.stdout:8/389: write d6/f13 [1259621,106727] 0 2024-02-16T19:24:08.620 INFO:tasks.workunit.client.0.smithi040.stdout:8/390: creat d6/d64/d50/f74 x:0 0 0 2024-02-16T19:24:08.620 INFO:tasks.workunit.client.0.smithi040.stdout:8/391: truncate d6/d7/d15/f47 4744057 0 2024-02-16T19:24:08.622 INFO:tasks.workunit.client.0.smithi040.stdout:9/405: dwrite d3/d4/d11/d29/f56 [0,4194304] 0 2024-02-16T19:24:08.622 INFO:tasks.workunit.client.0.smithi040.stdout:0/400: dwrite db/de/d6a/f82 [0,4194304] 0 2024-02-16T19:24:08.622 INFO:tasks.workunit.client.0.smithi040.stdout:0/401: dread - db/d2f/d1d/f80 zero size 2024-02-16T19:24:08.625 INFO:tasks.workunit.client.0.smithi040.stdout:6/344: dwrite d0/d30/d32/f21 [0,4194304] 0 2024-02-16T19:24:08.625 INFO:tasks.workunit.client.0.smithi040.stdout:0/402: creat db/d25/d34/f8a x:0 0 0 2024-02-16T19:24:08.627 INFO:tasks.workunit.client.0.smithi040.stdout:2/300: read d5/d7/d12/f15 [6556548,118198] 0 2024-02-16T19:24:08.631 INFO:tasks.workunit.client.0.smithi040.stdout:0/403: unlink db/cc 0 2024-02-16T19:24:08.632 INFO:tasks.workunit.client.0.smithi040.stdout:0/404: truncate db/d2f/d1d/d21/d2e/d85/f86 605399 0 2024-02-16T19:24:08.632 INFO:tasks.workunit.client.0.smithi040.stdout:0/405: truncate db/d2f/f70 804604 0 2024-02-16T19:24:08.632 INFO:tasks.workunit.client.0.smithi040.stdout:2/301: mkdir d5/d5c 0 2024-02-16T19:24:08.632 INFO:tasks.workunit.client.0.smithi040.stdout:1/384: write f8 [673637,2640] 0 2024-02-16T19:24:08.632 INFO:tasks.workunit.client.0.smithi040.stdout:1/385: chown f14 93 1 2024-02-16T19:24:08.632 INFO:tasks.workunit.client.0.smithi040.stdout:6/345: dread d0/d1/f1f [0,4194304] 0 2024-02-16T19:24:08.635 INFO:tasks.workunit.client.0.smithi040.stdout:1/386: rename fb to d17/d26/f7d 0 2024-02-16T19:24:08.636 INFO:tasks.workunit.client.0.smithi040.stdout:1/387: mkdir d17/d26/d5f/d78/d7e 0 2024-02-16T19:24:08.636 INFO:tasks.workunit.client.0.smithi040.stdout:1/388: readlink d17/d3c/l50 0 2024-02-16T19:24:08.639 INFO:tasks.workunit.client.0.smithi040.stdout:2/302: read d5/d7/d1e/d31/f34 [1536178,107415] 0 2024-02-16T19:24:08.639 INFO:tasks.workunit.client.0.smithi040.stdout:7/324: dwrite d21/d4d/f56 [0,4194304] 0 2024-02-16T19:24:08.639 INFO:tasks.workunit.client.0.smithi040.stdout:7/325: truncate d21/f3c 1023474 0 2024-02-16T19:24:08.642 INFO:tasks.workunit.client.0.smithi040.stdout:1/389: truncate d17/f36 1260550 0 2024-02-16T19:24:08.645 INFO:tasks.workunit.client.0.smithi040.stdout:2/303: creat d5/d7/d1e/d4c/d52/f5d x:0 0 0 2024-02-16T19:24:08.646 INFO:tasks.workunit.client.0.smithi040.stdout:1/390: getdents d17/d43/d38/d4a 0 2024-02-16T19:24:08.648 INFO:tasks.workunit.client.0.smithi040.stdout:2/304: rename d5/d7/d1e/d31/l51 to d5/d7/d12/l5e 0 2024-02-16T19:24:08.650 INFO:tasks.workunit.client.0.smithi040.stdout:1/391: creat d17/d3c/d74/f7f x:0 0 0 2024-02-16T19:24:08.654 INFO:tasks.workunit.client.0.smithi040.stdout:5/318: sync 2024-02-16T19:24:08.656 INFO:tasks.workunit.client.0.smithi040.stdout:5/319: link d1/da/d1c/d21/d57/l4e d1/da/d1c/d21/l5b 0 2024-02-16T19:24:08.659 INFO:tasks.workunit.client.0.smithi040.stdout:9/406: dwrite d3/d4/d11/d29/f24 [4194304,4194304] 0 2024-02-16T19:24:08.659 INFO:tasks.workunit.client.0.smithi040.stdout:9/407: chown d3/d4/d11/d29/f52 882932 1 2024-02-16T19:24:08.659 INFO:tasks.workunit.client.0.smithi040.stdout:2/305: dread d5/d7/dd/f45 [0,4194304] 0 2024-02-16T19:24:08.660 INFO:tasks.workunit.client.0.smithi040.stdout:2/306: rmdir d5/d8 39 2024-02-16T19:24:08.661 INFO:tasks.workunit.client.0.smithi040.stdout:0/406: dwrite db/de/f38 [0,4194304] 0 2024-02-16T19:24:08.661 INFO:tasks.workunit.client.0.smithi040.stdout:0/407: truncate db/d2f/d1d/d21/d2e/d49/f87 742408 0 2024-02-16T19:24:08.662 INFO:tasks.workunit.client.0.smithi040.stdout:8/392: dwrite d6/d12/d22/f68 [0,4194304] 0 2024-02-16T19:24:08.666 INFO:tasks.workunit.client.0.smithi040.stdout:0/408: symlink db/d2f/d23/d6f/l8b 0 2024-02-16T19:24:08.668 INFO:tasks.workunit.client.0.smithi040.stdout:8/393: rename d6/f33 to d6/f75 0 2024-02-16T19:24:08.669 INFO:tasks.workunit.client.0.smithi040.stdout:8/394: write d6/f13 [510765,6387] 0 2024-02-16T19:24:08.671 INFO:tasks.workunit.client.0.smithi040.stdout:0/409: creat db/d2f/d1d/d81/f8c x:0 0 0 2024-02-16T19:24:08.679 INFO:tasks.workunit.client.0.smithi040.stdout:9/408: read d3/d4/d11/d29/d38/d46/f5b [2851381,62581] 0 2024-02-16T19:24:08.680 INFO:tasks.workunit.client.0.smithi040.stdout:6/346: dwrite d0/d1/d12/f47 [0,4194304] 0 2024-02-16T19:24:08.682 INFO:tasks.workunit.client.0.smithi040.stdout:6/347: mkdir d0/d1/d12/d3a/d70 0 2024-02-16T19:24:08.682 INFO:tasks.workunit.client.0.smithi040.stdout:6/348: chown d0/d1/d12/d3b/d48/d4f/l63 5514582 1 2024-02-16T19:24:08.683 INFO:tasks.workunit.client.0.smithi040.stdout:6/349: getdents d0/d1 0 2024-02-16T19:24:08.684 INFO:tasks.workunit.client.0.smithi040.stdout:6/350: write d0/d1/f1f [3445256,44377] 0 2024-02-16T19:24:08.684 INFO:tasks.workunit.client.0.smithi040.stdout:6/351: readlink d0/d1/l1e 0 2024-02-16T19:24:08.684 INFO:tasks.workunit.client.0.smithi040.stdout:6/352: write d0/f3f [715567,22085] 0 2024-02-16T19:24:08.685 INFO:tasks.workunit.client.0.smithi040.stdout:6/353: creat d0/d1/d12/d3b/d48/d4d/d6d/f71 x:0 0 0 2024-02-16T19:24:08.686 INFO:tasks.workunit.client.0.smithi040.stdout:0/410: dread db/de/d6a/f6e [0,4194304] 0 2024-02-16T19:24:08.686 INFO:tasks.workunit.client.0.smithi040.stdout:0/411: write db/de/f75 [899104,27861] 0 2024-02-16T19:24:08.686 INFO:tasks.workunit.client.0.smithi040.stdout:0/412: write db/fd [1751535,72066] 0 2024-02-16T19:24:08.689 INFO:tasks.workunit.client.0.smithi040.stdout:0/413: rename db/d2f/c64 to db/d25/d2c/c8d 0 2024-02-16T19:24:08.689 INFO:tasks.workunit.client.0.smithi040.stdout:0/414: readlink db/d25/d34/l62 0 2024-02-16T19:24:08.689 INFO:tasks.workunit.client.0.smithi040.stdout:0/415: fdatasync db/d25/d34/d5e/f6b 0 2024-02-16T19:24:08.690 INFO:tasks.workunit.client.0.smithi040.stdout:0/416: rename db/d2f/d1d/d21/d2e/d49/d5b/d67/c7c to db/d25/d34/d5e/c8e 0 2024-02-16T19:24:08.698 INFO:tasks.workunit.client.0.smithi040.stdout:6/354: dread d0/fe [0,4194304] 0 2024-02-16T19:24:08.699 INFO:tasks.workunit.client.0.smithi040.stdout:6/355: creat d0/d30/d5b/f72 x:0 0 0 2024-02-16T19:24:08.699 INFO:tasks.workunit.client.0.smithi040.stdout:6/356: write d0/f1d [1654205,41681] 0 2024-02-16T19:24:08.699 INFO:tasks.workunit.client.0.smithi040.stdout:6/357: chown d0/d30/d4b 7 1 2024-02-16T19:24:08.700 INFO:tasks.workunit.client.0.smithi040.stdout:6/358: write d0/d1/d12/d3b/f40 [504817,10208] 0 2024-02-16T19:24:08.701 INFO:tasks.workunit.client.0.smithi040.stdout:7/326: dwrite d21/d29/f30 [0,4194304] 0 2024-02-16T19:24:08.701 INFO:tasks.workunit.client.0.smithi040.stdout:7/327: readlink d21/d29/l58 0 2024-02-16T19:24:08.701 INFO:tasks.workunit.client.0.smithi040.stdout:7/328: read - d21/f48 zero size 2024-02-16T19:24:08.703 INFO:tasks.workunit.client.0.smithi040.stdout:5/320: dwrite d1/f1d [0,4194304] 0 2024-02-16T19:24:08.703 INFO:tasks.workunit.client.0.smithi040.stdout:0/417: dread db/d2f/d1d/d21/f45 [0,4194304] 0 2024-02-16T19:24:08.703 INFO:tasks.workunit.client.0.smithi040.stdout:1/392: dwrite d17/d26/f53 [0,4194304] 0 2024-02-16T19:24:08.703 INFO:tasks.workunit.client.0.smithi040.stdout:0/418: truncate db/d2f/d1d/d21/d2e/d49/d5b/f79 770444 0 2024-02-16T19:24:08.703 INFO:tasks.workunit.client.0.smithi040.stdout:1/393: creat d17/d3c/f80 x:0 0 0 2024-02-16T19:24:08.703 INFO:tasks.workunit.client.0.smithi040.stdout:1/394: creat d17/d2d/f81 x:0 0 0 2024-02-16T19:24:08.706 INFO:tasks.workunit.client.0.smithi040.stdout:7/329: symlink d21/d4d/l65 0 2024-02-16T19:24:08.706 INFO:tasks.workunit.client.0.smithi040.stdout:7/330: creat d21/d4d/f66 x:0 0 0 2024-02-16T19:24:08.706 INFO:tasks.workunit.client.0.smithi040.stdout:5/321: creat d1/da/d1c/d21/d57/f5c x:0 0 0 2024-02-16T19:24:08.738 INFO:tasks.workunit.client.0.smithi040.stdout:0/419: creat db/d2f/d1d/d21/d2e/d49/d5b/d67/f8f x:0 0 0 2024-02-16T19:24:08.738 INFO:tasks.workunit.client.0.smithi040.stdout:2/307: dwrite d5/d7/dd/f2c [0,4194304] 0 2024-02-16T19:24:08.738 INFO:tasks.workunit.client.0.smithi040.stdout:2/308: stat d5/d7/d10/l21 0 2024-02-16T19:24:08.739 INFO:tasks.workunit.client.0.smithi040.stdout:8/395: dwrite d6/d12/f31 [0,4194304] 0 2024-02-16T19:24:08.739 INFO:tasks.workunit.client.0.smithi040.stdout:9/409: dwrite d3/d4/d11/d29/d38/d46/d6b/f6f [4194304,4194304] 0 2024-02-16T19:24:08.739 INFO:tasks.workunit.client.0.smithi040.stdout:9/410: fsync d3/d4/d11/d29/f48 0 2024-02-16T19:24:08.739 INFO:tasks.workunit.client.0.smithi040.stdout:1/395: creat d17/d26/d54/f82 x:0 0 0 2024-02-16T19:24:08.741 INFO:tasks.workunit.client.0.smithi040.stdout:5/322: rename d1/f55 to d1/d3c/f5d 0 2024-02-16T19:24:08.743 INFO:tasks.workunit.client.0.smithi040.stdout:1/396: dread d17/d2d/f2f [0,4194304] 0 2024-02-16T19:24:08.745 INFO:tasks.workunit.client.0.smithi040.stdout:0/420: dread db/d25/f4f [0,4194304] 0 2024-02-16T19:24:08.749 INFO:tasks.workunit.client.0.smithi040.stdout:7/331: dread f11 [0,4194304] 0 2024-02-16T19:24:08.749 INFO:tasks.workunit.client.0.smithi040.stdout:7/332: write f11 [1826847,131039] 0 2024-02-16T19:24:08.749 INFO:tasks.workunit.client.0.smithi040.stdout:7/333: chown d21/l53 1239628 1 2024-02-16T19:24:08.751 INFO:tasks.workunit.client.0.smithi040.stdout:8/396: link d6/d12/f14 d6/f76 0 2024-02-16T19:24:08.751 INFO:tasks.workunit.client.0.smithi040.stdout:8/397: chown d6/d12/d22/f23 16 1 2024-02-16T19:24:08.751 INFO:tasks.workunit.client.0.smithi040.stdout:8/398: creat d6/d7/d1d/f77 x:0 0 0 2024-02-16T19:24:08.751 INFO:tasks.workunit.client.0.smithi040.stdout:8/399: write d6/d7/d1d/f60 [778402,101220] 0 2024-02-16T19:24:08.752 INFO:tasks.workunit.client.0.smithi040.stdout:8/400: write d6/f43 [385950,28085] 0 2024-02-16T19:24:08.752 INFO:tasks.workunit.client.0.smithi040.stdout:8/401: chown d6/d7/d1d/d38/f3d 94935 1 2024-02-16T19:24:08.754 INFO:tasks.workunit.client.0.smithi040.stdout:9/411: link d3/f21 d3/d4/d55/d63/f80 0 2024-02-16T19:24:08.756 INFO:tasks.workunit.client.0.smithi040.stdout:8/402: dread d6/f46 [0,4194304] 0 2024-02-16T19:24:08.756 INFO:tasks.workunit.client.0.smithi040.stdout:6/359: dread d0/d30/d32/f21 [0,4194304] 0 2024-02-16T19:24:08.756 INFO:tasks.workunit.client.0.smithi040.stdout:2/309: rename d5/d7/d10/l21 to d5/d5c/l5f 0 2024-02-16T19:24:08.756 INFO:tasks.workunit.client.0.smithi040.stdout:2/310: fdatasync d5/d7/d12/d1b/f30 0 2024-02-16T19:24:08.757 INFO:tasks.workunit.client.0.smithi040.stdout:2/311: readlink d5/d7/d12/d1b/l2a 0 2024-02-16T19:24:08.757 INFO:tasks.workunit.client.0.smithi040.stdout:2/312: fsync d5/f3c 0 2024-02-16T19:24:08.758 INFO:tasks.workunit.client.0.smithi040.stdout:0/421: unlink db/d2f/d1d/d21/d2e/d49/l71 0 2024-02-16T19:24:08.758 INFO:tasks.workunit.client.0.smithi040.stdout:0/422: write db/d25/d2c/f5a [575595,19478] 0 2024-02-16T19:24:08.759 INFO:tasks.workunit.client.0.smithi040.stdout:0/423: dread db/d2f/d1d/d21/d2e/d49/f58 [0,4194304] 0 2024-02-16T19:24:08.762 INFO:tasks.workunit.client.0.smithi040.stdout:9/412: symlink d3/d4/d11/d71/l81 0 2024-02-16T19:24:08.762 INFO:tasks.workunit.client.0.smithi040.stdout:9/413: readlink d3/l16 0 2024-02-16T19:24:08.763 INFO:tasks.workunit.client.0.smithi040.stdout:9/414: fdatasync d3/d4/d11/d32/f5c 0 2024-02-16T19:24:08.763 INFO:tasks.workunit.client.0.smithi040.stdout:9/415: creat d3/d1c/f82 x:0 0 0 2024-02-16T19:24:08.764 INFO:tasks.workunit.client.0.smithi040.stdout:5/323: dwrite d1/d45/f52 [0,4194304] 0 2024-02-16T19:24:08.766 INFO:tasks.workunit.client.0.smithi040.stdout:6/360: rename d0/d1/d12/d3a to d0/d1/d12/d3b/d48/d73 0 2024-02-16T19:24:08.767 INFO:tasks.workunit.client.0.smithi040.stdout:0/424: mknod db/d2f/d1d/d21/d2e/d49/c90 0 2024-02-16T19:24:08.768 INFO:tasks.workunit.client.0.smithi040.stdout:7/334: symlink d21/l67 0 2024-02-16T19:24:08.768 INFO:tasks.workunit.client.0.smithi040.stdout:7/335: truncate d21/d4d/f66 769840 0 2024-02-16T19:24:08.771 INFO:tasks.workunit.client.0.smithi040.stdout:4/317: sync 2024-02-16T19:24:08.772 INFO:tasks.workunit.client.0.smithi040.stdout:2/313: dread d5/d7/f47 [0,4194304] 0 2024-02-16T19:24:08.775 INFO:tasks.workunit.client.0.smithi040.stdout:9/416: rename d3/d4/d55/d70/f7a to d3/d1c/d73/f83 0 2024-02-16T19:24:08.776 INFO:tasks.workunit.client.0.smithi040.stdout:6/361: symlink d0/d30/d4b/l74 0 2024-02-16T19:24:08.776 INFO:tasks.workunit.client.0.smithi040.stdout:6/362: chown d0/d1/d12/d3b/d48/d4f/f69 988 1 2024-02-16T19:24:08.777 INFO:tasks.workunit.client.0.smithi040.stdout:4/318: unlink f8 0 2024-02-16T19:24:08.778 INFO:tasks.workunit.client.0.smithi040.stdout:2/314: creat d5/d46/f60 x:0 0 0 2024-02-16T19:24:08.778 INFO:tasks.workunit.client.0.smithi040.stdout:5/324: write d1/da/d1c/d21/f2f [4789441,86903] 0 2024-02-16T19:24:08.778 INFO:tasks.workunit.client.0.smithi040.stdout:5/325: truncate d1/f2c 4241874 0 2024-02-16T19:24:08.779 INFO:tasks.workunit.client.0.smithi040.stdout:2/315: dread d5/d8/f38 [0,4194304] 0 2024-02-16T19:24:08.779 INFO:tasks.workunit.client.0.smithi040.stdout:2/316: creat d5/d46/f61 x:0 0 0 2024-02-16T19:24:08.780 INFO:tasks.workunit.client.0.smithi040.stdout:0/425: rename db/f1b to db/d25/d2c/d52/f91 0 2024-02-16T19:24:08.780 INFO:tasks.workunit.client.0.smithi040.stdout:0/426: stat db/d2f/d1d/d21/f51 0 2024-02-16T19:24:08.780 INFO:tasks.workunit.client.0.smithi040.stdout:0/427: write db/f7f [135440,21832] 0 2024-02-16T19:24:08.781 INFO:tasks.workunit.client.0.smithi040.stdout:6/363: creat d0/d1/d6f/f75 x:0 0 0 2024-02-16T19:24:08.781 INFO:tasks.workunit.client.0.smithi040.stdout:6/364: stat d0/d1/d12 0 2024-02-16T19:24:08.781 INFO:tasks.workunit.client.0.smithi040.stdout:6/365: write d0/f4 [4381948,70407] 0 2024-02-16T19:24:08.781 INFO:tasks.workunit.client.0.smithi040.stdout:6/366: fsync d0/d30/d5b/f72 0 2024-02-16T19:24:08.781 INFO:tasks.workunit.client.0.smithi040.stdout:0/428: dread db/d2f/d1d/d21/f51 [0,4194304] 0 2024-02-16T19:24:08.782 INFO:tasks.workunit.client.0.smithi040.stdout:0/429: fsync db/de/d6a/f82 0 2024-02-16T19:24:08.782 INFO:tasks.workunit.client.0.smithi040.stdout:0/430: creat db/d2f/d1d/d21/d2e/d49/d5b/d67/f92 x:0 0 0 2024-02-16T19:24:08.782 INFO:tasks.workunit.client.0.smithi040.stdout:0/431: read - db/d25/d34/f8a zero size 2024-02-16T19:24:08.783 INFO:tasks.workunit.client.0.smithi040.stdout:8/403: dwrite d6/d2b/d36/f4f [0,4194304] 0 2024-02-16T19:24:08.783 INFO:tasks.workunit.client.0.smithi040.stdout:7/336: mknod d21/c68 0 2024-02-16T19:24:08.783 INFO:tasks.workunit.client.0.smithi040.stdout:7/337: stat d21/d29/c2a 0 2024-02-16T19:24:08.784 INFO:tasks.workunit.client.0.smithi040.stdout:4/319: rename dd/de/d10/d36/f5b to dd/de/d10/d36/f63 0 2024-02-16T19:24:08.784 INFO:tasks.workunit.client.0.smithi040.stdout:3/446: sync 2024-02-16T19:24:08.786 INFO:tasks.workunit.client.0.smithi040.stdout:5/326: link d1/f17 d1/da/d51/f5e 0 2024-02-16T19:24:08.790 INFO:tasks.workunit.client.0.smithi040.stdout:4/320: dread dd/f52 [0,4194304] 0 2024-02-16T19:24:08.791 INFO:tasks.workunit.client.0.smithi040.stdout:2/317: mknod d5/d7/d1e/d4c/c62 0 2024-02-16T19:24:08.793 INFO:tasks.workunit.client.0.smithi040.stdout:9/417: rename d3/d4/d55/d31/f72 to d3/d4/d11/d32/f84 0 2024-02-16T19:24:08.793 INFO:tasks.workunit.client.0.smithi040.stdout:9/418: fdatasync d3/d4/d11/d29/d38/d46/f6a 0 2024-02-16T19:24:08.798 INFO:tasks.workunit.client.0.smithi040.stdout:0/432: mkdir db/d2f/d23/d93 0 2024-02-16T19:24:08.800 INFO:tasks.workunit.client.0.smithi040.stdout:3/447: creat d1/d4/d6e/f84 x:0 0 0 2024-02-16T19:24:08.802 INFO:tasks.workunit.client.0.smithi040.stdout:5/327: rename d1/da/d1c/d21/d57/f5c to d1/d45/f5f 0 2024-02-16T19:24:08.802 INFO:tasks.workunit.client.0.smithi040.stdout:5/328: write d1/f1d [4364578,10265] 0 2024-02-16T19:24:08.804 INFO:tasks.workunit.client.0.smithi040.stdout:9/419: mkdir d3/d4/d11/d29/d38/d46/d6b/d85 0 2024-02-16T19:24:08.805 INFO:tasks.workunit.client.0.smithi040.stdout:0/433: mkdir db/d2f/d1d/d21/d2e/d85/d89/d94 0 2024-02-16T19:24:08.807 INFO:tasks.workunit.client.0.smithi040.stdout:2/318: rename d5/d7/dd to d5/d7/d1e/d31/d4f/d63 0 2024-02-16T19:24:08.808 INFO:tasks.workunit.client.0.smithi040.stdout:5/329: mknod d1/c60 0 2024-02-16T19:24:08.809 INFO:tasks.workunit.client.0.smithi040.stdout:5/330: chown d1/d45/c59 4 1 2024-02-16T19:24:08.811 INFO:tasks.workunit.client.0.smithi040.stdout:6/367: dwrite d0/fe [0,4194304] 0 2024-02-16T19:24:08.811 INFO:tasks.workunit.client.0.smithi040.stdout:2/319: symlink d5/d7/d1e/l64 0 2024-02-16T19:24:08.811 INFO:tasks.workunit.client.0.smithi040.stdout:2/320: fsync d5/d7/f54 0 2024-02-16T19:24:08.813 INFO:tasks.workunit.client.0.smithi040.stdout:9/420: unlink d3/d1c/f82 0 2024-02-16T19:24:08.813 INFO:tasks.workunit.client.0.smithi040.stdout:9/421: truncate d3/d4/d11/d32/f42 1423540 0 2024-02-16T19:24:08.815 INFO:tasks.workunit.client.0.smithi040.stdout:2/321: unlink d5/d46/f61 0 2024-02-16T19:24:08.816 INFO:tasks.workunit.client.0.smithi040.stdout:9/422: mkdir d3/d4/d11/d29/d38/d86 0 2024-02-16T19:24:08.816 INFO:tasks.workunit.client.0.smithi040.stdout:9/423: chown d3/c10 193258511 1 2024-02-16T19:24:08.816 INFO:tasks.workunit.client.0.smithi040.stdout:4/321: dwrite dd/de/d10/f62 [0,4194304] 0 2024-02-16T19:24:08.816 INFO:tasks.workunit.client.0.smithi040.stdout:7/338: dwrite d21/d4d/f66 [0,4194304] 0 2024-02-16T19:24:08.817 INFO:tasks.workunit.client.0.smithi040.stdout:3/448: rename d1/d7/d33/d38 to d1/dc/d35/d48/d85 0 2024-02-16T19:24:08.817 INFO:tasks.workunit.client.0.smithi040.stdout:3/449: write d1/d7/d30/f81 [223730,45547] 0 2024-02-16T19:24:08.817 INFO:tasks.workunit.client.0.smithi040.stdout:3/450: chown d1/d4/f41 508802462 1 2024-02-16T19:24:08.817 INFO:tasks.workunit.client.0.smithi040.stdout:3/451: write d1/d4/f19 [1842041,93646] 0 2024-02-16T19:24:08.818 INFO:tasks.workunit.client.0.smithi040.stdout:8/404: dwrite d6/d64/d50/f74 [0,4194304] 0 2024-02-16T19:24:08.818 INFO:tasks.workunit.client.0.smithi040.stdout:2/322: rmdir d5/d7/d1e/d31 39 2024-02-16T19:24:08.819 INFO:tasks.workunit.client.0.smithi040.stdout:8/405: truncate d6/d7/f2e 223525 0 2024-02-16T19:24:08.819 INFO:tasks.workunit.client.0.smithi040.stdout:2/323: stat d5/d7/ce 0 2024-02-16T19:24:08.820 INFO:tasks.workunit.client.0.smithi040.stdout:9/424: stat d3/d4/d11/l6c 0 2024-02-16T19:24:08.820 INFO:tasks.workunit.client.0.smithi040.stdout:9/425: truncate d3/d4/d55/d31/f3b 5090397 0 2024-02-16T19:24:08.824 INFO:tasks.workunit.client.0.smithi040.stdout:5/331: read d1/d45/f49 [3213391,66005] 0 2024-02-16T19:24:08.827 INFO:tasks.workunit.client.0.smithi040.stdout:6/368: dwrite d0/d1/d12/d3b/d48/f38 [0,4194304] 0 2024-02-16T19:24:08.827 INFO:tasks.workunit.client.0.smithi040.stdout:3/452: dread d1/dc/d42/d4a/f50 [0,4194304] 0 2024-02-16T19:24:08.829 INFO:tasks.workunit.client.0.smithi040.stdout:4/322: truncate dd/f52 501915 0 2024-02-16T19:24:08.833 INFO:tasks.workunit.client.0.smithi040.stdout:0/434: rename db/f63 to db/d2f/d23/d6f/f95 0 2024-02-16T19:24:08.842 INFO:tasks.workunit.client.0.smithi040.stdout:8/406: mknod d6/d64/c78 0 2024-02-16T19:24:08.843 INFO:tasks.workunit.client.0.smithi040.stdout:6/369: mkdir d0/d1/d12/d3b/d48/d73/d76 0 2024-02-16T19:24:08.843 INFO:tasks.workunit.client.0.smithi040.stdout:6/370: chown d0/d30/d32/c2b 236519 1 2024-02-16T19:24:08.843 INFO:tasks.workunit.client.0.smithi040.stdout:6/371: write d0/f36 [273897,117090] 0 2024-02-16T19:24:08.844 INFO:tasks.workunit.client.0.smithi040.stdout:7/339: symlink d21/l69 0 2024-02-16T19:24:08.846 INFO:tasks.workunit.client.0.smithi040.stdout:3/453: unlink d1/dc/f1d 0 2024-02-16T19:24:08.846 INFO:tasks.workunit.client.0.smithi040.stdout:3/454: creat d1/dc/d35/d55/f86 x:0 0 0 2024-02-16T19:24:08.846 INFO:tasks.workunit.client.0.smithi040.stdout:3/455: creat d1/d7/d36/f87 x:0 0 0 2024-02-16T19:24:08.846 INFO:tasks.workunit.client.0.smithi040.stdout:3/456: chown d1/d7/d33/l22 83114 1 2024-02-16T19:24:08.849 INFO:tasks.workunit.client.0.smithi040.stdout:9/426: dwrite d3/d4/d55/f2e [0,4194304] 0 2024-02-16T19:24:08.849 INFO:tasks.workunit.client.0.smithi040.stdout:0/435: mknod db/d2f/d23/d56/c96 0 2024-02-16T19:24:08.849 INFO:tasks.workunit.client.0.smithi040.stdout:5/332: dwrite d1/d45/f49 [0,4194304] 0 2024-02-16T19:24:08.852 INFO:tasks.workunit.client.0.smithi040.stdout:0/436: dread db/d2f/f70 [0,4194304] 0 2024-02-16T19:24:08.852 INFO:tasks.workunit.client.0.smithi040.stdout:8/407: dread d6/d2b/d44/f49 [0,4194304] 0 2024-02-16T19:24:08.853 INFO:tasks.workunit.client.0.smithi040.stdout:8/408: write d6/d7/d15/f18 [4505579,70532] 0 2024-02-16T19:24:08.863 INFO:tasks.workunit.client.0.smithi040.stdout:0/437: dread db/f7f [0,4194304] 0 2024-02-16T19:24:08.874 INFO:tasks.workunit.client.0.smithi040.stdout:7/340: dwrite d21/f48 [0,4194304] 0 2024-02-16T19:24:08.875 INFO:tasks.workunit.client.0.smithi040.stdout:7/341: stat d21/d29 0 2024-02-16T19:24:08.875 INFO:tasks.workunit.client.0.smithi040.stdout:7/342: write d21/d29/f5c [163903,125886] 0 2024-02-16T19:24:08.875 INFO:tasks.workunit.client.0.smithi040.stdout:7/343: write d21/d4d/f5b [706808,29075] 0 2024-02-16T19:24:08.875 INFO:tasks.workunit.client.0.smithi040.stdout:6/372: dwrite d0/d30/d32/f2d [0,4194304] 0 2024-02-16T19:24:08.875 INFO:tasks.workunit.client.0.smithi040.stdout:9/427: dwrite d3/d4/d11/d29/f2b [0,4194304] 0 2024-02-16T19:24:08.879 INFO:tasks.workunit.client.0.smithi040.stdout:3/457: getdents d1/dc/d42/d5c 0 2024-02-16T19:24:08.879 INFO:tasks.workunit.client.0.smithi040.stdout:3/458: readlink d1/dc/d35/d48/d85/l4b 0 2024-02-16T19:24:08.879 INFO:tasks.workunit.client.0.smithi040.stdout:3/459: chown d1/d7/d33/c1e 0 1 2024-02-16T19:24:08.888 INFO:tasks.workunit.client.0.smithi040.stdout:5/333: symlink d1/da/d1c/d21/d57/l61 0 2024-02-16T19:24:08.892 INFO:tasks.workunit.client.0.smithi040.stdout:0/438: mknod db/de/d6a/c97 0 2024-02-16T19:24:08.893 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:08 smithi040 ceph-mon[29201]: pgmap v243: 65 pgs: 65 active+clean; 777 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 64 MiB/s rd, 96 MiB/s wr, 952 op/s 2024-02-16T19:24:08.896 INFO:tasks.workunit.client.0.smithi040.stdout:1/397: sync 2024-02-16T19:24:08.898 INFO:tasks.workunit.client.0.smithi040.stdout:7/344: mknod d21/d4d/c6a 0 2024-02-16T19:24:08.898 INFO:tasks.workunit.client.0.smithi040.stdout:7/345: chown d21/d4d/c6a 4906271 1 2024-02-16T19:24:08.898 INFO:tasks.workunit.client.0.smithi040.stdout:7/346: chown d21/f26 1 1 2024-02-16T19:24:08.901 INFO:tasks.workunit.client.0.smithi040.stdout:6/373: creat d0/d1/d12/d3b/d48/d73/f77 x:0 0 0 2024-02-16T19:24:08.920 INFO:tasks.workunit.client.0.smithi040.stdout:2/324: rename d5/d46 to d5/d5c/d65 0 2024-02-16T19:24:08.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/409: symlink d6/d7/d15/l79 0 2024-02-16T19:24:08.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/410: creat d6/d7/d1d/f7a x:0 0 0 2024-02-16T19:24:08.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/411: creat d6/d7/f7b x:0 0 0 2024-02-16T19:24:08.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/412: chown d6/d12/d22/f23 3228786 1 2024-02-16T19:24:08.921 INFO:tasks.workunit.client.0.smithi040.stdout:0/439: mkdir db/d2f/d23/d6f/d98 0 2024-02-16T19:24:08.922 INFO:tasks.workunit.client.0.smithi040.stdout:0/440: write db/f20 [1036855,31297] 0 2024-02-16T19:24:08.924 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:08 smithi163 ceph-mon[36452]: pgmap v243: 65 pgs: 65 active+clean; 777 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 64 MiB/s rd, 96 MiB/s wr, 952 op/s 2024-02-16T19:24:08.928 INFO:tasks.workunit.client.0.smithi040.stdout:3/460: getdents d1/d7 0 2024-02-16T19:24:08.928 INFO:tasks.workunit.client.0.smithi040.stdout:1/398: dwrite f12 [0,4194304] 0 2024-02-16T19:24:08.929 INFO:tasks.workunit.client.0.smithi040.stdout:6/374: dwrite d0/d1/d12/d3b/d48/d4d/d6d/f71 [0,4194304] 0 2024-02-16T19:24:08.931 INFO:tasks.workunit.client.0.smithi040.stdout:2/325: creat d5/d5c/f66 x:0 0 0 2024-02-16T19:24:08.931 INFO:tasks.workunit.client.0.smithi040.stdout:2/326: chown d5/d7/d1e/d31 2190 1 2024-02-16T19:24:08.931 INFO:tasks.workunit.client.0.smithi040.stdout:2/327: write d5/d8/d42/f50 [18550,118928] 0 2024-02-16T19:24:08.934 INFO:tasks.workunit.client.0.smithi040.stdout:8/413: truncate d6/f75 4144497 0 2024-02-16T19:24:08.934 INFO:tasks.workunit.client.0.smithi040.stdout:8/414: readlink d6/d7/d1d/l71 0 2024-02-16T19:24:08.936 INFO:tasks.workunit.client.0.smithi040.stdout:0/441: link db/d25/d34/d5e/c8e db/de/c99 0 2024-02-16T19:24:08.936 INFO:tasks.workunit.client.0.smithi040.stdout:0/442: write db/f7f [203144,100242] 0 2024-02-16T19:24:08.936 INFO:tasks.workunit.client.0.smithi040.stdout:0/443: readlink db/d2f/l3d 0 2024-02-16T19:24:08.937 INFO:tasks.workunit.client.0.smithi040.stdout:7/347: symlink d21/l6b 0 2024-02-16T19:24:08.937 INFO:tasks.workunit.client.0.smithi040.stdout:7/348: creat d21/d4d/f6c x:0 0 0 2024-02-16T19:24:08.937 INFO:tasks.workunit.client.0.smithi040.stdout:7/349: chown d21/d4d/f61 2 1 2024-02-16T19:24:08.938 INFO:tasks.workunit.client.0.smithi040.stdout:3/461: getdents d1/dc/d35 0 2024-02-16T19:24:08.938 INFO:tasks.workunit.client.0.smithi040.stdout:3/462: dread - d1/dc/d35/d48/f78 zero size 2024-02-16T19:24:08.938 INFO:tasks.workunit.client.0.smithi040.stdout:3/463: stat d1/dc/d35/l46 0 2024-02-16T19:24:08.941 INFO:tasks.workunit.client.0.smithi040.stdout:1/399: readlink d17/d26/d3d/l42 0 2024-02-16T19:24:08.945 INFO:tasks.workunit.client.0.smithi040.stdout:2/328: creat d5/d8/d42/f67 x:0 0 0 2024-02-16T19:24:08.946 INFO:tasks.workunit.client.0.smithi040.stdout:8/415: symlink d6/d2b/d44/l7c 0 2024-02-16T19:24:08.946 INFO:tasks.workunit.client.0.smithi040.stdout:8/416: chown d6/d2b/d44/l7c 1 1 2024-02-16T19:24:08.946 INFO:tasks.workunit.client.0.smithi040.stdout:8/417: creat d6/d64/d50/f7d x:0 0 0 2024-02-16T19:24:08.946 INFO:tasks.workunit.client.0.smithi040.stdout:2/329: dread d5/d8/f38 [0,4194304] 0 2024-02-16T19:24:08.948 INFO:tasks.workunit.client.0.smithi040.stdout:3/464: symlink d1/d4/d6e/l88 0 2024-02-16T19:24:08.951 INFO:tasks.workunit.client.0.smithi040.stdout:1/400: creat d17/d26/d3d/f83 x:0 0 0 2024-02-16T19:24:08.951 INFO:tasks.workunit.client.0.smithi040.stdout:1/401: dread - d17/d43/d38/d4a/f79 zero size 2024-02-16T19:24:08.951 INFO:tasks.workunit.client.0.smithi040.stdout:1/402: fdatasync d17/d29/f2b 0 2024-02-16T19:24:08.953 INFO:tasks.workunit.client.0.smithi040.stdout:3/465: dread d1/d4/f19 [0,4194304] 0 2024-02-16T19:24:08.953 INFO:tasks.workunit.client.0.smithi040.stdout:7/350: dwrite d21/f48 [0,4194304] 0 2024-02-16T19:24:08.954 INFO:tasks.workunit.client.0.smithi040.stdout:8/418: symlink d6/d64/d50/l7e 0 2024-02-16T19:24:08.954 INFO:tasks.workunit.client.0.smithi040.stdout:8/419: read - d6/d7/d1d/d38/f3d zero size 2024-02-16T19:24:08.955 INFO:tasks.workunit.client.0.smithi040.stdout:2/330: creat d5/d7/f68 x:0 0 0 2024-02-16T19:24:08.956 INFO:tasks.workunit.client.0.smithi040.stdout:1/403: creat d17/d26/d5f/f84 x:0 0 0 2024-02-16T19:24:08.958 INFO:tasks.workunit.client.0.smithi040.stdout:3/466: symlink d1/dc/d35/d53/d64/l89 0 2024-02-16T19:24:08.962 INFO:tasks.workunit.client.0.smithi040.stdout:7/351: symlink d21/d29/l6d 0 2024-02-16T19:24:08.963 INFO:tasks.workunit.client.0.smithi040.stdout:1/404: mknod d17/d26/d5f/c85 0 2024-02-16T19:24:08.963 INFO:tasks.workunit.client.0.smithi040.stdout:8/420: dread f4 [0,4194304] 0 2024-02-16T19:24:08.963 INFO:tasks.workunit.client.0.smithi040.stdout:8/421: creat d6/d2b/d44/f7f x:0 0 0 2024-02-16T19:24:08.964 INFO:tasks.workunit.client.0.smithi040.stdout:3/467: mkdir d1/d4/d6e/d8a 0 2024-02-16T19:24:08.966 INFO:tasks.workunit.client.0.smithi040.stdout:1/405: mkdir d17/d26/d4c/d86 0 2024-02-16T19:24:08.966 INFO:tasks.workunit.client.0.smithi040.stdout:1/406: chown d17/d26/d54/l77 10979 1 2024-02-16T19:24:08.969 INFO:tasks.workunit.client.0.smithi040.stdout:1/407: dread f14 [0,4194304] 0 2024-02-16T19:24:08.970 INFO:tasks.workunit.client.0.smithi040.stdout:3/468: mknod d1/dc/d35/d48/c8b 0 2024-02-16T19:24:08.972 INFO:tasks.workunit.client.0.smithi040.stdout:8/422: rmdir d6/d2b/d36 39 2024-02-16T19:24:08.975 INFO:tasks.workunit.client.0.smithi040.stdout:1/408: creat d17/d3c/d71/f87 x:0 0 0 2024-02-16T19:24:08.977 INFO:tasks.workunit.client.0.smithi040.stdout:7/352: dwrite d21/d29/f3f [0,4194304] 0 2024-02-16T19:24:08.977 INFO:tasks.workunit.client.0.smithi040.stdout:7/353: creat d21/d4d/f6e x:0 0 0 2024-02-16T19:24:08.977 INFO:tasks.workunit.client.0.smithi040.stdout:7/354: creat d21/d4d/f6f x:0 0 0 2024-02-16T19:24:08.977 INFO:tasks.workunit.client.0.smithi040.stdout:3/469: mkdir d1/d8c 0 2024-02-16T19:24:08.980 INFO:tasks.workunit.client.0.smithi040.stdout:1/409: rmdir d17/d26/d4c/d86 0 2024-02-16T19:24:08.980 INFO:tasks.workunit.client.0.smithi040.stdout:1/410: write d17/d26/d5f/f84 [982396,43539] 0 2024-02-16T19:24:08.980 INFO:tasks.workunit.client.0.smithi040.stdout:1/411: read d17/d26/f55 [265238,124588] 0 2024-02-16T19:24:08.985 INFO:tasks.workunit.client.0.smithi040.stdout:3/470: dread d1/d7/f2b [0,4194304] 0 2024-02-16T19:24:08.986 INFO:tasks.workunit.client.0.smithi040.stdout:3/471: mknod d1/dc/d35/d53/d64/d65/c8d 0 2024-02-16T19:24:08.987 INFO:tasks.workunit.client.0.smithi040.stdout:6/375: truncate d0/d1/d12/d3b/d48/f38 1212562 0 2024-02-16T19:24:08.987 INFO:tasks.workunit.client.0.smithi040.stdout:6/376: chown d0/d1 1952151 1 2024-02-16T19:24:08.988 INFO:tasks.workunit.client.0.smithi040.stdout:3/472: mkdir d1/d8c/d8e 0 2024-02-16T19:24:08.988 INFO:tasks.workunit.client.0.smithi040.stdout:3/473: chown d1/d7/c21 3 1 2024-02-16T19:24:08.988 INFO:tasks.workunit.client.0.smithi040.stdout:3/474: chown d1/d7/d33/f5e 5094 1 2024-02-16T19:24:08.988 INFO:tasks.workunit.client.0.smithi040.stdout:3/475: write d1/dc/d35/d55/f74 [800588,95451] 0 2024-02-16T19:24:08.991 INFO:tasks.workunit.client.0.smithi040.stdout:8/423: dwrite d6/d7/d1d/f7a [0,4194304] 0 2024-02-16T19:24:08.992 INFO:tasks.workunit.client.0.smithi040.stdout:8/424: creat d6/d2b/f80 x:0 0 0 2024-02-16T19:24:08.992 INFO:tasks.workunit.client.0.smithi040.stdout:8/425: readlink d6/l62 0 2024-02-16T19:24:08.992 INFO:tasks.workunit.client.0.smithi040.stdout:8/426: write d6/f24 [98262,96582] 0 2024-02-16T19:24:08.992 INFO:tasks.workunit.client.0.smithi040.stdout:8/427: dread - d6/d2b/f80 zero size 2024-02-16T19:24:08.996 INFO:tasks.workunit.client.0.smithi040.stdout:1/412: dread d17/d26/d5f/d63/f66 [0,4194304] 0 2024-02-16T19:24:08.997 INFO:tasks.workunit.client.0.smithi040.stdout:1/413: write d17/d26/f4b [172002,37846] 0 2024-02-16T19:24:08.999 INFO:tasks.workunit.client.0.smithi040.stdout:8/428: mkdir d6/d64/d50/d69/d81 0 2024-02-16T19:24:09.000 INFO:tasks.workunit.client.0.smithi040.stdout:8/429: readlink d6/lf 0 2024-02-16T19:24:09.000 INFO:tasks.workunit.client.0.smithi040.stdout:8/430: chown d6/d12/f5a 20257 1 2024-02-16T19:24:09.000 INFO:tasks.workunit.client.0.smithi040.stdout:8/431: creat d6/d2b/d44/f82 x:0 0 0 2024-02-16T19:24:09.001 INFO:tasks.workunit.client.0.smithi040.stdout:8/432: mknod d6/d2b/d36/c83 0 2024-02-16T19:24:09.001 INFO:tasks.workunit.client.0.smithi040.stdout:8/433: creat d6/d64/d50/f84 x:0 0 0 2024-02-16T19:24:09.005 INFO:tasks.workunit.client.0.smithi040.stdout:4/323: rename f0 to dd/de/d10/d13/d32/f64 0 2024-02-16T19:24:09.006 INFO:tasks.workunit.client.0.smithi040.stdout:9/428: rename d3/d4/d55/l2f to d3/d4/d11/d29/d36/l87 0 2024-02-16T19:24:09.006 INFO:tasks.workunit.client.0.smithi040.stdout:9/429: dread - d3/d4/f79 zero size 2024-02-16T19:24:09.007 INFO:tasks.workunit.client.0.smithi040.stdout:4/324: symlink dd/d58/l65 0 2024-02-16T19:24:09.008 INFO:tasks.workunit.client.0.smithi040.stdout:3/476: dwrite d1/f34 [4194304,4194304] 0 2024-02-16T19:24:09.010 INFO:tasks.workunit.client.0.smithi040.stdout:2/331: rename d5/d7/d1e/d31/d4f/d63/f45 to d5/d8/f69 0 2024-02-16T19:24:09.011 INFO:tasks.workunit.client.0.smithi040.stdout:4/325: rename dd/f23 to dd/f66 0 2024-02-16T19:24:09.013 INFO:tasks.workunit.client.0.smithi040.stdout:1/414: write f5 [3605686,73872] 0 2024-02-16T19:24:09.013 INFO:tasks.workunit.client.0.smithi040.stdout:3/477: symlink d1/dc/d42/d4a/l8f 0 2024-02-16T19:24:09.014 INFO:tasks.workunit.client.0.smithi040.stdout:7/355: getdents d21/d29 0 2024-02-16T19:24:09.017 INFO:tasks.workunit.client.0.smithi040.stdout:6/377: rename d0/d37/l62 to d0/d30/l78 0 2024-02-16T19:24:09.017 INFO:tasks.workunit.client.0.smithi040.stdout:6/378: readlink d0/d1/d12/d3b/d48/d4d/l6e 0 2024-02-16T19:24:09.018 INFO:tasks.workunit.client.0.smithi040.stdout:4/326: creat dd/de/d45/d37/f67 x:0 0 0 2024-02-16T19:24:09.019 INFO:tasks.workunit.client.0.smithi040.stdout:2/332: dread d5/f3c [0,4194304] 0 2024-02-16T19:24:09.019 INFO:tasks.workunit.client.0.smithi040.stdout:1/415: unlink f8 0 2024-02-16T19:24:09.019 INFO:tasks.workunit.client.0.smithi040.stdout:8/434: dwrite d6/d12/d22/f68 [0,4194304] 0 2024-02-16T19:24:09.020 INFO:tasks.workunit.client.0.smithi040.stdout:3/478: unlink d1/dc/l1c 0 2024-02-16T19:24:09.022 INFO:tasks.workunit.client.0.smithi040.stdout:6/379: mkdir d0/d1/d12/d3b/d79 0 2024-02-16T19:24:09.023 INFO:tasks.workunit.client.0.smithi040.stdout:4/327: truncate dd/de/d10/d13/d32/f64 2309951 0 2024-02-16T19:24:09.024 INFO:tasks.workunit.client.0.smithi040.stdout:2/333: creat d5/d7/d12/f6a x:0 0 0 2024-02-16T19:24:09.028 INFO:tasks.workunit.client.0.smithi040.stdout:3/479: creat d1/dc/d35/d48/d85/f90 x:0 0 0 2024-02-16T19:24:09.028 INFO:tasks.workunit.client.0.smithi040.stdout:3/480: creat d1/d4/d6e/f91 x:0 0 0 2024-02-16T19:24:09.028 INFO:tasks.workunit.client.0.smithi040.stdout:3/481: stat d1/d7/d36/f87 0 2024-02-16T19:24:09.028 INFO:tasks.workunit.client.0.smithi040.stdout:3/482: chown d1/dc 150 1 2024-02-16T19:24:09.029 INFO:tasks.workunit.client.0.smithi040.stdout:7/356: symlink d21/l70 0 2024-02-16T19:24:09.030 INFO:tasks.workunit.client.0.smithi040.stdout:6/380: mknod d0/c7a 0 2024-02-16T19:24:09.032 INFO:tasks.workunit.client.0.smithi040.stdout:2/334: truncate d5/d7/d1e/f3b 116442 0 2024-02-16T19:24:09.033 INFO:tasks.workunit.client.0.smithi040.stdout:1/416: dread d17/d2d/f2f [4194304,4194304] 0 2024-02-16T19:24:09.035 INFO:tasks.workunit.client.0.smithi040.stdout:6/381: symlink d0/d1/d12/d3b/d48/d73/l7b 0 2024-02-16T19:24:09.036 INFO:tasks.workunit.client.0.smithi040.stdout:3/483: getdents d1/d4 0 2024-02-16T19:24:09.036 INFO:tasks.workunit.client.0.smithi040.stdout:3/484: getdents d1/d4 0 2024-02-16T19:24:09.037 INFO:tasks.workunit.client.0.smithi040.stdout:3/485: truncate d1/dc/d42/d4a/f61 1453088 0 2024-02-16T19:24:09.037 INFO:tasks.workunit.client.0.smithi040.stdout:3/486: dread - d1/dc/d35/d48/f78 zero size 2024-02-16T19:24:09.037 INFO:tasks.workunit.client.0.smithi040.stdout:3/487: creat d1/d7/d30/f92 x:0 0 0 2024-02-16T19:24:09.041 INFO:tasks.workunit.client.0.smithi040.stdout:3/488: stat d1/d4/c8 0 2024-02-16T19:24:09.044 INFO:tasks.workunit.client.0.smithi040.stdout:7/357: dread d21/d4d/f66 [0,4194304] 0 2024-02-16T19:24:09.044 INFO:tasks.workunit.client.0.smithi040.stdout:7/358: write d21/f32 [699389,45873] 0 2024-02-16T19:24:09.046 INFO:tasks.workunit.client.0.smithi040.stdout:7/359: rename d21/d29/c52 to d21/d4d/c71 0 2024-02-16T19:24:09.046 INFO:tasks.workunit.client.0.smithi040.stdout:7/360: stat d21/d29/f59 0 2024-02-16T19:24:09.048 INFO:tasks.workunit.client.0.smithi040.stdout:7/361: mknod d21/c72 0 2024-02-16T19:24:09.048 INFO:tasks.workunit.client.0.smithi040.stdout:7/362: write d21/d4d/f6f [879848,44750] 0 2024-02-16T19:24:09.049 INFO:tasks.workunit.client.0.smithi040.stdout:7/363: creat d21/d29/f73 x:0 0 0 2024-02-16T19:24:09.050 INFO:tasks.workunit.client.0.smithi040.stdout:7/364: write d21/d4d/f6f [478880,91028] 0 2024-02-16T19:24:09.050 INFO:tasks.workunit.client.0.smithi040.stdout:7/365: chown d21/d29/f4c 511723132 1 2024-02-16T19:24:09.050 INFO:tasks.workunit.client.0.smithi040.stdout:7/366: write f1d [1866744,51992] 0 2024-02-16T19:24:09.051 INFO:tasks.workunit.client.0.smithi040.stdout:7/367: unlink l18 0 2024-02-16T19:24:09.059 INFO:tasks.workunit.client.0.smithi040.stdout:5/334: sync 2024-02-16T19:24:09.059 INFO:tasks.workunit.client.0.smithi040.stdout:8/435: dwrite d6/d12/d22/f68 [0,4194304] 0 2024-02-16T19:24:09.060 INFO:tasks.workunit.client.0.smithi040.stdout:4/328: dwrite dd/de/d10/d13/f15 [0,4194304] 0 2024-02-16T19:24:09.061 INFO:tasks.workunit.client.0.smithi040.stdout:4/329: chown dd/de/d10/f20 54946151 1 2024-02-16T19:24:09.061 INFO:tasks.workunit.client.0.smithi040.stdout:5/335: unlink f0 0 2024-02-16T19:24:09.064 INFO:tasks.workunit.client.0.smithi040.stdout:4/330: mkdir dd/d58/d68 0 2024-02-16T19:24:09.064 INFO:tasks.workunit.client.0.smithi040.stdout:1/417: dwrite d17/d3c/d74/f7f [0,4194304] 0 2024-02-16T19:24:09.065 INFO:tasks.workunit.client.0.smithi040.stdout:5/336: mkdir d1/d62 0 2024-02-16T19:24:09.067 INFO:tasks.workunit.client.0.smithi040.stdout:8/436: rename f4 to d6/d7/d15/f85 0 2024-02-16T19:24:09.067 INFO:tasks.workunit.client.0.smithi040.stdout:8/437: truncate d6/d64/f6d 2310872 0 2024-02-16T19:24:09.067 INFO:tasks.workunit.client.0.smithi040.stdout:8/438: creat d6/d12/d22/f86 x:0 0 0 2024-02-16T19:24:09.068 INFO:tasks.workunit.client.0.smithi040.stdout:4/331: creat dd/d58/d5e/f69 x:0 0 0 2024-02-16T19:24:09.068 INFO:tasks.workunit.client.0.smithi040.stdout:4/332: chown dd/de/d45/d37/c3a 7394 1 2024-02-16T19:24:09.070 INFO:tasks.workunit.client.0.smithi040.stdout:6/382: rmdir d0/d1/d12/d3b/d48/d73 39 2024-02-16T19:24:09.071 INFO:tasks.workunit.client.0.smithi040.stdout:7/368: dwrite d21/d29/f73 [0,4194304] 0 2024-02-16T19:24:09.076 INFO:tasks.workunit.client.0.smithi040.stdout:1/418: rename d17/d43/l60 to d17/d26/d5f/d78/l88 0 2024-02-16T19:24:09.077 INFO:tasks.workunit.client.0.smithi040.stdout:8/439: symlink d6/d64/d50/l87 0 2024-02-16T19:24:09.079 INFO:tasks.workunit.client.0.smithi040.stdout:4/333: link dd/d58/l65 dd/de/d10/d13/d32/l6a 0 2024-02-16T19:24:09.080 INFO:tasks.workunit.client.0.smithi040.stdout:6/383: symlink d0/d37/d4a/l7c 0 2024-02-16T19:24:09.084 INFO:tasks.workunit.client.0.smithi040.stdout:1/419: symlink d17/d43/d38/l89 0 2024-02-16T19:24:09.087 INFO:tasks.workunit.client.0.smithi040.stdout:8/440: mknod d6/d64/c88 0 2024-02-16T19:24:09.089 INFO:tasks.workunit.client.0.smithi040.stdout:4/334: creat dd/de/f6b x:0 0 0 2024-02-16T19:24:09.092 INFO:tasks.workunit.client.0.smithi040.stdout:8/441: creat d6/d12/d51/f89 x:0 0 0 2024-02-16T19:24:09.093 INFO:tasks.workunit.client.0.smithi040.stdout:8/442: truncate d6/d7/d1d/d38/f53 663298 0 2024-02-16T19:24:09.094 INFO:tasks.workunit.client.0.smithi040.stdout:4/335: creat dd/de/d10/d36/f6c x:0 0 0 2024-02-16T19:24:09.094 INFO:tasks.workunit.client.0.smithi040.stdout:4/336: chown dd/de/d45/l4d 80 1 2024-02-16T19:24:09.096 INFO:tasks.workunit.client.0.smithi040.stdout:5/337: rename d1/da/d1c/d21/d57 to d1/d45/d63 0 2024-02-16T19:24:09.098 INFO:tasks.workunit.client.0.smithi040.stdout:8/443: getdents d6/d2b/d36 0 2024-02-16T19:24:09.101 INFO:tasks.workunit.client.0.smithi040.stdout:4/337: unlink dd/de/d10/f47 0 2024-02-16T19:24:09.101 INFO:tasks.workunit.client.0.smithi040.stdout:4/338: chown dd/de/f5d 25942516 1 2024-02-16T19:24:09.101 INFO:tasks.workunit.client.0.smithi040.stdout:4/339: fdatasync dd/de/f5d 0 2024-02-16T19:24:09.102 INFO:tasks.workunit.client.0.smithi040.stdout:7/369: rename d21/d29/f3f to d21/d4d/f74 0 2024-02-16T19:24:09.103 INFO:tasks.workunit.client.0.smithi040.stdout:1/420: dwrite d17/d26/f53 [0,4194304] 0 2024-02-16T19:24:09.103 INFO:tasks.workunit.client.0.smithi040.stdout:1/421: creat d17/d26/d4c/f8a x:0 0 0 2024-02-16T19:24:09.104 INFO:tasks.workunit.client.0.smithi040.stdout:5/338: truncate d1/d3c/f43 3696302 0 2024-02-16T19:24:09.105 INFO:tasks.workunit.client.0.smithi040.stdout:7/370: rename d21 to d21/d75 22 2024-02-16T19:24:09.108 INFO:tasks.workunit.client.0.smithi040.stdout:5/339: creat d1/da/d1c/d21/f64 x:0 0 0 2024-02-16T19:24:09.110 INFO:tasks.workunit.client.0.smithi040.stdout:8/444: dwrite d6/d12/d51/f89 [0,4194304] 0 2024-02-16T19:24:09.110 INFO:tasks.workunit.client.0.smithi040.stdout:8/445: fdatasync d6/f76 0 2024-02-16T19:24:09.114 INFO:tasks.workunit.client.0.smithi040.stdout:7/371: link d21/f48 d21/f76 0 2024-02-16T19:24:09.115 INFO:tasks.workunit.client.0.smithi040.stdout:4/340: dread dd/f43 [0,4194304] 0 2024-02-16T19:24:09.119 INFO:tasks.workunit.client.0.smithi040.stdout:5/340: rename d1/da/f3e to d1/da/d1c/f65 0 2024-02-16T19:24:09.119 INFO:tasks.workunit.client.0.smithi040.stdout:5/341: write d1/f24 [1801713,118781] 0 2024-02-16T19:24:09.119 INFO:tasks.workunit.client.0.smithi040.stdout:5/342: write d1/da/f30 [646983,9372] 0 2024-02-16T19:24:09.122 INFO:tasks.workunit.client.0.smithi040.stdout:7/372: mkdir d21/d4d/d77 0 2024-02-16T19:24:09.123 INFO:tasks.workunit.client.0.smithi040.stdout:7/373: stat f11 0 2024-02-16T19:24:09.124 INFO:tasks.workunit.client.0.smithi040.stdout:8/446: dwrite d6/d12/f4b [0,4194304] 0 2024-02-16T19:24:09.126 INFO:tasks.workunit.client.0.smithi040.stdout:5/343: mknod d1/d3c/c66 0 2024-02-16T19:24:09.126 INFO:tasks.workunit.client.0.smithi040.stdout:7/374: symlink d21/d29/l78 0 2024-02-16T19:24:09.128 INFO:tasks.workunit.client.0.smithi040.stdout:4/341: getdents dd/de 0 2024-02-16T19:24:09.131 INFO:tasks.workunit.client.0.smithi040.stdout:8/447: mknod d6/d2b/d44/c8a 0 2024-02-16T19:24:09.132 INFO:tasks.workunit.client.0.smithi040.stdout:5/344: mknod d1/d3c/c67 0 2024-02-16T19:24:09.134 INFO:tasks.workunit.client.0.smithi040.stdout:7/375: mkdir d21/d29/d79 0 2024-02-16T19:24:09.135 INFO:tasks.workunit.client.0.smithi040.stdout:8/448: mkdir d6/d7/d1d/d38/d8b 0 2024-02-16T19:24:09.137 INFO:tasks.workunit.client.0.smithi040.stdout:5/345: creat d1/d3c/f68 x:0 0 0 2024-02-16T19:24:09.137 INFO:tasks.workunit.client.0.smithi040.stdout:5/346: chown d1/f17 1167292981 1 2024-02-16T19:24:09.138 INFO:tasks.workunit.client.0.smithi040.stdout:7/376: unlink c6 0 2024-02-16T19:24:09.139 INFO:tasks.workunit.client.0.smithi040.stdout:8/449: getdents d6/d64 0 2024-02-16T19:24:09.139 INFO:tasks.workunit.client.0.smithi040.stdout:8/450: creat d6/d64/d50/f8c x:0 0 0 2024-02-16T19:24:09.140 INFO:tasks.workunit.client.0.smithi040.stdout:5/347: mknod d1/d45/d63/c69 0 2024-02-16T19:24:09.141 INFO:tasks.workunit.client.0.smithi040.stdout:4/342: dwrite dd/de/d10/f20 [0,4194304] 0 2024-02-16T19:24:09.143 INFO:tasks.workunit.client.0.smithi040.stdout:4/343: creat dd/de/d10/f6d x:0 0 0 2024-02-16T19:24:09.144 INFO:tasks.workunit.client.0.smithi040.stdout:4/344: creat dd/f6e x:0 0 0 2024-02-16T19:24:09.146 INFO:tasks.workunit.client.0.smithi040.stdout:4/345: symlink dd/de/l6f 0 2024-02-16T19:24:09.148 INFO:tasks.workunit.client.0.smithi040.stdout:4/346: symlink dd/de/d10/d36/l70 0 2024-02-16T19:24:09.151 INFO:tasks.workunit.client.0.smithi040.stdout:7/377: dread f16 [0,4194304] 0 2024-02-16T19:24:09.151 INFO:tasks.workunit.client.0.smithi040.stdout:5/348: dread d1/fd [0,4194304] 0 2024-02-16T19:24:09.152 INFO:tasks.workunit.client.0.smithi040.stdout:5/349: stat d1/da/d1c/f65 0 2024-02-16T19:24:09.152 INFO:tasks.workunit.client.0.smithi040.stdout:8/451: dread d6/d2b/f48 [0,4194304] 0 2024-02-16T19:24:09.152 INFO:tasks.workunit.client.0.smithi040.stdout:8/452: creat d6/d2b/d44/f8d x:0 0 0 2024-02-16T19:24:09.153 INFO:tasks.workunit.client.0.smithi040.stdout:0/444: sync 2024-02-16T19:24:09.153 INFO:tasks.workunit.client.0.smithi040.stdout:0/445: write db/d25/f4f [2714214,58400] 0 2024-02-16T19:24:09.157 INFO:tasks.workunit.client.0.smithi040.stdout:8/453: symlink d6/d64/d50/l8e 0 2024-02-16T19:24:09.159 INFO:tasks.workunit.client.0.smithi040.stdout:0/446: creat db/d2f/d23/f9a x:0 0 0 2024-02-16T19:24:09.160 INFO:tasks.workunit.client.0.smithi040.stdout:7/378: mkdir d21/d4d/d77/d7a 0 2024-02-16T19:24:09.161 INFO:tasks.workunit.client.0.smithi040.stdout:5/350: rename d1/d45/c59 to d1/da/d1c/c6a 0 2024-02-16T19:24:09.164 INFO:tasks.workunit.client.0.smithi040.stdout:8/454: creat d6/d12/d51/d6e/f8f x:0 0 0 2024-02-16T19:24:09.166 INFO:tasks.workunit.client.0.smithi040.stdout:5/351: symlink d1/d62/l6b 0 2024-02-16T19:24:09.166 INFO:tasks.workunit.client.0.smithi040.stdout:5/352: fdatasync d1/d3c/f3f 0 2024-02-16T19:24:09.166 INFO:tasks.workunit.client.0.smithi040.stdout:5/353: readlink d1/da/d1c/d21/l47 0 2024-02-16T19:24:09.166 INFO:tasks.workunit.client.0.smithi040.stdout:5/354: write d1/f4c [361757,16400] 0 2024-02-16T19:24:09.166 INFO:tasks.workunit.client.0.smithi040.stdout:5/355: creat d1/da/d51/f6c x:0 0 0 2024-02-16T19:24:09.167 INFO:tasks.workunit.client.0.smithi040.stdout:4/347: dwrite dd/d58/d5e/f61 [0,4194304] 0 2024-02-16T19:24:09.167 INFO:tasks.workunit.client.0.smithi040.stdout:8/455: rename d6/d2b/d44/f82 to d6/d7/d1d/d38/d8b/f90 0 2024-02-16T19:24:09.170 INFO:tasks.workunit.client.0.smithi040.stdout:4/348: mkdir dd/de/d71 0 2024-02-16T19:24:09.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/349: symlink dd/de/d45/d37/l72 0 2024-02-16T19:24:09.173 INFO:tasks.workunit.client.0.smithi040.stdout:4/350: getdents dd/d58 0 2024-02-16T19:24:09.174 INFO:tasks.workunit.client.0.smithi040.stdout:4/351: rmdir dd/de/d45 39 2024-02-16T19:24:09.176 INFO:tasks.workunit.client.0.smithi040.stdout:4/352: truncate dd/de/d10/f55 2562196 0 2024-02-16T19:24:09.176 INFO:tasks.workunit.client.0.smithi040.stdout:4/353: write dd/de/f5d [898091,38208] 0 2024-02-16T19:24:09.176 INFO:tasks.workunit.client.0.smithi040.stdout:4/354: chown dd/d58/d68 723 1 2024-02-16T19:24:09.190 INFO:tasks.workunit.client.0.smithi040.stdout:0/447: dwrite db/d2f/f66 [0,4194304] 0 2024-02-16T19:24:09.190 INFO:tasks.workunit.client.0.smithi040.stdout:0/448: fdatasync db/de/d6a/f6e 0 2024-02-16T19:24:09.190 INFO:tasks.workunit.client.0.smithi040.stdout:5/356: dwrite d1/d45/f52 [4194304,4194304] 0 2024-02-16T19:24:09.191 INFO:tasks.workunit.client.0.smithi040.stdout:8/456: dwrite d6/d7/d15/f47 [4194304,4194304] 0 2024-02-16T19:24:09.193 INFO:tasks.workunit.client.0.smithi040.stdout:0/449: creat db/d2f/d23/d6f/d98/f9b x:0 0 0 2024-02-16T19:24:09.196 INFO:tasks.workunit.client.0.smithi040.stdout:5/357: rename d1/c11 to d1/d3c/c6d 0 2024-02-16T19:24:09.201 INFO:tasks.workunit.client.0.smithi040.stdout:8/457: write d6/d7/d1d/f7a [310671,15052] 0 2024-02-16T19:24:09.201 INFO:tasks.workunit.client.0.smithi040.stdout:0/450: mknod db/d25/d34/c9c 0 2024-02-16T19:24:09.201 INFO:tasks.workunit.client.0.smithi040.stdout:0/451: readlink db/d2f/d23/d3b/l4c 0 2024-02-16T19:24:09.201 INFO:tasks.workunit.client.0.smithi040.stdout:8/458: creat d6/d2b/d44/f91 x:0 0 0 2024-02-16T19:24:09.203 INFO:tasks.workunit.client.0.smithi040.stdout:8/459: truncate d6/f8 3270534 0 2024-02-16T19:24:09.203 INFO:tasks.workunit.client.0.smithi040.stdout:0/452: dread db/f20 [0,4194304] 0 2024-02-16T19:24:09.205 INFO:tasks.workunit.client.0.smithi040.stdout:5/358: truncate d1/da/d1c/f25 3197931 0 2024-02-16T19:24:09.205 INFO:tasks.workunit.client.0.smithi040.stdout:5/359: write d1/f2c [4597260,61586] 0 2024-02-16T19:24:09.205 INFO:tasks.workunit.client.0.smithi040.stdout:0/453: read db/d2f/d1d/f46 [875729,130278] 0 2024-02-16T19:24:09.205 INFO:tasks.workunit.client.0.smithi040.stdout:0/454: creat db/d2f/d1d/d21/d2e/d49/d5b/f9d x:0 0 0 2024-02-16T19:24:09.205 INFO:tasks.workunit.client.0.smithi040.stdout:0/455: stat db/d2f/d23/d56 0 2024-02-16T19:24:09.205 INFO:tasks.workunit.client.0.smithi040.stdout:0/456: fdatasync db/d2f/d1d/d21/d2e/d85/f86 0 2024-02-16T19:24:09.205 INFO:tasks.workunit.client.0.smithi040.stdout:0/457: creat db/d25/d34/f9e x:0 0 0 2024-02-16T19:24:09.205 INFO:tasks.workunit.client.0.smithi040.stdout:8/460: truncate d6/d2b/d36/f4f 27855 0 2024-02-16T19:24:09.207 INFO:tasks.workunit.client.0.smithi040.stdout:5/360: symlink d1/d45/d63/l6e 0 2024-02-16T19:24:09.210 INFO:tasks.workunit.client.0.smithi040.stdout:8/461: mknod d6/d12/c92 0 2024-02-16T19:24:09.212 INFO:tasks.workunit.client.0.smithi040.stdout:5/361: creat d1/d62/f6f x:0 0 0 2024-02-16T19:24:09.212 INFO:tasks.workunit.client.0.smithi040.stdout:5/362: fdatasync d1/da/d1c/d21/f42 0 2024-02-16T19:24:09.212 INFO:tasks.workunit.client.0.smithi040.stdout:5/363: chown d1/c34 17674031 1 2024-02-16T19:24:09.214 INFO:tasks.workunit.client.0.smithi040.stdout:8/462: mkdir d6/d12/d93 0 2024-02-16T19:24:09.214 INFO:tasks.workunit.client.0.smithi040.stdout:9/430: sync 2024-02-16T19:24:09.214 INFO:tasks.workunit.client.0.smithi040.stdout:2/335: sync 2024-02-16T19:24:09.214 INFO:tasks.workunit.client.0.smithi040.stdout:2/336: creat d5/d7/d1e/d4c/d52/f6b x:0 0 0 2024-02-16T19:24:09.214 INFO:tasks.workunit.client.0.smithi040.stdout:8/463: read d6/d7/f2e [177169,98225] 0 2024-02-16T19:24:09.215 INFO:tasks.workunit.client.0.smithi040.stdout:5/364: unlink d1/da/f30 0 2024-02-16T19:24:09.216 INFO:tasks.workunit.client.0.smithi040.stdout:5/365: chown d1/fd 100819 1 2024-02-16T19:24:09.217 INFO:tasks.workunit.client.0.smithi040.stdout:9/431: creat d3/d4/d11/d71/f88 x:0 0 0 2024-02-16T19:24:09.218 INFO:tasks.workunit.client.0.smithi040.stdout:2/337: creat d5/d7/d1e/d55/f6c x:0 0 0 2024-02-16T19:24:09.219 INFO:tasks.workunit.client.0.smithi040.stdout:0/458: getdents db/d2f/d1d/d21/d2e/d49/d5b 0 2024-02-16T19:24:09.219 INFO:tasks.workunit.client.0.smithi040.stdout:0/459: fsync db/fd 0 2024-02-16T19:24:09.219 INFO:tasks.workunit.client.0.smithi040.stdout:0/460: stat db/d2f/d1d/d21/d2e/d85/f86 0 2024-02-16T19:24:09.219 INFO:tasks.workunit.client.0.smithi040.stdout:0/461: chown db/de/f75 0 1 2024-02-16T19:24:09.221 INFO:tasks.workunit.client.0.smithi040.stdout:8/464: rename d6/d12/f31 to d6/d7/d1d/d38/d8b/f94 0 2024-02-16T19:24:09.221 INFO:tasks.workunit.client.0.smithi040.stdout:8/465: chown d6/d12/d22/f4e 403606607 1 2024-02-16T19:24:09.221 INFO:tasks.workunit.client.0.smithi040.stdout:8/466: readlink d6/d12/l2a 0 2024-02-16T19:24:09.222 INFO:tasks.workunit.client.0.smithi040.stdout:2/338: creat d5/d7/d1e/f6d x:0 0 0 2024-02-16T19:24:09.254 INFO:tasks.workunit.client.0.smithi040.stdout:5/366: dwrite d1/da/d1c/f65 [0,4194304] 0 2024-02-16T19:24:09.254 INFO:tasks.workunit.client.0.smithi040.stdout:2/339: fsync d5/d7/d1e/d55/f6c 0 2024-02-16T19:24:09.254 INFO:tasks.workunit.client.0.smithi040.stdout:2/340: dread - d5/d7/d1e/d55/f6c zero size 2024-02-16T19:24:09.255 INFO:tasks.workunit.client.0.smithi040.stdout:0/462: dwrite db/d2f/d1d/d81/f8c [0,4194304] 0 2024-02-16T19:24:09.255 INFO:tasks.workunit.client.0.smithi040.stdout:9/432: dwrite d3/d4/d55/d31/f35 [4194304,4194304] 0 2024-02-16T19:24:09.255 INFO:tasks.workunit.client.0.smithi040.stdout:8/467: dwrite d6/d7/d15/f47 [4194304,4194304] 0 2024-02-16T19:24:09.259 INFO:tasks.workunit.client.0.smithi040.stdout:0/463: link db/d2f/d23/d56/c96 db/d2f/d23/d93/c9f 0 2024-02-16T19:24:09.262 INFO:tasks.workunit.client.0.smithi040.stdout:0/464: symlink db/d25/d34/la0 0 2024-02-16T19:24:09.263 INFO:tasks.workunit.client.0.smithi040.stdout:0/465: chown db/de/d6a/f6e 1858250091 1 2024-02-16T19:24:09.265 INFO:tasks.workunit.client.0.smithi040.stdout:0/466: symlink db/d2f/d1d/d21/d2e/d85/d89/la1 0 2024-02-16T19:24:09.268 INFO:tasks.workunit.client.0.smithi040.stdout:1/422: sync 2024-02-16T19:24:09.268 INFO:tasks.workunit.client.0.smithi040.stdout:6/384: sync 2024-02-16T19:24:09.268 INFO:tasks.workunit.client.0.smithi040.stdout:3/489: sync 2024-02-16T19:24:09.269 INFO:tasks.workunit.client.0.smithi040.stdout:1/423: readlink d17/d43/l24 0 2024-02-16T19:24:09.269 INFO:tasks.workunit.client.0.smithi040.stdout:1/424: fdatasync d17/d3c/d74/f7f 0 2024-02-16T19:24:09.269 INFO:tasks.workunit.client.0.smithi040.stdout:1/425: truncate d17/d2d/f73 201424 0 2024-02-16T19:24:09.269 INFO:tasks.workunit.client.0.smithi040.stdout:1/426: truncate d17/d26/d3d/f83 781083 0 2024-02-16T19:24:09.269 INFO:tasks.workunit.client.0.smithi040.stdout:3/490: creat d1/dc/d42/d58/f93 x:0 0 0 2024-02-16T19:24:09.269 INFO:tasks.workunit.client.0.smithi040.stdout:5/367: read d1/da/d1c/f37 [513021,16692] 0 2024-02-16T19:24:09.269 INFO:tasks.workunit.client.0.smithi040.stdout:5/368: creat d1/da/f70 x:0 0 0 2024-02-16T19:24:09.270 INFO:tasks.workunit.client.0.smithi040.stdout:6/385: mkdir d0/d1/d12/d3b/d48/d73/d76/d7d 0 2024-02-16T19:24:09.273 INFO:tasks.workunit.client.0.smithi040.stdout:1/427: mkdir d17/d29/d8b 0 2024-02-16T19:24:09.273 INFO:tasks.workunit.client.0.smithi040.stdout:1/428: write d17/d2d/f2e [5419529,118848] 0 2024-02-16T19:24:09.274 INFO:tasks.workunit.client.0.smithi040.stdout:5/369: symlink d1/da/l71 0 2024-02-16T19:24:09.275 INFO:tasks.workunit.client.0.smithi040.stdout:6/386: mknod d0/d37/c7e 0 2024-02-16T19:24:09.276 INFO:tasks.workunit.client.0.smithi040.stdout:1/429: creat d17/d43/d38/d4a/f8c x:0 0 0 2024-02-16T19:24:09.278 INFO:tasks.workunit.client.0.smithi040.stdout:5/370: link d1/da/d1c/d21/f64 d1/d3c/f72 0 2024-02-16T19:24:09.281 INFO:tasks.workunit.client.0.smithi040.stdout:5/371: mkdir d1/da/d1c/d73 0 2024-02-16T19:24:09.282 INFO:tasks.workunit.client.0.smithi040.stdout:5/372: creat d1/d45/d63/f74 x:0 0 0 2024-02-16T19:24:09.285 INFO:tasks.workunit.client.0.smithi040.stdout:5/373: mkdir d1/d3c/d75 0 2024-02-16T19:24:09.287 INFO:tasks.workunit.client.0.smithi040.stdout:5/374: getdents d1/d45 0 2024-02-16T19:24:09.289 INFO:tasks.workunit.client.0.smithi040.stdout:0/467: read db/d25/d2c/d52/f91 [3499399,93413] 0 2024-02-16T19:24:09.289 INFO:tasks.workunit.client.0.smithi040.stdout:5/375: mknod d1/da/d1c/d21/c76 0 2024-02-16T19:24:09.289 INFO:tasks.workunit.client.0.smithi040.stdout:5/376: creat d1/d3c/f77 x:0 0 0 2024-02-16T19:24:09.291 INFO:tasks.workunit.client.0.smithi040.stdout:5/377: mknod d1/d45/d63/c78 0 2024-02-16T19:24:09.292 INFO:tasks.workunit.client.0.smithi040.stdout:5/378: write d1/da/f70 [379324,99967] 0 2024-02-16T19:24:09.305 INFO:tasks.workunit.client.0.smithi040.stdout:2/341: dwrite d5/d7/d12/d1b/f30 [0,4194304] 0 2024-02-16T19:24:09.307 INFO:tasks.workunit.client.0.smithi040.stdout:2/342: symlink d5/l6e 0 2024-02-16T19:24:09.308 INFO:tasks.workunit.client.0.smithi040.stdout:2/343: rename d5/d7/d10/f4d to d5/d7/d1e/d55/f6f 0 2024-02-16T19:24:09.313 INFO:tasks.workunit.client.0.smithi040.stdout:2/344: dread d5/f3c [0,4194304] 0 2024-02-16T19:24:09.316 INFO:tasks.workunit.client.0.smithi040.stdout:0/468: dwrite db/d2f/d23/f9a [0,4194304] 0 2024-02-16T19:24:09.318 INFO:tasks.workunit.client.0.smithi040.stdout:0/469: creat db/de/fa2 x:0 0 0 2024-02-16T19:24:09.319 INFO:tasks.workunit.client.0.smithi040.stdout:0/470: creat db/d2f/d1d/d21/d2e/d49/fa3 x:0 0 0 2024-02-16T19:24:09.319 INFO:tasks.workunit.client.0.smithi040.stdout:9/433: dwrite d3/d4/d11/d29/f2b [0,4194304] 0 2024-02-16T19:24:09.319 INFO:tasks.workunit.client.0.smithi040.stdout:1/430: dwrite d17/d2d/f73 [0,4194304] 0 2024-02-16T19:24:09.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/468: dwrite d6/f76 [0,4194304] 0 2024-02-16T19:24:09.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/469: creat d6/d2b/f95 x:0 0 0 2024-02-16T19:24:09.320 INFO:tasks.workunit.client.0.smithi040.stdout:6/387: dwrite d0/d1/d12/f2c [0,4194304] 0 2024-02-16T19:24:09.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/470: read - d6/d2b/f63 zero size 2024-02-16T19:24:09.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/471: write d6/d7/f32 [1039438,123596] 0 2024-02-16T19:24:09.321 INFO:tasks.workunit.client.0.smithi040.stdout:6/388: dread d0/d1/d12/d3b/f50 [0,4194304] 0 2024-02-16T19:24:09.321 INFO:tasks.workunit.client.0.smithi040.stdout:5/379: dwrite d1/f17 [0,4194304] 0 2024-02-16T19:24:09.321 INFO:tasks.workunit.client.0.smithi040.stdout:3/491: dwrite d1/d4/fa [4194304,4194304] 0 2024-02-16T19:24:09.321 INFO:tasks.workunit.client.0.smithi040.stdout:3/492: fsync d1/d4/d6e/f91 0 2024-02-16T19:24:09.323 INFO:tasks.workunit.client.0.smithi040.stdout:1/431: creat d17/d29/d8b/f8d x:0 0 0 2024-02-16T19:24:09.324 INFO:tasks.workunit.client.0.smithi040.stdout:8/472: dread d6/d7/d15/f67 [0,4194304] 0 2024-02-16T19:24:09.324 INFO:tasks.workunit.client.0.smithi040.stdout:8/473: creat d6/d12/d51/d6e/f96 x:0 0 0 2024-02-16T19:24:09.324 INFO:tasks.workunit.client.0.smithi040.stdout:8/474: fsync d6/d7/f2e 0 2024-02-16T19:24:09.324 INFO:tasks.workunit.client.0.smithi040.stdout:8/475: creat d6/d12/d22/f97 x:0 0 0 2024-02-16T19:24:09.335 INFO:tasks.workunit.client.0.smithi040.stdout:5/380: chown d1/da/d1c/f26 377312424 1 2024-02-16T19:24:09.336 INFO:tasks.workunit.client.0.smithi040.stdout:5/381: write d1/f24 [253063,91104] 0 2024-02-16T19:24:09.336 INFO:tasks.workunit.client.0.smithi040.stdout:5/382: readlink d1/d62/l6b 0 2024-02-16T19:24:09.336 INFO:tasks.workunit.client.0.smithi040.stdout:5/383: stat d1/da/d1c/d21/l47 0 2024-02-16T19:24:09.336 INFO:tasks.workunit.client.0.smithi040.stdout:5/384: creat d1/da/d51/f79 x:0 0 0 2024-02-16T19:24:09.336 INFO:tasks.workunit.client.0.smithi040.stdout:1/432: write d17/d3c/d74/f7f [3729444,34489] 0 2024-02-16T19:24:09.337 INFO:tasks.workunit.client.0.smithi040.stdout:3/493: truncate d1/d4/f19 604210 0 2024-02-16T19:24:09.337 INFO:tasks.workunit.client.0.smithi040.stdout:3/494: write d1/dc/d35/d48/f78 [65534,102166] 0 2024-02-16T19:24:09.339 INFO:tasks.workunit.client.0.smithi040.stdout:6/389: dwrite d0/d30/d4b/f57 [0,4194304] 0 2024-02-16T19:24:09.339 INFO:tasks.workunit.client.0.smithi040.stdout:0/471: dwrite db/d2f/d1d/d21/f51 [0,4194304] 0 2024-02-16T19:24:09.340 INFO:tasks.workunit.client.0.smithi040.stdout:0/472: write db/d2f/d1d/f46 [403701,34361] 0 2024-02-16T19:24:09.340 INFO:tasks.workunit.client.0.smithi040.stdout:0/473: truncate db/d2f/d23/d6f/f95 1852764 0 2024-02-16T19:24:09.340 INFO:tasks.workunit.client.0.smithi040.stdout:9/434: link d3/d4/d11/d29/d36/c69 d3/d4/c89 0 2024-02-16T19:24:09.341 INFO:tasks.workunit.client.0.smithi040.stdout:9/435: dread - d3/d4/d55/d31/f4c zero size 2024-02-16T19:24:09.344 INFO:tasks.workunit.client.0.smithi040.stdout:8/476: rename d6/l35 to d6/d64/d50/l98 0 2024-02-16T19:24:09.344 INFO:tasks.workunit.client.0.smithi040.stdout:8/477: dread - d6/d2b/d44/f8d zero size 2024-02-16T19:24:09.346 INFO:tasks.workunit.client.0.smithi040.stdout:5/385: mknod d1/d45/d63/c7a 0 2024-02-16T19:24:09.348 INFO:tasks.workunit.client.0.smithi040.stdout:2/345: getdents d5/d7/d10 0 2024-02-16T19:24:09.348 INFO:tasks.workunit.client.0.smithi040.stdout:2/346: write d5/d5c/d65/f60 [923961,81586] 0 2024-02-16T19:24:09.348 INFO:tasks.workunit.client.0.smithi040.stdout:2/347: creat d5/d7/d1e/f70 x:0 0 0 2024-02-16T19:24:09.349 INFO:tasks.workunit.client.0.smithi040.stdout:1/433: mknod d17/d3c/d6b/c8e 0 2024-02-16T19:24:09.350 INFO:tasks.workunit.client.0.smithi040.stdout:1/434: stat d17/d26/f55 0 2024-02-16T19:24:09.350 INFO:tasks.workunit.client.0.smithi040.stdout:1/435: truncate d17/d2d/f81 548564 0 2024-02-16T19:24:09.352 INFO:tasks.workunit.client.0.smithi040.stdout:0/474: symlink db/d25/d34/la4 0 2024-02-16T19:24:09.354 INFO:tasks.workunit.client.0.smithi040.stdout:9/436: mkdir d3/d4/d55/d8a 0 2024-02-16T19:24:09.355 INFO:tasks.workunit.client.0.smithi040.stdout:6/390: rename d0/d1/d12/f47 to d0/d1/d12/d3b/d48/f7f 0 2024-02-16T19:24:09.356 INFO:tasks.workunit.client.0.smithi040.stdout:3/495: dwrite d1/d4/f25 [0,4194304] 0 2024-02-16T19:24:09.356 INFO:tasks.workunit.client.0.smithi040.stdout:8/478: creat d6/d64/d50/d69/d81/f99 x:0 0 0 2024-02-16T19:24:09.358 INFO:tasks.workunit.client.0.smithi040.stdout:2/348: link d5/d7/d1e/d31/l58 d5/l71 0 2024-02-16T19:24:09.361 INFO:tasks.workunit.client.0.smithi040.stdout:6/391: creat d0/d1/d12/d3b/d48/d73/d76/d7d/f80 x:0 0 0 2024-02-16T19:24:09.361 INFO:tasks.workunit.client.0.smithi040.stdout:7/379: sync 2024-02-16T19:24:09.361 INFO:tasks.workunit.client.0.smithi040.stdout:4/355: sync 2024-02-16T19:24:09.361 INFO:tasks.workunit.client.0.smithi040.stdout:9/437: write d3/d4/d11/d29/f24 [4989637,80861] 0 2024-02-16T19:24:09.363 INFO:tasks.workunit.client.0.smithi040.stdout:3/496: symlink d1/d4/d6e/d8a/l94 0 2024-02-16T19:24:09.363 INFO:tasks.workunit.client.0.smithi040.stdout:3/497: chown d1/dc/d42/d58/f76 332948 1 2024-02-16T19:24:09.364 INFO:tasks.workunit.client.0.smithi040.stdout:3/498: readlink d1/d7/d33/l22 0 2024-02-16T19:24:09.365 INFO:tasks.workunit.client.0.smithi040.stdout:8/479: link d6/f8 d6/d2b/f9a 0 2024-02-16T19:24:09.366 INFO:tasks.workunit.client.0.smithi040.stdout:8/480: creat d6/d7/f9b x:0 0 0 2024-02-16T19:24:09.370 INFO:tasks.workunit.client.0.smithi040.stdout:5/386: rename d1/da/d1c/f40 to d1/d45/d63/f7b 0 2024-02-16T19:24:09.373 INFO:tasks.workunit.client.0.smithi040.stdout:6/392: symlink d0/d1/d12/d3b/l81 0 2024-02-16T19:24:09.373 INFO:tasks.workunit.client.0.smithi040.stdout:3/499: symlink d1/l95 0 2024-02-16T19:24:09.375 INFO:tasks.workunit.client.0.smithi040.stdout:9/438: dread d3/d4/d11/d29/f2b [0,4194304] 0 2024-02-16T19:24:09.375 INFO:tasks.workunit.client.0.smithi040.stdout:9/439: chown d3/d4/d11/d29/d36/f3a 137263 1 2024-02-16T19:24:09.375 INFO:tasks.workunit.client.0.smithi040.stdout:2/349: getdents d5/d7/d1e/d55 0 2024-02-16T19:24:09.375 INFO:tasks.workunit.client.0.smithi040.stdout:0/475: dwrite db/d2f/d1d/d21/f51 [0,4194304] 0 2024-02-16T19:24:09.380 INFO:tasks.workunit.client.0.smithi040.stdout:1/436: rename d17/d2d/f33 to d17/d3c/d6b/f8f 0 2024-02-16T19:24:09.380 INFO:tasks.workunit.client.0.smithi040.stdout:5/387: symlink d1/da/d51/l7c 0 2024-02-16T19:24:09.387 INFO:tasks.workunit.client.0.smithi040.stdout:3/500: symlink d1/l96 0 2024-02-16T19:24:09.391 INFO:tasks.workunit.client.0.smithi040.stdout:2/350: symlink d5/d7/d1e/d4c/d52/l72 0 2024-02-16T19:24:09.393 INFO:tasks.workunit.client.0.smithi040.stdout:0/476: mkdir db/d2f/d23/d93/da5 0 2024-02-16T19:24:09.395 INFO:tasks.workunit.client.0.smithi040.stdout:6/393: rename d0/d30/l46 to d0/l82 0 2024-02-16T19:24:09.396 INFO:tasks.workunit.client.0.smithi040.stdout:5/388: mknod d1/d3c/c7d 0 2024-02-16T19:24:09.398 INFO:tasks.workunit.client.0.smithi040.stdout:3/501: unlink d1/dc/d35/d48/f78 0 2024-02-16T19:24:09.398 INFO:tasks.workunit.client.0.smithi040.stdout:3/502: truncate d1/f3 4783770 0 2024-02-16T19:24:09.398 INFO:tasks.workunit.client.0.smithi040.stdout:3/503: chown d1/d7/f4e 131120236 1 2024-02-16T19:24:09.398 INFO:tasks.workunit.client.0.smithi040.stdout:3/504: creat d1/d7/d36/f97 x:0 0 0 2024-02-16T19:24:09.398 INFO:tasks.workunit.client.0.smithi040.stdout:3/505: write d1/dc/f3c [1846632,15858] 0 2024-02-16T19:24:09.399 INFO:tasks.workunit.client.0.smithi040.stdout:2/351: unlink d5/d7/d1e/d31/l4b 0 2024-02-16T19:24:09.400 INFO:tasks.workunit.client.0.smithi040.stdout:0/477: link db/d2f/d1d/d21/l6c db/d2f/d1d/d81/la6 0 2024-02-16T19:24:09.401 INFO:tasks.workunit.client.0.smithi040.stdout:6/394: truncate d0/f35 1128202 0 2024-02-16T19:24:09.404 INFO:tasks.workunit.client.0.smithi040.stdout:1/437: rename d17/d26/f4b to d17/d26/d5f/d63/f90 0 2024-02-16T19:24:09.407 INFO:tasks.workunit.client.0.smithi040.stdout:3/506: link d1/d7/f4e d1/dc/d35/d53/d64/d65/f98 0 2024-02-16T19:24:09.407 INFO:tasks.workunit.client.0.smithi040.stdout:3/507: readlink d1/dc/d35/l49 0 2024-02-16T19:24:09.407 INFO:tasks.workunit.client.0.smithi040.stdout:3/508: truncate d1/d4/d6e/f84 73874 0 2024-02-16T19:24:09.408 INFO:tasks.workunit.client.0.smithi040.stdout:2/352: mknod d5/d5c/c73 0 2024-02-16T19:24:09.410 INFO:tasks.workunit.client.0.smithi040.stdout:1/438: symlink d17/d29/d8b/l91 0 2024-02-16T19:24:09.410 INFO:tasks.workunit.client.0.smithi040.stdout:3/509: read d1/dc/d42/d5c/f6d [162832,61741] 0 2024-02-16T19:24:09.412 INFO:tasks.workunit.client.0.smithi040.stdout:2/353: unlink d5/d7/d1e/f3b 0 2024-02-16T19:24:09.412 INFO:tasks.workunit.client.0.smithi040.stdout:2/354: write d5/f3a [267058,3445] 0 2024-02-16T19:24:09.414 INFO:tasks.workunit.client.0.smithi040.stdout:2/355: link d5/d7/d1e/d4c/d52/f6b d5/d7/f74 0 2024-02-16T19:24:09.414 INFO:tasks.workunit.client.0.smithi040.stdout:2/356: readlink d5/d7/l32 0 2024-02-16T19:24:09.416 INFO:tasks.workunit.client.0.smithi040.stdout:8/481: dwrite d6/f76 [0,4194304] 0 2024-02-16T19:24:09.416 INFO:tasks.workunit.client.0.smithi040.stdout:8/482: creat d6/d2b/f9c x:0 0 0 2024-02-16T19:24:09.416 INFO:tasks.workunit.client.0.smithi040.stdout:2/357: creat d5/d5c/d65/f75 x:0 0 0 2024-02-16T19:24:09.416 INFO:tasks.workunit.client.0.smithi040.stdout:2/358: write d5/d7/d1e/f6d [582738,16072] 0 2024-02-16T19:24:09.417 INFO:tasks.workunit.client.0.smithi040.stdout:1/439: rename d17/d43/d38/d4a to d17/d43/d92 0 2024-02-16T19:24:09.417 INFO:tasks.workunit.client.0.smithi040.stdout:1/440: truncate d17/d2d/f81 1180271 0 2024-02-16T19:24:09.417 INFO:tasks.workunit.client.0.smithi040.stdout:1/441: chown f14 109 1 2024-02-16T19:24:09.417 INFO:tasks.workunit.client.0.smithi040.stdout:1/442: write f14 [362320,33985] 0 2024-02-16T19:24:09.418 INFO:tasks.workunit.client.0.smithi040.stdout:8/483: creat d6/d7/d1d/d38/d8b/f9d x:0 0 0 2024-02-16T19:24:09.420 INFO:tasks.workunit.client.0.smithi040.stdout:3/510: rename d1/dc/l24 to d1/dc/d42/d58/l99 0 2024-02-16T19:24:09.420 INFO:tasks.workunit.client.0.smithi040.stdout:8/484: symlink d6/d12/d22/l9e 0 2024-02-16T19:24:09.420 INFO:tasks.workunit.client.0.smithi040.stdout:8/485: stat d6/d7/d1d/d38/d8b/f94 0 2024-02-16T19:24:09.421 INFO:tasks.workunit.client.0.smithi040.stdout:3/511: symlink d1/d4/d6e/d8a/l9a 0 2024-02-16T19:24:09.422 INFO:tasks.workunit.client.0.smithi040.stdout:7/380: dwrite d21/d4d/f66 [0,4194304] 0 2024-02-16T19:24:09.423 INFO:tasks.workunit.client.0.smithi040.stdout:3/512: mkdir d1/d4/d9b 0 2024-02-16T19:24:09.423 INFO:tasks.workunit.client.0.smithi040.stdout:7/381: readlink d21/d29/l2c 0 2024-02-16T19:24:09.426 INFO:tasks.workunit.client.0.smithi040.stdout:3/513: truncate d1/d7/d36/f7b 808389 0 2024-02-16T19:24:09.427 INFO:tasks.workunit.client.0.smithi040.stdout:3/514: symlink d1/d8c/l9c 0 2024-02-16T19:24:09.429 INFO:tasks.workunit.client.0.smithi040.stdout:1/443: dread f1 [0,4194304] 0 2024-02-16T19:24:09.432 INFO:tasks.workunit.client.0.smithi040.stdout:5/389: dwrite d1/d3c/f72 [0,4194304] 0 2024-02-16T19:24:09.433 INFO:tasks.workunit.client.0.smithi040.stdout:9/440: dwrite d3/d4/d55/f5a [0,4194304] 0 2024-02-16T19:24:09.433 INFO:tasks.workunit.client.0.smithi040.stdout:0/478: dwrite db/d2f/d1d/d21/f45 [0,4194304] 0 2024-02-16T19:24:09.433 INFO:tasks.workunit.client.0.smithi040.stdout:6/395: dwrite d0/d1/d12/d3b/d48/d4d/d6d/f71 [4194304,4194304] 0 2024-02-16T19:24:09.436 INFO:tasks.workunit.client.0.smithi040.stdout:1/444: dread f14 [0,4194304] 0 2024-02-16T19:24:09.436 INFO:tasks.workunit.client.0.smithi040.stdout:1/445: write d17/d3c/d71/f87 [679760,124760] 0 2024-02-16T19:24:09.439 INFO:tasks.workunit.client.0.smithi040.stdout:9/441: mkdir d3/d8b 0 2024-02-16T19:24:09.439 INFO:tasks.workunit.client.0.smithi040.stdout:9/442: stat d3/d4/d11/d29/l77 0 2024-02-16T19:24:09.439 INFO:tasks.workunit.client.0.smithi040.stdout:9/443: creat d3/d4/d4a/f8c x:0 0 0 2024-02-16T19:24:09.440 INFO:tasks.workunit.client.0.smithi040.stdout:9/444: dread - d3/d4/d55/d31/f4c zero size 2024-02-16T19:24:09.440 INFO:tasks.workunit.client.0.smithi040.stdout:9/445: fdatasync d3/d4/d11/d29/f56 0 2024-02-16T19:24:09.444 INFO:tasks.workunit.client.0.smithi040.stdout:3/515: dwrite d1/f3 [4194304,4194304] 0 2024-02-16T19:24:09.445 INFO:tasks.workunit.client.0.smithi040.stdout:6/396: mkdir d0/d1/d12/d3b/d48/d73/d70/d83 0 2024-02-16T19:24:09.448 INFO:tasks.workunit.client.0.smithi040.stdout:1/446: dread f12 [0,4194304] 0 2024-02-16T19:24:09.449 INFO:tasks.workunit.client.0.smithi040.stdout:1/447: truncate d17/f61 594739 0 2024-02-16T19:24:09.449 INFO:tasks.workunit.client.0.smithi040.stdout:5/390: read d1/da/f35 [784092,126259] 0 2024-02-16T19:24:09.449 INFO:tasks.workunit.client.0.smithi040.stdout:6/397: write d0/f33 [4116464,26394] 0 2024-02-16T19:24:09.452 INFO:tasks.workunit.client.0.smithi040.stdout:2/359: getdents d5/d7/d1e/d4c/d52 0 2024-02-16T19:24:09.453 INFO:tasks.workunit.client.0.smithi040.stdout:0/479: dwrite db/d25/f4f [0,4194304] 0 2024-02-16T19:24:09.454 INFO:tasks.workunit.client.0.smithi040.stdout:9/446: creat d3/d4/d11/d29/f8d x:0 0 0 2024-02-16T19:24:09.454 INFO:tasks.workunit.client.0.smithi040.stdout:9/447: fdatasync d3/d4/d4a/f8c 0 2024-02-16T19:24:09.454 INFO:tasks.workunit.client.0.smithi040.stdout:2/360: dread d5/d5c/d65/f60 [0,4194304] 0 2024-02-16T19:24:09.455 INFO:tasks.workunit.client.0.smithi040.stdout:2/361: write d5/d7/d1e/d55/f6f [1818022,119068] 0 2024-02-16T19:24:09.455 INFO:tasks.workunit.client.0.smithi040.stdout:1/448: truncate d17/f37 3977264 0 2024-02-16T19:24:09.455 INFO:tasks.workunit.client.0.smithi040.stdout:1/449: stat d17/d26/d5f/d78 0 2024-02-16T19:24:09.455 INFO:tasks.workunit.client.0.smithi040.stdout:1/450: stat fd 0 2024-02-16T19:24:09.455 INFO:tasks.workunit.client.0.smithi040.stdout:1/451: truncate d17/d43/d92/f8c 236112 0 2024-02-16T19:24:09.455 INFO:tasks.workunit.client.0.smithi040.stdout:1/452: stat d17/d3c/d6b/c8e 0 2024-02-16T19:24:09.458 INFO:tasks.workunit.client.0.smithi040.stdout:6/398: rename d0/d30/d32/l1b to d0/d37/d4a/l84 0 2024-02-16T19:24:09.461 INFO:tasks.workunit.client.0.smithi040.stdout:0/480: mkdir db/d2f/d1d/d21/da7 0 2024-02-16T19:24:09.461 INFO:tasks.workunit.client.0.smithi040.stdout:0/481: write db/f20 [1711941,53379] 0 2024-02-16T19:24:09.462 INFO:tasks.workunit.client.0.smithi040.stdout:9/448: truncate d3/f21 79648 0 2024-02-16T19:24:09.462 INFO:tasks.workunit.client.0.smithi040.stdout:9/449: fsync d3/d4/d11/d29/d36/f75 0 2024-02-16T19:24:09.465 INFO:tasks.workunit.client.0.smithi040.stdout:1/453: mkdir d17/d29/d8b/d93 0 2024-02-16T19:24:09.465 INFO:tasks.workunit.client.0.smithi040.stdout:1/454: stat d17/d26/d54/c6c 0 2024-02-16T19:24:09.465 INFO:tasks.workunit.client.0.smithi040.stdout:1/455: write d17/d26/d5f/d63/f75 [882838,39390] 0 2024-02-16T19:24:09.465 INFO:tasks.workunit.client.0.smithi040.stdout:1/456: creat d17/f94 x:0 0 0 2024-02-16T19:24:09.465 INFO:tasks.workunit.client.0.smithi040.stdout:1/457: chown d17/d43/d38/f7b 47667 1 2024-02-16T19:24:09.465 INFO:tasks.workunit.client.0.smithi040.stdout:9/450: write d3/d4/d11/d29/d38/f54 [90578,13594] 0 2024-02-16T19:24:09.466 INFO:tasks.workunit.client.0.smithi040.stdout:6/399: rename d0/d30/d5b/f72 to d0/d30/d5b/f85 0 2024-02-16T19:24:09.467 INFO:tasks.workunit.client.0.smithi040.stdout:6/400: stat d0/d1/d12/d3b/d48/d73/f42 0 2024-02-16T19:24:09.467 INFO:tasks.workunit.client.0.smithi040.stdout:2/362: write d5/d7/d1e/d31/f43 [1845465,34931] 0 2024-02-16T19:24:09.467 INFO:tasks.workunit.client.0.smithi040.stdout:2/363: creat d5/d7/d12/f76 x:0 0 0 2024-02-16T19:24:09.470 INFO:tasks.workunit.client.0.smithi040.stdout:9/451: unlink d3/d4/d11/f53 0 2024-02-16T19:24:09.471 INFO:tasks.workunit.client.0.smithi040.stdout:6/401: symlink d0/d1/l86 0 2024-02-16T19:24:09.474 INFO:tasks.workunit.client.0.smithi040.stdout:0/482: dread db/d2f/d1d/d21/f51 [0,4194304] 0 2024-02-16T19:24:09.475 INFO:tasks.workunit.client.0.smithi040.stdout:5/391: dwrite d1/f2c [0,4194304] 0 2024-02-16T19:24:09.481 INFO:tasks.workunit.client.0.smithi040.stdout:8/486: rmdir d6/d12 39 2024-02-16T19:24:09.482 INFO:tasks.workunit.client.0.smithi040.stdout:2/364: mknod d5/d5c/d65/c77 0 2024-02-16T19:24:09.482 INFO:tasks.workunit.client.0.smithi040.stdout:2/365: truncate d5/d7/d1e/d55/f6c 129321 0 2024-02-16T19:24:09.483 INFO:tasks.workunit.client.0.smithi040.stdout:9/452: mkdir d3/d4/d11/d32/d8e 0 2024-02-16T19:24:09.487 INFO:tasks.workunit.client.0.smithi040.stdout:5/392: truncate d1/da/d1c/f25 842353 0 2024-02-16T19:24:09.487 INFO:tasks.workunit.client.0.smithi040.stdout:5/393: dread - d1/d3c/f68 zero size 2024-02-16T19:24:09.487 INFO:tasks.workunit.client.0.smithi040.stdout:5/394: fdatasync d1/da/d1c/d21/f42 0 2024-02-16T19:24:09.488 INFO:tasks.workunit.client.0.smithi040.stdout:8/487: symlink d6/d12/d93/l9f 0 2024-02-16T19:24:09.488 INFO:tasks.workunit.client.0.smithi040.stdout:8/488: getdents d6/d64 0 2024-02-16T19:24:09.489 INFO:tasks.workunit.client.0.smithi040.stdout:2/366: mkdir d5/d7/d1e/d31/d78 0 2024-02-16T19:24:09.490 INFO:tasks.workunit.client.0.smithi040.stdout:9/453: creat d3/d4/d55/d8a/f8f x:0 0 0 2024-02-16T19:24:09.490 INFO:tasks.workunit.client.0.smithi040.stdout:9/454: dread - d3/d4/d55/d31/f4c zero size 2024-02-16T19:24:09.490 INFO:tasks.workunit.client.0.smithi040.stdout:9/455: chown d3/d4/d11/d29/d38/f54 376295 1 2024-02-16T19:24:09.491 INFO:tasks.workunit.client.0.smithi040.stdout:6/402: rmdir d0/d30/d4b 39 2024-02-16T19:24:09.495 INFO:tasks.workunit.client.0.smithi040.stdout:2/367: dread d5/d7/d1e/f25 [0,4194304] 0 2024-02-16T19:24:09.495 INFO:tasks.workunit.client.0.smithi040.stdout:2/368: chown d5/d7/d12/f6a 173745 1 2024-02-16T19:24:09.498 INFO:tasks.workunit.client.0.smithi040.stdout:5/395: rmdir d1/d3c/d75 0 2024-02-16T19:24:09.500 INFO:tasks.workunit.client.0.smithi040.stdout:2/369: dread d5/d7/d1e/f25 [0,4194304] 0 2024-02-16T19:24:09.500 INFO:tasks.workunit.client.0.smithi040.stdout:8/489: link d6/d7/d1d/d38/d8b/f9d d6/d7/d1d/d38/d8b/fa0 0 2024-02-16T19:24:09.501 INFO:tasks.workunit.client.0.smithi040.stdout:6/403: mkdir d0/d1/d12/d3b/d48/d4d/d6d/d87 0 2024-02-16T19:24:09.503 INFO:tasks.workunit.client.0.smithi040.stdout:2/370: creat d5/f79 x:0 0 0 2024-02-16T19:24:09.503 INFO:tasks.workunit.client.0.smithi040.stdout:2/371: write d5/d7/d1e/f6d [866983,20729] 0 2024-02-16T19:24:09.503 INFO:tasks.workunit.client.0.smithi040.stdout:2/372: readlink d5/d7/d12/d1b/l35 0 2024-02-16T19:24:09.503 INFO:tasks.workunit.client.0.smithi040.stdout:6/404: stat d0/d30/d32/l6b 0 2024-02-16T19:24:09.504 INFO:tasks.workunit.client.0.smithi040.stdout:4/356: sync 2024-02-16T19:24:09.505 INFO:tasks.workunit.client.0.smithi040.stdout:2/373: link d5/f3a d5/d8/d42/f7a 0 2024-02-16T19:24:09.508 INFO:tasks.workunit.client.0.smithi040.stdout:6/405: rename d0/d1/d12/d3b/d48/d73/f3d to d0/d1/d12/d3b/d79/f88 0 2024-02-16T19:24:09.508 INFO:tasks.workunit.client.0.smithi040.stdout:6/406: dread - d0/d30/d32/f56 zero size 2024-02-16T19:24:09.511 INFO:tasks.workunit.client.0.smithi040.stdout:0/483: rmdir db/d2f/d1d/d21 39 2024-02-16T19:24:09.525 INFO:tasks.workunit.client.0.smithi040.stdout:1/458: read d17/f37 [3604678,101799] 0 2024-02-16T19:24:09.530 INFO:tasks.workunit.client.0.smithi040.stdout:1/459: mknod d17/d26/d3d/c95 0 2024-02-16T19:24:09.530 INFO:tasks.workunit.client.0.smithi040.stdout:0/484: dread db/de/f7d [0,4194304] 0 2024-02-16T19:24:09.530 INFO:tasks.workunit.client.0.smithi040.stdout:0/485: readlink db/d2f/d1d/d21/l6c 0 2024-02-16T19:24:09.532 INFO:tasks.workunit.client.0.smithi040.stdout:6/407: dread d0/d1/d12/f28 [8388608,4194304] 0 2024-02-16T19:24:09.532 INFO:tasks.workunit.client.0.smithi040.stdout:6/408: chown d0/d37/c7e 22 1 2024-02-16T19:24:09.532 INFO:tasks.workunit.client.0.smithi040.stdout:6/409: chown d0/d1/d12/d3b/l81 179 1 2024-02-16T19:24:09.533 INFO:tasks.workunit.client.0.smithi040.stdout:6/410: fdatasync d0/d1/d12/f2c 0 2024-02-16T19:24:09.533 INFO:tasks.workunit.client.0.smithi040.stdout:0/486: creat db/d2f/d1d/d21/d2e/d83/fa8 x:0 0 0 2024-02-16T19:24:09.533 INFO:tasks.workunit.client.0.smithi040.stdout:1/460: getdents d17/d3c/d71 0 2024-02-16T19:24:09.533 INFO:tasks.workunit.client.0.smithi040.stdout:6/411: getdents d0/d30 0 2024-02-16T19:24:09.534 INFO:tasks.workunit.client.0.smithi040.stdout:6/412: fdatasync d0/d1/d12/d3b/d48/d73/d76/d7d/f80 0 2024-02-16T19:24:09.534 INFO:tasks.workunit.client.0.smithi040.stdout:6/413: chown d0/d30/d32 114 1 2024-02-16T19:24:09.534 INFO:tasks.workunit.client.0.smithi040.stdout:6/414: dread - d0/d30/d32/f56 zero size 2024-02-16T19:24:09.534 INFO:tasks.workunit.client.0.smithi040.stdout:6/415: fsync d0/f36 0 2024-02-16T19:24:09.535 INFO:tasks.workunit.client.0.smithi040.stdout:0/487: rename db/d2f/d1d/d21/f45 to db/d2f/d1d/d21/d2e/d49/fa9 0 2024-02-16T19:24:09.536 INFO:tasks.workunit.client.0.smithi040.stdout:6/416: mkdir d0/d1/d12/d3b/d48/d4d/d89 0 2024-02-16T19:24:09.536 INFO:tasks.workunit.client.0.smithi040.stdout:6/417: creat d0/d30/d5b/f8a x:0 0 0 2024-02-16T19:24:09.537 INFO:tasks.workunit.client.0.smithi040.stdout:0/488: rename db/d25/d34/f9e to db/d2f/d1d/d21/d2e/d83/faa 0 2024-02-16T19:24:09.539 INFO:tasks.workunit.client.0.smithi040.stdout:6/418: mkdir d0/d1/d12/d3b/d48/d4f/d8b 0 2024-02-16T19:24:09.541 INFO:tasks.workunit.client.0.smithi040.stdout:0/489: rmdir db/d2f/d1d/d21/da7 0 2024-02-16T19:24:09.549 INFO:tasks.workunit.client.0.smithi040.stdout:4/357: dwrite dd/de/f6b [0,4194304] 0 2024-02-16T19:24:09.550 INFO:tasks.workunit.client.0.smithi040.stdout:4/358: getdents dd/de/d45 0 2024-02-16T19:24:09.552 INFO:tasks.workunit.client.0.smithi040.stdout:4/359: creat dd/de/d45/f73 x:0 0 0 2024-02-16T19:24:09.552 INFO:tasks.workunit.client.0.smithi040.stdout:5/396: dwrite d1/da/d1c/d21/f64 [0,4194304] 0 2024-02-16T19:24:09.552 INFO:tasks.workunit.client.0.smithi040.stdout:2/374: dwrite d5/d7/d1e/d4c/d52/f6b [0,4194304] 0 2024-02-16T19:24:09.552 INFO:tasks.workunit.client.0.smithi040.stdout:8/490: dwrite d6/d2b/f9c [0,4194304] 0 2024-02-16T19:24:09.553 INFO:tasks.workunit.client.0.smithi040.stdout:1/461: dwrite d17/d26/d54/f5e [0,4194304] 0 2024-02-16T19:24:09.553 INFO:tasks.workunit.client.0.smithi040.stdout:1/462: chown d17/d3c/d6b/c8e 4651331 1 2024-02-16T19:24:09.553 INFO:tasks.workunit.client.0.smithi040.stdout:4/360: creat dd/de/d45/f74 x:0 0 0 2024-02-16T19:24:09.557 INFO:tasks.workunit.client.0.smithi040.stdout:0/490: write db/de/d6a/f82 [1689435,87694] 0 2024-02-16T19:24:09.557 INFO:tasks.workunit.client.0.smithi040.stdout:0/491: chown db/d2f/d1d/d21/d2e/d49 65840145 1 2024-02-16T19:24:09.558 INFO:tasks.workunit.client.0.smithi040.stdout:2/375: rename d5/d7/d10/l11 to d5/d7/d1e/d55/l7b 0 2024-02-16T19:24:09.559 INFO:tasks.workunit.client.0.smithi040.stdout:2/376: dread d5/d8/d42/f50 [0,4194304] 0 2024-02-16T19:24:09.559 INFO:tasks.workunit.client.0.smithi040.stdout:2/377: chown d5/d7/d1e/d31/d78 50 1 2024-02-16T19:24:09.565 INFO:tasks.workunit.client.0.smithi040.stdout:8/491: truncate d6/f13 290467 0 2024-02-16T19:24:09.569 INFO:tasks.workunit.client.0.smithi040.stdout:1/463: creat d17/d29/d8b/d93/f96 x:0 0 0 2024-02-16T19:24:09.570 INFO:tasks.workunit.client.0.smithi040.stdout:4/361: unlink dd/de/d10/f6d 0 2024-02-16T19:24:09.570 INFO:tasks.workunit.client.0.smithi040.stdout:4/362: readlink dd/de/l5a 0 2024-02-16T19:24:09.571 INFO:tasks.workunit.client.0.smithi040.stdout:0/492: truncate db/d2f/d1d/d21/d2e/d49/d5b/f79 747678 0 2024-02-16T19:24:09.571 INFO:tasks.workunit.client.0.smithi040.stdout:0/493: write db/d2f/d1d/f46 [314283,24904] 0 2024-02-16T19:24:09.574 INFO:tasks.workunit.client.0.smithi040.stdout:8/492: write d6/d12/d51/f89 [1984545,64069] 0 2024-02-16T19:24:09.574 INFO:tasks.workunit.client.0.smithi040.stdout:4/363: mknod dd/de/d10/c75 0 2024-02-16T19:24:09.574 INFO:tasks.workunit.client.0.smithi040.stdout:0/494: rmdir db/d2f/d1d/d21/d2e/d49/d5b/d67 39 2024-02-16T19:24:09.574 INFO:tasks.workunit.client.0.smithi040.stdout:0/495: truncate db/d2f/d23/d3b/f47 879835 0 2024-02-16T19:24:09.575 INFO:tasks.workunit.client.0.smithi040.stdout:8/493: write d6/d2b/f80 [252252,51068] 0 2024-02-16T19:24:09.577 INFO:tasks.workunit.client.0.smithi040.stdout:6/419: rmdir d0/d1/d12/d3b 39 2024-02-16T19:24:09.577 INFO:tasks.workunit.client.0.smithi040.stdout:6/420: getdents d0/d1/d12/d3b/d48/d4d/d6d/d87 0 2024-02-16T19:24:09.577 INFO:tasks.workunit.client.0.smithi040.stdout:6/421: write d0/f1d [660548,16419] 0 2024-02-16T19:24:09.578 INFO:tasks.workunit.client.0.smithi040.stdout:0/496: symlink db/lab 0 2024-02-16T19:24:09.578 INFO:tasks.workunit.client.0.smithi040.stdout:0/497: chown db/de/f7d 16090449 1 2024-02-16T19:24:09.579 INFO:tasks.workunit.client.0.smithi040.stdout:2/378: dwrite d5/d7/d12/d1b/f30 [0,4194304] 0 2024-02-16T19:24:09.580 INFO:tasks.workunit.client.0.smithi040.stdout:6/422: dread d0/d1/d12/d3b/f40 [0,4194304] 0 2024-02-16T19:24:09.580 INFO:tasks.workunit.client.0.smithi040.stdout:8/494: creat d6/d12/d51/fa1 x:0 0 0 2024-02-16T19:24:09.581 INFO:tasks.workunit.client.0.smithi040.stdout:4/364: symlink dd/de/d45/d37/l76 0 2024-02-16T19:24:09.582 INFO:tasks.workunit.client.0.smithi040.stdout:1/464: dread d17/d26/d54/f5e [0,4194304] 0 2024-02-16T19:24:09.583 INFO:tasks.workunit.client.0.smithi040.stdout:1/465: read - d17/d43/d92/f6e zero size 2024-02-16T19:24:09.599 INFO:tasks.workunit.client.0.smithi040.stdout:2/379: mknod d5/d7/d1e/d4c/d52/c7c 0 2024-02-16T19:24:09.599 INFO:tasks.workunit.client.0.smithi040.stdout:2/380: dread - d5/d7/d12/f6a zero size 2024-02-16T19:24:09.600 INFO:tasks.workunit.client.0.smithi040.stdout:6/423: unlink d0/d37/c7e 0 2024-02-16T19:24:09.603 INFO:tasks.workunit.client.0.smithi040.stdout:1/466: mkdir d17/d3c/d74/d97 0 2024-02-16T19:24:09.603 INFO:tasks.workunit.client.0.smithi040.stdout:1/467: write d17/d26/d3d/f56 [531557,123823] 0 2024-02-16T19:24:09.604 INFO:tasks.workunit.client.0.smithi040.stdout:2/381: creat d5/d7/d1e/d31/f7d x:0 0 0 2024-02-16T19:24:09.604 INFO:tasks.workunit.client.0.smithi040.stdout:2/382: write d5/d8/d42/f67 [364324,129431] 0 2024-02-16T19:24:09.604 INFO:tasks.workunit.client.0.smithi040.stdout:2/383: readlink d5/d7/d12/d1b/l2a 0 2024-02-16T19:24:09.607 INFO:tasks.workunit.client.0.smithi040.stdout:1/468: link d17/d26/f7a d17/d26/d4c/f98 0 2024-02-16T19:24:09.607 INFO:tasks.workunit.client.0.smithi040.stdout:1/469: creat d17/d29/d8b/d93/f99 x:0 0 0 2024-02-16T19:24:09.607 INFO:tasks.workunit.client.0.smithi040.stdout:7/382: sync 2024-02-16T19:24:09.609 INFO:tasks.workunit.client.0.smithi040.stdout:1/470: mkdir d17/d26/d9a 0 2024-02-16T19:24:09.609 INFO:tasks.workunit.client.0.smithi040.stdout:1/471: fdatasync d17/d26/d4c/f8a 0 2024-02-16T19:24:09.610 INFO:tasks.workunit.client.0.smithi040.stdout:7/383: mkdir d21/d4d/d7b 0 2024-02-16T19:24:09.610 INFO:tasks.workunit.client.0.smithi040.stdout:6/424: dread d0/d1/d12/f2c [0,4194304] 0 2024-02-16T19:24:09.610 INFO:tasks.workunit.client.0.smithi040.stdout:6/425: chown d0/c3e 13918 1 2024-02-16T19:24:09.610 INFO:tasks.workunit.client.0.smithi040.stdout:6/426: rmdir d0/d1/d12/d3b/d48/d4d/d6d 39 2024-02-16T19:24:09.613 INFO:tasks.workunit.client.0.smithi040.stdout:6/427: link d0/d1/d12/d3b/d48/d4d/l6e d0/d1/d12/d3b/d48/d73/d70/d83/l8c 0 2024-02-16T19:24:09.613 INFO:tasks.workunit.client.0.smithi040.stdout:6/428: creat d0/d30/d32/f8d x:0 0 0 2024-02-16T19:24:09.618 INFO:tasks.workunit.client.0.smithi040.stdout:6/429: creat d0/d1/d12/d3b/d48/d4d/d6d/d87/f8e x:0 0 0 2024-02-16T19:24:09.618 INFO:tasks.workunit.client.0.smithi040.stdout:6/430: write d0/d1/d12/f66 [451131,32934] 0 2024-02-16T19:24:09.620 INFO:tasks.workunit.client.0.smithi040.stdout:8/495: rmdir d6/d12/d51 39 2024-02-16T19:24:09.620 INFO:tasks.workunit.client.0.smithi040.stdout:4/365: rmdir dd/de/d45/d37 39 2024-02-16T19:24:09.623 INFO:tasks.workunit.client.0.smithi040.stdout:8/496: stat d6/d12/d51/c6a 0 2024-02-16T19:24:09.623 INFO:tasks.workunit.client.0.smithi040.stdout:8/497: write d6/d7/d1d/f7a [4157132,126009] 0 2024-02-16T19:24:09.625 INFO:tasks.workunit.client.0.smithi040.stdout:8/498: truncate d6/d64/f6d 1359035 0 2024-02-16T19:24:09.625 INFO:tasks.workunit.client.0.smithi040.stdout:8/499: readlink d6/d2b/d36/l58 0 2024-02-16T19:24:09.625 INFO:tasks.workunit.client.0.smithi040.stdout:8/500: fdatasync d6/d7/d1d/d38/d8b/fa0 0 2024-02-16T19:24:09.627 INFO:tasks.workunit.client.0.smithi040.stdout:8/501: unlink d6/f13 0 2024-02-16T19:24:09.630 INFO:tasks.workunit.client.0.smithi040.stdout:8/502: creat d6/d12/d51/d6e/fa2 x:0 0 0 2024-02-16T19:24:09.630 INFO:tasks.workunit.client.0.smithi040.stdout:8/503: fdatasync d6/d7/f9b 0 2024-02-16T19:24:09.635 INFO:tasks.workunit.client.0.smithi040.stdout:2/384: dwrite d5/d7/d12/d1b/f30 [4194304,4194304] 0 2024-02-16T19:24:09.638 INFO:tasks.workunit.client.0.smithi040.stdout:8/504: dread d6/d7/d1d/f52 [0,4194304] 0 2024-02-16T19:24:09.648 INFO:tasks.workunit.client.0.smithi040.stdout:8/505: dread d6/d2b/d44/f49 [0,4194304] 0 2024-02-16T19:24:09.648 INFO:tasks.workunit.client.0.smithi040.stdout:8/506: fdatasync d6/d12/d51/d6e/fa2 0 2024-02-16T19:24:09.648 INFO:tasks.workunit.client.0.smithi040.stdout:8/507: fdatasync d6/d7/d15/f6b 0 2024-02-16T19:24:09.648 INFO:tasks.workunit.client.0.smithi040.stdout:8/508: write d6/d7/d1d/d38/f3d [367956,111457] 0 2024-02-16T19:24:09.651 INFO:tasks.workunit.client.0.smithi040.stdout:8/509: rename d6/d7/d1d/d38 to d6/d2b/da3 0 2024-02-16T19:24:09.651 INFO:tasks.workunit.client.0.smithi040.stdout:8/510: write d6/d7/f2e [95595,56966] 0 2024-02-16T19:24:09.653 INFO:tasks.workunit.client.0.smithi040.stdout:8/511: rename d6/d64/d50/d69/d81/f99 to d6/d64/d50/d69/fa4 0 2024-02-16T19:24:09.653 INFO:tasks.workunit.client.0.smithi040.stdout:8/512: readlink d6/d7/le 0 2024-02-16T19:24:09.654 INFO:tasks.workunit.client.0.smithi040.stdout:8/513: link d6/f9 d6/fa5 0 2024-02-16T19:24:09.659 INFO:tasks.workunit.client.0.smithi040.stdout:7/384: dwrite d21/d4d/f66 [0,4194304] 0 2024-02-16T19:24:09.659 INFO:tasks.workunit.client.0.smithi040.stdout:7/385: getdents d21/d4d/d77/d7a 0 2024-02-16T19:24:09.661 INFO:tasks.workunit.client.0.smithi040.stdout:7/386: rename d21/f26 to d21/d29/d79/f7c 0 2024-02-16T19:24:09.661 INFO:tasks.workunit.client.0.smithi040.stdout:7/387: truncate f1e 1259379 0 2024-02-16T19:24:09.661 INFO:tasks.workunit.client.0.smithi040.stdout:8/514: dread d6/f75 [0,4194304] 0 2024-02-16T19:24:09.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/388: mkdir d21/d4d/d7d 0 2024-02-16T19:24:09.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/389: write fa [1492383,99550] 0 2024-02-16T19:24:09.662 INFO:tasks.workunit.client.0.smithi040.stdout:1/472: dwrite d17/d26/d4c/f98 [0,4194304] 0 2024-02-16T19:24:09.662 INFO:tasks.workunit.client.0.smithi040.stdout:1/473: chown d17/d26/c51 5813 1 2024-02-16T19:24:09.662 INFO:tasks.workunit.client.0.smithi040.stdout:0/498: dwrite db/d2f/d1d/d21/d2e/d49/d5b/f79 [0,4194304] 0 2024-02-16T19:24:09.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/366: dwrite dd/de/f54 [0,4194304] 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:6/431: dwrite d0/d30/d5b/f85 [0,4194304] 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:8/515: mkdir d6/d12/d22/da6 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/385: dwrite d5/d7/d1e/f6d [0,4194304] 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/386: chown d5/d7/d1e/d4c/d52 9 1 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/387: creat d5/d7/d1e/f7e x:0 0 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/388: creat d5/d5c/f7f x:0 0 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/389: creat d5/d7/d1e/f80 x:0 0 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/390: stat d5/d7/l2e 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/391: truncate d5/d7/d1e/f70 42061 0 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/392: chown d5/d7/d12/d1b 43294180 1 2024-02-16T19:24:09.663 INFO:tasks.workunit.client.0.smithi040.stdout:2/393: chown d5/d7/d1e/d31/f7d 3855 1 2024-02-16T19:24:09.669 INFO:tasks.workunit.client.0.smithi040.stdout:1/474: unlink d17/d26/d54/f5e 0 2024-02-16T19:24:09.688 INFO:tasks.workunit.client.0.smithi040.stdout:0/499: dwrite db/d2f/d1d/d21/f51 [0,4194304] 0 2024-02-16T19:24:09.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/432: dwrite d0/d1/d12/d3b/d79/f88 [4194304,4194304] 0 2024-02-16T19:24:09.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/433: chown d0/d1/d12/d3b/c41 778416 1 2024-02-16T19:24:09.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/434: write d0/d1/d12/d3b/d48/d73/f42 [4629282,85509] 0 2024-02-16T19:24:09.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/435: truncate d0/f2e 5552115 0 2024-02-16T19:24:09.703 INFO:tasks.workunit.client.0.smithi040.stdout:6/436: dwrite d0/f36 [0,4194304] 0 2024-02-16T19:24:09.776 INFO:tasks.workunit.client.0.smithi040.stdout:4/367: rename dd/de/d45/d37 to dd/de/d10/d13/d77 0 2024-02-16T19:24:09.776 INFO:tasks.workunit.client.0.smithi040.stdout:4/368: fsync dd/de/d45/f74 0 2024-02-16T19:24:09.788 INFO:tasks.workunit.client.0.smithi040.stdout:3/516: sync 2024-02-16T19:24:09.789 INFO:tasks.workunit.client.0.smithi040.stdout:3/517: fsync d1/d7/d36/f87 0 2024-02-16T19:24:09.790 INFO:tasks.workunit.client.0.smithi040.stdout:2/394: unlink d5/d7/f54 0 2024-02-16T19:24:09.791 INFO:tasks.workunit.client.0.smithi040.stdout:2/395: write d5/d7/d1e/d55/f6f [494310,9959] 0 2024-02-16T19:24:09.792 INFO:tasks.workunit.client.0.smithi040.stdout:7/390: mkdir d21/d7e 0 2024-02-16T19:24:09.792 INFO:tasks.workunit.client.0.smithi040.stdout:9/456: sync 2024-02-16T19:24:09.793 INFO:tasks.workunit.client.0.smithi040.stdout:9/457: chown d3/d4/d55/d31/l47 7597332 1 2024-02-16T19:24:09.794 INFO:tasks.workunit.client.0.smithi040.stdout:0/500: link db/d2f/d23/c37 db/d2f/d1d/d21/cac 0 2024-02-16T19:24:09.796 INFO:tasks.workunit.client.0.smithi040.stdout:5/397: sync 2024-02-16T19:24:09.799 INFO:tasks.workunit.client.0.smithi040.stdout:6/437: mkdir d0/d1/d12/d3b/d48/d4d/d6d/d87/d8f 0 2024-02-16T19:24:09.814 INFO:tasks.workunit.client.0.smithi040.stdout:5/398: dwrite d1/d3c/f77 [0,4194304] 0 2024-02-16T19:24:09.814 INFO:tasks.workunit.client.0.smithi040.stdout:5/399: fsync d1/da/d51/f58 0 2024-02-16T19:24:09.814 INFO:tasks.workunit.client.0.smithi040.stdout:5/400: fdatasync d1/d45/f52 0 2024-02-16T19:24:09.814 INFO:tasks.workunit.client.0.smithi040.stdout:5/401: write d1/da/d51/f58 [204001,66468] 0 2024-02-16T19:24:09.814 INFO:tasks.workunit.client.0.smithi040.stdout:5/402: chown d1/d45/f52 152000 1 2024-02-16T19:24:09.814 INFO:tasks.workunit.client.0.smithi040.stdout:5/403: fsync d1/da/d1c/f26 0 2024-02-16T19:24:09.823 INFO:tasks.workunit.client.0.smithi040.stdout:5/404: dread d1/f17 [0,4194304] 0 2024-02-16T19:24:10.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:10 smithi040 ceph-mon[29201]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:24:10.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:10 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:10.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:10 smithi040 ceph-mon[29201]: pgmap v244: 65 pgs: 65 active+clean; 777 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 64 MiB/s rd, 93 MiB/s wr, 634 op/s 2024-02-16T19:24:10.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:10 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:24:10.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:10 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:10.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:10 smithi163 ceph-mon[36452]: pgmap v244: 65 pgs: 65 active+clean; 777 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 64 MiB/s rd, 93 MiB/s wr, 634 op/s 2024-02-16T19:24:13.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:13 smithi163 ceph-mon[36452]: pgmap v245: 65 pgs: 65 active+clean; 1.2 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 112 MiB/s rd, 150 MiB/s wr, 716 op/s 2024-02-16T19:24:13.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:13 smithi040 ceph-mon[29201]: pgmap v245: 65 pgs: 65 active+clean; 1.2 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 112 MiB/s rd, 150 MiB/s wr, 716 op/s 2024-02-16T19:24:14.211 INFO:tasks.workunit.client.0.smithi040.stdout:8/516: rename d6/d7/d1d/f60 to d6/d64/d50/d69/d81/fa7 0 2024-02-16T19:24:14.408 INFO:tasks.workunit.client.0.smithi040.stdout:1/475: rename d17/l18 to d17/d3c/l9b 0 2024-02-16T19:24:14.409 INFO:tasks.workunit.client.0.smithi040.stdout:8/517: mknod d6/d12/d22/ca8 0 2024-02-16T19:24:14.411 INFO:tasks.workunit.client.0.smithi040.stdout:6/438: rename d0/d1/d12/d3b/d48/d4d/d89 to d0/d1/d6f/d90 0 2024-02-16T19:24:14.412 INFO:tasks.workunit.client.0.smithi040.stdout:6/439: link d0/d1/d6f/f75 d0/d30/d5b/f91 0 2024-02-16T19:24:14.413 INFO:tasks.workunit.client.0.smithi040.stdout:6/440: truncate d0/d30/d32/f56 665165 0 2024-02-16T19:24:14.415 INFO:tasks.workunit.client.0.smithi040.stdout:6/441: link d0/d30/d32/f8d d0/d30/d5b/f92 0 2024-02-16T19:24:14.416 INFO:tasks.workunit.client.0.smithi040.stdout:8/518: dread d6/d64/d50/f74 [0,4194304] 0 2024-02-16T19:24:14.417 INFO:tasks.workunit.client.0.smithi040.stdout:1/476: dread d17/d2d/f73 [0,4194304] 0 2024-02-16T19:24:14.419 INFO:tasks.workunit.client.0.smithi040.stdout:8/519: rename d6/d7/d15/f6b to d6/d12/d93/fa9 0 2024-02-16T19:24:14.420 INFO:tasks.workunit.client.0.smithi040.stdout:8/520: creat d6/d64/faa x:0 0 0 2024-02-16T19:24:14.420 INFO:tasks.workunit.client.0.smithi040.stdout:8/521: dread - d6/d64/d50/f7d zero size 2024-02-16T19:24:14.421 INFO:tasks.workunit.client.0.smithi040.stdout:8/522: truncate d6/d7/d15/f2f 4501968 0 2024-02-16T19:24:14.434 INFO:tasks.workunit.client.0.smithi040.stdout:8/523: write d6/d2b/f9c [2645169,58838] 0 2024-02-16T19:24:14.442 INFO:tasks.workunit.client.0.smithi040.stdout:1/477: dwrite d17/d26/d54/f82 [0,4194304] 0 2024-02-16T19:24:14.442 INFO:tasks.workunit.client.0.smithi040.stdout:1/478: write f5 [2786673,101504] 0 2024-02-16T19:24:14.443 INFO:tasks.workunit.client.0.smithi040.stdout:1/479: write d17/d26/d4c/f8a [297030,34667] 0 2024-02-16T19:24:14.480 INFO:tasks.workunit.client.0.smithi040.stdout:8/524: symlink d6/d12/d93/lab 0 2024-02-16T19:24:14.480 INFO:tasks.workunit.client.0.smithi040.stdout:8/525: chown d6/d7/d1d/f52 46309 1 2024-02-16T19:24:14.480 INFO:tasks.workunit.client.0.smithi040.stdout:8/526: write d6/d7/f70 [1009673,63591] 0 2024-02-16T19:24:14.483 INFO:tasks.workunit.client.0.smithi040.stdout:5/405: rmdir d1/d3c 39 2024-02-16T19:24:14.483 INFO:tasks.workunit.client.0.smithi040.stdout:7/391: stat d21/f4a 0 2024-02-16T19:24:14.484 INFO:tasks.workunit.client.0.smithi040.stdout:7/392: truncate d21/d4d/f74 3898594 0 2024-02-16T19:24:14.484 INFO:tasks.workunit.client.0.smithi040.stdout:7/393: chown d21/d4d/c51 5054540 1 2024-02-16T19:24:14.499 INFO:tasks.workunit.client.0.smithi040.stdout:7/394: rename d21/f32 to d21/d29/d79/f7f 0 2024-02-16T19:24:14.500 INFO:tasks.workunit.client.0.smithi040.stdout:6/442: rmdir d0/d1/d6f 39 2024-02-16T19:24:14.500 INFO:tasks.workunit.client.0.smithi040.stdout:1/480: rmdir d17/d3c 39 2024-02-16T19:24:14.500 INFO:tasks.workunit.client.0.smithi040.stdout:1/481: stat d17 0 2024-02-16T19:24:14.505 INFO:tasks.workunit.client.0.smithi040.stdout:2/396: mkdir d5/d7/d1e/d4c/d52/d81 0 2024-02-16T19:24:14.505 INFO:tasks.workunit.client.0.smithi040.stdout:6/443: creat d0/d30/d4b/f93 x:0 0 0 2024-02-16T19:24:14.507 INFO:tasks.workunit.client.0.smithi040.stdout:3/518: symlink d1/dc/d42/d5c/l9d 0 2024-02-16T19:24:14.507 INFO:tasks.workunit.client.0.smithi040.stdout:3/519: chown d1/dc/d42/d4a/l59 2188221 1 2024-02-16T19:24:14.507 INFO:tasks.workunit.client.0.smithi040.stdout:8/527: dwrite d6/d2b/d44/f7f [0,4194304] 0 2024-02-16T19:24:14.507 INFO:tasks.workunit.client.0.smithi040.stdout:8/528: write d6/d7/d15/f1c [4388196,127060] 0 2024-02-16T19:24:14.507 INFO:tasks.workunit.client.0.smithi040.stdout:5/406: dwrite d1/d45/d63/f7b [4194304,4194304] 0 2024-02-16T19:24:14.508 INFO:tasks.workunit.client.0.smithi040.stdout:6/444: creat d0/d1/d12/d3b/d48/d4f/d8b/f94 x:0 0 0 2024-02-16T19:24:14.508 INFO:tasks.workunit.client.0.smithi040.stdout:6/445: chown d0/d1/d12/f28 479922786 1 2024-02-16T19:24:14.508 INFO:tasks.workunit.client.0.smithi040.stdout:6/446: getdents d0/d30 0 2024-02-16T19:24:14.509 INFO:tasks.workunit.client.0.smithi040.stdout:0/501: creat db/d25/d2c/fad x:0 0 0 2024-02-16T19:24:14.509 INFO:tasks.workunit.client.0.smithi040.stdout:0/502: chown db/d2f/d1d/d21/d2e/d85/d89/la1 202 1 2024-02-16T19:24:14.510 INFO:tasks.workunit.client.0.smithi040.stdout:3/520: creat d1/dc/d35/d48/d85/f9e x:0 0 0 2024-02-16T19:24:14.511 INFO:tasks.workunit.client.0.smithi040.stdout:3/521: stat d1/d4/f41 0 2024-02-16T19:24:14.512 INFO:tasks.workunit.client.0.smithi040.stdout:8/529: mknod d6/d64/d50/d69/cac 0 2024-02-16T19:24:14.513 INFO:tasks.workunit.client.0.smithi040.stdout:9/458: rmdir d3/d4/d11/d29/d38/d46 39 2024-02-16T19:24:14.513 INFO:tasks.workunit.client.0.smithi040.stdout:9/459: write d3/d4/d11/d29/d38/f54 [1216172,35409] 0 2024-02-16T19:24:14.513 INFO:tasks.workunit.client.0.smithi040.stdout:9/460: truncate d3/d4/d11/d71/f88 760084 0 2024-02-16T19:24:14.515 INFO:tasks.workunit.client.0.smithi040.stdout:4/369: rmdir dd/de/d10/d13/d32 39 2024-02-16T19:24:14.516 INFO:tasks.workunit.client.0.smithi040.stdout:2/397: write d5/f3c [3445041,33737] 0 2024-02-16T19:24:14.516 INFO:tasks.workunit.client.0.smithi040.stdout:2/398: chown d5/d7/d1e/d31/f43 234601448 1 2024-02-16T19:24:14.516 INFO:tasks.workunit.client.0.smithi040.stdout:2/399: write d5/d7/d1e/d31/f7d [211795,35782] 0 2024-02-16T19:24:14.517 INFO:tasks.workunit.client.0.smithi040.stdout:5/407: mkdir d1/da/d7e 0 2024-02-16T19:24:14.517 INFO:tasks.workunit.client.0.smithi040.stdout:8/530: write d6/d12/f1f [2860413,115822] 0 2024-02-16T19:24:14.520 INFO:tasks.workunit.client.0.smithi040.stdout:6/447: creat d0/d1/d12/d3b/d48/d4f/d8b/f95 x:0 0 0 2024-02-16T19:24:14.520 INFO:tasks.workunit.client.0.smithi040.stdout:6/448: write d0/d1/f52 [680156,82066] 0 2024-02-16T19:24:14.520 INFO:tasks.workunit.client.0.smithi040.stdout:6/449: dread - d0/d30/d5b/f8a zero size 2024-02-16T19:24:14.526 INFO:tasks.workunit.client.0.smithi040.stdout:1/482: dwrite d17/f37 [0,4194304] 0 2024-02-16T19:24:14.526 INFO:tasks.workunit.client.0.smithi040.stdout:0/503: mknod db/d2f/d23/cae 0 2024-02-16T19:24:14.526 INFO:tasks.workunit.client.0.smithi040.stdout:0/504: dread - db/d2f/d1d/f80 zero size 2024-02-16T19:24:14.528 INFO:tasks.workunit.client.0.smithi040.stdout:3/522: symlink d1/dc/d35/d48/l9f 0 2024-02-16T19:24:14.528 INFO:tasks.workunit.client.0.smithi040.stdout:3/523: chown d1/d4/c8 3977594 1 2024-02-16T19:24:14.530 INFO:tasks.workunit.client.0.smithi040.stdout:4/370: dread dd/f19 [0,4194304] 0 2024-02-16T19:24:14.530 INFO:tasks.workunit.client.0.smithi040.stdout:4/371: write dd/de/d10/d13/d77/f3b [4612275,113295] 0 2024-02-16T19:24:14.531 INFO:tasks.workunit.client.0.smithi040.stdout:3/524: dread d1/d7/d36/f7b [0,4194304] 0 2024-02-16T19:24:14.532 INFO:tasks.workunit.client.0.smithi040.stdout:9/461: creat d3/d4/d11/d29/d38/d46/d6b/f90 x:0 0 0 2024-02-16T19:24:14.534 INFO:tasks.workunit.client.0.smithi040.stdout:2/400: creat d5/d7/d1e/d55/f82 x:0 0 0 2024-02-16T19:24:14.534 INFO:tasks.workunit.client.0.smithi040.stdout:2/401: fsync d5/d7/d1e/d31/f7d 0 2024-02-16T19:24:14.535 INFO:tasks.workunit.client.0.smithi040.stdout:8/531: dwrite d6/d64/d50/f84 [0,4194304] 0 2024-02-16T19:24:14.535 INFO:tasks.workunit.client.0.smithi040.stdout:8/532: read - d6/d7/d15/f61 zero size 2024-02-16T19:24:14.536 INFO:tasks.workunit.client.0.smithi040.stdout:2/402: dread d5/f3a [0,4194304] 0 2024-02-16T19:24:14.537 INFO:tasks.workunit.client.0.smithi040.stdout:6/450: creat d0/d1/d12/d3b/d79/f96 x:0 0 0 2024-02-16T19:24:14.540 INFO:tasks.workunit.client.0.smithi040.stdout:1/483: mknod d17/d29/d8b/d93/c9c 0 2024-02-16T19:24:14.540 INFO:tasks.workunit.client.0.smithi040.stdout:1/484: write d17/d43/f6a [2021030,126814] 0 2024-02-16T19:24:14.540 INFO:tasks.workunit.client.0.smithi040.stdout:0/505: symlink db/d2f/d23/d93/laf 0 2024-02-16T19:24:14.542 INFO:tasks.workunit.client.0.smithi040.stdout:4/372: symlink dd/l78 0 2024-02-16T19:24:14.543 INFO:tasks.workunit.client.0.smithi040.stdout:3/525: rename d1/d7/d33/f6c to d1/dc/d42/d4a/fa0 0 2024-02-16T19:24:14.545 INFO:tasks.workunit.client.0.smithi040.stdout:9/462: mknod d3/c91 0 2024-02-16T19:24:14.547 INFO:tasks.workunit.client.0.smithi040.stdout:8/533: mknod d6/d2b/d36/cad 0 2024-02-16T19:24:14.547 INFO:tasks.workunit.client.0.smithi040.stdout:8/534: stat d6/d7/d1d 0 2024-02-16T19:24:14.547 INFO:tasks.workunit.client.0.smithi040.stdout:0/506: rmdir db/d2f/d1d/d21/d2e/d83 39 2024-02-16T19:24:14.549 INFO:tasks.workunit.client.0.smithi040.stdout:4/373: symlink dd/de/d10/l79 0 2024-02-16T19:24:14.549 INFO:tasks.workunit.client.0.smithi040.stdout:4/374: write dd/de/d10/d36/f63 [1320675,129081] 0 2024-02-16T19:24:14.550 INFO:tasks.workunit.client.0.smithi040.stdout:1/485: dread f5 [0,4194304] 0 2024-02-16T19:24:14.550 INFO:tasks.workunit.client.0.smithi040.stdout:1/486: chown d17/d26/d3d 557777867 1 2024-02-16T19:24:14.553 INFO:tasks.workunit.client.0.smithi040.stdout:2/403: rename d5/l71 to d5/d7/d1e/d55/l83 0 2024-02-16T19:24:14.555 INFO:tasks.workunit.client.0.smithi040.stdout:3/526: mknod d1/dc/d42/d5c/ca1 0 2024-02-16T19:24:14.556 INFO:tasks.workunit.client.0.smithi040.stdout:8/535: read d6/d64/d50/f84 [1554241,110293] 0 2024-02-16T19:24:14.556 INFO:tasks.workunit.client.0.smithi040.stdout:9/463: mknod d3/d4/d11/d29/d38/d46/d6b/c92 0 2024-02-16T19:24:14.556 INFO:tasks.workunit.client.0.smithi040.stdout:9/464: write d3/d4/f57 [603925,53917] 0 2024-02-16T19:24:14.559 INFO:tasks.workunit.client.0.smithi040.stdout:6/451: dwrite d0/d1/d12/d3b/d79/f88 [0,4194304] 0 2024-02-16T19:24:14.559 INFO:tasks.workunit.client.0.smithi040.stdout:1/487: unlink d17/d26/d54/f82 0 2024-02-16T19:24:14.560 INFO:tasks.workunit.client.0.smithi040.stdout:4/375: dread dd/de/f54 [0,4194304] 0 2024-02-16T19:24:14.561 INFO:tasks.workunit.client.0.smithi040.stdout:4/376: chown dd/de/d10/d13/f38 705355707 1 2024-02-16T19:24:14.561 INFO:tasks.workunit.client.0.smithi040.stdout:3/527: mknod d1/dc/d35/d53/d64/d65/ca2 0 2024-02-16T19:24:14.563 INFO:tasks.workunit.client.0.smithi040.stdout:8/536: mknod d6/cae 0 2024-02-16T19:24:14.567 INFO:tasks.workunit.client.0.smithi040.stdout:9/465: unlink d3/d4/d11/d71/l81 0 2024-02-16T19:24:14.567 INFO:tasks.workunit.client.0.smithi040.stdout:9/466: read d3/d4/f8 [3217194,72931] 0 2024-02-16T19:24:14.571 INFO:tasks.workunit.client.0.smithi040.stdout:6/452: creat d0/d1/d12/d3b/f97 x:0 0 0 2024-02-16T19:24:14.571 INFO:tasks.workunit.client.0.smithi040.stdout:3/528: creat d1/d7/fa3 x:0 0 0 2024-02-16T19:24:14.571 INFO:tasks.workunit.client.0.smithi040.stdout:3/529: fdatasync d1/dc/fd 0 2024-02-16T19:24:14.571 INFO:tasks.workunit.client.0.smithi040.stdout:3/530: chown d1/dc/d35/d48/d85 382341 1 2024-02-16T19:24:14.571 INFO:tasks.workunit.client.0.smithi040.stdout:3/531: chown d1/d7/d30/f63 0 1 2024-02-16T19:24:14.576 INFO:tasks.workunit.client.0.smithi040.stdout:1/488: dwrite d17/d2d/f81 [0,4194304] 0 2024-02-16T19:24:14.582 INFO:tasks.workunit.client.0.smithi040.stdout:6/453: dread d0/d30/d5b/f85 [0,4194304] 0 2024-02-16T19:24:14.585 INFO:tasks.workunit.client.0.smithi040.stdout:6/454: unlink d0/d30/d32/c2b 0 2024-02-16T19:24:14.586 INFO:tasks.workunit.client.0.smithi040.stdout:6/455: write d0/f1d [1007508,14576] 0 2024-02-16T19:24:14.586 INFO:tasks.workunit.client.0.smithi040.stdout:6/456: write d0/d30/d5b/f92 [696449,35015] 0 2024-02-16T19:24:14.587 INFO:tasks.workunit.client.0.smithi040.stdout:6/457: creat d0/d37/d4a/f98 x:0 0 0 2024-02-16T19:24:14.589 INFO:tasks.workunit.client.0.smithi040.stdout:6/458: truncate d0/fe 156511 0 2024-02-16T19:24:14.591 INFO:tasks.workunit.client.0.smithi040.stdout:6/459: truncate d0/f55 2400770 0 2024-02-16T19:24:14.599 INFO:tasks.workunit.client.0.smithi040.stdout:3/532: dwrite d1/dc/d42/d4a/fa0 [0,4194304] 0 2024-02-16T19:24:14.599 INFO:tasks.workunit.client.0.smithi040.stdout:9/467: dwrite d3/d4/d11/d29/d38/d46/f6a [0,4194304] 0 2024-02-16T19:24:14.599 INFO:tasks.workunit.client.0.smithi040.stdout:3/533: mknod d1/dc/d42/ca4 0 2024-02-16T19:24:14.600 INFO:tasks.workunit.client.0.smithi040.stdout:1/489: dwrite d17/d2d/f2f [4194304,4194304] 0 2024-02-16T19:24:14.601 INFO:tasks.workunit.client.0.smithi040.stdout:9/468: mknod d3/d4/d11/d32/c93 0 2024-02-16T19:24:14.602 INFO:tasks.workunit.client.0.smithi040.stdout:1/490: creat d17/d3c/f9d x:0 0 0 2024-02-16T19:24:14.603 INFO:tasks.workunit.client.0.smithi040.stdout:6/460: read d0/f36 [1554024,79074] 0 2024-02-16T19:24:14.603 INFO:tasks.workunit.client.0.smithi040.stdout:6/461: dread - d0/d1/d12/d3b/d48/d4f/d8b/f94 zero size 2024-02-16T19:24:14.607 INFO:tasks.workunit.client.0.smithi040.stdout:6/462: creat d0/d1/d12/d3b/d48/d4d/d6d/f99 x:0 0 0 2024-02-16T19:24:14.607 INFO:tasks.workunit.client.0.smithi040.stdout:9/469: getdents d3/d4/d11/d71 0 2024-02-16T19:24:14.610 INFO:tasks.workunit.client.0.smithi040.stdout:6/463: mkdir d0/d1/d12/d3b/d48/d4f/d9a 0 2024-02-16T19:24:14.610 INFO:tasks.workunit.client.0.smithi040.stdout:6/464: chown d0/d1/d12/d3b/d48/d4d/d6d 52955464 1 2024-02-16T19:24:14.611 INFO:tasks.workunit.client.0.smithi040.stdout:6/465: creat d0/d1/d6f/d90/f9b x:0 0 0 2024-02-16T19:24:14.613 INFO:tasks.workunit.client.0.smithi040.stdout:6/466: creat d0/d1/f9c x:0 0 0 2024-02-16T19:24:14.613 INFO:tasks.workunit.client.0.smithi040.stdout:6/467: truncate d0/d1/d12/d3b/f50 1332490 0 2024-02-16T19:24:14.613 INFO:tasks.workunit.client.0.smithi040.stdout:6/468: chown d0/d1/d12/f28 14734 1 2024-02-16T19:24:14.613 INFO:tasks.workunit.client.0.smithi040.stdout:6/469: dread - d0/d30/d5b/f91 zero size 2024-02-16T19:24:14.616 INFO:tasks.workunit.client.0.smithi040.stdout:1/491: dwrite d17/d43/f5d [0,4194304] 0 2024-02-16T19:24:14.617 INFO:tasks.workunit.client.0.smithi040.stdout:9/470: dread d3/d4/d11/d29/f56 [0,4194304] 0 2024-02-16T19:24:14.635 INFO:tasks.workunit.client.0.smithi040.stdout:1/492: creat d17/d3c/d74/d97/f9e x:0 0 0 2024-02-16T19:24:14.636 INFO:tasks.workunit.client.0.smithi040.stdout:6/470: rmdir d0/d30/d32 39 2024-02-16T19:24:14.636 INFO:tasks.workunit.client.0.smithi040.stdout:6/471: dread - d0/d30/d5b/f91 zero size 2024-02-16T19:24:14.637 INFO:tasks.workunit.client.0.smithi040.stdout:6/472: truncate d0/d1/d12/d3b/f97 319587 0 2024-02-16T19:24:14.637 INFO:tasks.workunit.client.0.smithi040.stdout:6/473: fsync d0/d30/d5b/f8a 0 2024-02-16T19:24:14.638 INFO:tasks.workunit.client.0.smithi040.stdout:9/471: creat d3/d4/d11/d29/d38/d46/f94 x:0 0 0 2024-02-16T19:24:14.638 INFO:tasks.workunit.client.0.smithi040.stdout:9/472: getdents d3/d4/d11/d32/d8e 0 2024-02-16T19:24:14.638 INFO:tasks.workunit.client.0.smithi040.stdout:9/473: fdatasync d3/d4/d11/d29/d36/f3a 0 2024-02-16T19:24:14.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/493: link d17/d43/d38/f72 d17/d26/d54/f9f 0 2024-02-16T19:24:14.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/494: read d17/d3c/d74/f7f [3119132,72057] 0 2024-02-16T19:24:14.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/495: readlink d17/d29/d8b/l91 0 2024-02-16T19:24:14.643 INFO:tasks.workunit.client.0.smithi040.stdout:1/496: creat d17/d3c/d71/fa0 x:0 0 0 2024-02-16T19:24:14.664 INFO:tasks.workunit.client.0.smithi040.stdout:6/474: dwrite d0/f36 [0,4194304] 0 2024-02-16T19:24:14.664 INFO:tasks.workunit.client.0.smithi040.stdout:9/474: dwrite d3/d4/d55/d31/f5e [0,4194304] 0 2024-02-16T19:24:14.665 INFO:tasks.workunit.client.0.smithi040.stdout:6/475: mkdir d0/d1/d12/d3b/d48/d4f/d9d 0 2024-02-16T19:24:14.672 INFO:tasks.workunit.client.0.smithi040.stdout:6/476: read d0/d37/d4a/f5e [3337600,122576] 0 2024-02-16T19:24:14.681 INFO:tasks.workunit.client.0.smithi040.stdout:9/475: dwrite d3/d4/d11/d29/d38/d46/f6a [0,4194304] 0 2024-02-16T19:24:14.683 INFO:tasks.workunit.client.0.smithi040.stdout:9/476: unlink d3/d4/d55/d31/c59 0 2024-02-16T19:24:14.684 INFO:tasks.workunit.client.0.smithi040.stdout:9/477: creat d3/d4/d55/d8a/f95 x:0 0 0 2024-02-16T19:24:14.684 INFO:tasks.workunit.client.0.smithi040.stdout:9/478: write d3/d4/d55/d31/f4c [685915,59332] 0 2024-02-16T19:24:14.685 INFO:tasks.workunit.client.0.smithi040.stdout:9/479: dread - d3/d4/f79 zero size 2024-02-16T19:24:14.685 INFO:tasks.workunit.client.0.smithi040.stdout:9/480: creat d3/d4/f96 x:0 0 0 2024-02-16T19:24:14.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/477: dwrite d0/d30/d32/f8d [0,4194304] 0 2024-02-16T19:24:14.690 INFO:tasks.workunit.client.0.smithi040.stdout:9/481: dread d3/d4/d11/d29/d38/d46/f5b [0,4194304] 0 2024-02-16T19:24:14.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/478: write d0/d1/d12/d3b/f40 [515688,86926] 0 2024-02-16T19:24:14.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/479: write d0/d1/d12/d3b/f50 [1603923,54451] 0 2024-02-16T19:24:14.691 INFO:tasks.workunit.client.0.smithi040.stdout:9/482: write d3/d4/d11/d71/f88 [455693,126727] 0 2024-02-16T19:24:14.691 INFO:tasks.workunit.client.0.smithi040.stdout:9/483: write d3/d4/d55/f39 [1555781,58394] 0 2024-02-16T19:24:14.691 INFO:tasks.workunit.client.0.smithi040.stdout:9/484: dread - d3/d4/d11/d32/f84 zero size 2024-02-16T19:24:14.695 INFO:tasks.workunit.client.0.smithi040.stdout:9/485: dread d3/d4/d11/d71/f88 [0,4194304] 0 2024-02-16T19:24:14.695 INFO:tasks.workunit.client.0.smithi040.stdout:9/486: creat d3/d4/d11/d29/d38/d46/d6b/f97 x:0 0 0 2024-02-16T19:24:14.697 INFO:tasks.workunit.client.0.smithi040.stdout:9/487: creat d3/d4/d55/f98 x:0 0 0 2024-02-16T19:24:14.699 INFO:tasks.workunit.client.0.smithi040.stdout:9/488: mkdir d3/d4/d55/d8a/d99 0 2024-02-16T19:24:15.136 INFO:tasks.workunit.client.0.smithi040.stdout:7/395: sync 2024-02-16T19:24:15.139 INFO:tasks.workunit.client.0.smithi040.stdout:7/396: dread d21/d29/f4c [0,4194304] 0 2024-02-16T19:24:15.139 INFO:tasks.workunit.client.0.smithi040.stdout:7/397: stat d21/c68 0 2024-02-16T19:24:15.141 INFO:tasks.workunit.client.0.smithi040.stdout:7/398: dread d21/d29/f4c [0,4194304] 0 2024-02-16T19:24:15.142 INFO:tasks.workunit.client.0.smithi040.stdout:7/399: getdents d21/d29 0 2024-02-16T19:24:15.143 INFO:tasks.workunit.client.0.smithi040.stdout:7/400: creat d21/d29/d79/f80 x:0 0 0 2024-02-16T19:24:15.145 INFO:tasks.workunit.client.0.smithi040.stdout:7/401: mkdir d21/d7e/d81 0 2024-02-16T19:24:15.146 INFO:tasks.workunit.client.0.smithi040.stdout:7/402: link d21/d4d/f56 d21/d4d/d77/d7a/f82 0 2024-02-16T19:24:15.147 INFO:tasks.workunit.client.0.smithi040.stdout:7/403: creat d21/d4d/d7d/f83 x:0 0 0 2024-02-16T19:24:15.164 INFO:tasks.workunit.client.0.smithi040.stdout:7/404: dwrite d21/d29/f73 [4194304,4194304] 0 2024-02-16T19:24:15.166 INFO:tasks.workunit.client.0.smithi040.stdout:7/405: creat d21/d7e/f84 x:0 0 0 2024-02-16T19:24:15.173 INFO:tasks.workunit.client.0.smithi040.stdout:7/406: dread d21/d29/f73 [0,4194304] 0 2024-02-16T19:24:15.178 INFO:tasks.workunit.client.0.smithi040.stdout:7/407: write f16 [820161,85995] 0 2024-02-16T19:24:15.186 INFO:tasks.workunit.client.0.smithi040.stdout:7/408: write d21/d4d/f4b [1844982,116987] 0 2024-02-16T19:24:15.202 INFO:tasks.workunit.client.0.smithi040.stdout:7/409: dwrite f17 [0,4194304] 0 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:0/507: sync 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:5/408: sync 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:3/534: sync 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:1/497: sync 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:8/537: sync 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:2/404: sync 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:0/508: chown db/de/d6a/f6e 1 1 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:3/535: fdatasync d1/d4/d6e/f91 0 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:1/498: fsync d17/d2d/f73 0 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:8/538: chown d6/d2b/f34 749 1 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:8/539: fdatasync d6/d7/f7b 0 2024-02-16T19:24:15.208 INFO:tasks.workunit.client.0.smithi040.stdout:8/540: stat d6/d64/d50/d69/d81/fa7 0 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:8/541: readlink d6/lf 0 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:8/542: write d6/d2b/da3/d8b/f90 [442247,38937] 0 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:4/377: sync 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:9/489: sync 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:0/509: write db/d2f/d1d/d21/d2e/d49/fa3 [114405,118761] 0 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:9/490: chown d3/d4/f96 28289 1 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:0/510: fsync db/d2f/d23/d3b/f47 0 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:6/480: sync 2024-02-16T19:24:15.209 INFO:tasks.workunit.client.0.smithi040.stdout:0/511: truncate db/d2f/d1d/d21/d2e/d49/d5b/f5d 2707283 0 2024-02-16T19:24:15.210 INFO:tasks.workunit.client.0.smithi040.stdout:6/481: chown d0/f36 307941 1 2024-02-16T19:24:15.210 INFO:tasks.workunit.client.0.smithi040.stdout:6/482: readlink d0/d37/d4a/l84 0 2024-02-16T19:24:15.210 INFO:tasks.workunit.client.0.smithi040.stdout:2/405: symlink d5/d7/d1e/d31/d4f/d63/l84 0 2024-02-16T19:24:15.210 INFO:tasks.workunit.client.0.smithi040.stdout:7/410: dread f1d [0,4194304] 0 2024-02-16T19:24:15.211 INFO:tasks.workunit.client.0.smithi040.stdout:4/378: mkdir dd/de/d10/d13/d32/d7a 0 2024-02-16T19:24:15.213 INFO:tasks.workunit.client.0.smithi040.stdout:9/491: truncate d3/d4/d55/d31/f5e 3536369 0 2024-02-16T19:24:15.213 INFO:tasks.workunit.client.0.smithi040.stdout:8/543: dread d6/d64/f6d [0,4194304] 0 2024-02-16T19:24:15.213 INFO:tasks.workunit.client.0.smithi040.stdout:0/512: getdents db/d2f/d23/d6f 0 2024-02-16T19:24:15.214 INFO:tasks.workunit.client.0.smithi040.stdout:0/513: fsync db/d2f/d1d/d21/d2e/d49/d5b/f5d 0 2024-02-16T19:24:15.214 INFO:tasks.workunit.client.0.smithi040.stdout:0/514: truncate db/d2f/d23/d6f/d98/f9b 458749 0 2024-02-16T19:24:15.214 INFO:tasks.workunit.client.0.smithi040.stdout:0/515: write db/d25/d2c/fad [360915,65533] 0 2024-02-16T19:24:15.214 INFO:tasks.workunit.client.0.smithi040.stdout:0/516: fsync db/de/d6a/f82 0 2024-02-16T19:24:15.214 INFO:tasks.workunit.client.0.smithi040.stdout:5/409: rmdir d1/da 39 2024-02-16T19:24:15.214 INFO:tasks.workunit.client.0.smithi040.stdout:5/410: fdatasync d1/d3c/f43 0 2024-02-16T19:24:15.215 INFO:tasks.workunit.client.0.smithi040.stdout:6/483: creat d0/d1/d12/d3b/d48/f9e x:0 0 0 2024-02-16T19:24:15.215 INFO:tasks.workunit.client.0.smithi040.stdout:6/484: chown d0/d1/d12/d3b/d48/f9e 220824 1 2024-02-16T19:24:15.216 INFO:tasks.workunit.client.0.smithi040.stdout:3/536: write d1/d7/f4e [809461,66839] 0 2024-02-16T19:24:15.217 INFO:tasks.workunit.client.0.smithi040.stdout:2/406: link d5/d5c/l5f d5/d7/d1e/d31/l85 0 2024-02-16T19:24:15.217 INFO:tasks.workunit.client.0.smithi040.stdout:2/407: chown d5/d7/d12/d1b/l2a 0 1 2024-02-16T19:24:15.219 INFO:tasks.workunit.client.0.smithi040.stdout:3/537: dread d1/d7/d36/f45 [0,4194304] 0 2024-02-16T19:24:15.219 INFO:tasks.workunit.client.0.smithi040.stdout:3/538: fdatasync d1/dc/d42/d58/f76 0 2024-02-16T19:24:15.219 INFO:tasks.workunit.client.0.smithi040.stdout:3/539: fdatasync d1/dc/d42/d58/f72 0 2024-02-16T19:24:15.220 INFO:tasks.workunit.client.0.smithi040.stdout:3/540: chown d1/dc/d35/d53/d64/d65/f98 3396149 1 2024-02-16T19:24:15.220 INFO:tasks.workunit.client.0.smithi040.stdout:4/379: fdatasync dd/de/d45/f5c 0 2024-02-16T19:24:15.220 INFO:tasks.workunit.client.0.smithi040.stdout:4/380: chown dd/de/d10/l21 0 1 2024-02-16T19:24:15.224 INFO:tasks.workunit.client.0.smithi040.stdout:9/492: link d3/d4/d11/d32/c93 d3/c9a 0 2024-02-16T19:24:15.224 INFO:tasks.workunit.client.0.smithi040.stdout:8/544: truncate d6/d7/d15/f85 2712911 0 2024-02-16T19:24:15.224 INFO:tasks.workunit.client.0.smithi040.stdout:0/517: write db/d25/f4f [1635314,33282] 0 2024-02-16T19:24:15.224 INFO:tasks.workunit.client.0.smithi040.stdout:0/518: readlink db/d2f/d1d/l31 0 2024-02-16T19:24:15.225 INFO:tasks.workunit.client.0.smithi040.stdout:5/411: link d1/d45/d63/c56 d1/da/d1c/d73/c7f 0 2024-02-16T19:24:15.225 INFO:tasks.workunit.client.0.smithi040.stdout:1/499: dwrite d17/d2d/f81 [0,4194304] 0 2024-02-16T19:24:15.227 INFO:tasks.workunit.client.0.smithi040.stdout:6/485: creat d0/d30/d5b/f9f x:0 0 0 2024-02-16T19:24:15.232 INFO:tasks.workunit.client.0.smithi040.stdout:2/408: creat d5/d7/d10/f86 x:0 0 0 2024-02-16T19:24:15.232 INFO:tasks.workunit.client.0.smithi040.stdout:2/409: read d5/d7/d1e/f25 [5747390,8258] 0 2024-02-16T19:24:15.234 INFO:tasks.workunit.client.0.smithi040.stdout:3/541: mkdir d1/d4/d6e/d8a/da5 0 2024-02-16T19:24:15.234 INFO:tasks.workunit.client.0.smithi040.stdout:3/542: write d1/d7/d30/f40 [999048,27691] 0 2024-02-16T19:24:15.234 INFO:tasks.workunit.client.0.smithi040.stdout:3/543: write d1/d7/d36/f7b [1424714,124283] 0 2024-02-16T19:24:15.234 INFO:tasks.workunit.client.0.smithi040.stdout:3/544: write d1/d7/fa3 [133701,81138] 0 2024-02-16T19:24:15.236 INFO:tasks.workunit.client.0.smithi040.stdout:4/381: rename dd/de/f44 to dd/de/d10/d13/d77/f7b 0 2024-02-16T19:24:15.237 INFO:tasks.workunit.client.0.smithi040.stdout:5/412: read d1/d3c/f77 [3650177,20403] 0 2024-02-16T19:24:15.238 INFO:tasks.workunit.client.0.smithi040.stdout:7/411: mknod d21/c85 0 2024-02-16T19:24:15.240 INFO:tasks.workunit.client.0.smithi040.stdout:9/493: creat d3/d4/d55/d31/f9b x:0 0 0 2024-02-16T19:24:15.241 INFO:tasks.workunit.client.0.smithi040.stdout:8/545: creat d6/d64/d50/d69/faf x:0 0 0 2024-02-16T19:24:15.244 INFO:tasks.workunit.client.0.smithi040.stdout:6/486: link d0/d1/f16 d0/fa0 0 2024-02-16T19:24:15.246 INFO:tasks.workunit.client.0.smithi040.stdout:2/410: creat d5/d8/d42/f87 x:0 0 0 2024-02-16T19:24:15.246 INFO:tasks.workunit.client.0.smithi040.stdout:2/411: dread - d5/d7/d1e/d55/f82 zero size 2024-02-16T19:24:15.247 INFO:tasks.workunit.client.0.smithi040.stdout:3/545: creat d1/dc/d35/fa6 x:0 0 0 2024-02-16T19:24:15.247 INFO:tasks.workunit.client.0.smithi040.stdout:3/546: stat d1/dc/d35/d55/f74 0 2024-02-16T19:24:15.249 INFO:tasks.workunit.client.0.smithi040.stdout:4/382: chown dd/de/d45/c48 86514 1 2024-02-16T19:24:15.249 INFO:tasks.workunit.client.0.smithi040.stdout:4/383: chown dd/de/d10/d13/d77/l76 76 1 2024-02-16T19:24:15.251 INFO:tasks.workunit.client.0.smithi040.stdout:9/494: mknod d3/d4/d11/c9c 0 2024-02-16T19:24:15.252 INFO:tasks.workunit.client.0.smithi040.stdout:0/519: dwrite db/d2f/d1d/d21/d2e/d49/fa9 [0,4194304] 0 2024-02-16T19:24:15.253 INFO:tasks.workunit.client.0.smithi040.stdout:8/546: creat d6/d2b/da3/d8b/fb0 x:0 0 0 2024-02-16T19:24:15.255 INFO:tasks.workunit.client.0.smithi040.stdout:6/487: mknod d0/d1/d12/d3b/d48/d4d/ca1 0 2024-02-16T19:24:15.256 INFO:tasks.workunit.client.0.smithi040.stdout:2/412: symlink d5/d7/d1e/d31/d78/l88 0 2024-02-16T19:24:15.256 INFO:tasks.workunit.client.0.smithi040.stdout:2/413: readlink d5/d7/l32 0 2024-02-16T19:24:15.257 INFO:tasks.workunit.client.0.smithi040.stdout:8/547: dread d6/d2b/f80 [0,4194304] 0 2024-02-16T19:24:15.257 INFO:tasks.workunit.client.0.smithi040.stdout:5/413: dwrite d1/da/f2e [0,4194304] 0 2024-02-16T19:24:15.257 INFO:tasks.workunit.client.0.smithi040.stdout:5/414: write d1/d62/f6f [604498,77449] 0 2024-02-16T19:24:15.258 INFO:tasks.workunit.client.0.smithi040.stdout:7/412: rename d21/l5e to d21/d4d/d7b/l86 0 2024-02-16T19:24:15.260 INFO:tasks.workunit.client.0.smithi040.stdout:1/500: dwrite d17/d26/d3d/f56 [0,4194304] 0 2024-02-16T19:24:15.263 INFO:tasks.workunit.client.0.smithi040.stdout:3/547: dread d1/d7/f4e [0,4194304] 0 2024-02-16T19:24:15.263 INFO:tasks.workunit.client.0.smithi040.stdout:3/548: stat d1/d7/f4e 0 2024-02-16T19:24:15.264 INFO:tasks.workunit.client.0.smithi040.stdout:7/413: unlink f7 0 2024-02-16T19:24:15.266 INFO:tasks.workunit.client.0.smithi040.stdout:7/414: fsync d21/d29/f4c 0 2024-02-16T19:24:15.266 INFO:tasks.workunit.client.0.smithi040.stdout:1/501: dread d17/d26/d54/f9f [0,4194304] 0 2024-02-16T19:24:15.267 INFO:tasks.workunit.client.0.smithi040.stdout:9/495: rename d3/d4/d55/f5a to d3/d4/d55/d8a/d99/f9d 0 2024-02-16T19:24:15.267 INFO:tasks.workunit.client.0.smithi040.stdout:9/496: truncate d3/d4/d55/d8a/f8f 662450 0 2024-02-16T19:24:15.267 INFO:tasks.workunit.client.0.smithi040.stdout:9/497: write d3/d4/d11/d29/d38/d46/f94 [187895,106810] 0 2024-02-16T19:24:15.267 INFO:tasks.workunit.client.0.smithi040.stdout:9/498: write d3/d4/f57 [1417625,80715] 0 2024-02-16T19:24:15.269 INFO:tasks.workunit.client.0.smithi040.stdout:7/415: mkdir d21/d4d/d77/d87 0 2024-02-16T19:24:15.269 INFO:tasks.workunit.client.0.smithi040.stdout:7/416: stat d21/d7e/d81 0 2024-02-16T19:24:15.269 INFO:tasks.workunit.client.0.smithi040.stdout:5/415: rmdir d1/d3c 39 2024-02-16T19:24:15.270 INFO:tasks.workunit.client.0.smithi040.stdout:8/548: read d6/fa5 [2303235,73888] 0 2024-02-16T19:24:15.270 INFO:tasks.workunit.client.0.smithi040.stdout:0/520: dread db/d2f/f66 [0,4194304] 0 2024-02-16T19:24:15.270 INFO:tasks.workunit.client.0.smithi040.stdout:0/521: chown db/d2f/d23/d3b 627005 1 2024-02-16T19:24:15.271 INFO:tasks.workunit.client.0.smithi040.stdout:0/522: chown db/d2f/d23/l29 4340 1 2024-02-16T19:24:15.271 INFO:tasks.workunit.client.0.smithi040.stdout:4/384: dwrite dd/de/d10/f2e [4194304,4194304] 0 2024-02-16T19:24:15.271 INFO:tasks.workunit.client.0.smithi040.stdout:4/385: stat dd/f52 0 2024-02-16T19:24:15.271 INFO:tasks.workunit.client.0.smithi040.stdout:7/417: rename f1d to d21/d4d/d7d/f88 0 2024-02-16T19:24:15.271 INFO:tasks.workunit.client.0.smithi040.stdout:7/418: readlink d21/d29/l2c 0 2024-02-16T19:24:15.272 INFO:tasks.workunit.client.0.smithi040.stdout:8/549: link d6/d2b/f9c d6/d64/fb1 0 2024-02-16T19:24:15.272 INFO:tasks.workunit.client.0.smithi040.stdout:8/550: fsync d6/f9 0 2024-02-16T19:24:15.273 INFO:tasks.workunit.client.0.smithi040.stdout:0/523: mknod db/de/d6a/cb0 0 2024-02-16T19:24:15.273 INFO:tasks.workunit.client.0.smithi040.stdout:0/524: chown db/d25/l5f 6 1 2024-02-16T19:24:15.275 INFO:tasks.workunit.client.0.smithi040.stdout:4/386: symlink dd/de/d10/d36/l7c 0 2024-02-16T19:24:15.277 INFO:tasks.workunit.client.0.smithi040.stdout:7/419: symlink d21/d29/l89 0 2024-02-16T19:24:15.278 INFO:tasks.workunit.client.0.smithi040.stdout:8/551: mknod d6/d2b/da3/d8b/cb2 0 2024-02-16T19:24:15.278 INFO:tasks.workunit.client.0.smithi040.stdout:5/416: write d1/da/f2e [2505773,68653] 0 2024-02-16T19:24:15.279 INFO:tasks.workunit.client.0.smithi040.stdout:4/387: mknod dd/d58/d68/c7d 0 2024-02-16T19:24:15.279 INFO:tasks.workunit.client.0.smithi040.stdout:4/388: dread - dd/f6e zero size 2024-02-16T19:24:15.280 INFO:tasks.workunit.client.0.smithi040.stdout:7/420: truncate d21/f35 4535028 0 2024-02-16T19:24:15.283 INFO:tasks.workunit.client.0.smithi040.stdout:5/417: creat d1/da/f80 x:0 0 0 2024-02-16T19:24:15.283 INFO:tasks.workunit.client.0.smithi040.stdout:5/418: truncate d1/da/d51/f79 148604 0 2024-02-16T19:24:15.283 INFO:tasks.workunit.client.0.smithi040.stdout:5/419: truncate d1/da/d1c/d21/f64 4629631 0 2024-02-16T19:24:15.283 INFO:tasks.workunit.client.0.smithi040.stdout:5/420: write d1/d3c/f68 [431185,5855] 0 2024-02-16T19:24:15.286 INFO:tasks.workunit.client.0.smithi040.stdout:1/502: dread d17/d26/f7d [4194304,4194304] 0 2024-02-16T19:24:15.288 INFO:tasks.workunit.client.0.smithi040.stdout:5/421: rename d1/d62/f6f to d1/da/d51/f81 0 2024-02-16T19:24:15.289 INFO:tasks.workunit.client.0.smithi040.stdout:5/422: dread d1/da/d51/f79 [0,4194304] 0 2024-02-16T19:24:15.289 INFO:tasks.workunit.client.0.smithi040.stdout:5/423: fdatasync d1/d3c/f72 0 2024-02-16T19:24:15.290 INFO:tasks.workunit.client.0.smithi040.stdout:9/499: read d3/d4/d11/d29/d38/d46/f6a [532146,101097] 0 2024-02-16T19:24:15.290 INFO:tasks.workunit.client.0.smithi040.stdout:6/488: dwrite d0/d1/d12/d3b/d48/d4d/d6d/f99 [0,4194304] 0 2024-02-16T19:24:15.290 INFO:tasks.workunit.client.0.smithi040.stdout:6/489: creat d0/d1/d12/d3b/d79/fa2 x:0 0 0 2024-02-16T19:24:15.292 INFO:tasks.workunit.client.0.smithi040.stdout:2/414: dwrite d5/d5c/d65/f60 [0,4194304] 0 2024-02-16T19:24:15.293 INFO:tasks.workunit.client.0.smithi040.stdout:2/415: chown d5/d7/d1e 43949 1 2024-02-16T19:24:15.301 INFO:tasks.workunit.client.0.smithi040.stdout:6/490: symlink d0/d1/d12/d3b/d48/d4d/d6d/d87/la3 0 2024-02-16T19:24:15.302 INFO:tasks.workunit.client.0.smithi040.stdout:8/552: dwrite d6/fa5 [0,4194304] 0 2024-02-16T19:24:15.302 INFO:tasks.workunit.client.0.smithi040.stdout:8/553: chown d6/d2b/d44/f7f 86299 1 2024-02-16T19:24:15.304 INFO:tasks.workunit.client.0.smithi040.stdout:3/549: dwrite d1/d7/d36/f7c [0,4194304] 0 2024-02-16T19:24:15.307 INFO:tasks.workunit.client.0.smithi040.stdout:2/416: mkdir d5/d7/d89 0 2024-02-16T19:24:15.311 INFO:tasks.workunit.client.0.smithi040.stdout:5/424: rename d1/d3c/f3f to d1/da/d1c/f82 0 2024-02-16T19:24:15.311 INFO:tasks.workunit.client.0.smithi040.stdout:5/425: stat d1/d45/d63/f74 0 2024-02-16T19:24:15.318 INFO:tasks.workunit.client.0.smithi040.stdout:6/491: mknod d0/d1/d12/d3b/d48/d4d/d6d/d87/ca4 0 2024-02-16T19:24:15.318 INFO:tasks.workunit.client.0.smithi040.stdout:6/492: dread - d0/d1/d12/d3b/d48/d73/d76/d7d/f80 zero size 2024-02-16T19:24:15.318 INFO:tasks.workunit.client.0.smithi040.stdout:6/493: chown d0/d1/d12/d3b/d48/d4d/d6d/d87/f8e 1841366 1 2024-02-16T19:24:15.323 INFO:tasks.workunit.client.0.smithi040.stdout:2/417: mknod d5/d5c/d65/c8a 0 2024-02-16T19:24:15.331 INFO:tasks.workunit.client.0.smithi040.stdout:0/525: dwrite db/d25/f4f [0,4194304] 0 2024-02-16T19:24:15.331 INFO:tasks.workunit.client.0.smithi040.stdout:0/526: readlink db/d2f/d1d/d21/d2e/d85/d89/la1 0 2024-02-16T19:24:15.331 INFO:tasks.workunit.client.0.smithi040.stdout:8/554: dwrite d6/d7/d1d/f29 [0,4194304] 0 2024-02-16T19:24:15.331 INFO:tasks.workunit.client.0.smithi040.stdout:8/555: chown d6/d64/d50/f84 78218669 1 2024-02-16T19:24:15.331 INFO:tasks.workunit.client.0.smithi040.stdout:8/556: chown d6/d12/d51/fa1 0 1 2024-02-16T19:24:15.331 INFO:tasks.workunit.client.0.smithi040.stdout:8/557: dread - d6/d7/f9b zero size 2024-02-16T19:24:15.331 INFO:tasks.workunit.client.0.smithi040.stdout:8/558: truncate d6/d12/d51/d6e/f96 819401 0 2024-02-16T19:24:15.331 INFO:tasks.workunit.client.0.smithi040.stdout:4/389: dwrite dd/d58/d5e/f69 [0,4194304] 0 2024-02-16T19:24:15.332 INFO:tasks.workunit.client.0.smithi040.stdout:9/500: rename d3/d4/f79 to d3/d1c/f9e 0 2024-02-16T19:24:15.332 INFO:tasks.workunit.client.0.smithi040.stdout:9/501: creat d3/d4/d55/d8a/f9f x:0 0 0 2024-02-16T19:24:15.332 INFO:tasks.workunit.client.0.smithi040.stdout:0/527: read db/f7f [14658,63608] 0 2024-02-16T19:24:15.332 INFO:tasks.workunit.client.0.smithi040.stdout:0/528: fsync db/de/d6a/f6e 0 2024-02-16T19:24:15.333 INFO:tasks.workunit.client.0.smithi040.stdout:6/494: getdents d0/d1/d12/d3b/d48/d73/d76/d7d 0 2024-02-16T19:24:15.333 INFO:tasks.workunit.client.0.smithi040.stdout:8/559: dread d6/d7/f32 [0,4194304] 0 2024-02-16T19:24:15.334 INFO:tasks.workunit.client.0.smithi040.stdout:2/418: unlink d5/d7/d12/d1b/f30 0 2024-02-16T19:24:15.334 INFO:tasks.workunit.client.0.smithi040.stdout:8/560: dread d6/d2b/da3/f53 [0,4194304] 0 2024-02-16T19:24:15.335 INFO:tasks.workunit.client.0.smithi040.stdout:0/529: write db/de/d6a/f6e [909618,101037] 0 2024-02-16T19:24:15.341 INFO:tasks.workunit.client.0.smithi040.stdout:3/550: rename d1/dc/d35/d48/c8b to d1/dc/d35/d53/d64/ca7 0 2024-02-16T19:24:15.344 INFO:tasks.workunit.client.0.smithi040.stdout:9/502: mknod d3/d4/d55/d70/ca0 0 2024-02-16T19:24:15.344 INFO:tasks.workunit.client.0.smithi040.stdout:9/503: chown d3/c25 0 1 2024-02-16T19:24:15.345 INFO:tasks.workunit.client.0.smithi040.stdout:5/426: mknod d1/da/d1c/c83 0 2024-02-16T19:24:15.348 INFO:tasks.workunit.client.0.smithi040.stdout:2/419: mkdir d5/d7/d1e/d31/d4f/d63/d8b 0 2024-02-16T19:24:15.349 INFO:tasks.workunit.client.0.smithi040.stdout:8/561: symlink d6/d12/d93/lb3 0 2024-02-16T19:24:15.349 INFO:tasks.workunit.client.0.smithi040.stdout:8/562: read d6/d7/d1d/f52 [166496,77341] 0 2024-02-16T19:24:15.350 INFO:tasks.workunit.client.0.smithi040.stdout:8/563: fsync d6/f8 0 2024-02-16T19:24:15.352 INFO:tasks.workunit.client.0.smithi040.stdout:4/390: unlink dd/de/d45/c39 0 2024-02-16T19:24:15.354 INFO:tasks.workunit.client.0.smithi040.stdout:0/530: rename db/de/f7d to db/d2f/d1d/d81/fb1 0 2024-02-16T19:24:15.357 INFO:tasks.workunit.client.0.smithi040.stdout:9/504: link d3/l22 d3/d4/d55/d70/la1 0 2024-02-16T19:24:15.358 INFO:tasks.workunit.client.0.smithi040.stdout:2/420: dread d5/d7/d12/d1b/f2f [0,4194304] 0 2024-02-16T19:24:15.359 INFO:tasks.workunit.client.0.smithi040.stdout:4/391: symlink dd/de/d10/d36/l7e 0 2024-02-16T19:24:15.363 INFO:tasks.workunit.client.0.smithi040.stdout:5/427: dwrite d1/d3c/f77 [4194304,4194304] 0 2024-02-16T19:24:15.363 INFO:tasks.workunit.client.0.smithi040.stdout:9/505: dread d3/d4/d11/d29/f48 [0,4194304] 0 2024-02-16T19:24:15.363 INFO:tasks.workunit.client.0.smithi040.stdout:9/506: dread - d3/d4/d11/d32/f84 zero size 2024-02-16T19:24:15.363 INFO:tasks.workunit.client.0.smithi040.stdout:3/551: rename d1/d4/f2e to d1/d7/d30/fa8 0 2024-02-16T19:24:15.363 INFO:tasks.workunit.client.0.smithi040.stdout:8/564: write d6/d2b/f34 [1298215,128896] 0 2024-02-16T19:24:15.363 INFO:tasks.workunit.client.0.smithi040.stdout:8/565: write d6/d7/d1d/f7a [1184935,38956] 0 2024-02-16T19:24:15.364 INFO:tasks.workunit.client.0.smithi040.stdout:8/566: chown d6/f8 7 1 2024-02-16T19:24:15.364 INFO:tasks.workunit.client.0.smithi040.stdout:8/567: readlink d6/d2b/l55 0 2024-02-16T19:24:15.364 INFO:tasks.workunit.client.0.smithi040.stdout:8/568: readlink d6/d64/d50/l8e 0 2024-02-16T19:24:15.364 INFO:tasks.workunit.client.0.smithi040.stdout:0/531: getdents db/d2f/d1d/d21/d2e 0 2024-02-16T19:24:15.368 INFO:tasks.workunit.client.0.smithi040.stdout:6/495: rmdir d0/d1 39 2024-02-16T19:24:15.368 INFO:tasks.workunit.client.0.smithi040.stdout:6/496: getdents d0/d1/d12/d3b/d48/d4f/d9d 0 2024-02-16T19:24:15.370 INFO:tasks.workunit.client.0.smithi040.stdout:4/392: creat dd/d58/d5e/f7f x:0 0 0 2024-02-16T19:24:15.370 INFO:tasks.workunit.client.0.smithi040.stdout:4/393: readlink dd/de/d10/d13/d77/l76 0 2024-02-16T19:24:15.370 INFO:tasks.workunit.client.0.smithi040.stdout:4/394: chown dd/de/d10/l21 2051 1 2024-02-16T19:24:15.373 INFO:tasks.workunit.client.0.smithi040.stdout:5/428: creat d1/da/d7e/f84 x:0 0 0 2024-02-16T19:24:15.373 INFO:tasks.workunit.client.0.smithi040.stdout:5/429: fsync d1/d3c/f43 0 2024-02-16T19:24:15.373 INFO:tasks.workunit.client.0.smithi040.stdout:5/430: truncate d1/da/d51/f6c 684643 0 2024-02-16T19:24:15.373 INFO:tasks.workunit.client.0.smithi040.stdout:5/431: dread - d1/da/d7e/f84 zero size 2024-02-16T19:24:15.373 INFO:tasks.workunit.client.0.smithi040.stdout:3/552: dread d1/dc/d42/d4a/fa0 [0,4194304] 0 2024-02-16T19:24:15.374 INFO:tasks.workunit.client.0.smithi040.stdout:3/553: write d1/dc/fd [289097,11863] 0 2024-02-16T19:24:15.374 INFO:tasks.workunit.client.0.smithi040.stdout:3/554: truncate d1/d7/d30/f7f 318300 0 2024-02-16T19:24:15.374 INFO:tasks.workunit.client.0.smithi040.stdout:9/507: mkdir d3/d4/d11/d71/da2 0 2024-02-16T19:24:15.377 INFO:tasks.workunit.client.0.smithi040.stdout:8/569: rmdir d6/d12/d22/da6 0 2024-02-16T19:24:15.378 INFO:tasks.workunit.client.0.smithi040.stdout:8/570: truncate d6/d12/d93/fa9 421113 0 2024-02-16T19:24:15.379 INFO:tasks.workunit.client.0.smithi040.stdout:2/421: dwrite d5/d7/d1e/f33 [0,4194304] 0 2024-02-16T19:24:15.380 INFO:tasks.workunit.client.0.smithi040.stdout:0/532: unlink db/c1e 0 2024-02-16T19:24:15.381 INFO:tasks.workunit.client.0.smithi040.stdout:6/497: rmdir d0/d1/d12/d3b/d48/d73/d70/d83 39 2024-02-16T19:24:15.382 INFO:tasks.workunit.client.0.smithi040.stdout:4/395: creat dd/de/f80 x:0 0 0 2024-02-16T19:24:15.382 INFO:tasks.workunit.client.0.smithi040.stdout:4/396: chown dd/d58/d68 6 1 2024-02-16T19:24:15.382 INFO:tasks.workunit.client.0.smithi040.stdout:4/397: truncate dd/de/d10/f55 3554863 0 2024-02-16T19:24:15.384 INFO:tasks.workunit.client.0.smithi040.stdout:5/432: truncate d1/f2c 4362717 0 2024-02-16T19:24:15.386 INFO:tasks.workunit.client.0.smithi040.stdout:3/555: creat d1/dc/d35/d53/fa9 x:0 0 0 2024-02-16T19:24:15.388 INFO:tasks.workunit.client.0.smithi040.stdout:9/508: mknod d3/d4/d11/d32/ca3 0 2024-02-16T19:24:15.391 INFO:tasks.workunit.client.0.smithi040.stdout:3/556: dread d1/d4/f4f [0,4194304] 0 2024-02-16T19:24:15.391 INFO:tasks.workunit.client.0.smithi040.stdout:3/557: write d1/dc/d42/d4a/f54 [1293982,77544] 0 2024-02-16T19:24:15.391 INFO:tasks.workunit.client.0.smithi040.stdout:2/422: mknod d5/d7/d12/c8c 0 2024-02-16T19:24:15.391 INFO:tasks.workunit.client.0.smithi040.stdout:2/423: chown d5/d7/ce 15640 1 2024-02-16T19:24:15.392 INFO:tasks.workunit.client.0.smithi040.stdout:4/398: write dd/de/d10/f20 [578640,49661] 0 2024-02-16T19:24:15.392 INFO:tasks.workunit.client.0.smithi040.stdout:4/399: chown dd/de/d10/d13/d32/d7a 141383 1 2024-02-16T19:24:15.394 INFO:tasks.workunit.client.0.smithi040.stdout:0/533: creat db/d2f/d1d/d21/d2e/d85/d89/d94/fb2 x:0 0 0 2024-02-16T19:24:15.398 INFO:tasks.workunit.client.0.smithi040.stdout:5/433: creat d1/da/d51/f85 x:0 0 0 2024-02-16T19:24:15.398 INFO:tasks.workunit.client.0.smithi040.stdout:5/434: stat d1/da/d51/f81 0 2024-02-16T19:24:15.399 INFO:tasks.workunit.client.0.smithi040.stdout:9/509: rmdir d3/d4/d55/d70 39 2024-02-16T19:24:15.399 INFO:tasks.workunit.client.0.smithi040.stdout:5/435: dread d1/da/d51/f79 [0,4194304] 0 2024-02-16T19:24:15.399 INFO:tasks.workunit.client.0.smithi040.stdout:3/558: truncate d1/d7/d36/f45 893521 0 2024-02-16T19:24:15.401 INFO:tasks.workunit.client.0.smithi040.stdout:2/424: rename d5/d7/d1e/d31/d78 to d5/d7/d1e/d55/d8d 0 2024-02-16T19:24:15.402 INFO:tasks.workunit.client.0.smithi040.stdout:2/425: fdatasync d5/d7/d1e/d4c/d52/f5d 0 2024-02-16T19:24:15.402 INFO:tasks.workunit.client.0.smithi040.stdout:6/498: dwrite d0/f4 [0,4194304] 0 2024-02-16T19:24:15.402 INFO:tasks.workunit.client.0.smithi040.stdout:6/499: fdatasync d0/d1/d12/d3b/f40 0 2024-02-16T19:24:15.402 INFO:tasks.workunit.client.0.smithi040.stdout:6/500: write d0/f4 [4494711,114263] 0 2024-02-16T19:24:15.404 INFO:tasks.workunit.client.0.smithi040.stdout:0/534: creat db/d2f/d1d/d21/d2e/d49/d5b/fb3 x:0 0 0 2024-02-16T19:24:15.404 INFO:tasks.workunit.client.0.smithi040.stdout:0/535: dread - db/d25/d2c/f6d zero size 2024-02-16T19:24:15.405 INFO:tasks.workunit.client.0.smithi040.stdout:5/436: getdents d1/da/d1c/d21 0 2024-02-16T19:24:15.405 INFO:tasks.workunit.client.0.smithi040.stdout:5/437: fdatasync d1/da/f2a 0 2024-02-16T19:24:15.407 INFO:tasks.workunit.client.0.smithi040.stdout:3/559: mkdir d1/dc/d35/d53/d64/d65/daa 0 2024-02-16T19:24:15.409 INFO:tasks.workunit.client.0.smithi040.stdout:2/426: link d5/d5c/f66 d5/d7/d1e/d31/d4f/d63/f8e 0 2024-02-16T19:24:15.411 INFO:tasks.workunit.client.0.smithi040.stdout:6/501: creat d0/d1/d12/d3b/d48/d73/d70/fa5 x:0 0 0 2024-02-16T19:24:15.412 INFO:tasks.workunit.client.0.smithi040.stdout:6/502: creat d0/d1/d12/d3b/d48/d73/d70/fa6 x:0 0 0 2024-02-16T19:24:15.412 INFO:tasks.workunit.client.0.smithi040.stdout:9/510: rmdir d3/d4/d11 39 2024-02-16T19:24:15.413 INFO:tasks.workunit.client.0.smithi040.stdout:4/400: rmdir dd/d58/d5e 39 2024-02-16T19:24:15.415 INFO:tasks.workunit.client.0.smithi040.stdout:0/536: getdents db/d2f/d23/d6f/d98 0 2024-02-16T19:24:15.417 INFO:tasks.workunit.client.0.smithi040.stdout:3/560: mknod d1/dc/d42/d4a/cab 0 2024-02-16T19:24:15.417 INFO:tasks.workunit.client.0.smithi040.stdout:3/561: chown d1/dc/d42/d4a/f50 395780 1 2024-02-16T19:24:15.419 INFO:tasks.workunit.client.0.smithi040.stdout:2/427: truncate d5/d7/d1e/f25 3644925 0 2024-02-16T19:24:15.422 INFO:tasks.workunit.client.0.smithi040.stdout:6/503: getdents d0/d1/d12/d3b/d48/d4f 0 2024-02-16T19:24:15.427 INFO:tasks.workunit.client.0.smithi040.stdout:4/401: truncate dd/de/d10/d13/d77/f3b 3862219 0 2024-02-16T19:24:15.428 INFO:tasks.workunit.client.0.smithi040.stdout:2/428: truncate d5/d7/d1e/d31/f43 173304 0 2024-02-16T19:24:15.428 INFO:tasks.workunit.client.0.smithi040.stdout:2/429: creat d5/d7/d10/f8f x:0 0 0 2024-02-16T19:24:15.428 INFO:tasks.workunit.client.0.smithi040.stdout:5/438: dwrite d1/da/d51/f6c [0,4194304] 0 2024-02-16T19:24:15.428 INFO:tasks.workunit.client.0.smithi040.stdout:5/439: readlink d1/da/l15 0 2024-02-16T19:24:15.428 INFO:tasks.workunit.client.0.smithi040.stdout:5/440: fdatasync d1/da/d51/f81 0 2024-02-16T19:24:15.428 INFO:tasks.workunit.client.0.smithi040.stdout:3/562: dread d1/d7/d36/f45 [0,4194304] 0 2024-02-16T19:24:15.428 INFO:tasks.workunit.client.0.smithi040.stdout:3/563: write d1/dc/d42/d58/f93 [849994,125314] 0 2024-02-16T19:24:15.434 INFO:tasks.workunit.client.0.smithi040.stdout:6/504: link d0/f55 d0/d1/d12/d3b/d48/d73/fa7 0 2024-02-16T19:24:15.436 INFO:tasks.workunit.client.0.smithi040.stdout:4/402: rename dd/de/d10/d13/f38 to dd/de/d10/d36/f81 0 2024-02-16T19:24:15.437 INFO:tasks.workunit.client.0.smithi040.stdout:4/403: fdatasync dd/f19 0 2024-02-16T19:24:15.442 INFO:tasks.workunit.client.0.smithi040.stdout:4/404: read dd/de/d10/f30 [1662757,98243] 0 2024-02-16T19:24:15.443 INFO:tasks.workunit.client.0.smithi040.stdout:6/505: creat d0/d1/d6f/d90/fa8 x:0 0 0 2024-02-16T19:24:15.445 INFO:tasks.workunit.client.0.smithi040.stdout:3/564: dwrite d1/d4/f19 [0,4194304] 0 2024-02-16T19:24:15.445 INFO:tasks.workunit.client.0.smithi040.stdout:3/565: read - d1/dc/d35/d48/d85/f90 zero size 2024-02-16T19:24:15.445 INFO:tasks.workunit.client.0.smithi040.stdout:3/566: write d1/dc/d42/d4a/f61 [102704,99357] 0 2024-02-16T19:24:15.445 INFO:tasks.workunit.client.0.smithi040.stdout:9/511: dwrite d3/d4/d11/d32/f58 [0,4194304] 0 2024-02-16T19:24:15.447 INFO:tasks.workunit.client.0.smithi040.stdout:2/430: rename d5/d7/d1e/d4c/d52/c5a to d5/d7/d1e/d55/d8d/c90 0 2024-02-16T19:24:15.447 INFO:tasks.workunit.client.0.smithi040.stdout:2/431: read - d5/d7/d1e/d4c/d52/f5d zero size 2024-02-16T19:24:15.449 INFO:tasks.workunit.client.0.smithi040.stdout:5/441: truncate d1/da/d1c/f1f 3813081 0 2024-02-16T19:24:15.449 INFO:tasks.workunit.client.0.smithi040.stdout:5/442: creat d1/da/d7e/f86 x:0 0 0 2024-02-16T19:24:15.449 INFO:tasks.workunit.client.0.smithi040.stdout:4/405: mkdir dd/de/d82 0 2024-02-16T19:24:15.450 INFO:tasks.workunit.client.0.smithi040.stdout:6/506: symlink d0/d30/d32/la9 0 2024-02-16T19:24:15.453 INFO:tasks.workunit.client.0.smithi040.stdout:2/432: dread d5/d7/d12/f15 [0,4194304] 0 2024-02-16T19:24:15.453 INFO:tasks.workunit.client.0.smithi040.stdout:3/567: mknod d1/dc/d42/d58/cac 0 2024-02-16T19:24:15.456 INFO:tasks.workunit.client.0.smithi040.stdout:9/512: mkdir d3/d4/d11/d29/d38/d86/da4 0 2024-02-16T19:24:15.456 INFO:tasks.workunit.client.0.smithi040.stdout:9/513: chown d3/d4/d11/d71 250786 1 2024-02-16T19:24:15.462 INFO:tasks.workunit.client.0.smithi040.stdout:4/406: truncate dd/de/f12 3605780 0 2024-02-16T19:24:15.463 INFO:tasks.workunit.client.0.smithi040.stdout:2/433: creat d5/d7/d10/d40/f91 x:0 0 0 2024-02-16T19:24:15.464 INFO:tasks.workunit.client.0.smithi040.stdout:2/434: write d5/d7/d1e/d4c/d52/f5d [352349,84862] 0 2024-02-16T19:24:15.465 INFO:tasks.workunit.client.0.smithi040.stdout:3/568: creat d1/dc/d42/d5c/fad x:0 0 0 2024-02-16T19:24:15.465 INFO:tasks.workunit.client.0.smithi040.stdout:3/569: truncate d1/dc/d35/d48/d85/f90 1046446 0 2024-02-16T19:24:15.466 INFO:tasks.workunit.client.0.smithi040.stdout:3/570: read d1/d7/d30/f63 [238661,81030] 0 2024-02-16T19:24:15.466 INFO:tasks.workunit.client.0.smithi040.stdout:3/571: read d1/dc/d42/d4a/f54 [694332,103111] 0 2024-02-16T19:24:15.467 INFO:tasks.workunit.client.0.smithi040.stdout:3/572: truncate d1/d7/d30/f63 996321 0 2024-02-16T19:24:15.467 INFO:tasks.workunit.client.0.smithi040.stdout:9/514: creat d3/d4/d55/d63/fa5 x:0 0 0 2024-02-16T19:24:15.467 INFO:tasks.workunit.client.0.smithi040.stdout:9/515: readlink d3/d4/d11/d29/l68 0 2024-02-16T19:24:15.467 INFO:tasks.workunit.client.0.smithi040.stdout:9/516: dread - d3/d4/d4a/f62 zero size 2024-02-16T19:24:15.468 INFO:tasks.workunit.client.0.smithi040.stdout:2/435: symlink d5/d5c/l92 0 2024-02-16T19:24:15.468 INFO:tasks.workunit.client.0.smithi040.stdout:2/436: stat d5/d7/c48 0 2024-02-16T19:24:15.468 INFO:tasks.workunit.client.0.smithi040.stdout:2/437: fdatasync d5/d7/d10/d40/f91 0 2024-02-16T19:24:15.471 INFO:tasks.workunit.client.0.smithi040.stdout:3/573: mkdir d1/d8c/dae 0 2024-02-16T19:24:15.472 INFO:tasks.workunit.client.0.smithi040.stdout:6/507: dwrite d0/d1/d12/d3b/d79/f96 [0,4194304] 0 2024-02-16T19:24:15.472 INFO:tasks.workunit.client.0.smithi040.stdout:6/508: stat d0/d1/l86 0 2024-02-16T19:24:15.472 INFO:tasks.workunit.client.0.smithi040.stdout:6/509: write d0/d1/d6f/f75 [922426,105296] 0 2024-02-16T19:24:15.475 INFO:tasks.workunit.client.0.smithi040.stdout:4/407: write dd/f43 [3034721,106352] 0 2024-02-16T19:24:15.476 INFO:tasks.workunit.client.0.smithi040.stdout:6/510: write d0/d1/d12/d3b/d48/f38 [258620,113046] 0 2024-02-16T19:24:15.478 INFO:tasks.workunit.client.0.smithi040.stdout:5/443: dread d1/f2c [0,4194304] 0 2024-02-16T19:24:15.478 INFO:tasks.workunit.client.0.smithi040.stdout:3/574: mknod d1/d4/d6e/d8a/caf 0 2024-02-16T19:24:15.479 INFO:tasks.workunit.client.0.smithi040.stdout:3/575: write d1/d7/d30/f81 [531793,36129] 0 2024-02-16T19:24:15.479 INFO:tasks.workunit.client.0.smithi040.stdout:3/576: dread d1/d4/d6e/f84 [0,4194304] 0 2024-02-16T19:24:15.482 INFO:tasks.workunit.client.0.smithi040.stdout:4/408: rmdir dd/de/d10/d36 39 2024-02-16T19:24:15.483 INFO:tasks.workunit.client.0.smithi040.stdout:4/409: write dd/de/d10/d36/f63 [797036,49958] 0 2024-02-16T19:24:15.484 INFO:tasks.workunit.client.0.smithi040.stdout:6/511: creat d0/d1/d12/d3b/d48/d4f/faa x:0 0 0 2024-02-16T19:24:15.485 INFO:tasks.workunit.client.0.smithi040.stdout:9/517: dwrite d3/d4/d55/d8a/d99/f9d [4194304,4194304] 0 2024-02-16T19:24:15.485 INFO:tasks.workunit.client.0.smithi040.stdout:9/518: chown d3/d4/d55/d31 50371502 1 2024-02-16T19:24:15.487 INFO:tasks.workunit.client.0.smithi040.stdout:5/444: symlink d1/da/d7e/l87 0 2024-02-16T19:24:15.487 INFO:tasks.workunit.client.0.smithi040.stdout:6/512: write d0/d30/d5b/f91 [816839,117088] 0 2024-02-16T19:24:15.487 INFO:tasks.workunit.client.0.smithi040.stdout:6/513: read - d0/d30/d5b/f8a zero size 2024-02-16T19:24:15.487 INFO:tasks.workunit.client.0.smithi040.stdout:6/514: write d0/d1/d12/d3b/d48/d4f/f68 [394467,113974] 0 2024-02-16T19:24:15.487 INFO:tasks.workunit.client.0.smithi040.stdout:6/515: readlink d0/d1/d12/d3b/d48/d73/d70/d83/l8c 0 2024-02-16T19:24:15.487 INFO:tasks.workunit.client.0.smithi040.stdout:6/516: chown d0/f4 396 1 2024-02-16T19:24:15.487 INFO:tasks.workunit.client.0.smithi040.stdout:4/410: symlink dd/de/d10/d13/d32/d7a/l83 0 2024-02-16T19:24:15.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:15 smithi163 ceph-mon[36452]: pgmap v246: 65 pgs: 65 active+clean; 1.3 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 126 MiB/s rd, 167 MiB/s wr, 397 op/s 2024-02-16T19:24:15.495 INFO:tasks.workunit.client.0.smithi040.stdout:2/438: write d5/d7/d1e/d31/f43 [1173987,65192] 0 2024-02-16T19:24:15.495 INFO:tasks.workunit.client.0.smithi040.stdout:6/517: write d0/d1/d12/d3b/d48/d4d/d6d/f71 [3450405,115786] 0 2024-02-16T19:24:15.501 INFO:tasks.workunit.client.0.smithi040.stdout:5/445: truncate d1/d3c/f43 2776679 0 2024-02-16T19:24:15.502 INFO:tasks.workunit.client.0.smithi040.stdout:4/411: creat dd/de/d71/f84 x:0 0 0 2024-02-16T19:24:15.502 INFO:tasks.workunit.client.0.smithi040.stdout:5/446: write d1/f24 [1494978,126705] 0 2024-02-16T19:24:15.503 INFO:tasks.workunit.client.0.smithi040.stdout:5/447: fsync d1/da/d1c/d21/f36 0 2024-02-16T19:24:15.503 INFO:tasks.workunit.client.0.smithi040.stdout:5/448: write d1/d3c/f68 [1266163,112928] 0 2024-02-16T19:24:15.503 INFO:tasks.workunit.client.0.smithi040.stdout:9/519: dwrite d3/d4/d11/d29/d38/d46/f5b [0,4194304] 0 2024-02-16T19:24:15.504 INFO:tasks.workunit.client.0.smithi040.stdout:2/439: mknod d5/d7/d1e/d55/d8d/c93 0 2024-02-16T19:24:15.504 INFO:tasks.workunit.client.0.smithi040.stdout:2/440: fdatasync d5/d8/f38 0 2024-02-16T19:24:15.506 INFO:tasks.workunit.client.0.smithi040.stdout:4/412: symlink dd/d58/l85 0 2024-02-16T19:24:15.507 INFO:tasks.workunit.client.0.smithi040.stdout:5/449: rename d1/d45/c4a to d1/d3c/c88 0 2024-02-16T19:24:15.508 INFO:tasks.workunit.client.0.smithi040.stdout:9/520: symlink d3/d1c/la6 0 2024-02-16T19:24:15.509 INFO:tasks.workunit.client.0.smithi040.stdout:4/413: creat dd/de/d10/d36/f86 x:0 0 0 2024-02-16T19:24:15.511 INFO:tasks.workunit.client.0.smithi040.stdout:2/441: rename d5/d5c/f66 to d5/d7/d1e/d55/f94 0 2024-02-16T19:24:15.512 INFO:tasks.workunit.client.0.smithi040.stdout:6/518: dwrite d0/d37/f5d [0,4194304] 0 2024-02-16T19:24:15.512 INFO:tasks.workunit.client.0.smithi040.stdout:5/450: unlink d1/d3c/c7d 0 2024-02-16T19:24:15.514 INFO:tasks.workunit.client.0.smithi040.stdout:4/414: rmdir dd/de/d82 0 2024-02-16T19:24:15.515 INFO:tasks.workunit.client.0.smithi040.stdout:2/442: unlink d5/d7/d1e/d55/d8d/c93 0 2024-02-16T19:24:15.519 INFO:tasks.workunit.client.0.smithi040.stdout:6/519: creat d0/d1/d12/d3b/d48/d4f/d9d/fab x:0 0 0 2024-02-16T19:24:15.519 INFO:tasks.workunit.client.0.smithi040.stdout:6/520: chown d0/d30/d5b/f91 584 1 2024-02-16T19:24:15.521 INFO:tasks.workunit.client.0.smithi040.stdout:5/451: mknod d1/d62/c89 0 2024-02-16T19:24:15.523 INFO:tasks.workunit.client.0.smithi040.stdout:4/415: symlink dd/d58/l87 0 2024-02-16T19:24:15.523 INFO:tasks.workunit.client.0.smithi040.stdout:4/416: readlink dd/de/l33 0 2024-02-16T19:24:15.523 INFO:tasks.workunit.client.0.smithi040.stdout:4/417: chown dd/d58/d5e/f61 2115575 1 2024-02-16T19:24:15.523 INFO:tasks.workunit.client.0.smithi040.stdout:4/418: chown dd/de/d10/d13/c42 43223749 1 2024-02-16T19:24:15.523 INFO:tasks.workunit.client.0.smithi040.stdout:4/419: write dd/d58/d5e/f7f [933061,77078] 0 2024-02-16T19:24:15.523 INFO:tasks.workunit.client.0.smithi040.stdout:4/420: dread - dd/de/d10/d13/d77/f67 zero size 2024-02-16T19:24:15.524 INFO:tasks.workunit.client.0.smithi040.stdout:6/521: mkdir d0/d1/d12/d3b/d48/d4f/d8b/dac 0 2024-02-16T19:24:15.525 INFO:tasks.workunit.client.0.smithi040.stdout:5/452: mknod d1/d62/c8a 0 2024-02-16T19:24:15.527 INFO:tasks.workunit.client.0.smithi040.stdout:9/521: rmdir d3/d4/d11/d32/d8e 0 2024-02-16T19:24:15.527 INFO:tasks.workunit.client.0.smithi040.stdout:9/522: write d3/f21 [988145,34637] 0 2024-02-16T19:24:15.528 INFO:tasks.workunit.client.0.smithi040.stdout:4/421: symlink dd/d58/l88 0 2024-02-16T19:24:15.528 INFO:tasks.workunit.client.0.smithi040.stdout:4/422: dread - dd/de/d10/d36/f86 zero size 2024-02-16T19:24:15.531 INFO:tasks.workunit.client.0.smithi040.stdout:5/453: dread d1/f4c [0,4194304] 0 2024-02-16T19:24:15.531 INFO:tasks.workunit.client.0.smithi040.stdout:5/454: creat d1/da/d51/f8b x:0 0 0 2024-02-16T19:24:15.531 INFO:tasks.workunit.client.0.smithi040.stdout:2/443: dwrite d5/d7/d12/f15 [0,4194304] 0 2024-02-16T19:24:15.531 INFO:tasks.workunit.client.0.smithi040.stdout:2/444: fsync d5/d7/d1e/d31/f43 0 2024-02-16T19:24:15.531 INFO:tasks.workunit.client.0.smithi040.stdout:2/445: truncate d5/d7/d1e/d55/f6c 1015054 0 2024-02-16T19:24:15.533 INFO:tasks.workunit.client.0.smithi040.stdout:9/523: getdents d3/d1c 0 2024-02-16T19:24:15.535 INFO:tasks.workunit.client.0.smithi040.stdout:5/455: dread d1/d45/d63/f7b [0,4194304] 0 2024-02-16T19:24:15.535 INFO:tasks.workunit.client.0.smithi040.stdout:5/456: fdatasync d1/f24 0 2024-02-16T19:24:15.535 INFO:tasks.workunit.client.0.smithi040.stdout:5/457: read - d1/d3c/f5d zero size 2024-02-16T19:24:15.539 INFO:tasks.workunit.client.0.smithi040.stdout:2/446: rename d5/d7/l2e to d5/d7/d10/d40/l95 0 2024-02-16T19:24:15.539 INFO:tasks.workunit.client.0.smithi040.stdout:6/522: rmdir d0/d1/d12/d3b/d48/d4d 39 2024-02-16T19:24:15.542 INFO:tasks.workunit.client.0.smithi040.stdout:9/524: mknod d3/d1c/d73/ca7 0 2024-02-16T19:24:15.546 INFO:tasks.workunit.client.0.smithi040.stdout:5/458: symlink d1/l8c 0 2024-02-16T19:24:15.546 INFO:tasks.workunit.client.0.smithi040.stdout:5/459: write d1/da/d51/f81 [632553,84878] 0 2024-02-16T19:24:15.547 INFO:tasks.workunit.client.0.smithi040.stdout:9/525: rename d3/d4/d11/d32 to d3/d4/d11/d71/da8 0 2024-02-16T19:24:15.547 INFO:tasks.workunit.client.0.smithi040.stdout:9/526: stat d3/d4/d55/f39 0 2024-02-16T19:24:15.549 INFO:tasks.workunit.client.0.smithi040.stdout:5/460: link d1/d45/c46 d1/d45/c8d 0 2024-02-16T19:24:15.549 INFO:tasks.workunit.client.0.smithi040.stdout:5/461: stat d1/da/d1c/d73 0 2024-02-16T19:24:15.550 INFO:tasks.workunit.client.0.smithi040.stdout:5/462: mknod d1/da/c8e 0 2024-02-16T19:24:15.552 INFO:tasks.workunit.client.0.smithi040.stdout:5/463: creat d1/da/d1c/d21/f8f x:0 0 0 2024-02-16T19:24:15.552 INFO:tasks.workunit.client.0.smithi040.stdout:5/464: readlink d1/da/d1c/d21/l47 0 2024-02-16T19:24:15.552 INFO:tasks.workunit.client.0.smithi040.stdout:2/447: dread d5/d7/f74 [0,4194304] 0 2024-02-16T19:24:15.552 INFO:tasks.workunit.client.0.smithi040.stdout:2/448: stat d5/d7/d1e/d31/c5b 0 2024-02-16T19:24:15.552 INFO:tasks.workunit.client.0.smithi040.stdout:2/449: fsync d5/d7/d1e/d31/f7d 0 2024-02-16T19:24:15.552 INFO:tasks.workunit.client.0.smithi040.stdout:5/465: mknod d1/da/d7e/c90 0 2024-02-16T19:24:15.553 INFO:tasks.workunit.client.0.smithi040.stdout:2/450: rmdir d5/d7/d12 39 2024-02-16T19:24:15.554 INFO:tasks.workunit.client.0.smithi040.stdout:2/451: chown d5/d7/d12 6781042 1 2024-02-16T19:24:15.554 INFO:tasks.workunit.client.0.smithi040.stdout:2/452: write d5/d7/d1e/d31/d4f/d63/f8e [617117,54730] 0 2024-02-16T19:24:15.554 INFO:tasks.workunit.client.0.smithi040.stdout:2/453: fsync d5/d7/d1e/d31/f7d 0 2024-02-16T19:24:15.554 INFO:tasks.workunit.client.0.smithi040.stdout:6/523: dread d0/fa0 [0,4194304] 0 2024-02-16T19:24:15.555 INFO:tasks.workunit.client.0.smithi040.stdout:5/466: creat d1/da/d1c/f91 x:0 0 0 2024-02-16T19:24:15.557 INFO:tasks.workunit.client.0.smithi040.stdout:9/527: dread d3/d4/d11/d29/d36/f75 [0,4194304] 0 2024-02-16T19:24:15.557 INFO:tasks.workunit.client.0.smithi040.stdout:6/524: unlink d0/f35 0 2024-02-16T19:24:15.559 INFO:tasks.workunit.client.0.smithi040.stdout:9/528: truncate d3/d4/f8 350121 0 2024-02-16T19:24:15.560 INFO:tasks.workunit.client.0.smithi040.stdout:6/525: mknod d0/d1/d12/d3b/d48/d73/d70/d83/cad 0 2024-02-16T19:24:15.560 INFO:tasks.workunit.client.0.smithi040.stdout:6/526: truncate d0/d1/d12/d3b/d48/f3c 26497 0 2024-02-16T19:24:15.561 INFO:tasks.workunit.client.0.smithi040.stdout:2/454: dread d5/d7/d1e/f25 [0,4194304] 0 2024-02-16T19:24:15.562 INFO:tasks.workunit.client.0.smithi040.stdout:9/529: rmdir d3/d4/d11/d71 39 2024-02-16T19:24:15.562 INFO:tasks.workunit.client.0.smithi040.stdout:5/467: write d1/da/d51/f5e [1779351,110665] 0 2024-02-16T19:24:15.564 INFO:tasks.workunit.client.0.smithi040.stdout:5/468: dread d1/da/d51/f79 [0,4194304] 0 2024-02-16T19:24:15.564 INFO:tasks.workunit.client.0.smithi040.stdout:5/469: fsync d1/f24 0 2024-02-16T19:24:15.566 INFO:tasks.workunit.client.0.smithi040.stdout:9/530: symlink d3/d4/d55/d8a/d99/la9 0 2024-02-16T19:24:15.566 INFO:tasks.workunit.client.0.smithi040.stdout:9/531: write d3/d4/d55/f66 [906050,127522] 0 2024-02-16T19:24:15.566 INFO:tasks.workunit.client.0.smithi040.stdout:9/532: write d3/d4/d11/d71/da8/f84 [195779,75805] 0 2024-02-16T19:24:15.567 INFO:tasks.workunit.client.0.smithi040.stdout:7/421: sync 2024-02-16T19:24:15.569 INFO:tasks.workunit.client.0.smithi040.stdout:6/527: dread d0/d1/f52 [0,4194304] 0 2024-02-16T19:24:15.569 INFO:tasks.workunit.client.0.smithi040.stdout:6/528: stat d0/d1/d12/d3b/d48/d4d/ca1 0 2024-02-16T19:24:15.570 INFO:tasks.workunit.client.0.smithi040.stdout:9/533: creat d3/d4/d11/d29/d38/d86/faa x:0 0 0 2024-02-16T19:24:15.571 INFO:tasks.workunit.client.0.smithi040.stdout:7/422: symlink d21/d4d/d77/d7a/l8a 0 2024-02-16T19:24:15.573 INFO:tasks.workunit.client.0.smithi040.stdout:9/534: creat d3/d4/d11/d29/d38/d86/fab x:0 0 0 2024-02-16T19:24:15.573 INFO:tasks.workunit.client.0.smithi040.stdout:9/535: fdatasync d3/d4/d55/f98 0 2024-02-16T19:24:15.573 INFO:tasks.workunit.client.0.smithi040.stdout:9/536: readlink d3/d4/d11/d29/l77 0 2024-02-16T19:24:15.573 INFO:tasks.workunit.client.0.smithi040.stdout:9/537: readlink d3/d4/d11/d29/l77 0 2024-02-16T19:24:15.574 INFO:tasks.workunit.client.0.smithi040.stdout:7/423: stat d21/l5f 0 2024-02-16T19:24:15.575 INFO:tasks.workunit.client.0.smithi040.stdout:9/538: write d3/d4/f8 [326850,95110] 0 2024-02-16T19:24:15.583 INFO:tasks.workunit.client.0.smithi040.stdout:2/455: dwrite d5/d7/d1e/d31/f43 [0,4194304] 0 2024-02-16T19:24:15.583 INFO:tasks.workunit.client.0.smithi040.stdout:5/470: dwrite d1/da/d51/f79 [0,4194304] 0 2024-02-16T19:24:15.583 INFO:tasks.workunit.client.0.smithi040.stdout:5/471: creat d1/da/d7e/f92 x:0 0 0 2024-02-16T19:24:15.583 INFO:tasks.workunit.client.0.smithi040.stdout:5/472: chown d1/da/l71 6 1 2024-02-16T19:24:15.585 INFO:tasks.workunit.client.0.smithi040.stdout:2/456: rename d5/d7/d1e/f7e to d5/d7/d1e/d4c/d52/f96 0 2024-02-16T19:24:15.588 INFO:tasks.workunit.client.0.smithi040.stdout:5/473: symlink d1/da/d7e/l93 0 2024-02-16T19:24:15.588 INFO:tasks.workunit.client.0.smithi040.stdout:9/539: dread d3/d4/d11/d29/f52 [0,4194304] 0 2024-02-16T19:24:15.590 INFO:tasks.workunit.client.0.smithi040.stdout:2/457: creat d5/d7/d1e/d31/d4f/d63/d8b/f97 x:0 0 0 2024-02-16T19:24:15.590 INFO:tasks.workunit.client.0.smithi040.stdout:2/458: write d5/d8/d42/f67 [365268,40233] 0 2024-02-16T19:24:15.592 INFO:tasks.workunit.client.0.smithi040.stdout:9/540: mkdir d3/d4/d11/d29/d38/d46/d6b/dac 0 2024-02-16T19:24:15.592 INFO:tasks.workunit.client.0.smithi040.stdout:9/541: creat d3/d4/d55/d31/fad x:0 0 0 2024-02-16T19:24:15.596 INFO:tasks.workunit.client.0.smithi040.stdout:7/424: dwrite d21/d29/f73 [0,4194304] 0 2024-02-16T19:24:15.599 INFO:tasks.workunit.client.0.smithi040.stdout:2/459: dread d5/d7/d1e/f33 [0,4194304] 0 2024-02-16T19:24:15.601 INFO:tasks.workunit.client.0.smithi040.stdout:7/425: rmdir d21/d29/d79 39 2024-02-16T19:24:15.601 INFO:tasks.workunit.client.0.smithi040.stdout:7/426: stat d21/l5f 0 2024-02-16T19:24:15.605 INFO:tasks.workunit.client.0.smithi040.stdout:7/427: mkdir d21/d8b 0 2024-02-16T19:24:15.607 INFO:tasks.workunit.client.0.smithi040.stdout:7/428: creat d21/d29/d79/f8c x:0 0 0 2024-02-16T19:24:15.607 INFO:tasks.workunit.client.0.smithi040.stdout:7/429: mknod d21/d29/c8d 0 2024-02-16T19:24:15.610 INFO:tasks.workunit.client.0.smithi040.stdout:5/474: dwrite d1/d45/f5f [0,4194304] 0 2024-02-16T19:24:15.617 INFO:tasks.workunit.client.0.smithi040.stdout:5/475: dread d1/f24 [0,4194304] 0 2024-02-16T19:24:15.618 INFO:tasks.workunit.client.0.smithi040.stdout:5/476: getdents d1/da/d7e 0 2024-02-16T19:24:15.618 INFO:tasks.workunit.client.0.smithi040.stdout:5/477: creat d1/da/d1c/f94 x:0 0 0 2024-02-16T19:24:15.619 INFO:tasks.workunit.client.0.smithi040.stdout:5/478: symlink d1/d3c/l95 0 2024-02-16T19:24:15.620 INFO:tasks.workunit.client.0.smithi040.stdout:9/542: dwrite d3/d4/d11/d29/f56 [4194304,4194304] 0 2024-02-16T19:24:15.620 INFO:tasks.workunit.client.0.smithi040.stdout:5/479: creat d1/f96 x:0 0 0 2024-02-16T19:24:15.621 INFO:tasks.workunit.client.0.smithi040.stdout:5/480: readlink d1/d45/d63/l4e 0 2024-02-16T19:24:15.621 INFO:tasks.workunit.client.0.smithi040.stdout:5/481: chown d1/da/d1c/d21/f64 1097141794 1 2024-02-16T19:24:15.621 INFO:tasks.workunit.client.0.smithi040.stdout:5/482: chown d1/da/f2e 4989410 1 2024-02-16T19:24:15.621 INFO:tasks.workunit.client.0.smithi040.stdout:5/483: mkdir d1/d3c/d97 0 2024-02-16T19:24:15.622 INFO:tasks.workunit.client.0.smithi040.stdout:6/529: dwrite d0/d30/d32/f56 [0,4194304] 0 2024-02-16T19:24:15.623 INFO:tasks.workunit.client.0.smithi040.stdout:5/484: creat d1/d62/f98 x:0 0 0 2024-02-16T19:24:15.623 INFO:tasks.workunit.client.0.smithi040.stdout:5/485: readlink d1/da/l33 0 2024-02-16T19:24:15.625 INFO:tasks.workunit.client.0.smithi040.stdout:2/460: dwrite d5/d7/d1e/d4c/d52/f6b [0,4194304] 0 2024-02-16T19:24:15.625 INFO:tasks.workunit.client.0.smithi040.stdout:2/461: fdatasync d5/d7/f74 0 2024-02-16T19:24:15.625 INFO:tasks.workunit.client.0.smithi040.stdout:2/462: fdatasync d5/d8/d42/f67 0 2024-02-16T19:24:15.625 INFO:tasks.workunit.client.0.smithi040.stdout:6/530: symlink d0/d1/d12/d3b/d48/d4f/d9d/lae 0 2024-02-16T19:24:15.625 INFO:tasks.workunit.client.0.smithi040.stdout:6/531: chown d0/d1/d12/d3b/d48/d4d/d6d 1500 1 2024-02-16T19:24:15.625 INFO:tasks.workunit.client.0.smithi040.stdout:9/543: write d3/d4/d11/d29/f2b [4133954,107861] 0 2024-02-16T19:24:15.625 INFO:tasks.workunit.client.0.smithi040.stdout:5/486: dread d1/da/f70 [0,4194304] 0 2024-02-16T19:24:15.625 INFO:tasks.workunit.client.0.smithi040.stdout:5/487: creat d1/da/f99 x:0 0 0 2024-02-16T19:24:15.627 INFO:tasks.workunit.client.0.smithi040.stdout:2/463: truncate d5/d7/d1e/f70 776696 0 2024-02-16T19:24:15.627 INFO:tasks.workunit.client.0.smithi040.stdout:2/464: write d5/d8/d42/f7a [523184,121222] 0 2024-02-16T19:24:15.627 INFO:tasks.workunit.client.0.smithi040.stdout:2/465: readlink d5/d7/d1e/d31/l85 0 2024-02-16T19:24:15.627 INFO:tasks.workunit.client.0.smithi040.stdout:2/466: chown d5/d7/d1e/d55/d8d/c90 1678991 1 2024-02-16T19:24:15.628 INFO:tasks.workunit.client.0.smithi040.stdout:6/532: symlink d0/d30/d5b/laf 0 2024-02-16T19:24:15.629 INFO:tasks.workunit.client.0.smithi040.stdout:9/544: mknod d3/d4/d55/d31/cae 0 2024-02-16T19:24:15.629 INFO:tasks.workunit.client.0.smithi040.stdout:9/545: fsync d3/d4/d55/d8a/f8f 0 2024-02-16T19:24:15.632 INFO:tasks.workunit.client.0.smithi040.stdout:9/546: mkdir d3/d4/d55/d6d/daf 0 2024-02-16T19:24:15.634 INFO:tasks.workunit.client.0.smithi040.stdout:9/547: rmdir d3/d4/d11/d29/d38/d86/da4 0 2024-02-16T19:24:15.634 INFO:tasks.workunit.client.0.smithi040.stdout:9/548: creat d3/d4/d55/d63/fb0 x:0 0 0 2024-02-16T19:24:15.634 INFO:tasks.workunit.client.0.smithi040.stdout:9/549: fsync d3/d4/f57 0 2024-02-16T19:24:15.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:15 smithi040 ceph-mon[29201]: pgmap v246: 65 pgs: 65 active+clean; 1.3 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 126 MiB/s rd, 167 MiB/s wr, 397 op/s 2024-02-16T19:24:15.667 INFO:tasks.workunit.client.0.smithi040.stdout:2/467: dwrite d5/d7/f74 [0,4194304] 0 2024-02-16T19:24:15.668 INFO:tasks.workunit.client.0.smithi040.stdout:2/468: mknod d5/d7/d1e/d31/d4f/d63/d8b/c98 0 2024-02-16T19:24:15.669 INFO:tasks.workunit.client.0.smithi040.stdout:5/488: dwrite d1/d45/d63/f7b [4194304,4194304] 0 2024-02-16T19:24:15.669 INFO:tasks.workunit.client.0.smithi040.stdout:6/533: dwrite d0/d1/d12/d3b/d48/d73/d70/fa6 [0,4194304] 0 2024-02-16T19:24:15.669 INFO:tasks.workunit.client.0.smithi040.stdout:6/534: creat d0/d37/d4a/fb0 x:0 0 0 2024-02-16T19:24:15.670 INFO:tasks.workunit.client.0.smithi040.stdout:9/550: dwrite d3/d4/d11/d71/da8/f42 [0,4194304] 0 2024-02-16T19:24:15.670 INFO:tasks.workunit.client.0.smithi040.stdout:2/469: symlink d5/d7/d1e/d31/d4f/l99 0 2024-02-16T19:24:15.670 INFO:tasks.workunit.client.0.smithi040.stdout:2/470: truncate d5/d7/d1e/d4c/d52/f96 593518 0 2024-02-16T19:24:15.670 INFO:tasks.workunit.client.0.smithi040.stdout:5/489: mkdir d1/d62/d9a 0 2024-02-16T19:24:15.673 INFO:tasks.workunit.client.0.smithi040.stdout:5/490: dread d1/da/d51/f81 [0,4194304] 0 2024-02-16T19:24:15.678 INFO:tasks.workunit.client.0.smithi040.stdout:9/551: creat d3/d1c/d73/fb1 x:0 0 0 2024-02-16T19:24:15.679 INFO:tasks.workunit.client.0.smithi040.stdout:9/552: truncate d3/d4/d55/d6d/f7d 588726 0 2024-02-16T19:24:15.680 INFO:tasks.workunit.client.0.smithi040.stdout:9/553: creat d3/d4/d55/d8a/d99/fb2 x:0 0 0 2024-02-16T19:24:15.682 INFO:tasks.workunit.client.0.smithi040.stdout:5/491: rmdir d1/d3c 39 2024-02-16T19:24:15.685 INFO:tasks.workunit.client.0.smithi040.stdout:9/554: dread d3/d4/d55/f23 [0,4194304] 0 2024-02-16T19:24:15.687 INFO:tasks.workunit.client.0.smithi040.stdout:9/555: creat d3/d1c/fb3 x:0 0 0 2024-02-16T19:24:15.687 INFO:tasks.workunit.client.0.smithi040.stdout:9/556: chown d3/d8b 1295655 1 2024-02-16T19:24:15.687 INFO:tasks.workunit.client.0.smithi040.stdout:9/557: write d3/d4/d11/d71/f88 [106731,123414] 0 2024-02-16T19:24:15.687 INFO:tasks.workunit.client.0.smithi040.stdout:9/558: write d3/d4/f96 [1043506,19850] 0 2024-02-16T19:24:15.688 INFO:tasks.workunit.client.0.smithi040.stdout:9/559: mkdir d3/d4/d4a/db4 0 2024-02-16T19:24:15.703 INFO:tasks.workunit.client.0.smithi040.stdout:5/492: dwrite d1/da/d51/f8b [0,4194304] 0 2024-02-16T19:24:15.703 INFO:tasks.workunit.client.0.smithi040.stdout:9/560: dwrite d3/d4/d55/d8a/f95 [0,4194304] 0 2024-02-16T19:24:15.705 INFO:tasks.workunit.client.0.smithi040.stdout:5/493: link d1/d62/f98 d1/d62/d9a/f9b 0 2024-02-16T19:24:15.705 INFO:tasks.workunit.client.0.smithi040.stdout:5/494: creat d1/f9c x:0 0 0 2024-02-16T19:24:15.705 INFO:tasks.workunit.client.0.smithi040.stdout:5/495: write d1/f24 [2618404,48014] 0 2024-02-16T19:24:15.706 INFO:tasks.workunit.client.0.smithi040.stdout:9/561: creat d3/d4/d11/d29/d38/d46/d6b/dac/fb5 x:0 0 0 2024-02-16T19:24:15.706 INFO:tasks.workunit.client.0.smithi040.stdout:9/562: chown d3/d4/d55/d31 14922 1 2024-02-16T19:24:15.706 INFO:tasks.workunit.client.0.smithi040.stdout:9/563: creat d3/d4/d55/d63/fb6 x:0 0 0 2024-02-16T19:24:15.707 INFO:tasks.workunit.client.0.smithi040.stdout:5/496: getdents d1/d3c 0 2024-02-16T19:24:15.707 INFO:tasks.workunit.client.0.smithi040.stdout:5/497: creat d1/da/d1c/d21/f9d x:0 0 0 2024-02-16T19:24:15.708 INFO:tasks.workunit.client.0.smithi040.stdout:5/498: read d1/d3c/f72 [4304103,99561] 0 2024-02-16T19:24:15.708 INFO:tasks.workunit.client.0.smithi040.stdout:5/499: readlink d1/da/d7e/l93 0 2024-02-16T19:24:15.708 INFO:tasks.workunit.client.0.smithi040.stdout:9/564: creat d3/d4/d55/d6d/fb7 x:0 0 0 2024-02-16T19:24:15.710 INFO:tasks.workunit.client.0.smithi040.stdout:5/500: mknod d1/da/d1c/d73/c9e 0 2024-02-16T19:24:15.710 INFO:tasks.workunit.client.0.smithi040.stdout:5/501: truncate d1/da/f99 229253 0 2024-02-16T19:24:15.710 INFO:tasks.workunit.client.0.smithi040.stdout:9/565: creat d3/d4/d11/fb8 x:0 0 0 2024-02-16T19:24:15.710 INFO:tasks.workunit.client.0.smithi040.stdout:9/566: readlink l2 0 2024-02-16T19:24:15.712 INFO:tasks.workunit.client.0.smithi040.stdout:5/502: rename d1/da/d7e/l87 to d1/da/d1c/l9f 0 2024-02-16T19:24:15.772 INFO:tasks.workunit.client.0.smithi040.stdout:2/471: dwrite d5/d7/d1e/f80 [0,4194304] 0 2024-02-16T19:24:15.772 INFO:tasks.workunit.client.0.smithi040.stdout:6/535: dwrite d0/d30/d32/f8d [0,4194304] 0 2024-02-16T19:24:15.772 INFO:tasks.workunit.client.0.smithi040.stdout:9/567: dwrite d3/d4/d11/d71/f88 [0,4194304] 0 2024-02-16T19:24:15.772 INFO:tasks.workunit.client.0.smithi040.stdout:9/568: chown d3/d4/d55/d31/l47 50 1 2024-02-16T19:24:15.772 INFO:tasks.workunit.client.0.smithi040.stdout:6/536: fsync d0/f55 0 2024-02-16T19:24:15.772 INFO:tasks.workunit.client.0.smithi040.stdout:6/537: chown d0/d1/d12/d3b/d48/f9e 88966143 1 2024-02-16T19:24:15.772 INFO:tasks.workunit.client.0.smithi040.stdout:6/538: write d0/d1/d12/d3b/d48/d4f/d8b/f95 [700033,61978] 0 2024-02-16T19:24:15.772 INFO:tasks.workunit.client.0.smithi040.stdout:5/503: dwrite d1/f24 [0,4194304] 0 2024-02-16T19:24:15.773 INFO:tasks.workunit.client.0.smithi040.stdout:5/504: chown d1/d45/d63/c78 1378 1 2024-02-16T19:24:15.780 INFO:tasks.workunit.client.0.smithi040.stdout:2/472: creat d5/d7/d10/d40/f9a x:0 0 0 2024-02-16T19:24:15.784 INFO:tasks.workunit.client.0.smithi040.stdout:5/505: creat d1/da/d7e/fa0 x:0 0 0 2024-02-16T19:24:15.785 INFO:tasks.workunit.client.0.smithi040.stdout:5/506: read d1/da/d51/f81 [370919,87181] 0 2024-02-16T19:24:15.785 INFO:tasks.workunit.client.0.smithi040.stdout:9/569: rmdir d3/d4/d55/d6d 39 2024-02-16T19:24:15.785 INFO:tasks.workunit.client.0.smithi040.stdout:9/570: dread - d3/d4/d11/fb8 zero size 2024-02-16T19:24:15.786 INFO:tasks.workunit.client.0.smithi040.stdout:9/571: write d3/d4/d55/d31/fad [763846,113138] 0 2024-02-16T19:24:15.787 INFO:tasks.workunit.client.0.smithi040.stdout:6/539: rmdir d0/d1/d12/d3b/d48/d73/d70 39 2024-02-16T19:24:15.787 INFO:tasks.workunit.client.0.smithi040.stdout:6/540: readlink d0/d30/d32/l6b 0 2024-02-16T19:24:15.787 INFO:tasks.workunit.client.0.smithi040.stdout:6/541: fsync d0/d30/d5b/f85 0 2024-02-16T19:24:15.787 INFO:tasks.workunit.client.0.smithi040.stdout:0/537: sync 2024-02-16T19:24:15.787 INFO:tasks.workunit.client.0.smithi040.stdout:7/430: sync 2024-02-16T19:24:15.787 INFO:tasks.workunit.client.0.smithi040.stdout:8/571: sync 2024-02-16T19:24:15.788 INFO:tasks.workunit.client.0.smithi040.stdout:3/577: sync 2024-02-16T19:24:15.788 INFO:tasks.workunit.client.0.smithi040.stdout:4/423: sync 2024-02-16T19:24:15.788 INFO:tasks.workunit.client.0.smithi040.stdout:1/503: sync 2024-02-16T19:24:15.792 INFO:tasks.workunit.client.0.smithi040.stdout:5/507: rename d1/da/f29 to d1/da/fa1 0 2024-02-16T19:24:15.794 INFO:tasks.workunit.client.0.smithi040.stdout:6/542: mkdir d0/d1/d6f/d90/db1 0 2024-02-16T19:24:15.795 INFO:tasks.workunit.client.0.smithi040.stdout:8/572: creat d6/d2b/d44/fb4 x:0 0 0 2024-02-16T19:24:15.796 INFO:tasks.workunit.client.0.smithi040.stdout:4/424: creat dd/d58/d68/f89 x:0 0 0 2024-02-16T19:24:15.799 INFO:tasks.workunit.client.0.smithi040.stdout:5/508: rename d1/d45/d63/f74 to d1/d62/d9a/fa2 0 2024-02-16T19:24:15.802 INFO:tasks.workunit.client.0.smithi040.stdout:6/543: creat d0/d37/fb2 x:0 0 0 2024-02-16T19:24:15.802 INFO:tasks.workunit.client.0.smithi040.stdout:6/544: truncate d0/d1/d12/d3b/d48/d73/d70/fa5 341302 0 2024-02-16T19:24:15.802 INFO:tasks.workunit.client.0.smithi040.stdout:6/545: fdatasync d0/d30/d32/f56 0 2024-02-16T19:24:15.808 INFO:tasks.workunit.client.0.smithi040.stdout:0/538: dread db/de/d6a/f82 [0,4194304] 0 2024-02-16T19:24:15.808 INFO:tasks.workunit.client.0.smithi040.stdout:2/473: dwrite d5/d7/d1e/d31/d4f/d63/f8e [0,4194304] 0 2024-02-16T19:24:15.808 INFO:tasks.workunit.client.0.smithi040.stdout:2/474: stat d5/d7/d1e/l64 0 2024-02-16T19:24:15.809 INFO:tasks.workunit.client.0.smithi040.stdout:0/539: fdatasync db/d2f/d1d/d21/d2e/d49/d5b/d67/f8f 0 2024-02-16T19:24:15.813 INFO:tasks.workunit.client.0.smithi040.stdout:0/540: link db/d2f/f66 db/d25/d34/d5e/fb4 0 2024-02-16T19:24:15.814 INFO:tasks.workunit.client.0.smithi040.stdout:0/541: mknod db/d2f/d1d/d21/d2e/d49/d5b/d67/cb5 0 2024-02-16T19:24:15.820 INFO:tasks.workunit.client.0.smithi040.stdout:6/546: write d0/d30/d32/f21 [1498658,121241] 0 2024-02-16T19:24:15.822 INFO:tasks.workunit.client.0.smithi040.stdout:6/547: creat d0/d1/d12/d3b/d48/d4f/d8b/dac/fb3 x:0 0 0 2024-02-16T19:24:15.822 INFO:tasks.workunit.client.0.smithi040.stdout:6/548: fsync d0/d1/d12/f66 0 2024-02-16T19:24:15.823 INFO:tasks.workunit.client.0.smithi040.stdout:5/509: dread d1/d3c/f72 [0,4194304] 0 2024-02-16T19:24:15.825 INFO:tasks.workunit.client.0.smithi040.stdout:5/510: link d1/f17 d1/d45/d63/fa3 0 2024-02-16T19:24:15.825 INFO:tasks.workunit.client.0.smithi040.stdout:5/511: truncate d1/da/d7e/f84 726327 0 2024-02-16T19:24:15.825 INFO:tasks.workunit.client.0.smithi040.stdout:5/512: readlink d1/l8c 0 2024-02-16T19:24:15.836 INFO:tasks.workunit.client.0.smithi040.stdout:3/578: dwrite d1/d7/f4e [4194304,4194304] 0 2024-02-16T19:24:15.838 INFO:tasks.workunit.client.0.smithi040.stdout:3/579: getdents d1/dc/d35 0 2024-02-16T19:24:15.839 INFO:tasks.workunit.client.0.smithi040.stdout:3/580: write d1/dc/d42/d4a/f7d [641360,56787] 0 2024-02-16T19:24:15.839 INFO:tasks.workunit.client.0.smithi040.stdout:9/572: dwrite d3/d4/d11/d29/d38/d46/f94 [0,4194304] 0 2024-02-16T19:24:15.839 INFO:tasks.workunit.client.0.smithi040.stdout:9/573: write d3/d4/d11/f67 [165888,17074] 0 2024-02-16T19:24:15.840 INFO:tasks.workunit.client.0.smithi040.stdout:9/574: stat d3/d1c/l40 0 2024-02-16T19:24:15.840 INFO:tasks.workunit.client.0.smithi040.stdout:9/575: creat d3/d1c/fb9 x:0 0 0 2024-02-16T19:24:15.840 INFO:tasks.workunit.client.0.smithi040.stdout:3/581: mknod d1/dc/d42/d4a/cb0 0 2024-02-16T19:24:15.840 INFO:tasks.workunit.client.0.smithi040.stdout:3/582: chown d1/dc/d35/d53/d64/d65/f98 16864 1 2024-02-16T19:24:15.841 INFO:tasks.workunit.client.0.smithi040.stdout:2/475: dwrite d5/d7/f74 [4194304,4194304] 0 2024-02-16T19:24:15.841 INFO:tasks.workunit.client.0.smithi040.stdout:2/476: fdatasync d5/d8/f38 0 2024-02-16T19:24:15.841 INFO:tasks.workunit.client.0.smithi040.stdout:4/425: dwrite dd/f66 [0,4194304] 0 2024-02-16T19:24:15.842 INFO:tasks.workunit.client.0.smithi040.stdout:0/542: dwrite f7 [0,4194304] 0 2024-02-16T19:24:15.843 INFO:tasks.workunit.client.0.smithi040.stdout:8/573: dwrite d6/d2b/da3/d8b/f9d [0,4194304] 0 2024-02-16T19:24:15.843 INFO:tasks.workunit.client.0.smithi040.stdout:8/574: stat d6/d12/l2a 0 2024-02-16T19:24:15.843 INFO:tasks.workunit.client.0.smithi040.stdout:8/575: write d6/d12/f1f [4545741,106433] 0 2024-02-16T19:24:15.843 INFO:tasks.workunit.client.0.smithi040.stdout:8/576: chown d6/d12/d93/fa9 676 1 2024-02-16T19:24:15.846 INFO:tasks.workunit.client.0.smithi040.stdout:9/576: unlink d3/d4/d55/f51 0 2024-02-16T19:24:15.846 INFO:tasks.workunit.client.0.smithi040.stdout:3/583: mknod d1/d7/d36/cb1 0 2024-02-16T19:24:15.846 INFO:tasks.workunit.client.0.smithi040.stdout:9/577: write d3/d1c/fb3 [927368,124358] 0 2024-02-16T19:24:15.846 INFO:tasks.workunit.client.0.smithi040.stdout:9/578: write d3/d4/f8 [303772,20257] 0 2024-02-16T19:24:15.847 INFO:tasks.workunit.client.0.smithi040.stdout:2/477: rename d5/d7/d1e/d31/d4f/d63/d8b/f97 to d5/d7/d12/d1b/f9b 0 2024-02-16T19:24:15.849 INFO:tasks.workunit.client.0.smithi040.stdout:8/577: write d6/d2b/f48 [303627,43759] 0 2024-02-16T19:24:15.855 INFO:tasks.workunit.client.0.smithi040.stdout:3/584: write d1/d4/fa [5746261,80164] 0 2024-02-16T19:24:15.859 INFO:tasks.workunit.client.0.smithi040.stdout:5/513: rmdir d1 39 2024-02-16T19:24:15.859 INFO:tasks.workunit.client.0.smithi040.stdout:5/514: getdents d1/d3c 0 2024-02-16T19:24:15.869 INFO:tasks.workunit.client.0.smithi040.stdout:9/579: mknod d3/d4/d55/d8a/cba 0 2024-02-16T19:24:15.870 INFO:tasks.workunit.client.0.smithi040.stdout:9/580: truncate d3/d4/d11/d29/d36/f3a 1057118 0 2024-02-16T19:24:15.871 INFO:tasks.workunit.client.0.smithi040.stdout:2/478: creat d5/d7/d1e/d4c/d52/f9c x:0 0 0 2024-02-16T19:24:15.872 INFO:tasks.workunit.client.0.smithi040.stdout:4/426: dwrite dd/de/f12 [0,4194304] 0 2024-02-16T19:24:15.873 INFO:tasks.workunit.client.0.smithi040.stdout:3/585: link d1/dc/d42/d4a/l59 d1/dc/d42/d4a/lb2 0 2024-02-16T19:24:15.874 INFO:tasks.workunit.client.0.smithi040.stdout:3/586: stat d1/dc/d42/d4a/d57/d79 0 2024-02-16T19:24:15.874 INFO:tasks.workunit.client.0.smithi040.stdout:0/543: dwrite db/d2f/d23/d6f/f78 [0,4194304] 0 2024-02-16T19:24:15.874 INFO:tasks.workunit.client.0.smithi040.stdout:0/544: truncate db/de/fa2 230980 0 2024-02-16T19:24:15.874 INFO:tasks.workunit.client.0.smithi040.stdout:6/549: dwrite d0/f2e [4194304,4194304] 0 2024-02-16T19:24:15.875 INFO:tasks.workunit.client.0.smithi040.stdout:5/515: mkdir d1/da/da4 0 2024-02-16T19:24:15.875 INFO:tasks.workunit.client.0.smithi040.stdout:5/516: fsync d1/da/f99 0 2024-02-16T19:24:15.877 INFO:tasks.workunit.client.0.smithi040.stdout:9/581: creat d3/d4/d11/d29/d38/d46/d6b/d85/fbb x:0 0 0 2024-02-16T19:24:15.879 INFO:tasks.workunit.client.0.smithi040.stdout:2/479: unlink d5/d7/f47 0 2024-02-16T19:24:15.879 INFO:tasks.workunit.client.0.smithi040.stdout:8/578: rmdir d6/d64/d50 39 2024-02-16T19:24:15.880 INFO:tasks.workunit.client.0.smithi040.stdout:8/579: chown d6/d2b/f9a 54283279 1 2024-02-16T19:24:15.883 INFO:tasks.workunit.client.0.smithi040.stdout:4/427: rename dd/de/d10/f55 to dd/de/d10/f8a 0 2024-02-16T19:24:15.883 INFO:tasks.workunit.client.0.smithi040.stdout:4/428: chown dd/de/d10/f20 12217 1 2024-02-16T19:24:15.886 INFO:tasks.workunit.client.0.smithi040.stdout:0/545: creat db/d2f/d23/d56/fb6 x:0 0 0 2024-02-16T19:24:15.888 INFO:tasks.workunit.client.0.smithi040.stdout:6/550: rename d0/d30/d5b/laf to d0/lb4 0 2024-02-16T19:24:15.888 INFO:tasks.workunit.client.0.smithi040.stdout:9/582: rename d3 to d3/dbc 22 2024-02-16T19:24:15.888 INFO:tasks.workunit.client.0.smithi040.stdout:6/551: chown d0/d1/d12/d3b/d48/d4f/d8b 1916291107 1 2024-02-16T19:24:15.888 INFO:tasks.workunit.client.0.smithi040.stdout:9/583: chown d3/d4/d55/d31/f5e 17 1 2024-02-16T19:24:15.889 INFO:tasks.workunit.client.0.smithi040.stdout:5/517: symlink d1/d3c/d97/la5 0 2024-02-16T19:24:15.889 INFO:tasks.workunit.client.0.smithi040.stdout:5/518: fsync d1/d45/d63/fa3 0 2024-02-16T19:24:15.890 INFO:tasks.workunit.client.0.smithi040.stdout:3/587: dwrite d1/dc/d42/d4a/f50 [0,4194304] 0 2024-02-16T19:24:15.891 INFO:tasks.workunit.client.0.smithi040.stdout:4/429: mknod dd/de/d10/d13/c8b 0 2024-02-16T19:24:15.891 INFO:tasks.workunit.client.0.smithi040.stdout:2/480: dread d5/d7/d1e/f6d [0,4194304] 0 2024-02-16T19:24:15.892 INFO:tasks.workunit.client.0.smithi040.stdout:9/584: link d3/d1c/d73/ca7 d3/d4/d55/d6d/cbd 0 2024-02-16T19:24:15.894 INFO:tasks.workunit.client.0.smithi040.stdout:6/552: rename d0/d1/d12/d3b/d48/d4f/faa to d0/d30/fb5 0 2024-02-16T19:24:15.894 INFO:tasks.workunit.client.0.smithi040.stdout:6/553: write d0/d1/d12/d3b/d48/d4f/d9d/fab [366007,104712] 0 2024-02-16T19:24:15.894 INFO:tasks.workunit.client.0.smithi040.stdout:6/554: dread - d0/d1/d12/d3b/d48/d4f/d8b/f94 zero size 2024-02-16T19:24:15.895 INFO:tasks.workunit.client.0.smithi040.stdout:5/519: creat d1/da/da4/fa6 x:0 0 0 2024-02-16T19:24:15.898 INFO:tasks.workunit.client.0.smithi040.stdout:3/588: mknod d1/dc/d35/d48/cb3 0 2024-02-16T19:24:15.900 INFO:tasks.workunit.client.0.smithi040.stdout:6/555: unlink d0/d37/d4a/fb0 0 2024-02-16T19:24:15.901 INFO:tasks.workunit.client.0.smithi040.stdout:4/430: dread dd/de/f1a [0,4194304] 0 2024-02-16T19:24:15.903 INFO:tasks.workunit.client.0.smithi040.stdout:8/580: rmdir d6/d64/d50/d69 39 2024-02-16T19:24:15.905 INFO:tasks.workunit.client.0.smithi040.stdout:3/589: dread d1/d7/d30/f7f [0,4194304] 0 2024-02-16T19:24:15.906 INFO:tasks.workunit.client.0.smithi040.stdout:4/431: mkdir dd/d58/d8c 0 2024-02-16T19:24:15.906 INFO:tasks.workunit.client.0.smithi040.stdout:2/481: read d5/d7/d1e/d31/d4f/d63/f2c [3675202,6608] 0 2024-02-16T19:24:15.907 INFO:tasks.workunit.client.0.smithi040.stdout:2/482: read d5/d8/d42/f50 [100265,66231] 0 2024-02-16T19:24:15.907 INFO:tasks.workunit.client.0.smithi040.stdout:0/546: dwrite db/d2f/d1d/d21/d2e/d49/d5b/d67/f8f [0,4194304] 0 2024-02-16T19:24:15.907 INFO:tasks.workunit.client.0.smithi040.stdout:9/585: dread d3/f14 [0,4194304] 0 2024-02-16T19:24:15.908 INFO:tasks.workunit.client.0.smithi040.stdout:5/520: mkdir d1/da/d1c/da7 0 2024-02-16T19:24:15.908 INFO:tasks.workunit.client.0.smithi040.stdout:5/521: stat d1/d45/d63/c69 0 2024-02-16T19:24:15.908 INFO:tasks.workunit.client.0.smithi040.stdout:5/522: stat d1/da/d1c/d21/f2f 0 2024-02-16T19:24:15.910 INFO:tasks.workunit.client.0.smithi040.stdout:3/590: symlink d1/dc/lb4 0 2024-02-16T19:24:15.912 INFO:tasks.workunit.client.0.smithi040.stdout:4/432: creat dd/d58/f8d x:0 0 0 2024-02-16T19:24:15.912 INFO:tasks.workunit.client.0.smithi040.stdout:2/483: dread d5/d7/d1e/f33 [0,4194304] 0 2024-02-16T19:24:15.915 INFO:tasks.workunit.client.0.smithi040.stdout:6/556: write d0/d1/d12/d3b/d79/f88 [666630,7874] 0 2024-02-16T19:24:15.917 INFO:tasks.workunit.client.0.smithi040.stdout:0/547: creat db/d2f/d1d/d21/d2e/d49/d5b/d67/fb7 x:0 0 0 2024-02-16T19:24:15.918 INFO:tasks.workunit.client.0.smithi040.stdout:5/523: dread d1/da/d1c/f26 [0,4194304] 0 2024-02-16T19:24:15.918 INFO:tasks.workunit.client.0.smithi040.stdout:8/581: dwrite d6/d12/d22/f86 [0,4194304] 0 2024-02-16T19:24:15.919 INFO:tasks.workunit.client.0.smithi040.stdout:8/582: fsync d6/d12/f14 0 2024-02-16T19:24:15.919 INFO:tasks.workunit.client.0.smithi040.stdout:3/591: link d1/d7/d30/f7f d1/d4/d6e/fb5 0 2024-02-16T19:24:15.921 INFO:tasks.workunit.client.0.smithi040.stdout:4/433: link dd/de/d10/d13/d77/f67 dd/f8e 0 2024-02-16T19:24:15.922 INFO:tasks.workunit.client.0.smithi040.stdout:0/548: mknod db/d2f/d1d/d81/cb8 0 2024-02-16T19:24:15.924 INFO:tasks.workunit.client.0.smithi040.stdout:9/586: mkdir d3/d4/dbe 0 2024-02-16T19:24:15.928 INFO:tasks.workunit.client.0.smithi040.stdout:8/583: creat d6/d7/fb5 x:0 0 0 2024-02-16T19:24:15.928 INFO:tasks.workunit.client.0.smithi040.stdout:8/584: write d6/d2b/d44/f91 [1011262,116324] 0 2024-02-16T19:24:15.929 INFO:tasks.workunit.client.0.smithi040.stdout:3/592: symlink d1/lb6 0 2024-02-16T19:24:15.931 INFO:tasks.workunit.client.0.smithi040.stdout:4/434: symlink dd/de/l8f 0 2024-02-16T19:24:15.931 INFO:tasks.workunit.client.0.smithi040.stdout:0/549: mknod db/d25/d34/cb9 0 2024-02-16T19:24:15.931 INFO:tasks.workunit.client.0.smithi040.stdout:0/550: write db/d25/d2c/f6d [763906,104125] 0 2024-02-16T19:24:15.931 INFO:tasks.workunit.client.0.smithi040.stdout:0/551: fsync db/d2f/d1d/d81/fb1 0 2024-02-16T19:24:15.931 INFO:tasks.workunit.client.0.smithi040.stdout:9/587: symlink d3/d1c/d73/lbf 0 2024-02-16T19:24:15.933 INFO:tasks.workunit.client.0.smithi040.stdout:5/524: symlink d1/da/d1c/la8 0 2024-02-16T19:24:15.933 INFO:tasks.workunit.client.0.smithi040.stdout:5/525: fdatasync d1/da/f80 0 2024-02-16T19:24:15.933 INFO:tasks.workunit.client.0.smithi040.stdout:5/526: fdatasync d1/d62/d9a/fa2 0 2024-02-16T19:24:15.934 INFO:tasks.workunit.client.0.smithi040.stdout:3/593: symlink d1/d4/lb7 0 2024-02-16T19:24:15.935 INFO:tasks.workunit.client.0.smithi040.stdout:0/552: symlink db/de/lba 0 2024-02-16T19:24:15.935 INFO:tasks.workunit.client.0.smithi040.stdout:0/553: chown db/d25/d2c/f5a 515886654 1 2024-02-16T19:24:15.936 INFO:tasks.workunit.client.0.smithi040.stdout:4/435: write dd/de/f54 [2330194,18180] 0 2024-02-16T19:24:15.936 INFO:tasks.workunit.client.0.smithi040.stdout:4/436: chown fb 4402194 1 2024-02-16T19:24:15.936 INFO:tasks.workunit.client.0.smithi040.stdout:0/554: mkdir db/d2f/d23/d56/dbb 0 2024-02-16T19:24:15.939 INFO:tasks.workunit.client.0.smithi040.stdout:5/527: mkdir d1/da/d1c/da9 0 2024-02-16T19:24:15.939 INFO:tasks.workunit.client.0.smithi040.stdout:8/585: read d6/f9 [641838,87760] 0 2024-02-16T19:24:15.939 INFO:tasks.workunit.client.0.smithi040.stdout:0/555: symlink db/d25/lbc 0 2024-02-16T19:24:15.941 INFO:tasks.workunit.client.0.smithi040.stdout:0/556: mknod db/d2f/d1d/d21/d2e/d85/cbd 0 2024-02-16T19:24:15.942 INFO:tasks.workunit.client.0.smithi040.stdout:0/557: dread - db/d2f/d1d/d21/d2e/d49/d5b/d67/fb7 zero size 2024-02-16T19:24:15.944 INFO:tasks.workunit.client.0.smithi040.stdout:0/558: rmdir db/d2f/d23/d93/da5 0 2024-02-16T19:24:15.945 INFO:tasks.workunit.client.0.smithi040.stdout:0/559: dread db/d2f/d23/d6f/d98/f9b [0,4194304] 0 2024-02-16T19:24:15.947 INFO:tasks.workunit.client.0.smithi040.stdout:0/560: getdents db/d2f/d1d/d21/d2e/d49/d5b/d67 0 2024-02-16T19:24:15.947 INFO:tasks.workunit.client.0.smithi040.stdout:2/484: dwrite d5/d8/d42/f67 [0,4194304] 0 2024-02-16T19:24:15.948 INFO:tasks.workunit.client.0.smithi040.stdout:3/594: dread d1/d7/d30/fa8 [0,4194304] 0 2024-02-16T19:24:15.950 INFO:tasks.workunit.client.0.smithi040.stdout:2/485: mkdir d5/d5c/d65/d9d 0 2024-02-16T19:24:15.951 INFO:tasks.workunit.client.0.smithi040.stdout:6/557: dwrite d0/d1/d12/d3b/d48/d4d/d6d/f99 [0,4194304] 0 2024-02-16T19:24:15.951 INFO:tasks.workunit.client.0.smithi040.stdout:9/588: dwrite d3/d4/d11/d29/d38/d46/d6b/f90 [0,4194304] 0 2024-02-16T19:24:15.952 INFO:tasks.workunit.client.0.smithi040.stdout:5/528: write d1/f13 [509584,101308] 0 2024-02-16T19:24:15.954 INFO:tasks.workunit.client.0.smithi040.stdout:3/595: rename d1/d7/d33/l22 to d1/dc/d42/d4a/d57/d79/lb8 0 2024-02-16T19:24:15.954 INFO:tasks.workunit.client.0.smithi040.stdout:3/596: fsync d1/d7/d30/fa8 0 2024-02-16T19:24:15.954 INFO:tasks.workunit.client.0.smithi040.stdout:3/597: truncate d1/dc/d42/d58/f72 2020423 0 2024-02-16T19:24:15.954 INFO:tasks.workunit.client.0.smithi040.stdout:3/598: stat d1/d7/d30/f40 0 2024-02-16T19:24:15.954 INFO:tasks.workunit.client.0.smithi040.stdout:3/599: readlink d1/d7/d30/l31 0 2024-02-16T19:24:15.954 INFO:tasks.workunit.client.0.smithi040.stdout:9/589: write d3/d4/d55/d8a/f8f [321823,49437] 0 2024-02-16T19:24:15.955 INFO:tasks.workunit.client.0.smithi040.stdout:2/486: mkdir d5/d7/d1e/d4c/d52/d81/d9e 0 2024-02-16T19:24:15.955 INFO:tasks.workunit.client.0.smithi040.stdout:2/487: creat d5/d7/d1e/d4c/d52/f9f x:0 0 0 2024-02-16T19:24:15.956 INFO:tasks.workunit.client.0.smithi040.stdout:3/600: dread d1/d4/d6e/f84 [0,4194304] 0 2024-02-16T19:24:15.963 INFO:tasks.workunit.client.0.smithi040.stdout:6/558: dread d0/f36 [0,4194304] 0 2024-02-16T19:24:15.964 INFO:tasks.workunit.client.0.smithi040.stdout:2/488: dread d5/d7/d1e/d31/f34 [0,4194304] 0 2024-02-16T19:24:15.967 INFO:tasks.workunit.client.0.smithi040.stdout:0/561: dwrite db/d2f/d23/d6f/f95 [0,4194304] 0 2024-02-16T19:24:15.970 INFO:tasks.workunit.client.0.smithi040.stdout:8/586: dwrite d6/d2b/d44/f91 [0,4194304] 0 2024-02-16T19:24:15.970 INFO:tasks.workunit.client.0.smithi040.stdout:8/587: write d6/d2b/da3/f53 [1007816,116862] 0 2024-02-16T19:24:15.970 INFO:tasks.workunit.client.0.smithi040.stdout:8/588: write d6/d64/fb1 [2160534,72624] 0 2024-02-16T19:24:15.974 INFO:tasks.workunit.client.0.smithi040.stdout:5/529: mkdir d1/da/d1c/d21/daa 0 2024-02-16T19:24:15.976 INFO:tasks.workunit.client.0.smithi040.stdout:3/601: creat d1/dc/d35/d53/d64/d65/fb9 x:0 0 0 2024-02-16T19:24:15.977 INFO:tasks.workunit.client.0.smithi040.stdout:6/559: creat d0/d1/d6f/d90/fb6 x:0 0 0 2024-02-16T19:24:15.977 INFO:tasks.workunit.client.0.smithi040.stdout:6/560: truncate d0/d30/d5b/f9f 273019 0 2024-02-16T19:24:15.984 INFO:tasks.workunit.client.0.smithi040.stdout:0/562: rename db/d2f/d1d/d81 to db/de/dbe 0 2024-02-16T19:24:15.984 INFO:tasks.workunit.client.0.smithi040.stdout:0/563: chown db/d2f/d23/c50 246632045 1 2024-02-16T19:24:15.988 INFO:tasks.workunit.client.0.smithi040.stdout:8/589: mkdir d6/d12/d51/db6 0 2024-02-16T19:24:15.989 INFO:tasks.workunit.client.0.smithi040.stdout:5/530: mkdir d1/d45/dab 0 2024-02-16T19:24:15.989 INFO:tasks.workunit.client.0.smithi040.stdout:5/531: readlink d1/da/d51/l7c 0 2024-02-16T19:24:15.989 INFO:tasks.workunit.client.0.smithi040.stdout:5/532: dread - d1/f9c zero size 2024-02-16T19:24:15.990 INFO:tasks.workunit.client.0.smithi040.stdout:5/533: stat d1/d3c/f43 0 2024-02-16T19:24:15.990 INFO:tasks.workunit.client.0.smithi040.stdout:5/534: dread - d1/d62/d9a/fa2 zero size 2024-02-16T19:24:15.990 INFO:tasks.workunit.client.0.smithi040.stdout:9/590: getdents d3/d4/d11/d29/d38/d46 0 2024-02-16T19:24:15.992 INFO:tasks.workunit.client.0.smithi040.stdout:0/564: unlink db/d25/d34/f8a 0 2024-02-16T19:24:15.993 INFO:tasks.workunit.client.0.smithi040.stdout:9/591: mknod d3/d1c/d73/cc0 0 2024-02-16T19:24:15.993 INFO:tasks.workunit.client.0.smithi040.stdout:9/592: write d3/f14 [934128,118133] 0 2024-02-16T19:24:15.994 INFO:tasks.workunit.client.0.smithi040.stdout:0/565: truncate db/fd 2499956 0 2024-02-16T19:24:15.995 INFO:tasks.workunit.client.0.smithi040.stdout:7/431: sync 2024-02-16T19:24:15.995 INFO:tasks.workunit.client.0.smithi040.stdout:1/504: sync 2024-02-16T19:24:15.995 INFO:tasks.workunit.client.0.smithi040.stdout:1/505: readlink d17/d29/l35 0 2024-02-16T19:24:15.995 INFO:tasks.workunit.client.0.smithi040.stdout:1/506: stat d17/d26/d5f/d63/f66 0 2024-02-16T19:24:15.998 INFO:tasks.workunit.client.0.smithi040.stdout:1/507: symlink d17/la1 0 2024-02-16T19:24:15.998 INFO:tasks.workunit.client.0.smithi040.stdout:1/508: truncate d17/d26/f7a 5238174 0 2024-02-16T19:24:15.999 INFO:tasks.workunit.client.0.smithi040.stdout:9/593: rmdir d3/d4/d55/d8a 39 2024-02-16T19:24:15.999 INFO:tasks.workunit.client.0.smithi040.stdout:9/594: write d3/f21 [1342870,120649] 0 2024-02-16T19:24:16.000 INFO:tasks.workunit.client.0.smithi040.stdout:9/595: symlink d3/d4/d55/d8a/lc1 0 2024-02-16T19:24:16.000 INFO:tasks.workunit.client.0.smithi040.stdout:9/596: write d3/d4/d11/f67 [533193,40321] 0 2024-02-16T19:24:16.002 INFO:tasks.workunit.client.0.smithi040.stdout:8/590: write d6/f3f [1655813,96556] 0 2024-02-16T19:24:16.004 INFO:tasks.workunit.client.0.smithi040.stdout:6/561: dwrite d0/d1/f9c [0,4194304] 0 2024-02-16T19:24:16.004 INFO:tasks.workunit.client.0.smithi040.stdout:6/562: chown d0/lb4 7 1 2024-02-16T19:24:16.004 INFO:tasks.workunit.client.0.smithi040.stdout:5/535: write d1/da/d51/f8b [2419626,82801] 0 2024-02-16T19:24:16.005 INFO:tasks.workunit.client.0.smithi040.stdout:7/432: write d21/d4d/f56 [3477838,128505] 0 2024-02-16T19:24:16.006 INFO:tasks.workunit.client.0.smithi040.stdout:8/591: mkdir d6/d12/d51/d6e/db7 0 2024-02-16T19:24:16.009 INFO:tasks.workunit.client.0.smithi040.stdout:8/592: dread d6/d64/d50/d69/d81/fa7 [0,4194304] 0 2024-02-16T19:24:16.011 INFO:tasks.workunit.client.0.smithi040.stdout:0/566: write db/d2f/d23/d6f/f95 [4105412,68489] 0 2024-02-16T19:24:16.011 INFO:tasks.workunit.client.0.smithi040.stdout:0/567: chown db/d25/d34/l62 98239860 1 2024-02-16T19:24:16.013 INFO:tasks.workunit.client.0.smithi040.stdout:6/563: rmdir d0/d1/d12/d3b/d48/d4f/d9a 0 2024-02-16T19:24:16.013 INFO:tasks.workunit.client.0.smithi040.stdout:5/536: dread d1/d3c/f77 [0,4194304] 0 2024-02-16T19:24:16.013 INFO:tasks.workunit.client.0.smithi040.stdout:3/602: dwrite d1/d7/d36/f45 [0,4194304] 0 2024-02-16T19:24:16.013 INFO:tasks.workunit.client.0.smithi040.stdout:3/603: chown d1/ff 224096331 1 2024-02-16T19:24:16.015 INFO:tasks.workunit.client.0.smithi040.stdout:7/433: write d21/f28 [4924650,101064] 0 2024-02-16T19:24:16.020 INFO:tasks.workunit.client.0.smithi040.stdout:0/568: mknod db/d25/d2c/cbf 0 2024-02-16T19:24:16.022 INFO:tasks.workunit.client.0.smithi040.stdout:6/564: link d0/d1/d12/d3b/d48/d4f/l63 d0/d30/d32/lb7 0 2024-02-16T19:24:16.022 INFO:tasks.workunit.client.0.smithi040.stdout:6/565: chown d0/d1/d12/f18 10 1 2024-02-16T19:24:16.022 INFO:tasks.workunit.client.0.smithi040.stdout:6/566: fsync d0/d1/d12/d3b/d48/d73/f42 0 2024-02-16T19:24:16.023 INFO:tasks.workunit.client.0.smithi040.stdout:5/537: rename d1/da/f70 to d1/da/d1c/d21/daa/fac 0 2024-02-16T19:24:16.023 INFO:tasks.workunit.client.0.smithi040.stdout:5/538: readlink d1/d62/l6b 0 2024-02-16T19:24:16.024 INFO:tasks.workunit.client.0.smithi040.stdout:7/434: stat l1a 0 2024-02-16T19:24:16.024 INFO:tasks.workunit.client.0.smithi040.stdout:7/435: truncate d21/d29/d79/f8c 780289 0 2024-02-16T19:24:16.025 INFO:tasks.workunit.client.0.smithi040.stdout:8/593: getdents d6/d12/d51 0 2024-02-16T19:24:16.025 INFO:tasks.workunit.client.0.smithi040.stdout:9/597: dwrite d3/d4/d55/f2e [0,4194304] 0 2024-02-16T19:24:16.025 INFO:tasks.workunit.client.0.smithi040.stdout:3/604: write d1/d4/f19 [3409573,31943] 0 2024-02-16T19:24:16.025 INFO:tasks.workunit.client.0.smithi040.stdout:3/605: chown d1/dc/c7a 10110869 1 2024-02-16T19:24:16.026 INFO:tasks.workunit.client.0.smithi040.stdout:1/509: dwrite d17/f28 [0,4194304] 0 2024-02-16T19:24:16.027 INFO:tasks.workunit.client.0.smithi040.stdout:0/569: dread db/de/fa2 [0,4194304] 0 2024-02-16T19:24:16.028 INFO:tasks.workunit.client.0.smithi040.stdout:6/567: creat d0/d1/d12/d3b/d48/d4d/d6d/fb8 x:0 0 0 2024-02-16T19:24:16.029 INFO:tasks.workunit.client.0.smithi040.stdout:7/436: dread d21/d29/f30 [4194304,4194304] 0 2024-02-16T19:24:16.035 INFO:tasks.workunit.client.0.smithi040.stdout:5/539: rename d1/da/d51/c5a to d1/d62/cad 0 2024-02-16T19:24:16.037 INFO:tasks.workunit.client.0.smithi040.stdout:8/594: creat d6/d64/d50/fb8 x:0 0 0 2024-02-16T19:24:16.039 INFO:tasks.workunit.client.0.smithi040.stdout:9/598: creat d3/d4/d11/d71/fc2 x:0 0 0 2024-02-16T19:24:16.040 INFO:tasks.workunit.client.0.smithi040.stdout:0/570: mkdir db/de/dc0 0 2024-02-16T19:24:16.044 INFO:tasks.workunit.client.0.smithi040.stdout:6/568: creat d0/d1/d12/d3b/d48/d4d/d6d/d87/d8f/fb9 x:0 0 0 2024-02-16T19:24:16.045 INFO:tasks.workunit.client.0.smithi040.stdout:3/606: dwrite d1/ff [0,4194304] 0 2024-02-16T19:24:16.045 INFO:tasks.workunit.client.0.smithi040.stdout:6/569: chown d0/d30/d4b 1 1 2024-02-16T19:24:16.045 INFO:tasks.workunit.client.0.smithi040.stdout:3/607: fdatasync d1/dc/d35/d55/f74 0 2024-02-16T19:24:16.047 INFO:tasks.workunit.client.0.smithi040.stdout:5/540: link d1/da/c1a d1/da/d1c/d21/daa/cae 0 2024-02-16T19:24:16.049 INFO:tasks.workunit.client.0.smithi040.stdout:0/571: link db/d2f/d23/d3b/f47 db/d25/d2c/d52/fc1 0 2024-02-16T19:24:16.052 INFO:tasks.workunit.client.0.smithi040.stdout:6/570: mkdir d0/d1/d12/d3b/d48/dba 0 2024-02-16T19:24:16.052 INFO:tasks.workunit.client.0.smithi040.stdout:6/571: fsync d0/d30/d4b/f93 0 2024-02-16T19:24:16.052 INFO:tasks.workunit.client.0.smithi040.stdout:6/572: readlink d0/d1/d12/d3b/d48/l2a 0 2024-02-16T19:24:16.053 INFO:tasks.workunit.client.0.smithi040.stdout:3/608: mknod d1/dc/d35/d53/d64/d65/cba 0 2024-02-16T19:24:16.053 INFO:tasks.workunit.client.0.smithi040.stdout:3/609: dread - d1/d4/d6e/f91 zero size 2024-02-16T19:24:16.055 INFO:tasks.workunit.client.0.smithi040.stdout:0/572: link db/de/dbe/la6 db/d2f/d1d/d21/d2e/d49/d5b/lc2 0 2024-02-16T19:24:16.056 INFO:tasks.workunit.client.0.smithi040.stdout:5/541: write d1/d45/f5f [3577922,58409] 0 2024-02-16T19:24:16.056 INFO:tasks.workunit.client.0.smithi040.stdout:5/542: chown d1/d3c/f43 1 1 2024-02-16T19:24:16.056 INFO:tasks.workunit.client.0.smithi040.stdout:5/543: write d1/da/d1c/f25 [1772278,94120] 0 2024-02-16T19:24:16.057 INFO:tasks.workunit.client.0.smithi040.stdout:3/610: mknod d1/d7/d30/cbb 0 2024-02-16T19:24:16.058 INFO:tasks.workunit.client.0.smithi040.stdout:0/573: mknod db/d2f/d1d/d21/d2e/d83/cc3 0 2024-02-16T19:24:16.060 INFO:tasks.workunit.client.0.smithi040.stdout:6/573: mknod d0/d1/d12/cbb 0 2024-02-16T19:24:16.060 INFO:tasks.workunit.client.0.smithi040.stdout:6/574: write d0/f2e [2150338,102117] 0 2024-02-16T19:24:16.062 INFO:tasks.workunit.client.0.smithi040.stdout:3/611: creat d1/dc/fbc x:0 0 0 2024-02-16T19:24:16.064 INFO:tasks.workunit.client.0.smithi040.stdout:3/612: truncate d1/d4/f41 3158120 0 2024-02-16T19:24:16.065 INFO:tasks.workunit.client.0.smithi040.stdout:8/595: dwrite d6/d12/d22/f42 [0,4194304] 0 2024-02-16T19:24:16.065 INFO:tasks.workunit.client.0.smithi040.stdout:5/544: write d1/da/d1c/d21/f64 [2377532,32294] 0 2024-02-16T19:24:16.066 INFO:tasks.workunit.client.0.smithi040.stdout:8/596: dread - d6/d12/d51/d6e/fa2 zero size 2024-02-16T19:24:16.066 INFO:tasks.workunit.client.0.smithi040.stdout:3/613: creat d1/d4/d6e/d8a/da5/fbd x:0 0 0 2024-02-16T19:24:16.067 INFO:tasks.workunit.client.0.smithi040.stdout:3/614: readlink d1/l96 0 2024-02-16T19:24:16.068 INFO:tasks.workunit.client.0.smithi040.stdout:5/545: mknod d1/d3c/d97/caf 0 2024-02-16T19:24:16.070 INFO:tasks.workunit.client.0.smithi040.stdout:8/597: mkdir d6/d12/d51/d6e/db9 0 2024-02-16T19:24:16.071 INFO:tasks.workunit.client.0.smithi040.stdout:0/574: read db/d25/d2c/d52/f55 [8382231,101647] 0 2024-02-16T19:24:16.071 INFO:tasks.workunit.client.0.smithi040.stdout:0/575: readlink db/d2f/d23/l48 0 2024-02-16T19:24:16.072 INFO:tasks.workunit.client.0.smithi040.stdout:8/598: creat d6/d2b/d36/fba x:0 0 0 2024-02-16T19:24:16.074 INFO:tasks.workunit.client.0.smithi040.stdout:0/576: creat db/d2f/d23/d6f/fc4 x:0 0 0 2024-02-16T19:24:16.074 INFO:tasks.workunit.client.0.smithi040.stdout:0/577: chown db/de/f75 166695 1 2024-02-16T19:24:16.074 INFO:tasks.workunit.client.0.smithi040.stdout:0/578: chown db/d2f/d23/cae 59 1 2024-02-16T19:24:16.076 INFO:tasks.workunit.client.0.smithi040.stdout:8/599: symlink d6/d2b/da3/d8b/lbb 0 2024-02-16T19:24:16.078 INFO:tasks.workunit.client.0.smithi040.stdout:8/600: symlink d6/d12/d51/db6/lbc 0 2024-02-16T19:24:16.078 INFO:tasks.workunit.client.0.smithi040.stdout:8/601: write d6/d7/fb5 [908414,67827] 0 2024-02-16T19:24:16.080 INFO:tasks.workunit.client.0.smithi040.stdout:0/579: unlink ca 0 2024-02-16T19:24:16.094 INFO:tasks.workunit.client.0.smithi040.stdout:3/615: dwrite d1/dc/d42/d4a/f61 [0,4194304] 0 2024-02-16T19:24:16.094 INFO:tasks.workunit.client.0.smithi040.stdout:6/575: dwrite d0/d1/d12/d3b/f40 [0,4194304] 0 2024-02-16T19:24:16.094 INFO:tasks.workunit.client.0.smithi040.stdout:6/576: chown d0/fe 1 1 2024-02-16T19:24:16.095 INFO:tasks.workunit.client.0.smithi040.stdout:7/437: dwrite d21/f35 [0,4194304] 0 2024-02-16T19:24:16.100 INFO:tasks.workunit.client.0.smithi040.stdout:6/577: mkdir d0/d1/d12/d3b/d48/d4d/d6d/dbc 0 2024-02-16T19:24:16.101 INFO:tasks.workunit.client.0.smithi040.stdout:7/438: readlink d21/l69 0 2024-02-16T19:24:16.102 INFO:tasks.workunit.client.0.smithi040.stdout:6/578: unlink d0/d1/d12/d3b/d48/d4d/d6d/d87/la3 0 2024-02-16T19:24:16.102 INFO:tasks.workunit.client.0.smithi040.stdout:6/579: readlink d0/d1/d12/d3b/d48/d4f/d9d/lae 0 2024-02-16T19:24:16.104 INFO:tasks.workunit.client.0.smithi040.stdout:7/439: link d21/d4d/d7b/l86 d21/d7e/l8e 0 2024-02-16T19:24:16.105 INFO:tasks.workunit.client.0.smithi040.stdout:7/440: fsync d21/f46 0 2024-02-16T19:24:16.107 INFO:tasks.workunit.client.0.smithi040.stdout:7/441: creat d21/d4d/d7b/f8f x:0 0 0 2024-02-16T19:24:16.112 INFO:tasks.workunit.client.0.smithi040.stdout:6/580: dread d0/f2e [0,4194304] 0 2024-02-16T19:24:16.112 INFO:tasks.workunit.client.0.smithi040.stdout:6/581: fdatasync d0/d1/d12/d3b/d48/f7f 0 2024-02-16T19:24:16.112 INFO:tasks.workunit.client.0.smithi040.stdout:6/582: mkdir d0/d1/d12/d3b/d48/d4f/d9d/dbd 0 2024-02-16T19:24:16.113 INFO:tasks.workunit.client.0.smithi040.stdout:6/583: read - d0/d1/d6f/d90/f9b zero size 2024-02-16T19:24:16.114 INFO:tasks.workunit.client.0.smithi040.stdout:6/584: mkdir d0/d30/d5b/dbe 0 2024-02-16T19:24:16.114 INFO:tasks.workunit.client.0.smithi040.stdout:6/585: write d0/d1/f1f [3094819,123745] 0 2024-02-16T19:24:16.114 INFO:tasks.workunit.client.0.smithi040.stdout:6/586: chown d0/d1/d6f/d90/db1 4108 1 2024-02-16T19:24:16.114 INFO:tasks.workunit.client.0.smithi040.stdout:6/587: chown d0/d30/d4b/f93 879217620 1 2024-02-16T19:24:16.116 INFO:tasks.workunit.client.0.smithi040.stdout:6/588: mknod d0/d1/d12/d3b/d48/d4f/d9d/dbd/cbf 0 2024-02-16T19:24:16.116 INFO:tasks.workunit.client.0.smithi040.stdout:6/589: readlink d0/lb4 0 2024-02-16T19:24:16.116 INFO:tasks.workunit.client.0.smithi040.stdout:7/442: dread d21/d4d/f4b [0,4194304] 0 2024-02-16T19:24:16.117 INFO:tasks.workunit.client.0.smithi040.stdout:6/590: mkdir d0/d1/d12/d3b/d48/dc0 0 2024-02-16T19:24:16.118 INFO:tasks.workunit.client.0.smithi040.stdout:6/591: mknod d0/d30/d5b/dbe/cc1 0 2024-02-16T19:24:16.125 INFO:tasks.workunit.client.0.smithi040.stdout:6/592: dread d0/d1/f16 [0,4194304] 0 2024-02-16T19:24:16.134 INFO:tasks.workunit.client.0.smithi040.stdout:6/593: dread d0/d1/d12/d3b/d48/f7f [4194304,4194304] 0 2024-02-16T19:24:16.135 INFO:tasks.workunit.client.0.smithi040.stdout:6/594: unlink d0/d30/d32/lb7 0 2024-02-16T19:24:16.138 INFO:tasks.workunit.client.0.smithi040.stdout:7/443: dwrite d21/d4d/d7d/f83 [0,4194304] 0 2024-02-16T19:24:16.139 INFO:tasks.workunit.client.0.smithi040.stdout:0/580: dwrite db/fd [0,4194304] 0 2024-02-16T19:24:16.139 INFO:tasks.workunit.client.0.smithi040.stdout:0/581: fsync db/d2f/d23/d6f/f95 0 2024-02-16T19:24:16.142 INFO:tasks.workunit.client.0.smithi040.stdout:7/444: symlink d21/d4d/d7b/l90 0 2024-02-16T19:24:16.150 INFO:tasks.workunit.client.0.smithi040.stdout:0/582: rename db/d2f/d23/d6f/l8b to db/d25/d34/d5e/lc5 0 2024-02-16T19:24:16.150 INFO:tasks.workunit.client.0.smithi040.stdout:0/583: write db/d2f/d1d/f46 [677515,66242] 0 2024-02-16T19:24:16.150 INFO:tasks.workunit.client.0.smithi040.stdout:0/584: fsync db/d2f/d23/d6f/f95 0 2024-02-16T19:24:16.166 INFO:tasks.workunit.client.0.smithi040.stdout:6/595: dwrite d0/d1/d12/d3b/d48/d73/fa7 [0,4194304] 0 2024-02-16T19:24:16.167 INFO:tasks.workunit.client.0.smithi040.stdout:6/596: mknod d0/d1/d6f/d90/cc2 0 2024-02-16T19:24:16.168 INFO:tasks.workunit.client.0.smithi040.stdout:6/597: creat d0/d37/d4a/fc3 x:0 0 0 2024-02-16T19:24:16.168 INFO:tasks.workunit.client.0.smithi040.stdout:6/598: write d0/d1/d12/d3b/d48/d4f/f68 [1436926,121052] 0 2024-02-16T19:24:16.168 INFO:tasks.workunit.client.0.smithi040.stdout:6/599: write d0/d30/d5b/f9f [598954,97636] 0 2024-02-16T19:24:16.168 INFO:tasks.workunit.client.0.smithi040.stdout:0/585: dwrite db/d2f/d1d/d21/d2e/d49/d5b/f5d [0,4194304] 0 2024-02-16T19:24:16.168 INFO:tasks.workunit.client.0.smithi040.stdout:7/445: dwrite d21/d4d/d7b/f8f [0,4194304] 0 2024-02-16T19:24:16.168 INFO:tasks.workunit.client.0.smithi040.stdout:7/446: write d21/d4d/f5b [1032650,28289] 0 2024-02-16T19:24:16.172 INFO:tasks.workunit.client.0.smithi040.stdout:0/586: stat db/d2f/c17 0 2024-02-16T19:24:16.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/437: sync 2024-02-16T19:24:16.176 INFO:tasks.workunit.client.0.smithi040.stdout:4/438: unlink dd/de/l6f 0 2024-02-16T19:24:16.176 INFO:tasks.workunit.client.0.smithi040.stdout:4/439: write dd/de/d10/d36/f63 [2396237,892] 0 2024-02-16T19:24:16.179 INFO:tasks.workunit.client.0.smithi040.stdout:4/440: write dd/de/d45/f53 [221763,57663] 0 2024-02-16T19:24:16.180 INFO:tasks.workunit.client.0.smithi040.stdout:4/441: creat dd/de/d10/d13/f90 x:0 0 0 2024-02-16T19:24:16.180 INFO:tasks.workunit.client.0.smithi040.stdout:4/442: readlink dd/d58/l65 0 2024-02-16T19:24:16.181 INFO:tasks.workunit.client.0.smithi040.stdout:4/443: link dd/de/d10/d13/d32/c40 dd/de/c91 0 2024-02-16T19:24:16.185 INFO:tasks.workunit.client.0.smithi040.stdout:6/600: dwrite d0/d30/d5b/f9f [0,4194304] 0 2024-02-16T19:24:16.186 INFO:tasks.workunit.client.0.smithi040.stdout:4/444: dread dd/de/d45/f53 [0,4194304] 0 2024-02-16T19:24:16.187 INFO:tasks.workunit.client.0.smithi040.stdout:6/601: mknod d0/d1/d12/d3b/d48/d4d/d6d/cc4 0 2024-02-16T19:24:16.187 INFO:tasks.workunit.client.0.smithi040.stdout:6/602: write d0/d1/d12/f2c [5053518,87377] 0 2024-02-16T19:24:16.188 INFO:tasks.workunit.client.0.smithi040.stdout:4/445: getdents dd 0 2024-02-16T19:24:16.188 INFO:tasks.workunit.client.0.smithi040.stdout:7/447: write d21/f4a [524311,12687] 0 2024-02-16T19:24:16.188 INFO:tasks.workunit.client.0.smithi040.stdout:7/448: chown d21/d4d/f6f 61958380 1 2024-02-16T19:24:16.190 INFO:tasks.workunit.client.0.smithi040.stdout:6/603: creat d0/d1/d12/d3b/d48/dba/fc5 x:0 0 0 2024-02-16T19:24:16.190 INFO:tasks.workunit.client.0.smithi040.stdout:6/604: fdatasync d0/d1/d12/d3b/f50 0 2024-02-16T19:24:16.190 INFO:tasks.workunit.client.0.smithi040.stdout:6/605: stat d0/d1/l1e 0 2024-02-16T19:24:16.191 INFO:tasks.workunit.client.0.smithi040.stdout:4/446: truncate dd/de/f12 3557910 0 2024-02-16T19:24:16.191 INFO:tasks.workunit.client.0.smithi040.stdout:4/447: stat dd 0 2024-02-16T19:24:16.193 INFO:tasks.workunit.client.0.smithi040.stdout:7/449: symlink d21/d4d/l91 0 2024-02-16T19:24:16.195 INFO:tasks.workunit.client.0.smithi040.stdout:6/606: truncate d0/f4 2427800 0 2024-02-16T19:24:16.195 INFO:tasks.workunit.client.0.smithi040.stdout:6/607: chown d0/fa0 384644 1 2024-02-16T19:24:16.196 INFO:tasks.workunit.client.0.smithi040.stdout:6/608: getdents d0/d30/d32 0 2024-02-16T19:24:16.199 INFO:tasks.workunit.client.0.smithi040.stdout:6/609: unlink d0/d1/d12/d3b/d48/d4f/d9d/lae 0 2024-02-16T19:24:16.202 INFO:tasks.workunit.client.0.smithi040.stdout:6/610: creat d0/d1/d12/d3b/d48/d4f/d8b/fc6 x:0 0 0 2024-02-16T19:24:16.202 INFO:tasks.workunit.client.0.smithi040.stdout:6/611: chown d0/d30/d4b/f57 2 1 2024-02-16T19:24:16.204 INFO:tasks.workunit.client.0.smithi040.stdout:6/612: symlink d0/d37/lc7 0 2024-02-16T19:24:16.205 INFO:tasks.workunit.client.0.smithi040.stdout:6/613: link d0/d30/d32/f56 d0/d1/d12/d3b/d48/fc8 0 2024-02-16T19:24:16.206 INFO:tasks.workunit.client.0.smithi040.stdout:6/614: creat d0/d1/d12/d3b/d48/d73/d70/fc9 x:0 0 0 2024-02-16T19:24:16.208 INFO:tasks.workunit.client.0.smithi040.stdout:6/615: unlink d0/d1/d12/d3b/d48/d73/d76/d7d/f80 0 2024-02-16T19:24:16.208 INFO:tasks.workunit.client.0.smithi040.stdout:6/616: fsync d0/d1/d12/d3b/d79/f96 0 2024-02-16T19:24:16.208 INFO:tasks.workunit.client.0.smithi040.stdout:6/617: creat d0/d30/d4b/fca x:0 0 0 2024-02-16T19:24:16.210 INFO:tasks.workunit.client.0.smithi040.stdout:6/618: link d0/d1/d12/d3b/c41 d0/d30/d32/ccb 0 2024-02-16T19:24:16.211 INFO:tasks.workunit.client.0.smithi040.stdout:6/619: mkdir d0/d30/d4b/dcc 0 2024-02-16T19:24:16.211 INFO:tasks.workunit.client.0.smithi040.stdout:6/620: fsync d0/d37/d4a/f98 0 2024-02-16T19:24:16.215 INFO:tasks.workunit.client.0.smithi040.stdout:4/448: dwrite dd/de/d45/f73 [0,4194304] 0 2024-02-16T19:24:16.218 INFO:tasks.workunit.client.0.smithi040.stdout:4/449: mknod dd/de/d45/c92 0 2024-02-16T19:24:16.219 INFO:tasks.workunit.client.0.smithi040.stdout:4/450: unlink dd/d58/d68/f89 0 2024-02-16T19:24:16.219 INFO:tasks.workunit.client.0.smithi040.stdout:7/450: dwrite d21/d4d/f56 [0,4194304] 0 2024-02-16T19:24:16.220 INFO:tasks.workunit.client.0.smithi040.stdout:7/451: chown fa 394 1 2024-02-16T19:24:16.220 INFO:tasks.workunit.client.0.smithi040.stdout:4/451: mkdir dd/de/d10/d13/d77/d93 0 2024-02-16T19:24:16.230 INFO:tasks.workunit.client.0.smithi040.stdout:6/621: dwrite d0/d1/d12/f28 [8388608,4194304] 0 2024-02-16T19:24:16.290 INFO:tasks.workunit.client.0.smithi040.stdout:7/452: symlink d21/d4d/l92 0 2024-02-16T19:24:16.290 INFO:tasks.workunit.client.0.smithi040.stdout:7/453: chown d21/c68 989678582 1 2024-02-16T19:24:16.293 INFO:tasks.workunit.client.0.smithi040.stdout:6/622: mkdir d0/d1/d12/d3b/d48/d4f/d9d/dcd 0 2024-02-16T19:24:16.293 INFO:tasks.workunit.client.0.smithi040.stdout:6/623: write d0/d1/d12/d3b/d79/f88 [3553688,37010] 0 2024-02-16T19:24:16.293 INFO:tasks.workunit.client.0.smithi040.stdout:6/624: fdatasync d0/d1/d12/d3b/d48/d4f/d8b/f94 0 2024-02-16T19:24:16.296 INFO:tasks.workunit.client.0.smithi040.stdout:6/625: dread d0/d1/d12/d3b/d48/f7f [4194304,4194304] 0 2024-02-16T19:24:16.300 INFO:tasks.workunit.client.0.smithi040.stdout:7/454: chown d21/f23 908577087 1 2024-02-16T19:24:16.303 INFO:tasks.workunit.client.0.smithi040.stdout:7/455: symlink d21/d4d/d77/l93 0 2024-02-16T19:24:16.303 INFO:tasks.workunit.client.0.smithi040.stdout:6/626: dread d0/fe [0,4194304] 0 2024-02-16T19:24:16.303 INFO:tasks.workunit.client.0.smithi040.stdout:4/452: getdents dd/de/d10/d13/d77 0 2024-02-16T19:24:16.303 INFO:tasks.workunit.client.0.smithi040.stdout:4/453: rename dd/de/d10/d13/d32/d7a to dd/de/d10/d13/d32/d7a/d94 22 2024-02-16T19:24:16.304 INFO:tasks.workunit.client.0.smithi040.stdout:4/454: stat dd/de/d10/d36/f63 0 2024-02-16T19:24:16.304 INFO:tasks.workunit.client.0.smithi040.stdout:7/456: creat d21/d4d/d7b/f94 x:0 0 0 2024-02-16T19:24:16.305 INFO:tasks.workunit.client.0.smithi040.stdout:4/455: creat dd/de/d10/d13/d32/f95 x:0 0 0 2024-02-16T19:24:16.311 INFO:tasks.workunit.client.0.smithi040.stdout:6/627: read d0/d30/d4b/f57 [660540,31949] 0 2024-02-16T19:24:16.313 INFO:tasks.workunit.client.0.smithi040.stdout:2/489: sync 2024-02-16T19:24:16.313 INFO:tasks.workunit.client.0.smithi040.stdout:2/490: truncate d5/f79 160001 0 2024-02-16T19:24:16.315 INFO:tasks.workunit.client.0.smithi040.stdout:6/628: truncate d0/d37/f5d 1531380 0 2024-02-16T19:24:16.315 INFO:tasks.workunit.client.0.smithi040.stdout:6/629: creat d0/d30/d5b/fce x:0 0 0 2024-02-16T19:24:16.316 INFO:tasks.workunit.client.0.smithi040.stdout:2/491: unlink d5/d7/d1e/d31/l85 0 2024-02-16T19:24:16.317 INFO:tasks.workunit.client.0.smithi040.stdout:4/456: rmdir dd/de 39 2024-02-16T19:24:16.319 INFO:tasks.workunit.client.0.smithi040.stdout:6/630: symlink d0/d1/d12/d3b/d48/d73/lcf 0 2024-02-16T19:24:16.321 INFO:tasks.workunit.client.0.smithi040.stdout:2/492: rename d5/d7/d1e/d55/d8d/c90 to d5/d5c/d65/ca0 0 2024-02-16T19:24:16.321 INFO:tasks.workunit.client.0.smithi040.stdout:2/493: chown d5/d8/d42/f67 411 1 2024-02-16T19:24:16.321 INFO:tasks.workunit.client.0.smithi040.stdout:2/494: truncate d5/d7/d1e/d31/f7d 680051 0 2024-02-16T19:24:16.322 INFO:tasks.workunit.client.0.smithi040.stdout:6/631: symlink d0/d1/d12/d3b/d48/dc0/ld0 0 2024-02-16T19:24:16.322 INFO:tasks.workunit.client.0.smithi040.stdout:6/632: dread - d0/d30/d4b/fca zero size 2024-02-16T19:24:16.322 INFO:tasks.workunit.client.0.smithi040.stdout:6/633: chown d0/d37/d4a/f98 2386688 1 2024-02-16T19:24:16.324 INFO:tasks.workunit.client.0.smithi040.stdout:6/634: creat d0/d1/d12/d3b/d48/d4f/d9d/dbd/fd1 x:0 0 0 2024-02-16T19:24:16.325 INFO:tasks.workunit.client.0.smithi040.stdout:4/457: dread dd/f66 [0,4194304] 0 2024-02-16T19:24:16.327 INFO:tasks.workunit.client.0.smithi040.stdout:4/458: creat dd/de/d10/d13/d77/f96 x:0 0 0 2024-02-16T19:24:16.327 INFO:tasks.workunit.client.0.smithi040.stdout:7/457: dwrite d21/f62 [0,4194304] 0 2024-02-16T19:24:16.328 INFO:tasks.workunit.client.0.smithi040.stdout:7/458: mknod d21/d4d/c95 0 2024-02-16T19:24:16.331 INFO:tasks.workunit.client.0.smithi040.stdout:2/495: dread d5/d7/d1e/d4c/d52/f6b [4194304,4194304] 0 2024-02-16T19:24:16.333 INFO:tasks.workunit.client.0.smithi040.stdout:6/635: dread d0/f55 [0,4194304] 0 2024-02-16T19:24:16.334 INFO:tasks.workunit.client.0.smithi040.stdout:2/496: rmdir d5/d7/d1e 39 2024-02-16T19:24:16.337 INFO:tasks.workunit.client.0.smithi040.stdout:2/497: creat d5/d7/fa1 x:0 0 0 2024-02-16T19:24:16.337 INFO:tasks.workunit.client.0.smithi040.stdout:4/459: dread dd/f1d [0,4194304] 0 2024-02-16T19:24:16.338 INFO:tasks.workunit.client.0.smithi040.stdout:6/636: rmdir d0/d1/d12/d3b/d48/dba 39 2024-02-16T19:24:16.339 INFO:tasks.workunit.client.0.smithi040.stdout:6/637: write d0/d1/d12/f2c [4846911,98003] 0 2024-02-16T19:24:16.340 INFO:tasks.workunit.client.0.smithi040.stdout:2/498: unlink d5/d5c/d65/l4e 0 2024-02-16T19:24:16.340 INFO:tasks.workunit.client.0.smithi040.stdout:7/459: dread d21/f24 [0,4194304] 0 2024-02-16T19:24:16.341 INFO:tasks.workunit.client.0.smithi040.stdout:4/460: dread dd/f66 [0,4194304] 0 2024-02-16T19:24:16.342 INFO:tasks.workunit.client.0.smithi040.stdout:2/499: rename d5/d7/d1e/d4c/d52/f5d to d5/d7/d1e/d4c/d52/fa2 0 2024-02-16T19:24:16.345 INFO:tasks.workunit.client.0.smithi040.stdout:4/461: rmdir dd/de/d10/d13/d77/d93 0 2024-02-16T19:24:16.345 INFO:tasks.workunit.client.0.smithi040.stdout:2/500: mkdir d5/d7/d1e/d55/da3 0 2024-02-16T19:24:16.348 INFO:tasks.workunit.client.0.smithi040.stdout:4/462: mknod dd/de/d10/d13/d32/c97 0 2024-02-16T19:24:16.349 INFO:tasks.workunit.client.0.smithi040.stdout:2/501: mknod d5/d7/d1e/d4c/d52/ca4 0 2024-02-16T19:24:16.350 INFO:tasks.workunit.client.0.smithi040.stdout:4/463: creat dd/de/d10/f98 x:0 0 0 2024-02-16T19:24:16.351 INFO:tasks.workunit.client.0.smithi040.stdout:2/502: mknod d5/d8/d42/ca5 0 2024-02-16T19:24:16.351 INFO:tasks.workunit.client.0.smithi040.stdout:2/503: chown d5/d7/d1e/d31/d4f/l99 1 1 2024-02-16T19:24:16.353 INFO:tasks.workunit.client.0.smithi040.stdout:2/504: mkdir d5/d7/d12/da6 0 2024-02-16T19:24:16.353 INFO:tasks.workunit.client.0.smithi040.stdout:2/505: chown d5/d8/d42/f7a 0 1 2024-02-16T19:24:16.355 INFO:tasks.workunit.client.0.smithi040.stdout:6/638: rename d0/d1/d12/d3b/d48/d4d/d6d to d0/d30/d32/dd2 0 2024-02-16T19:24:16.355 INFO:tasks.workunit.client.0.smithi040.stdout:6/639: truncate d0/d37/fb2 1003797 0 2024-02-16T19:24:16.357 INFO:tasks.workunit.client.0.smithi040.stdout:7/460: dwrite d21/d29/f2b [0,4194304] 0 2024-02-16T19:24:16.358 INFO:tasks.workunit.client.0.smithi040.stdout:2/506: dread d5/f3a [0,4194304] 0 2024-02-16T19:24:16.359 INFO:tasks.workunit.client.0.smithi040.stdout:7/461: dread d21/d29/f59 [0,4194304] 0 2024-02-16T19:24:16.363 INFO:tasks.workunit.client.0.smithi040.stdout:7/462: read d21/d29/d79/f8c [764294,70534] 0 2024-02-16T19:24:16.365 INFO:tasks.workunit.client.0.smithi040.stdout:7/463: mknod d21/d4d/d77/d7a/c96 0 2024-02-16T19:24:16.365 INFO:tasks.workunit.client.0.smithi040.stdout:7/464: readlink d21/l2d 0 2024-02-16T19:24:16.367 INFO:tasks.workunit.client.0.smithi040.stdout:7/465: rename d21/l2d to d21/d29/l97 0 2024-02-16T19:24:16.368 INFO:tasks.workunit.client.0.smithi040.stdout:7/466: creat d21/d7e/d81/f98 x:0 0 0 2024-02-16T19:24:16.369 INFO:tasks.workunit.client.0.smithi040.stdout:7/467: mkdir d21/d4d/d77/d7a/d99 0 2024-02-16T19:24:16.369 INFO:tasks.workunit.client.0.smithi040.stdout:7/468: chown d21/d29/d79/f7f 1 1 2024-02-16T19:24:16.378 INFO:tasks.workunit.client.0.smithi040.stdout:4/464: dwrite dd/de/f54 [0,4194304] 0 2024-02-16T19:24:16.381 INFO:tasks.workunit.client.0.smithi040.stdout:4/465: unlink dd/de/d45/f53 0 2024-02-16T19:24:16.384 INFO:tasks.workunit.client.0.smithi040.stdout:4/466: mkdir dd/d58/d8c/d99 0 2024-02-16T19:24:16.388 INFO:tasks.workunit.client.0.smithi040.stdout:4/467: write dd/f66 [1228114,99719] 0 2024-02-16T19:24:16.390 INFO:tasks.workunit.client.0.smithi040.stdout:2/507: dwrite d5/d7/d1e/d4c/d52/f9f [0,4194304] 0 2024-02-16T19:24:16.390 INFO:tasks.workunit.client.0.smithi040.stdout:6/640: dwrite d0/d1/d12/d3b/d48/f7f [8388608,4194304] 0 2024-02-16T19:24:16.390 INFO:tasks.workunit.client.0.smithi040.stdout:7/469: dwrite d21/d4d/f6f [0,4194304] 0 2024-02-16T19:24:16.400 INFO:tasks.workunit.client.0.smithi040.stdout:7/470: write d21/f35 [1446786,8702] 0 2024-02-16T19:24:16.403 INFO:tasks.workunit.client.0.smithi040.stdout:2/508: mknod d5/d7/d1e/d4c/ca7 0 2024-02-16T19:24:16.409 INFO:tasks.workunit.client.0.smithi040.stdout:2/509: symlink d5/d7/d1e/d55/d8d/la8 0 2024-02-16T19:24:16.409 INFO:tasks.workunit.client.0.smithi040.stdout:2/510: write d5/d8/d42/f7a [392293,75030] 0 2024-02-16T19:24:16.409 INFO:tasks.workunit.client.0.smithi040.stdout:2/511: truncate d5/d5c/f7f 295822 0 2024-02-16T19:24:16.424 INFO:tasks.workunit.client.0.smithi040.stdout:6/641: dwrite d0/d1/d6f/d90/fa8 [0,4194304] 0 2024-02-16T19:24:16.424 INFO:tasks.workunit.client.0.smithi040.stdout:2/512: dread d5/d7/d12/d1b/f2f [0,4194304] 0 2024-02-16T19:24:16.431 INFO:tasks.workunit.client.0.smithi040.stdout:6/642: creat d0/d1/d6f/d90/fd3 x:0 0 0 2024-02-16T19:24:16.457 INFO:tasks.workunit.client.0.smithi040.stdout:6/643: dwrite d0/f33 [0,4194304] 0 2024-02-16T19:24:16.458 INFO:tasks.workunit.client.0.smithi040.stdout:1/510: sync 2024-02-16T19:24:16.458 INFO:tasks.workunit.client.0.smithi040.stdout:1/511: chown d17/d26/d4c/f8a 14 1 2024-02-16T19:24:16.459 INFO:tasks.workunit.client.0.smithi040.stdout:6/644: creat d0/d1/d12/d3b/d79/fd4 x:0 0 0 2024-02-16T19:24:16.462 INFO:tasks.workunit.client.0.smithi040.stdout:1/512: link d17/d26/d54/f9f d17/d43/fa2 0 2024-02-16T19:24:16.462 INFO:tasks.workunit.client.0.smithi040.stdout:1/513: chown d17/f28 91448636 1 2024-02-16T19:24:16.464 INFO:tasks.workunit.client.0.smithi040.stdout:6/645: getdents d0/d1/d12/d3b/d48/dba 0 2024-02-16T19:24:16.467 INFO:tasks.workunit.client.0.smithi040.stdout:6/646: creat d0/d1/d12/d3b/d48/d73/d76/fd5 x:0 0 0 2024-02-16T19:24:16.468 INFO:tasks.workunit.client.0.smithi040.stdout:8/602: sync 2024-02-16T19:24:16.469 INFO:tasks.workunit.client.0.smithi040.stdout:5/546: sync 2024-02-16T19:24:16.469 INFO:tasks.workunit.client.0.smithi040.stdout:0/587: sync 2024-02-16T19:24:16.469 INFO:tasks.workunit.client.0.smithi040.stdout:5/547: write d1/d3c/f43 [3205960,90839] 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/603: link d6/d12/d93/fa9 d6/d2b/d44/fbd 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/604: readlink d6/d12/d93/l9f 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/605: readlink d6/d2b/da3/l5e 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/606: stat d6/fa5 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/607: fdatasync d6/d64/d50/f74 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/608: write d6/d2b/d36/fba [176633,94333] 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/609: creat d6/d64/d50/fbe x:0 0 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/610: write d6/d7/d15/f67 [1375224,31814] 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/611: fdatasync d6/d2b/da3/d8b/f9d 0 2024-02-16T19:24:16.473 INFO:tasks.workunit.client.0.smithi040.stdout:8/612: fsync d6/d7/f32 0 2024-02-16T19:24:16.474 INFO:tasks.workunit.client.0.smithi040.stdout:0/588: rmdir db/d25/d2c/d52 39 2024-02-16T19:24:16.479 INFO:tasks.workunit.client.0.smithi040.stdout:5/548: mkdir d1/da/db0 0 2024-02-16T19:24:16.479 INFO:tasks.workunit.client.0.smithi040.stdout:3/616: sync 2024-02-16T19:24:16.479 INFO:tasks.workunit.client.0.smithi040.stdout:9/599: sync 2024-02-16T19:24:16.479 INFO:tasks.workunit.client.0.smithi040.stdout:9/600: chown d3/d4/d55/d6d/f7d 31 1 2024-02-16T19:24:16.482 INFO:tasks.workunit.client.0.smithi040.stdout:0/589: rename db/f20 to db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/fc6 0 2024-02-16T19:24:16.483 INFO:tasks.workunit.client.0.smithi040.stdout:0/590: stat db/de/d6a/f6e 0 2024-02-16T19:24:16.485 INFO:tasks.workunit.client.0.smithi040.stdout:3/617: mknod d1/dc/d42/d4a/d57/d79/cbe 0 2024-02-16T19:24:16.485 INFO:tasks.workunit.client.0.smithi040.stdout:3/618: write d1/dc/d35/d55/f86 [272215,15101] 0 2024-02-16T19:24:16.487 INFO:tasks.workunit.client.0.smithi040.stdout:9/601: dread d3/d1c/fb3 [0,4194304] 0 2024-02-16T19:24:16.488 INFO:tasks.workunit.client.0.smithi040.stdout:1/514: dwrite d17/d26/f7d [8388608,4194304] 0 2024-02-16T19:24:16.489 INFO:tasks.workunit.client.0.smithi040.stdout:1/515: read d17/d26/d5f/d63/f75 [860687,101094] 0 2024-02-16T19:24:16.490 INFO:tasks.workunit.client.0.smithi040.stdout:6/647: dwrite d0/d1/f1f [4194304,4194304] 0 2024-02-16T19:24:16.490 INFO:tasks.workunit.client.0.smithi040.stdout:3/619: rename d1/dc/f29 to d1/dc/d35/fbf 0 2024-02-16T19:24:16.492 INFO:tasks.workunit.client.0.smithi040.stdout:6/648: write d0/f1d [1624722,56079] 0 2024-02-16T19:24:16.493 INFO:tasks.workunit.client.0.smithi040.stdout:9/602: mkdir d3/d4/d55/dc3 0 2024-02-16T19:24:16.496 INFO:tasks.workunit.client.0.smithi040.stdout:6/649: dread d0/f3f [0,4194304] 0 2024-02-16T19:24:16.496 INFO:tasks.workunit.client.0.smithi040.stdout:6/650: write d0/d1/d12/f66 [106375,35076] 0 2024-02-16T19:24:16.497 INFO:tasks.workunit.client.0.smithi040.stdout:8/613: dwrite d6/d2b/f63 [0,4194304] 0 2024-02-16T19:24:16.499 INFO:tasks.workunit.client.0.smithi040.stdout:3/620: mknod d1/d8c/d8e/cc0 0 2024-02-16T19:24:16.501 INFO:tasks.workunit.client.0.smithi040.stdout:6/651: dread d0/d1/d12/d3b/d48/d4f/d9d/fab [0,4194304] 0 2024-02-16T19:24:16.502 INFO:tasks.workunit.client.0.smithi040.stdout:5/549: dwrite d1/f17 [0,4194304] 0 2024-02-16T19:24:16.503 INFO:tasks.workunit.client.0.smithi040.stdout:5/550: dread d1/f4c [0,4194304] 0 2024-02-16T19:24:16.503 INFO:tasks.workunit.client.0.smithi040.stdout:5/551: write d1/da/d1c/f26 [3566080,37428] 0 2024-02-16T19:24:16.503 INFO:tasks.workunit.client.0.smithi040.stdout:5/552: dread - d1/da/d1c/d21/f9d zero size 2024-02-16T19:24:16.504 INFO:tasks.workunit.client.0.smithi040.stdout:0/591: dwrite db/de/f59 [0,4194304] 0 2024-02-16T19:24:16.507 INFO:tasks.workunit.client.0.smithi040.stdout:3/621: rmdir d1/dc/d42/d4a/d57 39 2024-02-16T19:24:16.507 INFO:tasks.workunit.client.0.smithi040.stdout:3/622: fsync d1/d4/f4f 0 2024-02-16T19:24:16.509 INFO:tasks.workunit.client.0.smithi040.stdout:6/652: truncate d0/d30/d32/dd2/f99 3421816 0 2024-02-16T19:24:16.509 INFO:tasks.workunit.client.0.smithi040.stdout:6/653: fsync d0/d37/f5d 0 2024-02-16T19:24:16.512 INFO:tasks.workunit.client.0.smithi040.stdout:8/614: dwrite d6/d7/f7b [0,4194304] 0 2024-02-16T19:24:16.516 INFO:tasks.workunit.client.0.smithi040.stdout:9/603: rmdir d3/d4/d55/d31 39 2024-02-16T19:24:16.516 INFO:tasks.workunit.client.0.smithi040.stdout:9/604: write d3/d4/d11/d29/d38/d46/f94 [4834615,98333] 0 2024-02-16T19:24:16.520 INFO:tasks.workunit.client.0.smithi040.stdout:3/623: dwrite d1/d7/fa3 [0,4194304] 0 2024-02-16T19:24:16.527 INFO:tasks.workunit.client.0.smithi040.stdout:5/553: rmdir d1/d3c 39 2024-02-16T19:24:16.528 INFO:tasks.workunit.client.0.smithi040.stdout:3/624: dread d1/d7/d36/f45 [0,4194304] 0 2024-02-16T19:24:16.528 INFO:tasks.workunit.client.0.smithi040.stdout:0/592: creat db/d2f/d23/d3b/fc7 x:0 0 0 2024-02-16T19:24:16.533 INFO:tasks.workunit.client.0.smithi040.stdout:6/654: creat d0/d1/d12/d3b/fd6 x:0 0 0 2024-02-16T19:24:16.536 INFO:tasks.workunit.client.0.smithi040.stdout:5/554: dread d1/d45/d63/f7b [0,4194304] 0 2024-02-16T19:24:16.536 INFO:tasks.workunit.client.0.smithi040.stdout:5/555: fdatasync d1/f17 0 2024-02-16T19:24:16.536 INFO:tasks.workunit.client.0.smithi040.stdout:5/556: fdatasync d1/da/d51/f81 0 2024-02-16T19:24:16.539 INFO:tasks.workunit.client.0.smithi040.stdout:8/615: mknod d6/d64/d50/d69/d81/cbf 0 2024-02-16T19:24:16.539 INFO:tasks.workunit.client.0.smithi040.stdout:8/616: dread - d6/d64/faa zero size 2024-02-16T19:24:16.539 INFO:tasks.workunit.client.0.smithi040.stdout:8/617: truncate d6/d2b/da3/f53 1656540 0 2024-02-16T19:24:16.541 INFO:tasks.workunit.client.0.smithi040.stdout:9/605: creat d3/d4/d11/d71/da2/fc4 x:0 0 0 2024-02-16T19:24:16.542 INFO:tasks.workunit.client.0.smithi040.stdout:9/606: chown d3/d4/d55/f66 5403021 1 2024-02-16T19:24:16.542 INFO:tasks.workunit.client.0.smithi040.stdout:9/607: creat d3/d4/d11/fc5 x:0 0 0 2024-02-16T19:24:16.542 INFO:tasks.workunit.client.0.smithi040.stdout:9/608: chown d3/d4/d55/l41 2865 1 2024-02-16T19:24:16.542 INFO:tasks.workunit.client.0.smithi040.stdout:9/609: write d3/d4/d55/d6d/f7c [29560,98368] 0 2024-02-16T19:24:16.542 INFO:tasks.workunit.client.0.smithi040.stdout:9/610: truncate d3/d4/d55/d31/f9b 669220 0 2024-02-16T19:24:16.543 INFO:tasks.workunit.client.0.smithi040.stdout:0/593: dwrite db/d2f/d1d/d21/d2e/d49/d5b/d67/f92 [0,4194304] 0 2024-02-16T19:24:16.550 INFO:tasks.workunit.client.0.smithi040.stdout:3/625: creat d1/d8c/dae/fc1 x:0 0 0 2024-02-16T19:24:16.550 INFO:tasks.workunit.client.0.smithi040.stdout:3/626: readlink d1/l83 0 2024-02-16T19:24:16.552 INFO:tasks.workunit.client.0.smithi040.stdout:6/655: symlink d0/d1/d12/d3b/d48/d4f/d8b/dac/ld7 0 2024-02-16T19:24:16.552 INFO:tasks.workunit.client.0.smithi040.stdout:6/656: stat d0/f2e 0 2024-02-16T19:24:16.554 INFO:tasks.workunit.client.0.smithi040.stdout:5/557: rename d1/d45/d63 to d1/d3c/d97/db1 0 2024-02-16T19:24:16.558 INFO:tasks.workunit.client.0.smithi040.stdout:9/611: truncate d3/d4/d11/f19 290988 0 2024-02-16T19:24:16.558 INFO:tasks.workunit.client.0.smithi040.stdout:9/612: chown d3/d4/d55/d6d/f7c 1 1 2024-02-16T19:24:16.558 INFO:tasks.workunit.client.0.smithi040.stdout:9/613: truncate d3/d1c/d73/fb1 448787 0 2024-02-16T19:24:16.565 INFO:tasks.workunit.client.0.smithi040.stdout:8/618: dwrite d6/d2b/f9c [4194304,4194304] 0 2024-02-16T19:24:16.565 INFO:tasks.workunit.client.0.smithi040.stdout:0/594: dwrite db/d2f/d1d/d21/d2e/d83/fa8 [0,4194304] 0 2024-02-16T19:24:16.568 INFO:tasks.workunit.client.0.smithi040.stdout:0/595: dread db/d2f/d1d/f46 [0,4194304] 0 2024-02-16T19:24:16.569 INFO:tasks.workunit.client.0.smithi040.stdout:0/596: chown db/d25/d34/la4 1 1 2024-02-16T19:24:16.569 INFO:tasks.workunit.client.0.smithi040.stdout:0/597: truncate db/de/dbe/fb1 5175580 0 2024-02-16T19:24:16.569 INFO:tasks.workunit.client.0.smithi040.stdout:0/598: getdents db/de/dc0 0 2024-02-16T19:24:16.569 INFO:tasks.workunit.client.0.smithi040.stdout:0/599: chown db/d2f/f77 29 1 2024-02-16T19:24:16.569 INFO:tasks.workunit.client.0.smithi040.stdout:5/558: write d1/f17 [3661769,56856] 0 2024-02-16T19:24:16.569 INFO:tasks.workunit.client.0.smithi040.stdout:5/559: chown d1/d62/d9a/f9b 98905184 1 2024-02-16T19:24:16.581 INFO:tasks.workunit.client.0.smithi040.stdout:6/657: unlink d0/d30/d32/f2d 0 2024-02-16T19:24:16.585 INFO:tasks.workunit.client.0.smithi040.stdout:9/614: rename d3/d4/d11/d29/d38/d46/f6e to d3/d4/d11/fc6 0 2024-02-16T19:24:16.585 INFO:tasks.workunit.client.0.smithi040.stdout:9/615: chown d3/d4/d11/d29/d38/d46/d6b/d85 493439090 1 2024-02-16T19:24:16.585 INFO:tasks.workunit.client.0.smithi040.stdout:9/616: creat d3/d4/d55/fc7 x:0 0 0 2024-02-16T19:24:16.585 INFO:tasks.workunit.client.0.smithi040.stdout:9/617: creat d3/d4/d11/d29/d38/d46/d6b/fc8 x:0 0 0 2024-02-16T19:24:16.588 INFO:tasks.workunit.client.0.smithi040.stdout:8/619: symlink d6/d2b/d36/lc0 0 2024-02-16T19:24:16.592 INFO:tasks.workunit.client.0.smithi040.stdout:0/600: symlink db/lc8 0 2024-02-16T19:24:16.596 INFO:tasks.workunit.client.0.smithi040.stdout:8/620: dread d6/d12/f1f [0,4194304] 0 2024-02-16T19:24:16.596 INFO:tasks.workunit.client.0.smithi040.stdout:5/560: mkdir d1/da/d7e/db2 0 2024-02-16T19:24:16.596 INFO:tasks.workunit.client.0.smithi040.stdout:5/561: write d1/da/d1c/f82 [275164,757] 0 2024-02-16T19:24:16.598 INFO:tasks.workunit.client.0.smithi040.stdout:6/658: unlink d0/d1/d12/d3b/d48/d4f/d8b/f95 0 2024-02-16T19:24:16.605 INFO:tasks.workunit.client.0.smithi040.stdout:8/621: read d6/d12/f14 [2738168,54611] 0 2024-02-16T19:24:16.606 INFO:tasks.workunit.client.0.smithi040.stdout:9/618: mkdir d3/d4/d11/d71/dc9 0 2024-02-16T19:24:16.607 INFO:tasks.workunit.client.0.smithi040.stdout:0/601: mknod db/d25/d2c/d52/cc9 0 2024-02-16T19:24:16.610 INFO:tasks.workunit.client.0.smithi040.stdout:5/562: creat d1/d45/dab/fb3 x:0 0 0 2024-02-16T19:24:16.614 INFO:tasks.workunit.client.0.smithi040.stdout:8/622: unlink d6/d2b/c73 0 2024-02-16T19:24:16.616 INFO:tasks.workunit.client.0.smithi040.stdout:0/602: read db/d2f/f66 [3871495,8742] 0 2024-02-16T19:24:16.616 INFO:tasks.workunit.client.0.smithi040.stdout:9/619: mkdir d3/d4/d11/d71/dc9/dca 0 2024-02-16T19:24:16.616 INFO:tasks.workunit.client.0.smithi040.stdout:9/620: dread - d3/d4/d11/d29/d38/d46/d6b/d85/fbb zero size 2024-02-16T19:24:16.616 INFO:tasks.workunit.client.0.smithi040.stdout:9/621: fsync d3/d4/d11/d71/da8/f5c 0 2024-02-16T19:24:16.617 INFO:tasks.workunit.client.0.smithi040.stdout:5/563: unlink d1/da/d1c/c83 0 2024-02-16T19:24:16.626 INFO:tasks.workunit.client.0.smithi040.stdout:8/623: mknod d6/d2b/cc1 0 2024-02-16T19:24:16.626 INFO:tasks.workunit.client.0.smithi040.stdout:8/624: write d6/d64/d50/f7d [166275,56864] 0 2024-02-16T19:24:16.627 INFO:tasks.workunit.client.0.smithi040.stdout:0/603: symlink db/d2f/d23/d56/dbb/lca 0 2024-02-16T19:24:16.630 INFO:tasks.workunit.client.0.smithi040.stdout:5/564: dwrite d1/f9c [0,4194304] 0 2024-02-16T19:24:16.642 INFO:tasks.workunit.client.0.smithi040.stdout:5/565: dread d1/d3c/d97/db1/fa3 [0,4194304] 0 2024-02-16T19:24:16.642 INFO:tasks.workunit.client.0.smithi040.stdout:5/566: readlink d1/d45/l48 0 2024-02-16T19:24:16.644 INFO:tasks.workunit.client.0.smithi040.stdout:5/567: link d1/da/c8e d1/cb4 0 2024-02-16T19:24:16.647 INFO:tasks.workunit.client.0.smithi040.stdout:5/568: truncate d1/da/f35 3444795 0 2024-02-16T19:24:16.647 INFO:tasks.workunit.client.0.smithi040.stdout:5/569: fsync d1/da/f2a 0 2024-02-16T19:24:16.652 INFO:tasks.workunit.client.0.smithi040.stdout:5/570: rename d1/d62/cad to d1/da/d7e/db2/cb5 0 2024-02-16T19:24:16.652 INFO:tasks.workunit.client.0.smithi040.stdout:5/571: chown d1/d3c/l95 196979863 1 2024-02-16T19:24:16.653 INFO:tasks.workunit.client.0.smithi040.stdout:8/625: dwrite d6/d2b/d36/fba [0,4194304] 0 2024-02-16T19:24:16.653 INFO:tasks.workunit.client.0.smithi040.stdout:8/626: chown d6/d12/c92 136363402 1 2024-02-16T19:24:16.655 INFO:tasks.workunit.client.0.smithi040.stdout:0/604: dwrite db/d2f/d1d/d21/d2e/d49/d5b/fb3 [0,4194304] 0 2024-02-16T19:24:16.655 INFO:tasks.workunit.client.0.smithi040.stdout:0/605: creat db/d25/d2c/fcb x:0 0 0 2024-02-16T19:24:16.658 INFO:tasks.workunit.client.0.smithi040.stdout:8/627: symlink d6/d2b/da3/d8b/lc2 0 2024-02-16T19:24:16.667 INFO:tasks.workunit.client.0.smithi040.stdout:0/606: creat db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/fcc x:0 0 0 2024-02-16T19:24:16.667 INFO:tasks.workunit.client.0.smithi040.stdout:0/607: write db/de/d6a/f82 [4677008,113265] 0 2024-02-16T19:24:16.667 INFO:tasks.workunit.client.0.smithi040.stdout:0/608: write db/d25/d2c/d52/f57 [535400,123504] 0 2024-02-16T19:24:16.670 INFO:tasks.workunit.client.0.smithi040.stdout:8/628: rename d6/d12/f14 to d6/d12/d51/d6e/fc3 0 2024-02-16T19:24:16.670 INFO:tasks.workunit.client.0.smithi040.stdout:8/629: chown d6/d2b/d44/c8a 884 1 2024-02-16T19:24:16.670 INFO:tasks.workunit.client.0.smithi040.stdout:8/630: creat d6/d64/fc4 x:0 0 0 2024-02-16T19:24:16.673 INFO:tasks.workunit.client.0.smithi040.stdout:9/622: rmdir d3/d4/d11/d29/d38/d46/d6b 39 2024-02-16T19:24:16.674 INFO:tasks.workunit.client.0.smithi040.stdout:8/631: truncate d6/d12/d51/d6e/f96 969337 0 2024-02-16T19:24:16.674 INFO:tasks.workunit.client.0.smithi040.stdout:8/632: chown d6/d2b/da3/d8b/lbb 60156 1 2024-02-16T19:24:16.676 INFO:tasks.workunit.client.0.smithi040.stdout:0/609: dread db/de/dbe/f8c [0,4194304] 0 2024-02-16T19:24:16.676 INFO:tasks.workunit.client.0.smithi040.stdout:0/610: write db/d25/d2c/d52/f91 [964989,27257] 0 2024-02-16T19:24:16.676 INFO:tasks.workunit.client.0.smithi040.stdout:8/633: dread d6/d7/d1d/f52 [0,4194304] 0 2024-02-16T19:24:16.677 INFO:tasks.workunit.client.0.smithi040.stdout:9/623: rename d3/d4/c89 to d3/d4/d55/d8a/ccb 0 2024-02-16T19:24:16.677 INFO:tasks.workunit.client.0.smithi040.stdout:9/624: dread - d3/d4/d11/d29/d38/d46/d6b/fc8 zero size 2024-02-16T19:24:16.677 INFO:tasks.workunit.client.0.smithi040.stdout:9/625: read d3/d4/d11/d29/f2b [121847,14144] 0 2024-02-16T19:24:16.680 INFO:tasks.workunit.client.0.smithi040.stdout:0/611: creat db/d2f/d1d/d21/d2e/d83/fcd x:0 0 0 2024-02-16T19:24:16.680 INFO:tasks.workunit.client.0.smithi040.stdout:0/612: chown db/d25/f4f 14764147 1 2024-02-16T19:24:16.682 INFO:tasks.workunit.client.0.smithi040.stdout:8/634: unlink d6/d2b/da3/d8b/fa0 0 2024-02-16T19:24:16.684 INFO:tasks.workunit.client.0.smithi040.stdout:9/626: unlink d3/d4/d11/d29/d38/d46/l7f 0 2024-02-16T19:24:16.686 INFO:tasks.workunit.client.0.smithi040.stdout:0/613: mknod db/d2f/d1d/d21/d2e/d49/d5b/cce 0 2024-02-16T19:24:16.688 INFO:tasks.workunit.client.0.smithi040.stdout:8/635: mknod d6/d12/d51/db6/cc5 0 2024-02-16T19:24:16.694 INFO:tasks.workunit.client.0.smithi040.stdout:9/627: dread d3/d4/d55/f2e [0,4194304] 0 2024-02-16T19:24:16.694 INFO:tasks.workunit.client.0.smithi040.stdout:9/628: getdents d3/d4/d55/d6d/daf 0 2024-02-16T19:24:16.694 INFO:tasks.workunit.client.0.smithi040.stdout:9/629: dread - d3/d4/d11/fc5 zero size 2024-02-16T19:24:16.694 INFO:tasks.workunit.client.0.smithi040.stdout:9/630: write d3/d4/d11/d29/d38/d46/d6b/fc8 [131505,48373] 0 2024-02-16T19:24:16.694 INFO:tasks.workunit.client.0.smithi040.stdout:9/631: fdatasync d3/d4/d11/fb8 0 2024-02-16T19:24:16.717 INFO:tasks.workunit.client.0.smithi040.stdout:8/636: dwrite d6/d64/fc4 [0,4194304] 0 2024-02-16T19:24:16.717 INFO:tasks.workunit.client.0.smithi040.stdout:9/632: dwrite d3/d1c/f3f [8388608,4194304] 0 2024-02-16T19:24:16.717 INFO:tasks.workunit.client.0.smithi040.stdout:8/637: write d6/d2b/f63 [4488271,93197] 0 2024-02-16T19:24:16.724 INFO:tasks.workunit.client.0.smithi040.stdout:8/638: truncate d6/d7/d15/f47 3530999 0 2024-02-16T19:24:16.726 INFO:tasks.workunit.client.0.smithi040.stdout:0/614: rmdir db/d25/d2c 39 2024-02-16T19:24:16.726 INFO:tasks.workunit.client.0.smithi040.stdout:0/615: write db/d2f/d23/d3b/f47 [1035460,95134] 0 2024-02-16T19:24:16.726 INFO:tasks.workunit.client.0.smithi040.stdout:0/616: write db/f7f [54002,31677] 0 2024-02-16T19:24:16.731 INFO:tasks.workunit.client.0.smithi040.stdout:9/633: dwrite d3/d4/d55/d6d/fb7 [0,4194304] 0 2024-02-16T19:24:16.732 INFO:tasks.workunit.client.0.smithi040.stdout:8/639: mkdir d6/d12/d51/d6e/db7/dc6 0 2024-02-16T19:24:16.743 INFO:tasks.workunit.client.0.smithi040.stdout:9/634: unlink d3/d1c/la6 0 2024-02-16T19:24:16.745 INFO:tasks.workunit.client.0.smithi040.stdout:8/640: mknod d6/d64/d50/d69/cc7 0 2024-02-16T19:24:16.751 INFO:tasks.workunit.client.0.smithi040.stdout:9/635: mkdir d3/d4/d55/d70/dcc 0 2024-02-16T19:24:16.761 INFO:tasks.workunit.client.0.smithi040.stdout:9/636: dread d3/d4/d55/f2e [0,4194304] 0 2024-02-16T19:24:16.761 INFO:tasks.workunit.client.0.smithi040.stdout:9/637: dread - d3/d1c/f9e zero size 2024-02-16T19:24:16.771 INFO:tasks.workunit.client.0.smithi040.stdout:8/641: dwrite d6/d12/d51/d6e/f8f [0,4194304] 0 2024-02-16T19:24:16.772 INFO:tasks.workunit.client.0.smithi040.stdout:8/642: dread d6/d64/d50/d69/d81/fa7 [0,4194304] 0 2024-02-16T19:24:16.787 INFO:tasks.workunit.client.0.smithi040.stdout:8/643: link d6/d64/d50/f8c d6/d7/d15/fc8 0 2024-02-16T19:24:16.787 INFO:tasks.workunit.client.0.smithi040.stdout:8/644: truncate d6/d12/d22/f86 4429307 0 2024-02-16T19:24:16.791 INFO:tasks.workunit.client.0.smithi040.stdout:8/645: unlink d6/d2b/da3/c5c 0 2024-02-16T19:24:16.797 INFO:tasks.workunit.client.0.smithi040.stdout:8/646: rmdir d6 39 2024-02-16T19:24:16.800 INFO:tasks.workunit.client.0.smithi040.stdout:8/647: mkdir d6/d12/d51/d6e/db9/dc9 0 2024-02-16T19:24:16.800 INFO:tasks.workunit.client.0.smithi040.stdout:8/648: chown d6/d7/d1d/l25 3734849 1 2024-02-16T19:24:16.800 INFO:tasks.workunit.client.0.smithi040.stdout:8/649: readlink d6/d12/l2a 0 2024-02-16T19:24:16.804 INFO:tasks.workunit.client.0.smithi040.stdout:8/650: link d6/d7/d15/f18 d6/d7/fca 0 2024-02-16T19:24:16.815 INFO:tasks.workunit.client.0.smithi040.stdout:0/617: dwrite db/d2f/d1d/d21/f51 [0,4194304] 0 2024-02-16T19:24:16.815 INFO:tasks.workunit.client.0.smithi040.stdout:0/618: truncate db/d2f/d1d/d21/d2e/d49/f87 1508159 0 2024-02-16T19:24:16.815 INFO:tasks.workunit.client.0.smithi040.stdout:0/619: chown db/de/dbe/f8c 5312 1 2024-02-16T19:24:16.816 INFO:tasks.workunit.client.0.smithi040.stdout:0/620: getdents db/de/dbe 0 2024-02-16T19:24:16.816 INFO:tasks.workunit.client.0.smithi040.stdout:0/621: chown db/d25/d34/d5e/c8e 1888277 1 2024-02-16T19:24:16.818 INFO:tasks.workunit.client.0.smithi040.stdout:0/622: creat db/d2f/d23/fcf x:0 0 0 2024-02-16T19:24:16.832 INFO:tasks.workunit.client.0.smithi040.stdout:4/468: sync 2024-02-16T19:24:16.832 INFO:tasks.workunit.client.0.smithi040.stdout:7/471: sync 2024-02-16T19:24:16.832 INFO:tasks.workunit.client.0.smithi040.stdout:4/469: chown dd/d58/d5e 256468261 1 2024-02-16T19:24:16.832 INFO:tasks.workunit.client.0.smithi040.stdout:2/513: sync 2024-02-16T19:24:16.832 INFO:tasks.workunit.client.0.smithi040.stdout:1/516: sync 2024-02-16T19:24:16.832 INFO:tasks.workunit.client.0.smithi040.stdout:1/517: write d17/d26/d4c/f8a [837149,38332] 0 2024-02-16T19:24:16.834 INFO:tasks.workunit.client.0.smithi040.stdout:0/623: dwrite db/d2f/d1d/d21/d2e/d85/f86 [0,4194304] 0 2024-02-16T19:24:16.836 INFO:tasks.workunit.client.0.smithi040.stdout:7/472: getdents d21 0 2024-02-16T19:24:16.837 INFO:tasks.workunit.client.0.smithi040.stdout:4/470: getdents dd/de/d71 0 2024-02-16T19:24:16.837 INFO:tasks.workunit.client.0.smithi040.stdout:4/471: chown dd/de/d10/d13/d77/f96 387 1 2024-02-16T19:24:16.837 INFO:tasks.workunit.client.0.smithi040.stdout:4/472: truncate dd/de/d10/d13/d77/f67 555185 0 2024-02-16T19:24:16.837 INFO:tasks.workunit.client.0.smithi040.stdout:4/473: write dd/de/d10/d36/f81 [1418279,20234] 0 2024-02-16T19:24:16.837 INFO:tasks.workunit.client.0.smithi040.stdout:4/474: stat dd/l78 0 2024-02-16T19:24:16.839 INFO:tasks.workunit.client.0.smithi040.stdout:2/514: unlink d5/d7/d12/c8c 0 2024-02-16T19:24:16.842 INFO:tasks.workunit.client.0.smithi040.stdout:1/518: unlink d17/d3c/c57 0 2024-02-16T19:24:16.848 INFO:tasks.workunit.client.0.smithi040.stdout:0/624: rename db/de/dbe/f8c to db/d25/d34/d5e/fd0 0 2024-02-16T19:24:16.850 INFO:tasks.workunit.client.0.smithi040.stdout:7/473: truncate d21/f24 2249182 0 2024-02-16T19:24:16.858 INFO:tasks.workunit.client.0.smithi040.stdout:2/515: rename d5/d7/d12/d1b/f2f to d5/d7/d10/d40/fa9 0 2024-02-16T19:24:16.858 INFO:tasks.workunit.client.0.smithi040.stdout:2/516: write d5/d7/d12/f76 [315440,26370] 0 2024-02-16T19:24:16.863 INFO:tasks.workunit.client.0.smithi040.stdout:4/475: dwrite dd/f52 [0,4194304] 0 2024-02-16T19:24:16.863 INFO:tasks.workunit.client.0.smithi040.stdout:0/625: write db/d2f/d23/f40 [3591900,130740] 0 2024-02-16T19:24:16.863 INFO:tasks.workunit.client.0.smithi040.stdout:2/517: read d5/d7/d1e/d55/f6c [926577,45974] 0 2024-02-16T19:24:16.867 INFO:tasks.workunit.client.0.smithi040.stdout:2/518: dread d5/d8/f69 [0,4194304] 0 2024-02-16T19:24:16.869 INFO:tasks.workunit.client.0.smithi040.stdout:1/519: dread d17/d26/d4c/f98 [0,4194304] 0 2024-02-16T19:24:16.871 INFO:tasks.workunit.client.0.smithi040.stdout:7/474: dwrite d21/d29/f2b [0,4194304] 0 2024-02-16T19:24:16.871 INFO:tasks.workunit.client.0.smithi040.stdout:7/475: write d21/d29/d79/f7f [1536581,13060] 0 2024-02-16T19:24:16.876 INFO:tasks.workunit.client.0.smithi040.stdout:4/476: rename dd/de/d10/f98 to dd/de/d71/f9a 0 2024-02-16T19:24:16.878 INFO:tasks.workunit.client.0.smithi040.stdout:1/520: creat d17/d29/d8b/d93/fa3 x:0 0 0 2024-02-16T19:24:16.879 INFO:tasks.workunit.client.0.smithi040.stdout:1/521: fdatasync d17/d3c/d6b/f8f 0 2024-02-16T19:24:16.879 INFO:tasks.workunit.client.0.smithi040.stdout:1/522: creat d17/d3c/d71/fa4 x:0 0 0 2024-02-16T19:24:16.881 INFO:tasks.workunit.client.0.smithi040.stdout:7/476: symlink d21/d4d/l9a 0 2024-02-16T19:24:16.881 INFO:tasks.workunit.client.0.smithi040.stdout:7/477: chown d21/d29/f50 236780360 1 2024-02-16T19:24:16.885 INFO:tasks.workunit.client.0.smithi040.stdout:4/477: getdents dd/de/d45 0 2024-02-16T19:24:16.886 INFO:tasks.workunit.client.0.smithi040.stdout:0/626: dwrite db/de/f44 [0,4194304] 0 2024-02-16T19:24:16.888 INFO:tasks.workunit.client.0.smithi040.stdout:4/478: rename dd/de/d45/c48 to dd/d58/c9b 0 2024-02-16T19:24:16.888 INFO:tasks.workunit.client.0.smithi040.stdout:2/519: dwrite d5/d7/d1e/d4c/d52/f9c [0,4194304] 0 2024-02-16T19:24:16.893 INFO:tasks.workunit.client.0.smithi040.stdout:1/523: write d17/f37 [1952914,115563] 0 2024-02-16T19:24:16.893 INFO:tasks.workunit.client.0.smithi040.stdout:1/524: chown d17/d43/d38 37579236 1 2024-02-16T19:24:16.893 INFO:tasks.workunit.client.0.smithi040.stdout:1/525: stat d17/c65 0 2024-02-16T19:24:16.903 INFO:tasks.workunit.client.0.smithi040.stdout:7/478: dwrite d21/d29/f50 [0,4194304] 0 2024-02-16T19:24:16.916 INFO:tasks.workunit.client.0.smithi040.stdout:2/520: mkdir d5/d7/d1e/d4c/d52/daa 0 2024-02-16T19:24:16.916 INFO:tasks.workunit.client.0.smithi040.stdout:2/521: stat d5/d5c/d65/c8a 0 2024-02-16T19:24:16.918 INFO:tasks.workunit.client.0.smithi040.stdout:1/526: symlink d17/d29/la5 0 2024-02-16T19:24:16.918 INFO:tasks.workunit.client.0.smithi040.stdout:1/527: read d17/f25 [1080816,89642] 0 2024-02-16T19:24:16.918 INFO:tasks.workunit.client.0.smithi040.stdout:1/528: chown d17/d3c/d71/fa0 0 1 2024-02-16T19:24:16.918 INFO:tasks.workunit.client.0.smithi040.stdout:1/529: write f1 [1128559,68436] 0 2024-02-16T19:24:16.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/651: truncate d6/d12/d22/f42 39009 0 2024-02-16T19:24:16.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/652: stat d6/d12/d22/f86 0 2024-02-16T19:24:16.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/653: stat d6/d7/l37 0 2024-02-16T19:24:16.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/654: write d6/d7/d1d/f52 [296615,117085] 0 2024-02-16T19:24:16.921 INFO:tasks.workunit.client.0.smithi040.stdout:8/655: read - d6/d12/d51/fa1 zero size 2024-02-16T19:24:16.922 INFO:tasks.workunit.client.0.smithi040.stdout:2/522: dread d5/d5c/f7f [0,4194304] 0 2024-02-16T19:24:16.924 INFO:tasks.workunit.client.0.smithi040.stdout:1/530: mkdir d17/d26/d5f/da6 0 2024-02-16T19:24:16.924 INFO:tasks.workunit.client.0.smithi040.stdout:1/531: write d17/d3c/d71/f87 [803447,130229] 0 2024-02-16T19:24:16.924 INFO:tasks.workunit.client.0.smithi040.stdout:0/627: dwrite db/d2f/d1d/d21/d2e/d49/f58 [0,4194304] 0 2024-02-16T19:24:16.925 INFO:tasks.workunit.client.0.smithi040.stdout:4/479: dwrite dd/de/d45/f74 [0,4194304] 0 2024-02-16T19:24:16.925 INFO:tasks.workunit.client.0.smithi040.stdout:4/480: dread - dd/f6e zero size 2024-02-16T19:24:16.926 INFO:tasks.workunit.client.0.smithi040.stdout:1/532: dread d17/d43/fa2 [0,4194304] 0 2024-02-16T19:24:16.927 INFO:tasks.workunit.client.0.smithi040.stdout:8/656: mkdir d6/d2b/da3/dcb 0 2024-02-16T19:24:16.927 INFO:tasks.workunit.client.0.smithi040.stdout:8/657: write d6/d2b/f95 [958142,112875] 0 2024-02-16T19:24:16.927 INFO:tasks.workunit.client.0.smithi040.stdout:8/658: fdatasync d6/d64/d50/f74 0 2024-02-16T19:24:16.929 INFO:tasks.workunit.client.0.smithi040.stdout:2/523: link d5/d7/c26 d5/d7/d1e/d31/d4f/cab 0 2024-02-16T19:24:16.951 INFO:tasks.workunit.client.0.smithi040.stdout:1/533: creat d17/d26/d5f/d63/fa7 x:0 0 0 2024-02-16T19:24:16.951 INFO:tasks.workunit.client.0.smithi040.stdout:1/534: readlink d17/d3c/l50 0 2024-02-16T19:24:16.953 INFO:tasks.workunit.client.0.smithi040.stdout:8/659: mkdir d6/d12/d51/d6e/db7/dc6/dcc 0 2024-02-16T19:24:16.955 INFO:tasks.workunit.client.0.smithi040.stdout:2/524: creat d5/d5c/fac x:0 0 0 2024-02-16T19:24:16.955 INFO:tasks.workunit.client.0.smithi040.stdout:2/525: truncate d5/d8/d42/f7a 707148 0 2024-02-16T19:24:16.955 INFO:tasks.workunit.client.0.smithi040.stdout:2/526: chown d5/d5c/d65/c77 32541 1 2024-02-16T19:24:16.955 INFO:tasks.workunit.client.0.smithi040.stdout:2/527: readlink d5/d7/d12/l5e 0 2024-02-16T19:24:16.956 INFO:tasks.workunit.client.0.smithi040.stdout:0/628: getdents db/d2f/d1d/d21/d2e/d49/d5b 0 2024-02-16T19:24:16.956 INFO:tasks.workunit.client.0.smithi040.stdout:0/629: chown db/d2f/d1d/d21/d2e/d85/f86 0 1 2024-02-16T19:24:16.959 INFO:tasks.workunit.client.0.smithi040.stdout:3/627: sync 2024-02-16T19:24:16.959 INFO:tasks.workunit.client.0.smithi040.stdout:6/659: sync 2024-02-16T19:24:16.961 INFO:tasks.workunit.client.0.smithi040.stdout:4/481: link dd/d58/l65 dd/de/d45/l9c 0 2024-02-16T19:24:16.961 INFO:tasks.workunit.client.0.smithi040.stdout:4/482: truncate dd/de/d10/d36/f63 3172192 0 2024-02-16T19:24:16.962 INFO:tasks.workunit.client.0.smithi040.stdout:2/528: dread d5/f3c [0,4194304] 0 2024-02-16T19:24:16.963 INFO:tasks.workunit.client.0.smithi040.stdout:1/535: creat d17/d26/d5f/d78/fa8 x:0 0 0 2024-02-16T19:24:16.965 INFO:tasks.workunit.client.0.smithi040.stdout:8/660: mknod d6/d12/d51/d6e/db9/ccd 0 2024-02-16T19:24:16.968 INFO:tasks.workunit.client.0.smithi040.stdout:0/630: rename db/d25/d2c/cbf to db/de/dc0/cd1 0 2024-02-16T19:24:16.968 INFO:tasks.workunit.client.0.smithi040.stdout:0/631: truncate db/d2f/d1d/d21/d2e/d49/d5b/f9d 95949 0 2024-02-16T19:24:16.968 INFO:tasks.workunit.client.0.smithi040.stdout:0/632: write db/de/d6a/f6e [4670569,32680] 0 2024-02-16T19:24:16.972 INFO:tasks.workunit.client.0.smithi040.stdout:3/628: rmdir d1/d4/d9b 0 2024-02-16T19:24:16.972 INFO:tasks.workunit.client.0.smithi040.stdout:3/629: fsync d1/dc/d35/fbf 0 2024-02-16T19:24:16.972 INFO:tasks.workunit.client.0.smithi040.stdout:1/536: dread d17/d43/f5d [0,4194304] 0 2024-02-16T19:24:16.972 INFO:tasks.workunit.client.0.smithi040.stdout:1/537: write d17/d43/d92/f6e [140558,43615] 0 2024-02-16T19:24:16.974 INFO:tasks.workunit.client.0.smithi040.stdout:4/483: truncate dd/de/d10/d13/f15 2532669 0 2024-02-16T19:24:16.976 INFO:tasks.workunit.client.0.smithi040.stdout:0/633: write db/d2f/d1d/d21/d2e/d49/fa9 [1388153,40412] 0 2024-02-16T19:24:16.976 INFO:tasks.workunit.client.0.smithi040.stdout:6/660: dwrite d0/d1/d12/d3b/d79/fa2 [0,4194304] 0 2024-02-16T19:24:16.979 INFO:tasks.workunit.client.0.smithi040.stdout:8/661: mkdir d6/d12/d51/d6e/db7/dc6/dce 0 2024-02-16T19:24:16.981 INFO:tasks.workunit.client.0.smithi040.stdout:8/662: dread d6/f24 [0,4194304] 0 2024-02-16T19:24:16.983 INFO:tasks.workunit.client.0.smithi040.stdout:3/630: mkdir d1/d4/d6e/dc2 0 2024-02-16T19:24:16.985 INFO:tasks.workunit.client.0.smithi040.stdout:1/538: mkdir d17/d3c/d74/d97/da9 0 2024-02-16T19:24:16.985 INFO:tasks.workunit.client.0.smithi040.stdout:1/539: stat d17/d3c/f80 0 2024-02-16T19:24:16.986 INFO:tasks.workunit.client.0.smithi040.stdout:9/638: sync 2024-02-16T19:24:16.986 INFO:tasks.workunit.client.0.smithi040.stdout:5/572: sync 2024-02-16T19:24:16.991 INFO:tasks.workunit.client.0.smithi040.stdout:0/634: symlink db/d2f/d1d/d21/d2e/ld2 0 2024-02-16T19:24:17.000 INFO:tasks.workunit.client.0.smithi040.stdout:6/661: creat d0/d1/d6f/d90/fd8 x:0 0 0 2024-02-16T19:24:17.004 INFO:tasks.workunit.client.0.smithi040.stdout:1/540: mkdir d17/d43/d92/daa 0 2024-02-16T19:24:17.006 INFO:tasks.workunit.client.0.smithi040.stdout:3/631: dwrite d1/dc/fbc [0,4194304] 0 2024-02-16T19:24:17.006 INFO:tasks.workunit.client.0.smithi040.stdout:5/573: dwrite d1/da/f2e [0,4194304] 0 2024-02-16T19:24:17.006 INFO:tasks.workunit.client.0.smithi040.stdout:5/574: write d1/da/d51/f6c [4453695,10361] 0 2024-02-16T19:24:17.007 INFO:tasks.workunit.client.0.smithi040.stdout:9/639: symlink d3/d4/dbe/lcd 0 2024-02-16T19:24:17.010 INFO:tasks.workunit.client.0.smithi040.stdout:0/635: creat db/d2f/d1d/fd3 x:0 0 0 2024-02-16T19:24:17.010 INFO:tasks.workunit.client.0.smithi040.stdout:0/636: readlink db/d2f/l3f 0 2024-02-16T19:24:17.011 INFO:tasks.workunit.client.0.smithi040.stdout:9/640: dread d3/f21 [0,4194304] 0 2024-02-16T19:24:17.011 INFO:tasks.workunit.client.0.smithi040.stdout:9/641: fdatasync d3/d1c/fb3 0 2024-02-16T19:24:17.016 INFO:tasks.workunit.client.0.smithi040.stdout:1/541: mkdir d17/d26/d4c/dab 0 2024-02-16T19:24:17.018 INFO:tasks.workunit.client.0.smithi040.stdout:8/663: rename d6/d64/d50/d69/d81 to d6/d12/d51/d6e/db9/dcf 0 2024-02-16T19:24:17.018 INFO:tasks.workunit.client.0.smithi040.stdout:8/664: write d6/d7/f27 [6086996,83198] 0 2024-02-16T19:24:17.021 INFO:tasks.workunit.client.0.smithi040.stdout:9/642: read d3/d4/d11/d71/da8/f58 [2759866,92047] 0 2024-02-16T19:24:17.021 INFO:tasks.workunit.client.0.smithi040.stdout:9/643: fsync d3/d4/d11/d71/da8/f84 0 2024-02-16T19:24:17.024 INFO:tasks.workunit.client.0.smithi040.stdout:3/632: creat d1/d8c/fc3 x:0 0 0 2024-02-16T19:24:17.024 INFO:tasks.workunit.client.0.smithi040.stdout:3/633: write d1/d4/f25 [5471591,112185] 0 2024-02-16T19:24:17.028 INFO:tasks.workunit.client.0.smithi040.stdout:0/637: symlink db/d25/ld4 0 2024-02-16T19:24:17.030 INFO:tasks.workunit.client.0.smithi040.stdout:5/575: dread d1/da/f35 [0,4194304] 0 2024-02-16T19:24:17.033 INFO:tasks.workunit.client.0.smithi040.stdout:6/662: getdents d0/d30/d4b 0 2024-02-16T19:24:17.033 INFO:tasks.workunit.client.0.smithi040.stdout:6/663: chown d0/d1/d12/d3b/d48/d4f/d9d/dbd/cbf 142562643 1 2024-02-16T19:24:17.033 INFO:tasks.workunit.client.0.smithi040.stdout:6/664: chown d0/d1/d6f/d90/f9b 1373178900 1 2024-02-16T19:24:17.033 INFO:tasks.workunit.client.0.smithi040.stdout:6/665: dread - d0/d1/d6f/d90/fd8 zero size 2024-02-16T19:24:17.034 INFO:tasks.workunit.client.0.smithi040.stdout:9/644: read d3/d4/d11/d29/d38/d46/d6b/f6f [1803918,16815] 0 2024-02-16T19:24:17.034 INFO:tasks.workunit.client.0.smithi040.stdout:9/645: fsync d3/d4/d11/d29/f48 0 2024-02-16T19:24:17.036 INFO:tasks.workunit.client.0.smithi040.stdout:8/665: unlink d6/d12/d51/d6e/db9/ccd 0 2024-02-16T19:24:17.037 INFO:tasks.workunit.client.0.smithi040.stdout:8/666: chown d6/d12/d22/l9e 1300396 1 2024-02-16T19:24:17.037 INFO:tasks.workunit.client.0.smithi040.stdout:8/667: chown d6/d2b/da3/l5e 553556 1 2024-02-16T19:24:17.037 INFO:tasks.workunit.client.0.smithi040.stdout:2/529: dwrite d5/f79 [0,4194304] 0 2024-02-16T19:24:17.037 INFO:tasks.workunit.client.0.smithi040.stdout:1/542: dwrite d17/d2d/f81 [0,4194304] 0 2024-02-16T19:24:17.041 INFO:tasks.workunit.client.0.smithi040.stdout:3/634: mkdir d1/d4/dc4 0 2024-02-16T19:24:17.045 INFO:tasks.workunit.client.0.smithi040.stdout:0/638: mkdir db/d2f/d23/d3b/dd5 0 2024-02-16T19:24:17.045 INFO:tasks.workunit.client.0.smithi040.stdout:0/639: write db/d2f/f77 [448867,62836] 0 2024-02-16T19:24:17.047 INFO:tasks.workunit.client.0.smithi040.stdout:6/666: creat d0/d30/d5b/dbe/fd9 x:0 0 0 2024-02-16T19:24:17.048 INFO:tasks.workunit.client.0.smithi040.stdout:9/646: truncate d3/d4/d55/d6d/f7d 508801 0 2024-02-16T19:24:17.054 INFO:tasks.workunit.client.0.smithi040.stdout:5/576: dwrite d1/f96 [0,4194304] 0 2024-02-16T19:24:17.054 INFO:tasks.workunit.client.0.smithi040.stdout:8/668: dwrite d6/d7/d1d/f52 [0,4194304] 0 2024-02-16T19:24:17.055 INFO:tasks.workunit.client.0.smithi040.stdout:2/530: unlink d5/d7/d1e/d4c/ca7 0 2024-02-16T19:24:17.061 INFO:tasks.workunit.client.0.smithi040.stdout:1/543: mkdir d17/d26/d5f/d78/dac 0 2024-02-16T19:24:17.063 INFO:tasks.workunit.client.0.smithi040.stdout:3/635: truncate d1/d7/d33/f5e 727576 0 2024-02-16T19:24:17.063 INFO:tasks.workunit.client.0.smithi040.stdout:3/636: stat d1/dc/d35/d48/d85/f9e 0 2024-02-16T19:24:17.065 INFO:tasks.workunit.client.0.smithi040.stdout:0/640: mknod db/d25/cd6 0 2024-02-16T19:24:17.066 INFO:tasks.workunit.client.0.smithi040.stdout:1/544: dread d17/d26/d5f/d63/f75 [0,4194304] 0 2024-02-16T19:24:17.066 INFO:tasks.workunit.client.0.smithi040.stdout:9/647: dwrite d3/d4/d4a/f8c [0,4194304] 0 2024-02-16T19:24:17.066 INFO:tasks.workunit.client.0.smithi040.stdout:9/648: creat d3/d4/d11/d71/da2/fce x:0 0 0 2024-02-16T19:24:17.067 INFO:tasks.workunit.client.0.smithi040.stdout:6/667: creat d0/d1/d6f/fda x:0 0 0 2024-02-16T19:24:17.076 INFO:tasks.workunit.client.0.smithi040.stdout:8/669: mknod d6/d12/d51/cd0 0 2024-02-16T19:24:17.078 INFO:tasks.workunit.client.0.smithi040.stdout:8/670: write d6/d2b/da3/f53 [622350,99766] 0 2024-02-16T19:24:17.083 INFO:tasks.workunit.client.0.smithi040.stdout:5/577: dwrite d1/fd [0,4194304] 0 2024-02-16T19:24:17.083 INFO:tasks.workunit.client.0.smithi040.stdout:5/578: chown d1/da/d1c/d21/l50 6 1 2024-02-16T19:24:17.083 INFO:tasks.workunit.client.0.smithi040.stdout:5/579: fdatasync d1/d45/f52 0 2024-02-16T19:24:17.083 INFO:tasks.workunit.client.0.smithi040.stdout:5/580: write d1/da/d1c/f26 [1910275,76840] 0 2024-02-16T19:24:17.083 INFO:tasks.workunit.client.0.smithi040.stdout:5/581: fsync d1/da/d1c/d21/f8f 0 2024-02-16T19:24:17.084 INFO:tasks.workunit.client.0.smithi040.stdout:5/582: chown d1/da/d1c/da7 15338 1 2024-02-16T19:24:17.084 INFO:tasks.workunit.client.0.smithi040.stdout:5/583: chown d1/d3c/d97/db1/c69 65 1 2024-02-16T19:24:17.084 INFO:tasks.workunit.client.0.smithi040.stdout:5/584: dread - d1/da/d7e/f86 zero size 2024-02-16T19:24:17.084 INFO:tasks.workunit.client.0.smithi040.stdout:1/545: dwrite d17/d3c/f80 [0,4194304] 0 2024-02-16T19:24:17.092 INFO:tasks.workunit.client.0.smithi040.stdout:5/585: write d1/da/f2e [3171157,83956] 0 2024-02-16T19:24:17.092 INFO:tasks.workunit.client.0.smithi040.stdout:2/531: link d5/d7/d1e/d31/c3e d5/d7/d89/cad 0 2024-02-16T19:24:17.095 INFO:tasks.workunit.client.0.smithi040.stdout:2/532: dread d5/d7/d12/f76 [0,4194304] 0 2024-02-16T19:24:17.095 INFO:tasks.workunit.client.0.smithi040.stdout:2/533: write d5/d7/d1e/f33 [4892433,107469] 0 2024-02-16T19:24:17.099 INFO:tasks.workunit.client.0.smithi040.stdout:3/637: rename d1/d4/l1a to d1/d4/d6e/d8a/lc5 0 2024-02-16T19:24:17.100 INFO:tasks.workunit.client.0.smithi040.stdout:1/546: dwrite d17/d29/d8b/f8d [0,4194304] 0 2024-02-16T19:24:17.103 INFO:tasks.workunit.client.0.smithi040.stdout:0/641: creat db/d2f/d1d/d21/d2e/d49/d5b/fd7 x:0 0 0 2024-02-16T19:24:17.105 INFO:tasks.workunit.client.0.smithi040.stdout:0/642: read db/d2f/f3e [4464957,118069] 0 2024-02-16T19:24:17.111 INFO:tasks.workunit.client.0.smithi040.stdout:9/649: creat d3/d4/d55/d8a/fcf x:0 0 0 2024-02-16T19:24:17.112 INFO:tasks.workunit.client.0.smithi040.stdout:6/668: symlink d0/d1/d12/d3b/ldb 0 2024-02-16T19:24:17.115 INFO:tasks.workunit.client.0.smithi040.stdout:2/534: dwrite d5/d7/d1e/f25 [0,4194304] 0 2024-02-16T19:24:17.118 INFO:tasks.workunit.client.0.smithi040.stdout:8/671: rmdir d6/d12/d51 39 2024-02-16T19:24:17.118 INFO:tasks.workunit.client.0.smithi040.stdout:8/672: write d6/d64/fc4 [4708620,36952] 0 2024-02-16T19:24:17.118 INFO:tasks.workunit.client.0.smithi040.stdout:8/673: write d6/f75 [3276756,46749] 0 2024-02-16T19:24:17.120 INFO:tasks.workunit.client.0.smithi040.stdout:5/586: creat d1/d62/fb6 x:0 0 0 2024-02-16T19:24:17.120 INFO:tasks.workunit.client.0.smithi040.stdout:5/587: chown d1/d3c/l95 115 1 2024-02-16T19:24:17.123 INFO:tasks.workunit.client.0.smithi040.stdout:1/547: dwrite d17/d26/d3d/f56 [0,4194304] 0 2024-02-16T19:24:17.123 INFO:tasks.workunit.client.0.smithi040.stdout:1/548: chown d17/d43 13235070 1 2024-02-16T19:24:17.125 INFO:tasks.workunit.client.0.smithi040.stdout:3/638: link d1/dc/d35/d48/d85/f9e d1/dc/fc6 0 2024-02-16T19:24:17.125 INFO:tasks.workunit.client.0.smithi040.stdout:3/639: read - d1/d8c/dae/fc1 zero size 2024-02-16T19:24:17.125 INFO:tasks.workunit.client.0.smithi040.stdout:3/640: write d1/d7/d30/f40 [1129678,16079] 0 2024-02-16T19:24:17.131 INFO:tasks.workunit.client.0.smithi040.stdout:0/643: unlink db/d2f/d23/d93/laf 0 2024-02-16T19:24:17.131 INFO:tasks.workunit.client.0.smithi040.stdout:0/644: write db/d2f/d1d/d21/d2e/d49/d5b/f5d [4266671,12675] 0 2024-02-16T19:24:17.131 INFO:tasks.workunit.client.0.smithi040.stdout:0/645: chown db/de/dbe 5188 1 2024-02-16T19:24:17.132 INFO:tasks.workunit.client.0.smithi040.stdout:9/650: symlink d3/d4/d11/ld0 0 2024-02-16T19:24:17.134 INFO:tasks.workunit.client.0.smithi040.stdout:6/669: mknod d0/d37/cdc 0 2024-02-16T19:24:17.135 INFO:tasks.workunit.client.0.smithi040.stdout:6/670: creat d0/d1/d6f/fdd x:0 0 0 2024-02-16T19:24:17.135 INFO:tasks.workunit.client.0.smithi040.stdout:6/671: read - d0/d30/d5b/dbe/fd9 zero size 2024-02-16T19:24:17.141 INFO:tasks.workunit.client.0.smithi040.stdout:9/651: dread d3/d4/d11/d29/d38/d46/f6a [0,4194304] 0 2024-02-16T19:24:17.142 INFO:tasks.workunit.client.0.smithi040.stdout:9/652: write d3/d4/d55/d31/f3b [4236401,17166] 0 2024-02-16T19:24:17.142 INFO:tasks.workunit.client.0.smithi040.stdout:9/653: fsync d3/d4/d11/d29/d38/d86/faa 0 2024-02-16T19:24:17.146 INFO:tasks.workunit.client.0.smithi040.stdout:0/646: dwrite db/d2f/d23/f9a [4194304,4194304] 0 2024-02-16T19:24:17.146 INFO:tasks.workunit.client.0.smithi040.stdout:2/535: mkdir d5/d7/d12/d1b/dae 0 2024-02-16T19:24:17.152 INFO:tasks.workunit.client.0.smithi040.stdout:8/674: symlink d6/d2b/da3/dcb/ld1 0 2024-02-16T19:24:17.154 INFO:tasks.workunit.client.0.smithi040.stdout:5/588: symlink d1/da/db0/lb7 0 2024-02-16T19:24:17.155 INFO:tasks.workunit.client.0.smithi040.stdout:6/672: dwrite d0/f2e [0,4194304] 0 2024-02-16T19:24:17.155 INFO:tasks.workunit.client.0.smithi040.stdout:6/673: truncate d0/d1/d12/d3b/d48/d4f/d9d/fab 1155241 0 2024-02-16T19:24:17.160 INFO:tasks.workunit.client.0.smithi040.stdout:1/549: link d17/d26/d4c/f5b d17/d2d/fad 0 2024-02-16T19:24:17.162 INFO:tasks.workunit.client.0.smithi040.stdout:0/647: dread db/d2f/d1d/d21/d2e/d49/d5b/fb3 [0,4194304] 0 2024-02-16T19:24:17.163 INFO:tasks.workunit.client.0.smithi040.stdout:3/641: unlink d1/lb6 0 2024-02-16T19:24:17.165 INFO:tasks.workunit.client.0.smithi040.stdout:3/642: dread d1/dc/d42/d58/f93 [0,4194304] 0 2024-02-16T19:24:17.166 INFO:tasks.workunit.client.0.smithi040.stdout:9/654: creat d3/d4/d55/d6d/daf/fd1 x:0 0 0 2024-02-16T19:24:17.169 INFO:tasks.workunit.client.0.smithi040.stdout:3/643: read d1/f37 [131382,80569] 0 2024-02-16T19:24:17.170 INFO:tasks.workunit.client.0.smithi040.stdout:2/536: rename d5/d7/d1e/d55/l7b to d5/d8/laf 0 2024-02-16T19:24:17.172 INFO:tasks.workunit.client.0.smithi040.stdout:8/675: link d6/d2b/da3/dcb/ld1 d6/d2b/ld2 0 2024-02-16T19:24:17.173 INFO:tasks.workunit.client.0.smithi040.stdout:7/479: sync 2024-02-16T19:24:17.173 INFO:tasks.workunit.client.0.smithi040.stdout:4/484: sync 2024-02-16T19:24:17.173 INFO:tasks.workunit.client.0.smithi040.stdout:4/485: fdatasync dd/f43 0 2024-02-16T19:24:17.174 INFO:tasks.workunit.client.0.smithi040.stdout:4/486: write dd/de/d71/f84 [322260,34842] 0 2024-02-16T19:24:17.174 INFO:tasks.workunit.client.0.smithi040.stdout:5/589: symlink d1/da/da4/lb8 0 2024-02-16T19:24:17.182 INFO:tasks.workunit.client.0.smithi040.stdout:0/648: creat db/de/dc0/fd8 x:0 0 0 2024-02-16T19:24:17.184 INFO:tasks.workunit.client.0.smithi040.stdout:9/655: creat d3/d4/d11/d71/da8/fd2 x:0 0 0 2024-02-16T19:24:17.185 INFO:tasks.workunit.client.0.smithi040.stdout:4/487: dread dd/de/f6b [0,4194304] 0 2024-02-16T19:24:17.185 INFO:tasks.workunit.client.0.smithi040.stdout:1/550: dwrite d17/f36 [0,4194304] 0 2024-02-16T19:24:17.187 INFO:tasks.workunit.client.0.smithi040.stdout:3/644: creat d1/dc/d35/d48/fc7 x:0 0 0 2024-02-16T19:24:17.188 INFO:tasks.workunit.client.0.smithi040.stdout:7/480: dwrite d21/d4d/f6c [0,4194304] 0 2024-02-16T19:24:17.190 INFO:tasks.workunit.client.0.smithi040.stdout:7/481: dread d21/d4d/f61 [0,4194304] 0 2024-02-16T19:24:17.190 INFO:tasks.workunit.client.0.smithi040.stdout:7/482: write d21/d29/f59 [527415,52037] 0 2024-02-16T19:24:17.190 INFO:tasks.workunit.client.0.smithi040.stdout:7/483: write d21/d7e/d81/f98 [310375,126711] 0 2024-02-16T19:24:17.190 INFO:tasks.workunit.client.0.smithi040.stdout:7/484: write d21/d4d/d7b/f94 [1000587,73999] 0 2024-02-16T19:24:17.190 INFO:tasks.workunit.client.0.smithi040.stdout:7/485: chown d21/d4d/f56 162935957 1 2024-02-16T19:24:17.191 INFO:tasks.workunit.client.0.smithi040.stdout:2/537: mknod d5/d7/d1e/d4c/cb0 0 2024-02-16T19:24:17.193 INFO:tasks.workunit.client.0.smithi040.stdout:8/676: symlink d6/d12/d51/d6e/db7/dc6/ld3 0 2024-02-16T19:24:17.193 INFO:tasks.workunit.client.0.smithi040.stdout:8/677: chown d6/d12/d51/db6 7998 1 2024-02-16T19:24:17.196 INFO:tasks.workunit.client.0.smithi040.stdout:5/590: symlink d1/da/db0/lb9 0 2024-02-16T19:24:17.197 INFO:tasks.workunit.client.0.smithi040.stdout:8/678: dread d6/f43 [0,4194304] 0 2024-02-16T19:24:17.199 INFO:tasks.workunit.client.0.smithi040.stdout:0/649: creat db/d25/d34/d5e/fd9 x:0 0 0 2024-02-16T19:24:17.201 INFO:tasks.workunit.client.0.smithi040.stdout:9/656: mkdir d3/dd3 0 2024-02-16T19:24:17.201 INFO:tasks.workunit.client.0.smithi040.stdout:9/657: dread - d3/d4/d55/fc7 zero size 2024-02-16T19:24:17.201 INFO:tasks.workunit.client.0.smithi040.stdout:9/658: creat d3/d4/d55/d63/fd4 x:0 0 0 2024-02-16T19:24:17.204 INFO:tasks.workunit.client.0.smithi040.stdout:2/538: write d5/d7/f74 [2926308,36000] 0 2024-02-16T19:24:17.204 INFO:tasks.workunit.client.0.smithi040.stdout:2/539: fdatasync d5/d7/f24 0 2024-02-16T19:24:17.204 INFO:tasks.workunit.client.0.smithi040.stdout:4/488: dwrite dd/de/d10/f4a [0,4194304] 0 2024-02-16T19:24:17.207 INFO:tasks.workunit.client.0.smithi040.stdout:0/650: write db/d25/f61 [1790515,91257] 0 2024-02-16T19:24:17.208 INFO:tasks.workunit.client.0.smithi040.stdout:0/651: truncate db/d2f/d23/d6f/fc4 653883 0 2024-02-16T19:24:17.208 INFO:tasks.workunit.client.0.smithi040.stdout:3/645: truncate d1/dc/fd 2017097 0 2024-02-16T19:24:17.209 INFO:tasks.workunit.client.0.smithi040.stdout:2/540: dread d5/f3a [0,4194304] 0 2024-02-16T19:24:17.213 INFO:tasks.workunit.client.0.smithi040.stdout:1/551: dwrite d17/d43/d92/f79 [0,4194304] 0 2024-02-16T19:24:17.215 INFO:tasks.workunit.client.0.smithi040.stdout:8/679: creat d6/d12/d51/d6e/db7/dc6/fd4 x:0 0 0 2024-02-16T19:24:17.215 INFO:tasks.workunit.client.0.smithi040.stdout:8/680: write d6/d7/d15/f18 [5898005,36017] 0 2024-02-16T19:24:17.216 INFO:tasks.workunit.client.0.smithi040.stdout:8/681: write d6/d12/d22/f4e [453890,110482] 0 2024-02-16T19:24:17.217 INFO:tasks.workunit.client.0.smithi040.stdout:8/682: chown d6/l20 54 1 2024-02-16T19:24:17.220 INFO:tasks.workunit.client.0.smithi040.stdout:2/541: dread d5/d7/d1e/d55/f94 [0,4194304] 0 2024-02-16T19:24:17.220 INFO:tasks.workunit.client.0.smithi040.stdout:2/542: dread - d5/d5c/d65/f75 zero size 2024-02-16T19:24:17.220 INFO:tasks.workunit.client.0.smithi040.stdout:9/659: rmdir d3/d4/d11/d71/dc9 39 2024-02-16T19:24:17.220 INFO:tasks.workunit.client.0.smithi040.stdout:9/660: fsync d3/d4/d55/d31/f9b 0 2024-02-16T19:24:17.220 INFO:tasks.workunit.client.0.smithi040.stdout:1/552: dread d17/d26/d5f/d63/f90 [0,4194304] 0 2024-02-16T19:24:17.220 INFO:tasks.workunit.client.0.smithi040.stdout:1/553: write d17/d3c/f9d [693957,18836] 0 2024-02-16T19:24:17.221 INFO:tasks.workunit.client.0.smithi040.stdout:4/489: dwrite dd/de/d10/d13/d77/f96 [0,4194304] 0 2024-02-16T19:24:17.221 INFO:tasks.workunit.client.0.smithi040.stdout:4/490: fdatasync dd/de/d10/d13/d32/f95 0 2024-02-16T19:24:17.228 INFO:tasks.workunit.client.0.smithi040.stdout:0/652: mknod db/de/dbe/cda 0 2024-02-16T19:24:17.229 INFO:tasks.workunit.client.0.smithi040.stdout:2/543: dread d5/d7/d1e/f25 [0,4194304] 0 2024-02-16T19:24:17.234 INFO:tasks.workunit.client.0.smithi040.stdout:1/554: dwrite d17/d2d/f2e [4194304,4194304] 0 2024-02-16T19:24:17.238 INFO:tasks.workunit.client.0.smithi040.stdout:8/683: mkdir d6/d64/d50/dd5 0 2024-02-16T19:24:17.238 INFO:tasks.workunit.client.0.smithi040.stdout:8/684: write d6/d2b/d44/f8d [1009758,129590] 0 2024-02-16T19:24:17.240 INFO:tasks.workunit.client.0.smithi040.stdout:9/661: mknod d3/cd5 0 2024-02-16T19:24:17.244 INFO:tasks.workunit.client.0.smithi040.stdout:0/653: dwrite db/d2f/d1d/d21/d2e/d83/fa8 [0,4194304] 0 2024-02-16T19:24:17.244 INFO:tasks.workunit.client.0.smithi040.stdout:0/654: creat db/d25/d34/d5e/fdb x:0 0 0 2024-02-16T19:24:17.247 INFO:tasks.workunit.client.0.smithi040.stdout:4/491: mkdir dd/d58/d8c/d99/d9d 0 2024-02-16T19:24:17.252 INFO:tasks.workunit.client.0.smithi040.stdout:2/544: creat d5/d7/d12/fb1 x:0 0 0 2024-02-16T19:24:17.255 INFO:tasks.workunit.client.0.smithi040.stdout:1/555: mknod d17/d26/d5f/d63/cae 0 2024-02-16T19:24:17.255 INFO:tasks.workunit.client.0.smithi040.stdout:1/556: write f5 [5074130,102848] 0 2024-02-16T19:24:17.255 INFO:tasks.workunit.client.0.smithi040.stdout:1/557: fdatasync d17/d26/d5f/d63/fa7 0 2024-02-16T19:24:17.256 INFO:tasks.workunit.client.0.smithi040.stdout:0/655: dread db/d25/f4f [0,4194304] 0 2024-02-16T19:24:17.257 INFO:tasks.workunit.client.0.smithi040.stdout:7/486: rename d21/d7e/d81 to d21/d4d/d77/d87/d9b 0 2024-02-16T19:24:17.260 INFO:tasks.workunit.client.0.smithi040.stdout:8/685: unlink d6/d64/c88 0 2024-02-16T19:24:17.260 INFO:tasks.workunit.client.0.smithi040.stdout:8/686: chown d6/d12/d51/fa1 0 1 2024-02-16T19:24:17.263 INFO:tasks.workunit.client.0.smithi040.stdout:9/662: mknod d3/d4/dbe/cd6 0 2024-02-16T19:24:17.264 INFO:tasks.workunit.client.0.smithi040.stdout:0/656: dread db/d2f/d1d/d21/d2e/d49/d5b/f79 [0,4194304] 0 2024-02-16T19:24:17.264 INFO:tasks.workunit.client.0.smithi040.stdout:1/558: write d17/d3c/f80 [3807058,71816] 0 2024-02-16T19:24:17.268 INFO:tasks.workunit.client.0.smithi040.stdout:7/487: read d21/f28 [2594650,34950] 0 2024-02-16T19:24:17.272 INFO:tasks.workunit.client.0.smithi040.stdout:9/663: write d3/d4/d11/f60 [773430,106782] 0 2024-02-16T19:24:17.280 INFO:tasks.workunit.client.0.smithi040.stdout:2/545: link d5/d7/d1e/d55/d8d/l88 d5/d7/d1e/d55/da3/lb2 0 2024-02-16T19:24:17.280 INFO:tasks.workunit.client.0.smithi040.stdout:2/546: chown d5/d7/d1e/d31/d4f/d63/f2c 9 1 2024-02-16T19:24:17.280 INFO:tasks.workunit.client.0.smithi040.stdout:2/547: chown d5/f79 4 1 2024-02-16T19:24:17.293 INFO:tasks.workunit.client.0.smithi040.stdout:8/687: mknod d6/d2b/d44/cd6 0 2024-02-16T19:24:17.293 INFO:tasks.workunit.client.0.smithi040.stdout:8/688: write d6/d64/d50/d69/fa4 [261629,16395] 0 2024-02-16T19:24:17.294 INFO:tasks.workunit.client.0.smithi040.stdout:0/657: mkdir db/d25/ddc 0 2024-02-16T19:24:17.297 INFO:tasks.workunit.client.0.smithi040.stdout:7/488: mkdir d21/d8b/d9c 0 2024-02-16T19:24:17.301 INFO:tasks.workunit.client.0.smithi040.stdout:9/664: getdents d3/d4/d55/d31 0 2024-02-16T19:24:17.304 INFO:tasks.workunit.client.0.smithi040.stdout:2/548: unlink d5/d7/d1e/d31/c5b 0 2024-02-16T19:24:17.306 INFO:tasks.workunit.client.0.smithi040.stdout:5/591: rename d1/d3c/d97 to d1/da/d1c/dba 0 2024-02-16T19:24:17.309 INFO:tasks.workunit.client.0.smithi040.stdout:8/689: creat d6/d12/fd7 x:0 0 0 2024-02-16T19:24:17.310 INFO:tasks.workunit.client.0.smithi040.stdout:8/690: dread d6/d64/d50/d69/fa4 [0,4194304] 0 2024-02-16T19:24:17.310 INFO:tasks.workunit.client.0.smithi040.stdout:0/658: mkdir db/d2f/d23/d6f/d98/ddd 0 2024-02-16T19:24:17.310 INFO:tasks.workunit.client.0.smithi040.stdout:0/659: chown db/d2f/d1d/d21/d2e/d49/d5b/f79 12 1 2024-02-16T19:24:17.310 INFO:tasks.workunit.client.0.smithi040.stdout:8/691: write d6/d12/d22/f4e [976750,111631] 0 2024-02-16T19:24:17.312 INFO:tasks.workunit.client.0.smithi040.stdout:1/559: symlink d17/d26/d3d/laf 0 2024-02-16T19:24:17.313 INFO:tasks.workunit.client.0.smithi040.stdout:1/560: dread d17/f61 [0,4194304] 0 2024-02-16T19:24:17.314 INFO:tasks.workunit.client.0.smithi040.stdout:1/561: chown d17/c65 176209 1 2024-02-16T19:24:17.323 INFO:tasks.workunit.client.0.smithi040.stdout:2/549: symlink d5/d7/d12/lb3 0 2024-02-16T19:24:17.324 INFO:tasks.workunit.client.0.smithi040.stdout:2/550: write d5/d8/d42/f50 [134719,20800] 0 2024-02-16T19:24:17.325 INFO:tasks.workunit.client.0.smithi040.stdout:3/646: rename d1/dc/d35/fa6 to d1/d4/d6e/fc8 0 2024-02-16T19:24:17.328 INFO:tasks.workunit.client.0.smithi040.stdout:1/562: creat d17/d26/d3d/fb0 x:0 0 0 2024-02-16T19:24:17.330 INFO:tasks.workunit.client.0.smithi040.stdout:2/551: rmdir d5/d7/d1e/d4c/d52/d81/d9e 0 2024-02-16T19:24:17.331 INFO:tasks.workunit.client.0.smithi040.stdout:5/592: dwrite d1/da/d1c/f91 [0,4194304] 0 2024-02-16T19:24:17.333 INFO:tasks.workunit.client.0.smithi040.stdout:8/692: dwrite d6/d64/d50/fbe [0,4194304] 0 2024-02-16T19:24:17.334 INFO:tasks.workunit.client.0.smithi040.stdout:3/647: creat d1/d4/d6e/fc9 x:0 0 0 2024-02-16T19:24:17.337 INFO:tasks.workunit.client.0.smithi040.stdout:0/660: rmdir db/d2f/d23/d93 39 2024-02-16T19:24:17.341 INFO:tasks.workunit.client.0.smithi040.stdout:1/563: link d17/d29/d8b/d93/fa3 d17/d26/d4c/dab/fb1 0 2024-02-16T19:24:17.346 INFO:tasks.workunit.client.0.smithi040.stdout:5/593: mkdir d1/d62/dbb 0 2024-02-16T19:24:17.351 INFO:tasks.workunit.client.0.smithi040.stdout:3/648: symlink d1/dc/d35/lca 0 2024-02-16T19:24:17.351 INFO:tasks.workunit.client.0.smithi040.stdout:3/649: stat d1/dc/d42/d4a/f7d 0 2024-02-16T19:24:17.355 INFO:tasks.workunit.client.0.smithi040.stdout:0/661: symlink db/d2f/d23/lde 0 2024-02-16T19:24:17.360 INFO:tasks.workunit.client.0.smithi040.stdout:9/665: write d3/d4/d11/f19 [326960,45573] 0 2024-02-16T19:24:17.361 INFO:tasks.workunit.client.0.smithi040.stdout:9/666: write d3/d4/d55/d31/f5e [4287448,59788] 0 2024-02-16T19:24:17.361 INFO:tasks.workunit.client.0.smithi040.stdout:9/667: creat d3/d4/d11/d71/da2/fd7 x:0 0 0 2024-02-16T19:24:17.361 INFO:tasks.workunit.client.0.smithi040.stdout:1/564: symlink d17/d3c/d74/lb2 0 2024-02-16T19:24:17.362 INFO:tasks.workunit.client.0.smithi040.stdout:4/492: rename dd/de/d10/d13 to dd/de/d10/d36/d9e 0 2024-02-16T19:24:17.363 INFO:tasks.workunit.client.0.smithi040.stdout:0/662: truncate db/d2f/f66 3728382 0 2024-02-16T19:24:17.365 INFO:tasks.workunit.client.0.smithi040.stdout:2/552: dwrite d5/f3c [0,4194304] 0 2024-02-16T19:24:17.366 INFO:tasks.workunit.client.0.smithi040.stdout:8/693: dwrite d6/d12/d51/d6e/fa2 [0,4194304] 0 2024-02-16T19:24:17.367 INFO:tasks.workunit.client.0.smithi040.stdout:5/594: dwrite d1/d3c/f77 [4194304,4194304] 0 2024-02-16T19:24:17.368 INFO:tasks.workunit.client.0.smithi040.stdout:1/565: symlink d17/d3c/d74/d97/lb3 0 2024-02-16T19:24:17.368 INFO:tasks.workunit.client.0.smithi040.stdout:1/566: write d17/d26/d54/f58 [987253,62309] 0 2024-02-16T19:24:17.368 INFO:tasks.workunit.client.0.smithi040.stdout:1/567: chown d17/f44 3052 1 2024-02-16T19:24:17.368 INFO:tasks.workunit.client.0.smithi040.stdout:1/568: write d17/d3c/d71/fa0 [623618,26112] 0 2024-02-16T19:24:17.371 INFO:tasks.workunit.client.0.smithi040.stdout:3/650: rename d1/dc/d35/d53/d64/ca7 to d1/dc/d35/d53/d64/d65/ccb 0 2024-02-16T19:24:17.371 INFO:tasks.workunit.client.0.smithi040.stdout:3/651: dread - d1/dc/d35/d48/d85/f9e zero size 2024-02-16T19:24:17.375 INFO:tasks.workunit.client.0.smithi040.stdout:4/493: link dd/de/d10/d36/d9e/d32/l6a dd/d58/d8c/d99/d9d/l9f 0 2024-02-16T19:24:17.378 INFO:tasks.workunit.client.0.smithi040.stdout:0/663: creat db/d25/fdf x:0 0 0 2024-02-16T19:24:17.378 INFO:tasks.workunit.client.0.smithi040.stdout:0/664: truncate db/d2f/d1d/f46 2082434 0 2024-02-16T19:24:17.380 INFO:tasks.workunit.client.0.smithi040.stdout:5/595: dread d1/da/d1c/f91 [0,4194304] 0 2024-02-16T19:24:17.380 INFO:tasks.workunit.client.0.smithi040.stdout:5/596: write d1/da/da4/fa6 [814318,39475] 0 2024-02-16T19:24:17.380 INFO:tasks.workunit.client.0.smithi040.stdout:5/597: write d1/f13 [3392162,6496] 0 2024-02-16T19:24:17.380 INFO:tasks.workunit.client.0.smithi040.stdout:5/598: fsync d1/d3c/f72 0 2024-02-16T19:24:17.387 INFO:tasks.workunit.client.0.smithi040.stdout:8/694: mkdir d6/d12/d93/dd8 0 2024-02-16T19:24:17.389 INFO:tasks.workunit.client.0.smithi040.stdout:2/553: dwrite d5/d7/d1e/d31/d4f/d63/f8e [4194304,4194304] 0 2024-02-16T19:24:17.395 INFO:tasks.workunit.client.0.smithi040.stdout:3/652: dwrite d1/dc/f3c [0,4194304] 0 2024-02-16T19:24:17.395 INFO:tasks.workunit.client.0.smithi040.stdout:3/653: chown d1/d8c/d8e/cc0 6374 1 2024-02-16T19:24:17.395 INFO:tasks.workunit.client.0.smithi040.stdout:3/654: write d1/dc/fd [1009252,76790] 0 2024-02-16T19:24:17.398 INFO:tasks.workunit.client.0.smithi040.stdout:1/569: symlink d17/d2d/lb4 0 2024-02-16T19:24:17.401 INFO:tasks.workunit.client.0.smithi040.stdout:9/668: rename d3/c9a to d3/d4/d11/d29/d38/d46/d6b/cd8 0 2024-02-16T19:24:17.401 INFO:tasks.workunit.client.0.smithi040.stdout:9/669: write d3/d4/d11/d29/d38/d46/d6b/f6f [2398525,112813] 0 2024-02-16T19:24:17.401 INFO:tasks.workunit.client.0.smithi040.stdout:9/670: fsync d3/d1c/f3f 0 2024-02-16T19:24:17.402 INFO:tasks.workunit.client.0.smithi040.stdout:4/494: dwrite dd/de/d10/d36/f81 [0,4194304] 0 2024-02-16T19:24:17.402 INFO:tasks.workunit.client.0.smithi040.stdout:4/495: fsync dd/f52 0 2024-02-16T19:24:17.403 INFO:tasks.workunit.client.0.smithi040.stdout:2/554: dwrite d5/d7/fa1 [0,4194304] 0 2024-02-16T19:24:17.404 INFO:tasks.workunit.client.0.smithi040.stdout:3/655: dread d1/dc/d35/fbf [0,4194304] 0 2024-02-16T19:24:17.406 INFO:tasks.workunit.client.0.smithi040.stdout:0/665: symlink db/d2f/le0 0 2024-02-16T19:24:17.409 INFO:tasks.workunit.client.0.smithi040.stdout:5/599: mknod d1/da/d1c/d21/cbc 0 2024-02-16T19:24:17.409 INFO:tasks.workunit.client.0.smithi040.stdout:5/600: fsync d1/d3c/f72 0 2024-02-16T19:24:17.411 INFO:tasks.workunit.client.0.smithi040.stdout:4/496: write dd/d58/d5e/f69 [3176910,909] 0 2024-02-16T19:24:17.411 INFO:tasks.workunit.client.0.smithi040.stdout:4/497: write dd/de/d10/d36/d9e/f90 [682015,22679] 0 2024-02-16T19:24:17.411 INFO:tasks.workunit.client.0.smithi040.stdout:4/498: write dd/f19 [3090628,52614] 0 2024-02-16T19:24:17.413 INFO:tasks.workunit.client.0.smithi040.stdout:8/695: link d6/d64/d50/l98 d6/d12/d51/db6/ld9 0 2024-02-16T19:24:17.422 INFO:tasks.workunit.client.0.smithi040.stdout:8/696: write d6/d12/f5a [76043,75827] 0 2024-02-16T19:24:17.422 INFO:tasks.workunit.client.0.smithi040.stdout:8/697: dread - d6/d12/d22/f97 zero size 2024-02-16T19:24:17.424 INFO:tasks.workunit.client.0.smithi040.stdout:1/570: mkdir d17/db5 0 2024-02-16T19:24:17.426 INFO:tasks.workunit.client.0.smithi040.stdout:9/671: symlink d3/d4/d55/ld9 0 2024-02-16T19:24:17.427 INFO:tasks.workunit.client.0.smithi040.stdout:5/601: dwrite d1/d3c/f72 [0,4194304] 0 2024-02-16T19:24:17.438 INFO:tasks.workunit.client.0.smithi040.stdout:5/602: read d1/da/d1c/d21/f64 [3219693,37939] 0 2024-02-16T19:24:17.438 INFO:tasks.workunit.client.0.smithi040.stdout:5/603: getdents d1/da/d1c/da7 0 2024-02-16T19:24:17.438 INFO:tasks.workunit.client.0.smithi040.stdout:5/604: chown d1/d3c/f43 29217689 1 2024-02-16T19:24:17.439 INFO:tasks.workunit.client.0.smithi040.stdout:2/555: mknod d5/d8/d42/cb4 0 2024-02-16T19:24:17.440 INFO:tasks.workunit.client.0.smithi040.stdout:1/571: dwrite d17/d26/d4c/dab/fb1 [0,4194304] 0 2024-02-16T19:24:17.440 INFO:tasks.workunit.client.0.smithi040.stdout:1/572: getdents d17/d3c/d74/d97/da9 0 2024-02-16T19:24:17.442 INFO:tasks.workunit.client.0.smithi040.stdout:3/656: mknod d1/d4/d6e/d8a/da5/ccc 0 2024-02-16T19:24:17.442 INFO:tasks.workunit.client.0.smithi040.stdout:3/657: readlink d1/d7/d30/l31 0 2024-02-16T19:24:17.442 INFO:tasks.workunit.client.0.smithi040.stdout:3/658: creat d1/d4/d6e/fcd x:0 0 0 2024-02-16T19:24:17.442 INFO:tasks.workunit.client.0.smithi040.stdout:3/659: chown d1/d7/d30/l31 105346502 1 2024-02-16T19:24:17.442 INFO:tasks.workunit.client.0.smithi040.stdout:3/660: fsync d1/d7/f4e 0 2024-02-16T19:24:17.449 INFO:tasks.workunit.client.0.smithi040.stdout:9/672: dwrite d3/d4/d55/d31/f5e [0,4194304] 0 2024-02-16T19:24:17.449 INFO:tasks.workunit.client.0.smithi040.stdout:4/499: mknod dd/ca0 0 2024-02-16T19:24:17.449 INFO:tasks.workunit.client.0.smithi040.stdout:4/500: creat dd/d58/fa1 x:0 0 0 2024-02-16T19:24:17.458 INFO:tasks.workunit.client.0.smithi040.stdout:5/605: creat d1/da/d1c/d73/fbd x:0 0 0 2024-02-16T19:24:17.460 INFO:tasks.workunit.client.0.smithi040.stdout:2/556: symlink d5/d5c/lb5 0 2024-02-16T19:24:17.464 INFO:tasks.workunit.client.0.smithi040.stdout:3/661: dwrite d1/d7/d36/f97 [0,4194304] 0 2024-02-16T19:24:17.464 INFO:tasks.workunit.client.0.smithi040.stdout:1/573: creat d17/d26/d3d/fb6 x:0 0 0 2024-02-16T19:24:17.465 INFO:tasks.workunit.client.0.smithi040.stdout:1/574: stat d17/d26/d4c/f8a 0 2024-02-16T19:24:17.469 INFO:tasks.workunit.client.0.smithi040.stdout:4/501: dwrite dd/de/d10/f1b [0,4194304] 0 2024-02-16T19:24:17.470 INFO:tasks.workunit.client.0.smithi040.stdout:4/502: fsync dd/de/d71/f84 0 2024-02-16T19:24:17.470 INFO:tasks.workunit.client.0.smithi040.stdout:4/503: write dd/d58/f8d [166237,116616] 0 2024-02-16T19:24:17.470 INFO:tasks.workunit.client.0.smithi040.stdout:0/666: rmdir db/d2f/d1d 39 2024-02-16T19:24:17.485 INFO:tasks.workunit.client.0.smithi040.stdout:9/673: creat d3/d4/d4a/fda x:0 0 0 2024-02-16T19:24:17.486 INFO:tasks.workunit.client.0.smithi040.stdout:9/674: stat d3/d4/d11/d29/d38/d46/d6b/f90 0 2024-02-16T19:24:17.488 INFO:tasks.workunit.client.0.smithi040.stdout:8/698: getdents d6/d2b/da3/d8b 0 2024-02-16T19:24:17.488 INFO:tasks.workunit.client.0.smithi040.stdout:8/699: chown d6/d64/faa 0 1 2024-02-16T19:24:17.491 INFO:tasks.workunit.client.0.smithi040.stdout:5/606: creat d1/da/d1c/d73/fbe x:0 0 0 2024-02-16T19:24:17.496 INFO:tasks.workunit.client.0.smithi040.stdout:9/675: read d3/d4/d11/d29/d38/d46/d6b/f90 [1151668,62870] 0 2024-02-16T19:24:17.499 INFO:tasks.workunit.client.0.smithi040.stdout:3/662: mkdir d1/d4/d6e/dc2/dce 0 2024-02-16T19:24:17.502 INFO:tasks.workunit.client.0.smithi040.stdout:1/575: mknod d17/d3c/d74/d97/cb7 0 2024-02-16T19:24:17.507 INFO:tasks.workunit.client.0.smithi040.stdout:0/667: creat db/de/dbe/fe1 x:0 0 0 2024-02-16T19:24:17.507 INFO:tasks.workunit.client.0.smithi040.stdout:0/668: dread - db/d2f/d1d/d21/d2e/d85/d89/d94/fb2 zero size 2024-02-16T19:24:17.507 INFO:tasks.workunit.client.0.smithi040.stdout:0/669: chown db/d25/d34/la0 61035997 1 2024-02-16T19:24:17.510 INFO:tasks.workunit.client.0.smithi040.stdout:5/607: creat d1/da/d51/fbf x:0 0 0 2024-02-16T19:24:17.514 INFO:tasks.workunit.client.0.smithi040.stdout:4/504: rmdir dd/de/d10/d36 39 2024-02-16T19:24:17.514 INFO:tasks.workunit.client.0.smithi040.stdout:4/505: fsync dd/de/f54 0 2024-02-16T19:24:17.514 INFO:tasks.workunit.client.0.smithi040.stdout:4/506: chown dd/de/f2a 617 1 2024-02-16T19:24:17.515 INFO:tasks.workunit.client.0.smithi040.stdout:2/557: getdents d5/d7/d1e/d55/d8d 0 2024-02-16T19:24:17.517 INFO:tasks.workunit.client.0.smithi040.stdout:5/608: mknod d1/d45/dab/cc0 0 2024-02-16T19:24:17.519 INFO:tasks.workunit.client.0.smithi040.stdout:9/676: dwrite d3/d4/f57 [0,4194304] 0 2024-02-16T19:24:17.520 INFO:tasks.workunit.client.0.smithi040.stdout:9/677: stat d3/d4/d11/ld0 0 2024-02-16T19:24:17.524 INFO:tasks.workunit.client.0.smithi040.stdout:2/558: link d5/d7/d1e/d4c/d52/f9f d5/d7/d10/fb6 0 2024-02-16T19:24:17.524 INFO:tasks.workunit.client.0.smithi040.stdout:2/559: chown d5/d7/d12/da6 11652 1 2024-02-16T19:24:17.528 INFO:tasks.workunit.client.0.smithi040.stdout:8/700: dwrite d6/d2b/f9c [0,4194304] 0 2024-02-16T19:24:17.529 INFO:tasks.workunit.client.0.smithi040.stdout:8/701: dread - d6/d12/d51/fa1 zero size 2024-02-16T19:24:17.529 INFO:tasks.workunit.client.0.smithi040.stdout:6/674: sync 2024-02-16T19:24:17.530 INFO:tasks.workunit.client.0.smithi040.stdout:7/489: sync 2024-02-16T19:24:17.532 INFO:tasks.workunit.client.0.smithi040.stdout:1/576: dwrite d17/d26/d3d/f56 [0,4194304] 0 2024-02-16T19:24:17.532 INFO:tasks.workunit.client.0.smithi040.stdout:1/577: chown d17/d26/d4c/f8a 14 1 2024-02-16T19:24:17.532 INFO:tasks.workunit.client.0.smithi040.stdout:1/578: chown d17/f36 6 1 2024-02-16T19:24:17.532 INFO:tasks.workunit.client.0.smithi040.stdout:1/579: chown d17/d3c/c46 2337990 1 2024-02-16T19:24:17.532 INFO:tasks.workunit.client.0.smithi040.stdout:1/580: readlink d17/l23 0 2024-02-16T19:24:17.532 INFO:tasks.workunit.client.0.smithi040.stdout:9/678: mknod d3/d4/d55/dc3/cdb 0 2024-02-16T19:24:17.535 INFO:tasks.workunit.client.0.smithi040.stdout:6/675: dread d0/d30/d5b/f91 [0,4194304] 0 2024-02-16T19:24:17.536 INFO:tasks.workunit.client.0.smithi040.stdout:1/581: creat d17/d29/fb8 x:0 0 0 2024-02-16T19:24:17.536 INFO:tasks.workunit.client.0.smithi040.stdout:1/582: readlink d17/d29/l47 0 2024-02-16T19:24:17.536 INFO:tasks.workunit.client.0.smithi040.stdout:1/583: getdents d17/d3c/d74/d97/da9 0 2024-02-16T19:24:17.538 INFO:tasks.workunit.client.0.smithi040.stdout:6/676: truncate d0/d1/d12/d3b/d48/d4f/d9d/fab 217086 0 2024-02-16T19:24:17.541 INFO:tasks.workunit.client.0.smithi040.stdout:1/584: creat d17/d26/fb9 x:0 0 0 2024-02-16T19:24:17.549 INFO:tasks.workunit.client.0.smithi040.stdout:1/585: dread d17/d26/d4c/f8a [0,4194304] 0 2024-02-16T19:24:17.549 INFO:tasks.workunit.client.0.smithi040.stdout:1/586: fdatasync d17/d43/d92/f8c 0 2024-02-16T19:24:17.549 INFO:tasks.workunit.client.0.smithi040.stdout:1/587: fdatasync d17/d43/f5d 0 2024-02-16T19:24:17.552 INFO:tasks.workunit.client.0.smithi040.stdout:7/490: read d21/d29/f2b [402896,109725] 0 2024-02-16T19:24:17.554 INFO:tasks.workunit.client.0.smithi040.stdout:7/491: truncate d21/d4d/d77/d7a/f82 965259 0 2024-02-16T19:24:17.555 INFO:tasks.workunit.client.0.smithi040.stdout:7/492: mknod d21/d4d/d7b/c9d 0 2024-02-16T19:24:17.556 INFO:tasks.workunit.client.0.smithi040.stdout:7/493: fsync d21/d29/d79/f7c 0 2024-02-16T19:24:17.556 INFO:tasks.workunit.client.0.smithi040.stdout:7/494: stat d21/f62 0 2024-02-16T19:24:17.557 INFO:tasks.workunit.client.0.smithi040.stdout:7/495: creat d21/d4d/d77/d87/d9b/f9e x:0 0 0 2024-02-16T19:24:17.559 INFO:tasks.workunit.client.0.smithi040.stdout:7/496: creat d21/f9f x:0 0 0 2024-02-16T19:24:17.567 INFO:tasks.workunit.client.0.smithi040.stdout:7/497: dread f1f [0,4194304] 0 2024-02-16T19:24:17.567 INFO:tasks.workunit.client.0.smithi040.stdout:7/498: fdatasync d21/d29/f5c 0 2024-02-16T19:24:17.571 INFO:tasks.workunit.client.0.smithi040.stdout:3/663: dwrite d1/dc/fd [0,4194304] 0 2024-02-16T19:24:17.571 INFO:tasks.workunit.client.0.smithi040.stdout:5/609: dwrite d1/da/d1c/f25 [0,4194304] 0 2024-02-16T19:24:17.571 INFO:tasks.workunit.client.0.smithi040.stdout:5/610: creat d1/da/da4/fc1 x:0 0 0 2024-02-16T19:24:17.571 INFO:tasks.workunit.client.0.smithi040.stdout:4/507: dwrite dd/de/d10/d36/d9e/d77/f67 [0,4194304] 0 2024-02-16T19:24:17.572 INFO:tasks.workunit.client.0.smithi040.stdout:0/670: dwrite db/d25/d2c/f5a [0,4194304] 0 2024-02-16T19:24:17.572 INFO:tasks.workunit.client.0.smithi040.stdout:2/560: dwrite d5/d7/d1e/d4c/d52/fa2 [0,4194304] 0 2024-02-16T19:24:17.572 INFO:tasks.workunit.client.0.smithi040.stdout:2/561: truncate d5/d5c/d65/f60 4881169 0 2024-02-16T19:24:17.572 INFO:tasks.workunit.client.0.smithi040.stdout:5/611: write d1/da/f2a [216626,53987] 0 2024-02-16T19:24:17.572 INFO:tasks.workunit.client.0.smithi040.stdout:5/612: truncate d1/da/d51/fbf 992176 0 2024-02-16T19:24:17.573 INFO:tasks.workunit.client.0.smithi040.stdout:0/671: read db/de/d6a/f82 [4540019,74366] 0 2024-02-16T19:24:17.573 INFO:tasks.workunit.client.0.smithi040.stdout:9/679: dwrite d3/d1c/fb9 [0,4194304] 0 2024-02-16T19:24:17.574 INFO:tasks.workunit.client.0.smithi040.stdout:7/499: dread d21/d29/f73 [0,4194304] 0 2024-02-16T19:24:17.587 INFO:tasks.workunit.client.0.smithi040.stdout:4/508: truncate dd/de/f6b 3659486 0 2024-02-16T19:24:17.589 INFO:tasks.workunit.client.0.smithi040.stdout:0/672: dwrite db/fd [4194304,4194304] 0 2024-02-16T19:24:17.591 INFO:tasks.workunit.client.0.smithi040.stdout:5/613: creat d1/da/d1c/dba/fc2 x:0 0 0 2024-02-16T19:24:17.591 INFO:tasks.workunit.client.0.smithi040.stdout:5/614: stat d1/da/d51/l7c 0 2024-02-16T19:24:17.591 INFO:tasks.workunit.client.0.smithi040.stdout:5/615: truncate d1/d62/d9a/f9b 425146 0 2024-02-16T19:24:17.594 INFO:tasks.workunit.client.0.smithi040.stdout:4/509: dread dd/f66 [0,4194304] 0 2024-02-16T19:24:17.594 INFO:tasks.workunit.client.0.smithi040.stdout:4/510: chown dd/de/d10/f20 2082 1 2024-02-16T19:24:17.594 INFO:tasks.workunit.client.0.smithi040.stdout:4/511: chown dd/de/d10/f2e 766631326 1 2024-02-16T19:24:17.594 INFO:tasks.workunit.client.0.smithi040.stdout:4/512: creat dd/de/d10/fa2 x:0 0 0 2024-02-16T19:24:17.597 INFO:tasks.workunit.client.0.smithi040.stdout:9/680: creat d3/fdc x:0 0 0 2024-02-16T19:24:17.600 INFO:tasks.workunit.client.0.smithi040.stdout:7/500: mkdir d21/d8b/d9c/da0 0 2024-02-16T19:24:17.604 INFO:tasks.workunit.client.0.smithi040.stdout:3/664: rmdir d1/dc/d35/d48 39 2024-02-16T19:24:17.608 INFO:tasks.workunit.client.0.smithi040.stdout:4/513: dwrite dd/de/d10/f4a [0,4194304] 0 2024-02-16T19:24:17.608 INFO:tasks.workunit.client.0.smithi040.stdout:4/514: chown dd/d58/l87 153673001 1 2024-02-16T19:24:17.608 INFO:tasks.workunit.client.0.smithi040.stdout:4/515: truncate dd/f52 4267671 0 2024-02-16T19:24:17.609 INFO:tasks.workunit.client.0.smithi040.stdout:5/616: truncate d1/da/f35 7102 0 2024-02-16T19:24:17.615 INFO:tasks.workunit.client.0.smithi040.stdout:6/677: rename d0/d1/d12/d3b/d48/d4f/d9d to d0/d1/d12/d3b/d48/dba/dde 0 2024-02-16T19:24:17.620 INFO:tasks.workunit.client.0.smithi040.stdout:4/516: creat dd/de/d71/fa3 x:0 0 0 2024-02-16T19:24:17.620 INFO:tasks.workunit.client.0.smithi040.stdout:4/517: fsync dd/d58/fa1 0 2024-02-16T19:24:17.620 INFO:tasks.workunit.client.0.smithi040.stdout:4/518: fdatasync dd/de/d10/d36/d9e/d32/f95 0 2024-02-16T19:24:17.622 INFO:tasks.workunit.client.0.smithi040.stdout:1/588: rename d17/d26/d3d/laf to d17/db5/lba 0 2024-02-16T19:24:17.622 INFO:tasks.workunit.client.0.smithi040.stdout:7/501: dwrite d21/d29/f37 [0,4194304] 0 2024-02-16T19:24:17.622 INFO:tasks.workunit.client.0.smithi040.stdout:5/617: dread d1/da/d1c/d21/f2f [4194304,4194304] 0 2024-02-16T19:24:17.622 INFO:tasks.workunit.client.0.smithi040.stdout:7/502: chown d21/d4d/d77/d7a/d99 28 1 2024-02-16T19:24:17.622 INFO:tasks.workunit.client.0.smithi040.stdout:7/503: write d21/d4d/d77/d87/d9b/f9e [792371,96964] 0 2024-02-16T19:24:17.625 INFO:tasks.workunit.client.0.smithi040.stdout:6/678: creat d0/d1/d12/d3b/d48/dba/fdf x:0 0 0 2024-02-16T19:24:17.631 INFO:tasks.workunit.client.0.smithi040.stdout:0/673: rename db/de/l4b to db/d25/ddc/le2 0 2024-02-16T19:24:17.632 INFO:tasks.workunit.client.0.smithi040.stdout:4/519: rmdir dd 39 2024-02-16T19:24:17.634 INFO:tasks.workunit.client.0.smithi040.stdout:1/589: dread d17/d2d/f73 [0,4194304] 0 2024-02-16T19:24:17.634 INFO:tasks.workunit.client.0.smithi040.stdout:7/504: mknod d21/d4d/d77/ca1 0 2024-02-16T19:24:17.635 INFO:tasks.workunit.client.0.smithi040.stdout:7/505: write d21/d4d/d77/d7a/f82 [1031893,36341] 0 2024-02-16T19:24:17.636 INFO:tasks.workunit.client.0.smithi040.stdout:5/618: dwrite d1/da/d1c/f37 [0,4194304] 0 2024-02-16T19:24:17.636 INFO:tasks.workunit.client.0.smithi040.stdout:5/619: chown d1/da/d7e/db2 16139287 1 2024-02-16T19:24:17.637 INFO:tasks.workunit.client.0.smithi040.stdout:5/620: readlink d1/da/l2d 0 2024-02-16T19:24:17.637 INFO:tasks.workunit.client.0.smithi040.stdout:6/679: write d0/d1/d12/d3b/d79/fa2 [2699642,9248] 0 2024-02-16T19:24:17.637 INFO:tasks.workunit.client.0.smithi040.stdout:6/680: fsync d0/d30/d4b/f93 0 2024-02-16T19:24:17.638 INFO:tasks.workunit.client.0.smithi040.stdout:3/665: rename d1/l96 to d1/d7/d33/lcf 0 2024-02-16T19:24:17.638 INFO:tasks.workunit.client.0.smithi040.stdout:3/666: creat d1/d4/d6e/d8a/da5/fd0 x:0 0 0 2024-02-16T19:24:17.640 INFO:tasks.workunit.client.0.smithi040.stdout:0/674: creat db/d2f/d1d/d21/fe3 x:0 0 0 2024-02-16T19:24:17.640 INFO:tasks.workunit.client.0.smithi040.stdout:0/675: chown db/de/dbe/fe1 6230023 1 2024-02-16T19:24:17.640 INFO:tasks.workunit.client.0.smithi040.stdout:0/676: stat db/d2f/d1d/d21/d2e/d49/d5b/d67/d84 0 2024-02-16T19:24:17.641 INFO:tasks.workunit.client.0.smithi040.stdout:4/520: mknod dd/de/d10/d36/d9e/ca4 0 2024-02-16T19:24:17.642 INFO:tasks.workunit.client.0.smithi040.stdout:1/590: symlink d17/d3c/d74/d97/da9/lbb 0 2024-02-16T19:24:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:17 smithi040 ceph-mon[29201]: pgmap v247: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 126 MiB/s rd, 158 MiB/s wr, 227 op/s 2024-02-16T19:24:17.651 INFO:tasks.workunit.client.0.smithi040.stdout:5/621: unlink d1/da/d1c/d21/c39 0 2024-02-16T19:24:17.651 INFO:tasks.workunit.client.0.smithi040.stdout:5/622: truncate d1/da/d51/f58 500356 0 2024-02-16T19:24:17.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/681: mknod d0/d30/d32/ce0 0 2024-02-16T19:24:17.655 INFO:tasks.workunit.client.0.smithi040.stdout:3/667: mkdir d1/dc/d35/d53/dd1 0 2024-02-16T19:24:17.658 INFO:tasks.workunit.client.0.smithi040.stdout:0/677: creat db/d2f/d1d/d21/d2e/d83/fe4 x:0 0 0 2024-02-16T19:24:17.660 INFO:tasks.workunit.client.0.smithi040.stdout:4/521: symlink dd/d58/d68/la5 0 2024-02-16T19:24:17.661 INFO:tasks.workunit.client.0.smithi040.stdout:4/522: chown dd/de/d10/d36/d9e 16741150 1 2024-02-16T19:24:17.661 INFO:tasks.workunit.client.0.smithi040.stdout:4/523: fdatasync dd/de/d10/d36/d9e/d77/f67 0 2024-02-16T19:24:17.661 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:17 smithi163 ceph-mon[36452]: pgmap v247: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 126 MiB/s rd, 158 MiB/s wr, 227 op/s 2024-02-16T19:24:17.662 INFO:tasks.workunit.client.0.smithi040.stdout:5/623: write d1/d45/f52 [3130189,90519] 0 2024-02-16T19:24:17.665 INFO:tasks.workunit.client.0.smithi040.stdout:1/591: dwrite d17/d2d/f2f [0,4194304] 0 2024-02-16T19:24:17.665 INFO:tasks.workunit.client.0.smithi040.stdout:1/592: stat c15 0 2024-02-16T19:24:17.665 INFO:tasks.workunit.client.0.smithi040.stdout:3/668: read d1/d7/d36/f7b [1297752,86263] 0 2024-02-16T19:24:17.665 INFO:tasks.workunit.client.0.smithi040.stdout:3/669: write d1/dc/d42/d58/f76 [268594,95172] 0 2024-02-16T19:24:17.683 INFO:tasks.workunit.client.0.smithi040.stdout:0/678: mknod db/d2f/d1d/d21/d2e/d85/d89/ce5 0 2024-02-16T19:24:17.686 INFO:tasks.workunit.client.0.smithi040.stdout:4/524: fdatasync dd/de/d45/f5c 0 2024-02-16T19:24:17.689 INFO:tasks.workunit.client.0.smithi040.stdout:0/679: dread db/d25/f4f [0,4194304] 0 2024-02-16T19:24:17.689 INFO:tasks.workunit.client.0.smithi040.stdout:0/680: creat db/d2f/d23/d56/fe6 x:0 0 0 2024-02-16T19:24:17.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/682: rename d0/d1/d12/d3b/d48/d4f/d8b to d0/d37/de1 0 2024-02-16T19:24:17.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/683: fdatasync d0/d1/d12/d3b/f40 0 2024-02-16T19:24:17.690 INFO:tasks.workunit.client.0.smithi040.stdout:6/684: creat d0/d1/d6f/fe2 x:0 0 0 2024-02-16T19:24:17.693 INFO:tasks.workunit.client.0.smithi040.stdout:1/593: creat d17/d26/d5f/d78/d7e/fbc x:0 0 0 2024-02-16T19:24:17.699 INFO:tasks.workunit.client.0.smithi040.stdout:4/525: write dd/de/d10/d36/d9e/d77/f59 [544906,82844] 0 2024-02-16T19:24:17.701 INFO:tasks.workunit.client.0.smithi040.stdout:4/526: write dd/de/f5d [255881,28044] 0 2024-02-16T19:24:17.702 INFO:tasks.workunit.client.0.smithi040.stdout:1/594: dread d17/d26/d4c/dab/fb1 [0,4194304] 0 2024-02-16T19:24:17.704 INFO:tasks.workunit.client.0.smithi040.stdout:0/681: symlink db/le7 0 2024-02-16T19:24:17.705 INFO:tasks.workunit.client.0.smithi040.stdout:1/595: read d17/d26/d4c/dab/fb1 [246763,67099] 0 2024-02-16T19:24:17.705 INFO:tasks.workunit.client.0.smithi040.stdout:1/596: truncate d17/d26/d5f/d63/fa7 620542 0 2024-02-16T19:24:17.706 INFO:tasks.workunit.client.0.smithi040.stdout:1/597: fsync d17/d2d/f81 0 2024-02-16T19:24:17.706 INFO:tasks.workunit.client.0.smithi040.stdout:1/598: chown d17/d3c/d6b/c8e 1 1 2024-02-16T19:24:17.706 INFO:tasks.workunit.client.0.smithi040.stdout:1/599: dread - d17/d3c/d74/d97/f9e zero size 2024-02-16T19:24:17.708 INFO:tasks.workunit.client.0.smithi040.stdout:3/670: rename d1/dc/l2f to d1/d8c/ld2 0 2024-02-16T19:24:17.711 INFO:tasks.workunit.client.0.smithi040.stdout:6/685: dwrite d0/d1/d6f/d90/fd3 [0,4194304] 0 2024-02-16T19:24:17.713 INFO:tasks.workunit.client.0.smithi040.stdout:6/686: dread d0/d1/d12/d3b/d48/dba/dde/fab [0,4194304] 0 2024-02-16T19:24:17.715 INFO:tasks.workunit.client.0.smithi040.stdout:4/527: symlink dd/de/d71/la6 0 2024-02-16T19:24:17.721 INFO:tasks.workunit.client.0.smithi040.stdout:3/671: dread d1/d7/f2b [0,4194304] 0 2024-02-16T19:24:17.723 INFO:tasks.workunit.client.0.smithi040.stdout:0/682: dwrite db/d25/d2c/f5a [0,4194304] 0 2024-02-16T19:24:17.723 INFO:tasks.workunit.client.0.smithi040.stdout:6/687: mknod d0/d30/d32/dd2/dbc/ce3 0 2024-02-16T19:24:17.723 INFO:tasks.workunit.client.0.smithi040.stdout:6/688: fdatasync d0/d30/d32/f56 0 2024-02-16T19:24:17.724 INFO:tasks.workunit.client.0.smithi040.stdout:6/689: chown d0/d1/d12/d3b/d48/d73/d76/fd5 1107 1 2024-02-16T19:24:17.730 INFO:tasks.workunit.client.0.smithi040.stdout:0/683: creat db/d2f/d1d/d21/d2e/d85/fe8 x:0 0 0 2024-02-16T19:24:17.730 INFO:tasks.workunit.client.0.smithi040.stdout:0/684: dread - db/d2f/d1d/d21/d2e/d85/d89/d94/fb2 zero size 2024-02-16T19:24:17.734 INFO:tasks.workunit.client.0.smithi040.stdout:7/506: write d21/d4d/f74 [3685712,114165] 0 2024-02-16T19:24:17.737 INFO:tasks.workunit.client.0.smithi040.stdout:0/685: link db/d25/d34/d5e/f6b db/d2f/d23/d93/fe9 0 2024-02-16T19:24:17.742 INFO:tasks.workunit.client.0.smithi040.stdout:7/507: dread f11 [0,4194304] 0 2024-02-16T19:24:17.743 INFO:tasks.workunit.client.0.smithi040.stdout:7/508: stat d21/c72 0 2024-02-16T19:24:17.743 INFO:tasks.workunit.client.0.smithi040.stdout:0/686: symlink db/d25/lea 0 2024-02-16T19:24:17.748 INFO:tasks.workunit.client.0.smithi040.stdout:0/687: mkdir db/d2f/d23/d3b/deb 0 2024-02-16T19:24:17.749 INFO:tasks.workunit.client.0.smithi040.stdout:7/509: dread d21/d4d/f6f [0,4194304] 0 2024-02-16T19:24:17.750 INFO:tasks.workunit.client.0.smithi040.stdout:6/690: rmdir d0/d1/d12/d3b/d48/d73/d76/d7d 0 2024-02-16T19:24:17.752 INFO:tasks.workunit.client.0.smithi040.stdout:7/510: write d21/d4d/f56 [518962,102832] 0 2024-02-16T19:24:17.754 INFO:tasks.workunit.client.0.smithi040.stdout:6/691: creat d0/d37/de1/dac/fe4 x:0 0 0 2024-02-16T19:24:17.754 INFO:tasks.workunit.client.0.smithi040.stdout:6/692: write d0/d1/d6f/fdd [511381,105835] 0 2024-02-16T19:24:17.756 INFO:tasks.workunit.client.0.smithi040.stdout:0/688: dread db/d2f/d1d/d21/d2e/d49/d5b/f5d [0,4194304] 0 2024-02-16T19:24:17.756 INFO:tasks.workunit.client.0.smithi040.stdout:0/689: creat db/d25/fec x:0 0 0 2024-02-16T19:24:17.756 INFO:tasks.workunit.client.0.smithi040.stdout:0/690: write db/de/dbe/fb1 [3107698,39681] 0 2024-02-16T19:24:17.759 INFO:tasks.workunit.client.0.smithi040.stdout:0/691: write db/d2f/d1d/d21/d2e/d49/d5b/f79 [1479619,62011] 0 2024-02-16T19:24:17.759 INFO:tasks.workunit.client.0.smithi040.stdout:0/692: creat db/d25/d34/d5e/fed x:0 0 0 2024-02-16T19:24:17.759 INFO:tasks.workunit.client.0.smithi040.stdout:7/511: mknod d21/d4d/ca2 0 2024-02-16T19:24:17.763 INFO:tasks.workunit.client.0.smithi040.stdout:6/693: dread d0/d30/d32/f8d [0,4194304] 0 2024-02-16T19:24:17.767 INFO:tasks.workunit.client.0.smithi040.stdout:0/693: rename db/d2f/d23/c24 to db/d2f/cee 0 2024-02-16T19:24:17.767 INFO:tasks.workunit.client.0.smithi040.stdout:0/694: fdatasync db/d2f/d1d/d21/fe3 0 2024-02-16T19:24:17.770 INFO:tasks.workunit.client.0.smithi040.stdout:7/512: rmdir d21/d4d/d7d 39 2024-02-16T19:24:17.770 INFO:tasks.workunit.client.0.smithi040.stdout:7/513: read - d21/f5d zero size 2024-02-16T19:24:17.772 INFO:tasks.workunit.client.0.smithi040.stdout:6/694: rename d0/d37/lc7 to d0/d30/d32/dd2/le5 0 2024-02-16T19:24:17.774 INFO:tasks.workunit.client.0.smithi040.stdout:0/695: symlink db/d2f/d1d/d21/lef 0 2024-02-16T19:24:17.776 INFO:tasks.workunit.client.0.smithi040.stdout:6/695: dread d0/d1/d12/d3b/d48/f38 [0,4194304] 0 2024-02-16T19:24:17.776 INFO:tasks.workunit.client.0.smithi040.stdout:6/696: write d0/d1/d12/d3b/d48/d73/fa7 [5062828,25144] 0 2024-02-16T19:24:17.776 INFO:tasks.workunit.client.0.smithi040.stdout:7/514: truncate d21/d4d/d7d/f88 3524034 0 2024-02-16T19:24:17.776 INFO:tasks.workunit.client.0.smithi040.stdout:7/515: write d21/f28 [1992481,119788] 0 2024-02-16T19:24:17.777 INFO:tasks.workunit.client.0.smithi040.stdout:7/516: chown d21/d4d/d77/d87/d9b 180791879 1 2024-02-16T19:24:17.778 INFO:tasks.workunit.client.0.smithi040.stdout:6/697: symlink d0/d30/d32/dd2/dbc/le6 0 2024-02-16T19:24:17.779 INFO:tasks.workunit.client.0.smithi040.stdout:7/517: dread d21/d29/d79/f7f [0,4194304] 0 2024-02-16T19:24:17.781 INFO:tasks.workunit.client.0.smithi040.stdout:0/696: write db/d2f/d1d/d21/d2e/d49/f58 [1947626,21968] 0 2024-02-16T19:24:17.782 INFO:tasks.workunit.client.0.smithi040.stdout:6/698: mkdir d0/d30/d32/dd2/dbc/de7 0 2024-02-16T19:24:17.784 INFO:tasks.workunit.client.0.smithi040.stdout:7/518: creat d21/d4d/d77/fa3 x:0 0 0 2024-02-16T19:24:17.788 INFO:tasks.workunit.client.0.smithi040.stdout:0/697: mkdir db/de/d6a/df0 0 2024-02-16T19:24:17.789 INFO:tasks.workunit.client.0.smithi040.stdout:6/699: truncate d0/d1/f1f 7919679 0 2024-02-16T19:24:17.790 INFO:tasks.workunit.client.0.smithi040.stdout:7/519: dread d21/d4d/f6c [0,4194304] 0 2024-02-16T19:24:17.792 INFO:tasks.workunit.client.0.smithi040.stdout:0/698: symlink db/lf1 0 2024-02-16T19:24:17.792 INFO:tasks.workunit.client.0.smithi040.stdout:0/699: dread - db/d2f/d1d/d21/d2e/d49/d5b/fd7 zero size 2024-02-16T19:24:17.792 INFO:tasks.workunit.client.0.smithi040.stdout:0/700: creat db/d2f/d1d/d21/d2e/d83/ff2 x:0 0 0 2024-02-16T19:24:17.792 INFO:tasks.workunit.client.0.smithi040.stdout:0/701: dread - db/d2f/d1d/d21/d2e/d85/d89/d94/fb2 zero size 2024-02-16T19:24:17.793 INFO:tasks.workunit.client.0.smithi040.stdout:6/700: rmdir d0/d37/de1 39 2024-02-16T19:24:17.800 INFO:tasks.workunit.client.0.smithi040.stdout:0/702: link db/le7 db/d2f/d1d/d21/d2e/d85/lf3 0 2024-02-16T19:24:17.806 INFO:tasks.workunit.client.0.smithi040.stdout:7/520: dwrite f9 [0,4194304] 0 2024-02-16T19:24:17.810 INFO:tasks.workunit.client.0.smithi040.stdout:0/703: creat db/d2f/d23/d6f/d98/ff4 x:0 0 0 2024-02-16T19:24:17.810 INFO:tasks.workunit.client.0.smithi040.stdout:0/704: readlink db/d25/d34/la0 0 2024-02-16T19:24:17.815 INFO:tasks.workunit.client.0.smithi040.stdout:0/705: rename db/d2f/d1d/d21/f51 to db/d2f/d23/d6f/ff5 0 2024-02-16T19:24:17.815 INFO:tasks.workunit.client.0.smithi040.stdout:0/706: chown db/de/d6a/cb0 2042 1 2024-02-16T19:24:17.817 INFO:tasks.workunit.client.0.smithi040.stdout:3/672: write d1/d7/d30/f7f [1125226,20333] 0 2024-02-16T19:24:17.819 INFO:tasks.workunit.client.0.smithi040.stdout:0/707: getdents db/de 0 2024-02-16T19:24:17.819 INFO:tasks.workunit.client.0.smithi040.stdout:0/708: write db/d2f/d1d/d21/d2e/d49/d5b/d67/fb7 [914494,2783] 0 2024-02-16T19:24:17.820 INFO:tasks.workunit.client.0.smithi040.stdout:3/673: dread d1/dc/d42/d4a/f7d [0,4194304] 0 2024-02-16T19:24:17.821 INFO:tasks.workunit.client.0.smithi040.stdout:0/709: truncate db/de/f59 2620176 0 2024-02-16T19:24:17.823 INFO:tasks.workunit.client.0.smithi040.stdout:3/674: unlink d1/d8c/d8e/cc0 0 2024-02-16T19:24:17.826 INFO:tasks.workunit.client.0.smithi040.stdout:7/521: dwrite d21/d7e/f84 [0,4194304] 0 2024-02-16T19:24:17.827 INFO:tasks.workunit.client.0.smithi040.stdout:0/710: link db/de/lba db/d2f/d23/d93/lf6 0 2024-02-16T19:24:17.827 INFO:tasks.workunit.client.0.smithi040.stdout:0/711: write db/d2f/d1d/d21/d2e/d85/fe8 [269887,109041] 0 2024-02-16T19:24:17.830 INFO:tasks.workunit.client.0.smithi040.stdout:7/522: dread d21/f28 [4194304,4194304] 0 2024-02-16T19:24:17.834 INFO:tasks.workunit.client.0.smithi040.stdout:7/523: rename f1e to d21/d4d/d7b/fa4 0 2024-02-16T19:24:17.836 INFO:tasks.workunit.client.0.smithi040.stdout:0/712: dread db/d2f/d1d/d21/d2e/d49/d5b/d67/fb7 [0,4194304] 0 2024-02-16T19:24:17.838 INFO:tasks.workunit.client.0.smithi040.stdout:7/524: unlink d21/d29/c2a 0 2024-02-16T19:24:17.845 INFO:tasks.workunit.client.0.smithi040.stdout:0/713: read db/d2f/d1d/d21/d2e/d49/d5b/d67/f92 [2473161,47414] 0 2024-02-16T19:24:17.847 INFO:tasks.workunit.client.0.smithi040.stdout:0/714: mknod db/d2f/d23/d6f/cf7 0 2024-02-16T19:24:17.849 INFO:tasks.workunit.client.0.smithi040.stdout:0/715: creat db/de/d6a/df0/ff8 x:0 0 0 2024-02-16T19:24:17.849 INFO:tasks.workunit.client.0.smithi040.stdout:0/716: write db/d2f/d1d/d21/d2e/d49/d5b/d67/fb7 [1309187,74223] 0 2024-02-16T19:24:17.851 INFO:tasks.workunit.client.0.smithi040.stdout:0/717: rename db/d25/d34/la4 to db/d2f/d1d/d21/lf9 0 2024-02-16T19:24:17.854 INFO:tasks.workunit.client.0.smithi040.stdout:0/718: rmdir db/d2f/d23/d6f/d98/ddd 0 2024-02-16T19:24:17.869 INFO:tasks.workunit.client.0.smithi040.stdout:3/675: dwrite d1/d4/d6e/f84 [0,4194304] 0 2024-02-16T19:24:17.873 INFO:tasks.workunit.client.0.smithi040.stdout:3/676: creat d1/d8c/d8e/fd3 x:0 0 0 2024-02-16T19:24:17.875 INFO:tasks.workunit.client.0.smithi040.stdout:3/677: chown d1/dc/d42/d4a/d57/d79/cbe 16 1 2024-02-16T19:24:17.875 INFO:tasks.workunit.client.0.smithi040.stdout:3/678: truncate d1/dc/d35/d48/fc7 416535 0 2024-02-16T19:24:17.884 INFO:tasks.workunit.client.0.smithi040.stdout:3/679: dread d1/dc/fd [0,4194304] 0 2024-02-16T19:24:17.884 INFO:tasks.workunit.client.0.smithi040.stdout:3/680: write d1/dc/d42/d4a/f50 [4213083,33829] 0 2024-02-16T19:24:17.884 INFO:tasks.workunit.client.0.smithi040.stdout:0/719: dwrite db/d25/d2c/d52/f91 [0,4194304] 0 2024-02-16T19:24:17.884 INFO:tasks.workunit.client.0.smithi040.stdout:0/720: chown db/d2f/d1d 200379 1 2024-02-16T19:24:17.884 INFO:tasks.workunit.client.0.smithi040.stdout:0/721: chown db/l7a 156 1 2024-02-16T19:24:17.885 INFO:tasks.workunit.client.0.smithi040.stdout:7/525: dwrite d21/d4d/f6f [0,4194304] 0 2024-02-16T19:24:17.885 INFO:tasks.workunit.client.0.smithi040.stdout:7/526: write d21/f62 [2888846,22945] 0 2024-02-16T19:24:17.885 INFO:tasks.workunit.client.0.smithi040.stdout:7/527: write d21/d29/f38 [561015,98926] 0 2024-02-16T19:24:17.895 INFO:tasks.workunit.client.0.smithi040.stdout:3/681: write d1/ff [2977869,67655] 0 2024-02-16T19:24:17.895 INFO:tasks.workunit.client.0.smithi040.stdout:3/682: chown d1/d7/c75 4756 1 2024-02-16T19:24:17.901 INFO:tasks.workunit.client.0.smithi040.stdout:3/683: dread d1/d7/d30/fa8 [0,4194304] 0 2024-02-16T19:24:17.901 INFO:tasks.workunit.client.0.smithi040.stdout:3/684: chown d1/d4/f41 8 1 2024-02-16T19:24:17.902 INFO:tasks.workunit.client.0.smithi040.stdout:3/685: symlink d1/d7/d30/ld4 0 2024-02-16T19:24:17.902 INFO:tasks.workunit.client.0.smithi040.stdout:3/686: symlink d1/dc/d35/d48/ld5 0 2024-02-16T19:24:17.902 INFO:tasks.workunit.client.0.smithi040.stdout:3/687: creat d1/d4/d6e/d8a/da5/fd6 x:0 0 0 2024-02-16T19:24:17.904 INFO:tasks.workunit.client.0.smithi040.stdout:3/688: symlink d1/d4/ld7 0 2024-02-16T19:24:17.905 INFO:tasks.workunit.client.0.smithi040.stdout:3/689: creat d1/d4/d6e/dc2/dce/fd8 x:0 0 0 2024-02-16T19:24:17.905 INFO:tasks.workunit.client.0.smithi040.stdout:3/690: dread - d1/d4/d6e/f91 zero size 2024-02-16T19:24:17.905 INFO:tasks.workunit.client.0.smithi040.stdout:7/528: dwrite d21/f48 [0,4194304] 0 2024-02-16T19:24:17.905 INFO:tasks.workunit.client.0.smithi040.stdout:0/722: dwrite db/d2f/f3e [4194304,4194304] 0 2024-02-16T19:24:17.907 INFO:tasks.workunit.client.0.smithi040.stdout:3/691: creat d1/dc/d35/d53/d64/d65/daa/fd9 x:0 0 0 2024-02-16T19:24:17.908 INFO:tasks.workunit.client.0.smithi040.stdout:0/723: creat db/de/d6a/ffa x:0 0 0 2024-02-16T19:24:17.908 INFO:tasks.workunit.client.0.smithi040.stdout:0/724: write db/d2f/d23/fcf [774310,62602] 0 2024-02-16T19:24:17.908 INFO:tasks.workunit.client.0.smithi040.stdout:0/725: chown db/d2f/d23/d6f/cf7 185 1 2024-02-16T19:24:17.909 INFO:tasks.workunit.client.0.smithi040.stdout:7/529: read d21/d4d/f6c [1288885,113185] 0 2024-02-16T19:24:17.911 INFO:tasks.workunit.client.0.smithi040.stdout:7/530: symlink d21/d4d/d7d/la5 0 2024-02-16T19:24:17.912 INFO:tasks.workunit.client.0.smithi040.stdout:7/531: creat d21/d4d/d7b/fa6 x:0 0 0 2024-02-16T19:24:17.912 INFO:tasks.workunit.client.0.smithi040.stdout:7/532: fsync d21/d29/f4c 0 2024-02-16T19:24:17.912 INFO:tasks.workunit.client.0.smithi040.stdout:7/533: dread - d21/f5d zero size 2024-02-16T19:24:17.913 INFO:tasks.workunit.client.0.smithi040.stdout:7/534: truncate d21/d4d/f4b 4137744 0 2024-02-16T19:24:17.925 INFO:tasks.workunit.client.0.smithi040.stdout:7/535: dread d21/d4d/d7b/f8f [0,4194304] 0 2024-02-16T19:24:17.932 INFO:tasks.workunit.client.0.smithi040.stdout:3/692: dwrite d1/d7/d36/f7c [0,4194304] 0 2024-02-16T19:24:17.947 INFO:tasks.workunit.client.0.smithi040.stdout:3/693: dwrite d1/f3 [0,4194304] 0 2024-02-16T19:24:17.963 INFO:tasks.workunit.client.0.smithi040.stdout:3/694: dwrite d1/dc/d42/d4a/f54 [0,4194304] 0 2024-02-16T19:24:17.972 INFO:tasks.workunit.client.0.smithi040.stdout:3/695: write d1/d4/f41 [1108356,10270] 0 2024-02-16T19:24:17.973 INFO:tasks.workunit.client.0.smithi040.stdout:3/696: rmdir d1/d7/d36 39 2024-02-16T19:24:17.973 INFO:tasks.workunit.client.0.smithi040.stdout:3/697: chown d1/dc/d42/d4a/f7d 31 1 2024-02-16T19:24:17.974 INFO:tasks.workunit.client.0.smithi040.stdout:3/698: mkdir d1/d8c/dda 0 2024-02-16T19:24:17.990 INFO:tasks.workunit.client.0.smithi040.stdout:3/699: dwrite d1/d7/d30/fa8 [0,4194304] 0 2024-02-16T19:24:18.002 INFO:tasks.workunit.client.0.smithi040.stdout:3/700: dwrite d1/dc/d35/d53/d64/d65/fb9 [0,4194304] 0 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:4/11: sync 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:0/24: sync 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:3/17: sync 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:7/1: sync 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:7/2: rename - no filename 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:7/3: fdatasync - no filename 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:1/0: sync 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:2/136: sync 2024-02-16T19:24:18.012 INFO:tasks.workunit.client.1.smithi163.stdout:1/1: write - no filename 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:1/2: chown . 626 1 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:1/3: chown . 26225 1 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:1/4: dwrite - no filename 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:1/5: stat - no entries 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:8/20: sync 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:6/5: sync 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:5/8: sync 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:8/21: chown d0/f4 40985 1 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:5/9: truncate d0/f1 379956 0 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:9/199: sync 2024-02-16T19:24:18.013 INFO:tasks.workunit.client.1.smithi163.stdout:5/10: write d0/f1 [1380934,90460] 0 2024-02-16T19:24:18.014 INFO:tasks.workunit.client.1.smithi163.stdout:4/12: creat f2 x:0 0 0 2024-02-16T19:24:18.014 INFO:tasks.workunit.client.1.smithi163.stdout:4/13: dread - f2 zero size 2024-02-16T19:24:18.014 INFO:tasks.workunit.client.1.smithi163.stdout:4/14: creat f3 x:0 0 0 2024-02-16T19:24:18.014 INFO:tasks.workunit.client.1.smithi163.stdout:7/4: symlink l0 0 2024-02-16T19:24:18.014 INFO:tasks.workunit.client.1.smithi163.stdout:7/5: fdatasync - no filename 2024-02-16T19:24:18.015 INFO:tasks.workunit.client.1.smithi163.stdout:1/6: creat f0 x:0 0 0 2024-02-16T19:24:18.015 INFO:tasks.workunit.client.1.smithi163.stdout:6/6: mknod d0/c1 0 2024-02-16T19:24:18.015 INFO:tasks.workunit.client.1.smithi163.stdout:6/7: chown d0 3614794 1 2024-02-16T19:24:18.015 INFO:tasks.workunit.client.1.smithi163.stdout:6/8: fsync - no filename 2024-02-16T19:24:18.015 INFO:tasks.workunit.client.1.smithi163.stdout:9/200: creat d5/de/d30/d43/f4b x:0 0 0 2024-02-16T19:24:18.018 INFO:tasks.workunit.client.1.smithi163.stdout:7/6: creat f1 x:0 0 0 2024-02-16T19:24:18.018 INFO:tasks.workunit.client.1.smithi163.stdout:7/7: rmdir - no directory 2024-02-16T19:24:18.018 INFO:tasks.workunit.client.1.smithi163.stdout:7/8: fdatasync f1 0 2024-02-16T19:24:18.018 INFO:tasks.workunit.client.1.smithi163.stdout:7/9: truncate f1 43897 0 2024-02-16T19:24:18.020 INFO:tasks.workunit.client.1.smithi163.stdout:6/9: creat d0/f2 x:0 0 0 2024-02-16T19:24:18.020 INFO:tasks.workunit.client.1.smithi163.stdout:7/10: dread f1 [0,4194304] 0 2024-02-16T19:24:18.021 INFO:tasks.workunit.client.1.smithi163.stdout:4/15: rename f3 to f4 0 2024-02-16T19:24:18.022 INFO:tasks.workunit.client.1.smithi163.stdout:7/11: dread f1 [0,4194304] 0 2024-02-16T19:24:18.023 INFO:tasks.workunit.client.1.smithi163.stdout:6/10: symlink d0/l3 0 2024-02-16T19:24:18.023 INFO:tasks.workunit.client.1.smithi163.stdout:6/11: readlink d0/l3 0 2024-02-16T19:24:18.023 INFO:tasks.workunit.client.1.smithi163.stdout:6/12: write d0/f2 [324393,83007] 0 2024-02-16T19:24:18.023 INFO:tasks.workunit.client.1.smithi163.stdout:6/13: write d0/f2 [952178,129] 0 2024-02-16T19:24:18.028 INFO:tasks.workunit.client.1.smithi163.stdout:1/7: fsync f0 0 2024-02-16T19:24:18.028 INFO:tasks.workunit.client.1.smithi163.stdout:1/8: stat f0 0 2024-02-16T19:24:18.028 INFO:tasks.workunit.client.1.smithi163.stdout:1/9: truncate f0 122317 0 2024-02-16T19:24:18.029 INFO:tasks.workunit.client.1.smithi163.stdout:3/18: write d0/f2 [31985,27447] 0 2024-02-16T19:24:18.030 INFO:tasks.workunit.client.1.smithi163.stdout:3/19: mknod d0/c4 0 2024-02-16T19:24:18.032 INFO:tasks.workunit.client.1.smithi163.stdout:9/201: write d5/de/d30/f3e [4074656,6888] 0 2024-02-16T19:24:18.033 INFO:tasks.workunit.client.1.smithi163.stdout:5/11: fsync d0/f1 0 2024-02-16T19:24:18.033 INFO:tasks.workunit.client.1.smithi163.stdout:5/12: stat d0 0 2024-02-16T19:24:18.033 INFO:tasks.workunit.client.1.smithi163.stdout:1/10: read f0 [49793,16255] 0 2024-02-16T19:24:18.035 INFO:tasks.workunit.client.1.smithi163.stdout:9/202: creat d5/de/d16/d47/f4c x:0 0 0 2024-02-16T19:24:18.036 INFO:tasks.workunit.client.1.smithi163.stdout:9/203: symlink d5/de/d16/d46/l4d 0 2024-02-16T19:24:18.037 INFO:tasks.workunit.client.1.smithi163.stdout:9/204: mkdir d5/de/d4e 0 2024-02-16T19:24:18.038 INFO:tasks.workunit.client.1.smithi163.stdout:9/205: rmdir d5 39 2024-02-16T19:24:18.039 INFO:tasks.workunit.client.1.smithi163.stdout:9/206: truncate d5/de/d16/f25 2720950 0 2024-02-16T19:24:18.039 INFO:tasks.workunit.client.1.smithi163.stdout:9/207: symlink d5/de/d30/l4f 0 2024-02-16T19:24:18.040 INFO:tasks.workunit.client.0.smithi040.stdout:3/701: dwrite d1/ff [0,4194304] 0 2024-02-16T19:24:18.040 INFO:tasks.workunit.client.1.smithi163.stdout:9/208: stat d5/c29 0 2024-02-16T19:24:18.040 INFO:tasks.workunit.client.1.smithi163.stdout:9/209: fdatasync d5/f1e 0 2024-02-16T19:24:18.041 INFO:tasks.workunit.client.1.smithi163.stdout:9/210: dread - d5/f2e zero size 2024-02-16T19:24:18.041 INFO:tasks.workunit.client.1.smithi163.stdout:9/211: getdents d5/de/d16 0 2024-02-16T19:24:18.044 INFO:tasks.workunit.client.1.smithi163.stdout:3/20: dread d0/f1 [0,4194304] 0 2024-02-16T19:24:18.048 INFO:tasks.workunit.client.1.smithi163.stdout:3/21: symlink d0/l5 0 2024-02-16T19:24:18.048 INFO:tasks.workunit.client.0.smithi040.stdout:3/702: creat d1/d7/d33/fdb x:0 0 0 2024-02-16T19:24:18.051 INFO:tasks.workunit.client.1.smithi163.stdout:2/137: dwrite d2/ff [4194304,4194304] 0 2024-02-16T19:24:18.056 INFO:tasks.workunit.client.1.smithi163.stdout:0/25: dwrite f3 [0,4194304] 0 2024-02-16T19:24:18.056 INFO:tasks.workunit.client.1.smithi163.stdout:5/13: truncate d0/f1 731808 0 2024-02-16T19:24:18.059 INFO:tasks.workunit.client.1.smithi163.stdout:0/26: mknod c9 0 2024-02-16T19:24:18.063 INFO:tasks.workunit.client.1.smithi163.stdout:0/27: mknod ca 0 2024-02-16T19:24:18.063 INFO:tasks.workunit.client.1.smithi163.stdout:7/12: dwrite f1 [0,4194304] 0 2024-02-16T19:24:18.063 INFO:tasks.workunit.client.1.smithi163.stdout:7/13: creat f2 x:0 0 0 2024-02-16T19:24:18.063 INFO:tasks.workunit.client.1.smithi163.stdout:7/14: creat f3 x:0 0 0 2024-02-16T19:24:18.063 INFO:tasks.workunit.client.1.smithi163.stdout:7/15: dread - f3 zero size 2024-02-16T19:24:18.063 INFO:tasks.workunit.client.1.smithi163.stdout:1/11: dwrite f0 [0,4194304] 0 2024-02-16T19:24:18.064 INFO:tasks.workunit.client.1.smithi163.stdout:4/16: dwrite f2 [0,4194304] 0 2024-02-16T19:24:18.064 INFO:tasks.workunit.client.1.smithi163.stdout:4/17: chown f4 775519 1 2024-02-16T19:24:18.064 INFO:tasks.workunit.client.1.smithi163.stdout:4/18: dread - f4 zero size 2024-02-16T19:24:18.067 INFO:tasks.workunit.client.1.smithi163.stdout:7/16: mknod c4 0 2024-02-16T19:24:18.067 INFO:tasks.workunit.client.1.smithi163.stdout:7/17: fdatasync f2 0 2024-02-16T19:24:18.067 INFO:tasks.workunit.client.1.smithi163.stdout:7/18: dread - f3 zero size 2024-02-16T19:24:18.068 INFO:tasks.workunit.client.1.smithi163.stdout:7/19: dread - f3 zero size 2024-02-16T19:24:18.072 INFO:tasks.workunit.client.1.smithi163.stdout:1/12: symlink l1 0 2024-02-16T19:24:18.081 INFO:tasks.workunit.client.1.smithi163.stdout:1/13: link f0 f2 0 2024-02-16T19:24:18.081 INFO:tasks.workunit.client.1.smithi163.stdout:1/14: creat f3 x:0 0 0 2024-02-16T19:24:18.089 INFO:tasks.workunit.client.1.smithi163.stdout:5/14: write d0/f1 [894527,56379] 0 2024-02-16T19:24:18.091 INFO:tasks.workunit.client.0.smithi040.stdout:3/703: dwrite d1/d7/d36/f87 [0,4194304] 0 2024-02-16T19:24:18.091 INFO:tasks.workunit.client.1.smithi163.stdout:6/14: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:18.092 INFO:tasks.workunit.client.1.smithi163.stdout:7/20: fsync f2 0 2024-02-16T19:24:18.094 INFO:tasks.workunit.client.0.smithi040.stdout:2/562: sync 2024-02-16T19:24:18.094 INFO:tasks.workunit.client.0.smithi040.stdout:3/704: creat d1/d8c/d8e/fdc x:0 0 0 2024-02-16T19:24:18.094 INFO:tasks.workunit.client.1.smithi163.stdout:1/15: write f2 [1172959,102256] 0 2024-02-16T19:24:18.095 INFO:tasks.workunit.client.1.smithi163.stdout:1/16: chown f0 339094 1 2024-02-16T19:24:18.095 INFO:tasks.workunit.client.0.smithi040.stdout:2/563: truncate d5/d7/d1e/f80 2463831 0 2024-02-16T19:24:18.097 INFO:tasks.workunit.client.0.smithi040.stdout:3/705: symlink d1/dc/d35/d55/ldd 0 2024-02-16T19:24:18.098 INFO:tasks.workunit.client.0.smithi040.stdout:2/564: symlink d5/d7/lb7 0 2024-02-16T19:24:18.098 INFO:tasks.workunit.client.0.smithi040.stdout:2/565: stat d5/d7/d10/d40/f91 0 2024-02-16T19:24:18.100 INFO:tasks.workunit.client.0.smithi040.stdout:3/706: mkdir d1/d8c/d8e/dde 0 2024-02-16T19:24:18.100 INFO:tasks.workunit.client.0.smithi040.stdout:3/707: readlink d1/dc/d42/d4a/l59 0 2024-02-16T19:24:18.100 INFO:tasks.workunit.client.0.smithi040.stdout:3/708: dread - d1/dc/d35/d53/fa9 zero size 2024-02-16T19:24:18.107 INFO:tasks.workunit.client.0.smithi040.stdout:8/702: sync 2024-02-16T19:24:18.108 INFO:tasks.workunit.client.1.smithi163.stdout:9/212: dwrite d5/de/d30/f3e [0,4194304] 0 2024-02-16T19:24:18.109 INFO:tasks.workunit.client.1.smithi163.stdout:9/213: readlink d5/de/l26 0 2024-02-16T19:24:18.111 INFO:tasks.workunit.client.1.smithi163.stdout:7/21: write f1 [608447,91764] 0 2024-02-16T19:24:18.111 INFO:tasks.workunit.client.1.smithi163.stdout:7/22: dread - f3 zero size 2024-02-16T19:24:18.112 INFO:tasks.workunit.client.1.smithi163.stdout:7/23: creat f5 x:0 0 0 2024-02-16T19:24:18.112 INFO:tasks.workunit.client.0.smithi040.stdout:2/566: creat d5/d7/d1e/d31/fb8 x:0 0 0 2024-02-16T19:24:18.113 INFO:tasks.workunit.client.0.smithi040.stdout:8/703: rename d6/d7/fb5 to d6/d12/d93/fda 0 2024-02-16T19:24:18.113 INFO:tasks.workunit.client.1.smithi163.stdout:9/214: mkdir d5/de/d30/d3d/d49/d50 0 2024-02-16T19:24:18.114 INFO:tasks.workunit.client.0.smithi040.stdout:2/567: mknod d5/d7/cb9 0 2024-02-16T19:24:18.115 INFO:tasks.workunit.client.0.smithi040.stdout:2/568: chown d5/d7/d1e/d31/c3e 6232 1 2024-02-16T19:24:18.115 INFO:tasks.workunit.client.0.smithi040.stdout:2/569: chown d5/d7/d1e/d31/d4f/d63/l84 1084 1 2024-02-16T19:24:18.116 INFO:tasks.workunit.client.0.smithi040.stdout:8/704: mknod d6/d2b/da3/d8b/cdb 0 2024-02-16T19:24:18.118 INFO:tasks.workunit.client.0.smithi040.stdout:2/570: rename d5/d8/cc to d5/d7/cba 0 2024-02-16T19:24:18.118 INFO:tasks.workunit.client.0.smithi040.stdout:2/571: truncate d5/d7/d12/f76 1205715 0 2024-02-16T19:24:18.118 INFO:tasks.workunit.client.0.smithi040.stdout:2/572: truncate d5/d7/fa1 4229905 0 2024-02-16T19:24:18.119 INFO:tasks.workunit.client.1.smithi163.stdout:7/24: unlink c4 0 2024-02-16T19:24:18.119 INFO:tasks.workunit.client.1.smithi163.stdout:7/25: chown f2 40 1 2024-02-16T19:24:18.119 INFO:tasks.workunit.client.1.smithi163.stdout:7/26: write f2 [795921,55322] 0 2024-02-16T19:24:18.120 INFO:tasks.workunit.client.0.smithi040.stdout:8/705: creat d6/fdc x:0 0 0 2024-02-16T19:24:18.120 INFO:tasks.workunit.client.1.smithi163.stdout:9/215: rmdir d5/de/d30/d31 39 2024-02-16T19:24:18.121 INFO:tasks.workunit.client.0.smithi040.stdout:2/573: truncate d5/d7/d1e/d4c/d52/f9f 4186441 0 2024-02-16T19:24:18.122 INFO:tasks.workunit.client.0.smithi040.stdout:8/706: rename d6/d2b/d44/f91 to d6/d12/d51/d6e/db7/dc6/fdd 0 2024-02-16T19:24:18.122 INFO:tasks.workunit.client.0.smithi040.stdout:8/707: creat d6/d12/d51/d6e/db7/dc6/fde x:0 0 0 2024-02-16T19:24:18.123 INFO:tasks.workunit.client.0.smithi040.stdout:8/708: write d6/fa5 [1581109,11070] 0 2024-02-16T19:24:18.123 INFO:tasks.workunit.client.0.smithi040.stdout:2/574: creat d5/d5c/d65/d9d/fbb x:0 0 0 2024-02-16T19:24:18.123 INFO:tasks.workunit.client.0.smithi040.stdout:2/575: chown d5/d8/c56 194554 1 2024-02-16T19:24:18.125 INFO:tasks.workunit.client.0.smithi040.stdout:8/709: mkdir d6/d12/d51/d6e/db7/ddf 0 2024-02-16T19:24:18.126 INFO:tasks.workunit.client.0.smithi040.stdout:2/576: mkdir d5/d7/d10/dbc 0 2024-02-16T19:24:18.128 INFO:tasks.workunit.client.0.smithi040.stdout:8/710: link d6/d7/d1d/c2d d6/d12/d51/d6e/ce0 0 2024-02-16T19:24:18.129 INFO:tasks.workunit.client.0.smithi040.stdout:2/577: rename d5/d7/d12/f76 to d5/d5c/d65/d9d/fbd 0 2024-02-16T19:24:18.129 INFO:tasks.workunit.client.0.smithi040.stdout:8/711: symlink d6/d64/d50/dd5/le1 0 2024-02-16T19:24:18.129 INFO:tasks.workunit.client.0.smithi040.stdout:8/712: chown d6/d12/d22/l9e 533 1 2024-02-16T19:24:18.130 INFO:tasks.workunit.client.0.smithi040.stdout:2/578: mkdir d5/d8/dbe 0 2024-02-16T19:24:18.185 INFO:tasks.workunit.client.1.smithi163.stdout:2/138: dwrite d2/d26/f2b [0,4194304] 0 2024-02-16T19:24:18.186 INFO:tasks.workunit.client.1.smithi163.stdout:0/28: dwrite f7 [0,4194304] 0 2024-02-16T19:24:18.186 INFO:tasks.workunit.client.1.smithi163.stdout:3/22: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:18.187 INFO:tasks.workunit.client.1.smithi163.stdout:4/19: dwrite f2 [4194304,4194304] 0 2024-02-16T19:24:18.187 INFO:tasks.workunit.client.1.smithi163.stdout:4/20: rmdir - no directory 2024-02-16T19:24:18.189 INFO:tasks.workunit.client.1.smithi163.stdout:2/139: symlink d2/d2a/l30 0 2024-02-16T19:24:18.189 INFO:tasks.workunit.client.1.smithi163.stdout:2/140: stat d2/l24 0 2024-02-16T19:24:18.193 INFO:tasks.workunit.client.1.smithi163.stdout:9/216: dwrite d5/de/ff [4194304,4194304] 0 2024-02-16T19:24:18.196 INFO:tasks.workunit.client.1.smithi163.stdout:0/29: symlink lb 0 2024-02-16T19:24:18.200 INFO:tasks.workunit.client.1.smithi163.stdout:3/23: unlink d0/c4 0 2024-02-16T19:24:18.202 INFO:tasks.workunit.client.1.smithi163.stdout:6/15: write d0/f2 [4401943,124196] 0 2024-02-16T19:24:18.203 INFO:tasks.workunit.client.1.smithi163.stdout:6/16: chown d0/l3 0 1 2024-02-16T19:24:18.203 INFO:tasks.workunit.client.1.smithi163.stdout:6/17: chown d0/f2 2334 1 2024-02-16T19:24:18.205 INFO:tasks.workunit.client.1.smithi163.stdout:2/141: link d2/f18 d2/d26/f31 0 2024-02-16T19:24:18.205 INFO:tasks.workunit.client.1.smithi163.stdout:2/142: write d2/d26/f27 [1066767,75111] 0 2024-02-16T19:24:18.207 INFO:tasks.workunit.client.1.smithi163.stdout:0/30: read f3 [46210,109356] 0 2024-02-16T19:24:18.207 INFO:tasks.workunit.client.1.smithi163.stdout:0/31: write f6 [5119318,94789] 0 2024-02-16T19:24:18.207 INFO:tasks.workunit.client.1.smithi163.stdout:9/217: creat d5/de/d30/d31/f51 x:0 0 0 2024-02-16T19:24:18.209 INFO:tasks.workunit.client.1.smithi163.stdout:7/27: dread f2 [0,4194304] 0 2024-02-16T19:24:18.210 INFO:tasks.workunit.client.1.smithi163.stdout:1/17: dread f2 [0,4194304] 0 2024-02-16T19:24:18.210 INFO:tasks.workunit.client.1.smithi163.stdout:1/18: read f0 [609769,94710] 0 2024-02-16T19:24:18.212 INFO:tasks.workunit.client.1.smithi163.stdout:6/18: rename d0/c1 to d0/c4 0 2024-02-16T19:24:18.214 INFO:tasks.workunit.client.1.smithi163.stdout:2/143: symlink d2/d26/l32 0 2024-02-16T19:24:18.215 INFO:tasks.workunit.client.0.smithi040.stdout:8/713: dwrite d6/d2b/f95 [0,4194304] 0 2024-02-16T19:24:18.217 INFO:tasks.workunit.client.1.smithi163.stdout:4/21: dwrite f4 [0,4194304] 0 2024-02-16T19:24:18.217 INFO:tasks.workunit.client.1.smithi163.stdout:0/32: dread f8 [0,4194304] 0 2024-02-16T19:24:18.218 INFO:tasks.workunit.client.1.smithi163.stdout:0/33: creat fc x:0 0 0 2024-02-16T19:24:18.218 INFO:tasks.workunit.client.1.smithi163.stdout:0/34: creat fd x:0 0 0 2024-02-16T19:24:18.218 INFO:tasks.workunit.client.1.smithi163.stdout:7/28: symlink l6 0 2024-02-16T19:24:18.224 INFO:tasks.workunit.client.1.smithi163.stdout:9/218: rmdir d5/de/d30/d43 39 2024-02-16T19:24:18.224 INFO:tasks.workunit.client.1.smithi163.stdout:9/219: stat d5/de/d16/d46/l4d 0 2024-02-16T19:24:18.224 INFO:tasks.workunit.client.1.smithi163.stdout:2/144: dread d2/f15 [0,4194304] 0 2024-02-16T19:24:18.224 INFO:tasks.workunit.client.1.smithi163.stdout:2/145: write d2/f5 [1245546,67351] 0 2024-02-16T19:24:18.230 INFO:tasks.workunit.client.1.smithi163.stdout:2/146: creat d2/d2a/f33 x:0 0 0 2024-02-16T19:24:18.232 INFO:tasks.workunit.client.1.smithi163.stdout:4/22: getdents . 0 2024-02-16T19:24:18.235 INFO:tasks.workunit.client.1.smithi163.stdout:2/147: mknod d2/c34 0 2024-02-16T19:24:18.235 INFO:tasks.workunit.client.1.smithi163.stdout:2/148: creat d2/d26/f35 x:0 0 0 2024-02-16T19:24:18.252 INFO:tasks.workunit.client.1.smithi163.stdout:0/35: fsync fd 0 2024-02-16T19:24:18.253 INFO:tasks.workunit.client.1.smithi163.stdout:0/36: mknod ce 0 2024-02-16T19:24:18.253 INFO:tasks.workunit.client.1.smithi163.stdout:0/37: creat ff x:0 0 0 2024-02-16T19:24:18.255 INFO:tasks.workunit.client.1.smithi163.stdout:4/23: truncate f2 1944126 0 2024-02-16T19:24:18.264 INFO:tasks.workunit.client.0.smithi040.stdout:9/681: sync 2024-02-16T19:24:18.271 INFO:tasks.workunit.client.1.smithi163.stdout:0/38: dwrite fc [0,4194304] 0 2024-02-16T19:24:18.271 INFO:tasks.workunit.client.1.smithi163.stdout:4/24: dread f4 [0,4194304] 0 2024-02-16T19:24:18.275 INFO:tasks.workunit.client.1.smithi163.stdout:4/25: rename f4 to f5 0 2024-02-16T19:24:18.276 INFO:tasks.workunit.client.1.smithi163.stdout:4/26: creat f6 x:0 0 0 2024-02-16T19:24:18.278 INFO:tasks.workunit.client.1.smithi163.stdout:4/27: mknod c7 0 2024-02-16T19:24:18.284 INFO:tasks.workunit.client.1.smithi163.stdout:4/28: dread f5 [0,4194304] 0 2024-02-16T19:24:18.287 INFO:tasks.workunit.client.1.smithi163.stdout:0/39: dwrite f8 [0,4194304] 0 2024-02-16T19:24:18.287 INFO:tasks.workunit.client.1.smithi163.stdout:0/40: write f7 [4871574,55473] 0 2024-02-16T19:24:18.287 INFO:tasks.workunit.client.1.smithi163.stdout:0/41: chown f6 5349515 1 2024-02-16T19:24:18.287 INFO:tasks.workunit.client.1.smithi163.stdout:0/42: dread - ff zero size 2024-02-16T19:24:18.287 INFO:tasks.workunit.client.1.smithi163.stdout:0/43: readlink lb 0 2024-02-16T19:24:18.287 INFO:tasks.workunit.client.1.smithi163.stdout:0/44: creat f10 x:0 0 0 2024-02-16T19:24:18.288 INFO:tasks.workunit.client.1.smithi163.stdout:4/29: rename f5 to f8 0 2024-02-16T19:24:18.288 INFO:tasks.workunit.client.1.smithi163.stdout:4/30: chown c1 289909 1 2024-02-16T19:24:18.298 INFO:tasks.workunit.client.1.smithi163.stdout:3/24: write d0/f1 [873540,67815] 0 2024-02-16T19:24:18.299 INFO:tasks.workunit.client.1.smithi163.stdout:3/25: stat d0/l5 0 2024-02-16T19:24:18.299 INFO:tasks.workunit.client.1.smithi163.stdout:3/26: creat d0/f6 x:0 0 0 2024-02-16T19:24:18.300 INFO:tasks.workunit.client.1.smithi163.stdout:3/27: mknod d0/c7 0 2024-02-16T19:24:18.319 INFO:tasks.workunit.client.1.smithi163.stdout:3/28: write d0/f2 [1279616,43518] 0 2024-02-16T19:24:18.319 INFO:tasks.workunit.client.1.smithi163.stdout:3/29: truncate d0/f2 4914575 0 2024-02-16T19:24:18.319 INFO:tasks.workunit.client.1.smithi163.stdout:3/30: creat d0/f8 x:0 0 0 2024-02-16T19:24:18.319 INFO:tasks.workunit.client.1.smithi163.stdout:3/31: write d0/f1 [4683050,37520] 0 2024-02-16T19:24:18.325 INFO:tasks.workunit.client.1.smithi163.stdout:9/220: dwrite d5/f2b [0,4194304] 0 2024-02-16T19:24:18.325 INFO:tasks.workunit.client.1.smithi163.stdout:6/19: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:18.325 INFO:tasks.workunit.client.1.smithi163.stdout:0/45: dwrite fd [0,4194304] 0 2024-02-16T19:24:18.325 INFO:tasks.workunit.client.1.smithi163.stdout:0/46: write f3 [4940354,100922] 0 2024-02-16T19:24:18.325 INFO:tasks.workunit.client.1.smithi163.stdout:0/47: readlink lb 0 2024-02-16T19:24:18.326 INFO:tasks.workunit.client.0.smithi040.stdout:9/682: dwrite d3/d4/f57 [0,4194304] 0 2024-02-16T19:24:18.326 INFO:tasks.workunit.client.1.smithi163.stdout:4/31: dwrite f2 [0,4194304] 0 2024-02-16T19:24:18.326 INFO:tasks.workunit.client.1.smithi163.stdout:2/149: dwrite d2/d26/f31 [0,4194304] 0 2024-02-16T19:24:18.332 INFO:tasks.workunit.client.0.smithi040.stdout:9/683: rename d3/d4/d11/d29/d38/d46/d6b/f90 to d3/d4/d11/fdd 0 2024-02-16T19:24:18.332 INFO:tasks.workunit.client.0.smithi040.stdout:9/684: chown d3/d1c/l45 243896 1 2024-02-16T19:24:18.335 INFO:tasks.workunit.client.0.smithi040.stdout:9/685: mkdir d3/d4/d11/d71/da8/dde 0 2024-02-16T19:24:18.337 INFO:tasks.workunit.client.1.smithi163.stdout:0/48: dread fd [0,4194304] 0 2024-02-16T19:24:18.349 INFO:tasks.workunit.client.1.smithi163.stdout:9/221: truncate d5/de/d16/f25 143611 0 2024-02-16T19:24:18.349 INFO:tasks.workunit.client.1.smithi163.stdout:9/222: chown d5/de/l22 0 1 2024-02-16T19:24:18.350 INFO:tasks.workunit.client.1.smithi163.stdout:4/32: getdents . 0 2024-02-16T19:24:18.351 INFO:tasks.workunit.client.1.smithi163.stdout:9/223: mkdir d5/de/d30/d31/d52 0 2024-02-16T19:24:18.363 INFO:tasks.workunit.client.1.smithi163.stdout:6/20: dwrite d0/f2 [4194304,4194304] 0 2024-02-16T19:24:18.363 INFO:tasks.workunit.client.1.smithi163.stdout:6/21: stat d0/c4 0 2024-02-16T19:24:18.364 INFO:tasks.workunit.client.1.smithi163.stdout:4/33: write f8 [3966113,102789] 0 2024-02-16T19:24:18.367 INFO:tasks.workunit.client.1.smithi163.stdout:9/224: dread d5/de/ff [4194304,4194304] 0 2024-02-16T19:24:18.368 INFO:tasks.workunit.client.1.smithi163.stdout:9/225: link d5/f14 d5/de/d16/f53 0 2024-02-16T19:24:18.376 INFO:tasks.workunit.client.0.smithi040.stdout:9/686: dwrite d3/d4/f96 [0,4194304] 0 2024-02-16T19:24:18.376 INFO:tasks.workunit.client.0.smithi040.stdout:9/687: read - d3/d4/d55/d63/fa5 zero size 2024-02-16T19:24:18.376 INFO:tasks.workunit.client.0.smithi040.stdout:9/688: fdatasync d3/d4/d4a/f62 0 2024-02-16T19:24:18.377 INFO:tasks.workunit.client.1.smithi163.stdout:0/49: dwrite f10 [0,4194304] 0 2024-02-16T19:24:18.377 INFO:tasks.workunit.client.1.smithi163.stdout:3/32: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:24:18.377 INFO:tasks.workunit.client.1.smithi163.stdout:6/22: read d0/f2 [6758794,106858] 0 2024-02-16T19:24:18.377 INFO:tasks.workunit.client.1.smithi163.stdout:6/23: creat d0/f5 x:0 0 0 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:8/714: sync 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:6/701: sync 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:4/528: sync 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:2/579: sync 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:0/726: sync 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:5/624: sync 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:5/625: chown d1/da/d1c/f65 733 1 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:1/600: sync 2024-02-16T19:24:18.382 INFO:tasks.workunit.client.0.smithi040.stdout:3/709: sync 2024-02-16T19:24:18.383 INFO:tasks.workunit.client.1.smithi163.stdout:9/226: dread d5/f2f [0,4194304] 0 2024-02-16T19:24:18.383 INFO:tasks.workunit.client.1.smithi163.stdout:2/150: dwrite d2/f15 [0,4194304] 0 2024-02-16T19:24:18.383 INFO:tasks.workunit.client.1.smithi163.stdout:9/227: truncate d5/f19 4916400 0 2024-02-16T19:24:18.383 INFO:tasks.workunit.client.0.smithi040.stdout:5/626: dread d1/da/d51/fbf [0,4194304] 0 2024-02-16T19:24:18.384 INFO:tasks.workunit.client.1.smithi163.stdout:3/33: symlink d0/l9 0 2024-02-16T19:24:18.384 INFO:tasks.workunit.client.1.smithi163.stdout:3/34: fdatasync d0/f6 0 2024-02-16T19:24:18.384 INFO:tasks.workunit.client.1.smithi163.stdout:3/35: chown d0/c7 507996 1 2024-02-16T19:24:18.384 INFO:tasks.workunit.client.1.smithi163.stdout:3/36: read d0/f2 [1740294,16394] 0 2024-02-16T19:24:18.385 INFO:tasks.workunit.client.0.smithi040.stdout:8/715: dread d6/d2b/da3/f3d [0,4194304] 0 2024-02-16T19:24:18.385 INFO:tasks.workunit.client.1.smithi163.stdout:4/34: dread f8 [0,4194304] 0 2024-02-16T19:24:18.388 INFO:tasks.workunit.client.1.smithi163.stdout:3/37: fdatasync d0/f2 0 2024-02-16T19:24:18.388 INFO:tasks.workunit.client.1.smithi163.stdout:3/38: readlink d0/l5 0 2024-02-16T19:24:18.390 INFO:tasks.workunit.client.0.smithi040.stdout:9/689: mknod d3/d4/d11/d71/dc9/cdf 0 2024-02-16T19:24:18.394 INFO:tasks.workunit.client.1.smithi163.stdout:6/24: mkdir d0/d6 0 2024-02-16T19:24:18.397 INFO:tasks.workunit.client.0.smithi040.stdout:4/529: creat dd/de/d45/fa7 x:0 0 0 2024-02-16T19:24:18.400 INFO:tasks.workunit.client.0.smithi040.stdout:2/580: symlink d5/d7/d10/dbc/lbf 0 2024-02-16T19:24:18.401 INFO:tasks.workunit.client.0.smithi040.stdout:6/702: symlink d0/d1/d12/d3b/d48/d73/d76/le8 0 2024-02-16T19:24:18.401 INFO:tasks.workunit.client.0.smithi040.stdout:6/703: read d0/d1/d12/d3b/f97 [232668,95845] 0 2024-02-16T19:24:18.401 INFO:tasks.workunit.client.0.smithi040.stdout:6/704: chown d0/d1/d12/d3b/f50 1245 1 2024-02-16T19:24:18.401 INFO:tasks.workunit.client.0.smithi040.stdout:6/705: chown d0/d1/d12/d3b/d48/d4f/f69 2758340 1 2024-02-16T19:24:18.401 INFO:tasks.workunit.client.0.smithi040.stdout:6/706: dread - d0/d1/d6f/fda zero size 2024-02-16T19:24:18.402 INFO:tasks.workunit.client.0.smithi040.stdout:0/727: symlink db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/lfb 0 2024-02-16T19:24:18.402 INFO:tasks.workunit.client.0.smithi040.stdout:0/728: write db/d2f/d23/d6f/f95 [2280655,61079] 0 2024-02-16T19:24:18.404 INFO:tasks.workunit.client.0.smithi040.stdout:1/601: unlink d17/d3c/d71/f87 0 2024-02-16T19:24:18.406 INFO:tasks.workunit.client.0.smithi040.stdout:1/602: dread d17/d26/d5f/d63/fa7 [0,4194304] 0 2024-02-16T19:24:18.406 INFO:tasks.workunit.client.0.smithi040.stdout:3/710: mkdir d1/dc/ddf 0 2024-02-16T19:24:18.406 INFO:tasks.workunit.client.0.smithi040.stdout:3/711: readlink d1/dc/d35/d53/d64/l73 0 2024-02-16T19:24:18.406 INFO:tasks.workunit.client.1.smithi163.stdout:2/151: rename d2/c25 to d2/c36 0 2024-02-16T19:24:18.409 INFO:tasks.workunit.client.0.smithi040.stdout:4/530: link dd/d58/l87 dd/de/d10/d36/la8 0 2024-02-16T19:24:18.409 INFO:tasks.workunit.client.0.smithi040.stdout:4/531: write dd/d58/d5e/f69 [4624692,19722] 0 2024-02-16T19:24:18.411 INFO:tasks.workunit.client.0.smithi040.stdout:0/729: mknod db/d2f/d23/d93/cfc 0 2024-02-16T19:24:18.411 INFO:tasks.workunit.client.0.smithi040.stdout:0/730: chown db/d2f/d23/d56/dbb 46 1 2024-02-16T19:24:18.411 INFO:tasks.workunit.client.1.smithi163.stdout:0/50: dwrite fd [0,4194304] 0 2024-02-16T19:24:18.413 INFO:tasks.workunit.client.1.smithi163.stdout:4/35: creat f9 x:0 0 0 2024-02-16T19:24:18.413 INFO:tasks.workunit.client.1.smithi163.stdout:4/36: chown f6 24246 1 2024-02-16T19:24:18.414 INFO:tasks.workunit.client.0.smithi040.stdout:9/690: getdents d3/d4/d11/d71/da2 0 2024-02-16T19:24:18.415 INFO:tasks.workunit.client.0.smithi040.stdout:1/603: dread d17/f2c [0,4194304] 0 2024-02-16T19:24:18.415 INFO:tasks.workunit.client.1.smithi163.stdout:3/39: mkdir d0/da 0 2024-02-16T19:24:18.415 INFO:tasks.workunit.client.1.smithi163.stdout:3/40: chown d0/da 802 1 2024-02-16T19:24:18.416 INFO:tasks.workunit.client.0.smithi040.stdout:4/532: getdents dd 0 2024-02-16T19:24:18.416 INFO:tasks.workunit.client.0.smithi040.stdout:6/707: write d0/d1/d6f/d90/fa8 [1732584,80451] 0 2024-02-16T19:24:18.418 INFO:tasks.workunit.client.0.smithi040.stdout:0/731: mknod db/d25/d2c/d52/cfd 0 2024-02-16T19:24:18.419 INFO:tasks.workunit.client.1.smithi163.stdout:2/152: creat d2/f37 x:0 0 0 2024-02-16T19:24:18.419 INFO:tasks.workunit.client.1.smithi163.stdout:2/153: write d2/d26/f28 [1313160,39445] 0 2024-02-16T19:24:18.419 INFO:tasks.workunit.client.1.smithi163.stdout:2/154: fdatasync d2/d26/f31 0 2024-02-16T19:24:18.420 INFO:tasks.workunit.client.0.smithi040.stdout:9/691: mkdir d3/d4/d55/d70/de0 0 2024-02-16T19:24:18.421 INFO:tasks.workunit.client.0.smithi040.stdout:8/716: dwrite d6/d2b/f95 [4194304,4194304] 0 2024-02-16T19:24:18.423 INFO:tasks.workunit.client.1.smithi163.stdout:9/228: truncate d5/de/d30/f3e 1857476 0 2024-02-16T19:24:18.423 INFO:tasks.workunit.client.1.smithi163.stdout:9/229: dread - d5/de/d30/d43/f4b zero size 2024-02-16T19:24:18.425 INFO:tasks.workunit.client.0.smithi040.stdout:5/627: dwrite d1/da/d1c/d21/f42 [0,4194304] 0 2024-02-16T19:24:18.425 INFO:tasks.workunit.client.0.smithi040.stdout:5/628: stat d1/da/d1c/d21/l54 0 2024-02-16T19:24:18.426 INFO:tasks.workunit.client.0.smithi040.stdout:1/604: link d17/d26/d4c/f8a d17/d26/d4c/dab/fbd 0 2024-02-16T19:24:18.426 INFO:tasks.workunit.client.0.smithi040.stdout:1/605: write d17/f28 [4484186,10302] 0 2024-02-16T19:24:18.426 INFO:tasks.workunit.client.0.smithi040.stdout:1/606: truncate d17/d26/d3d/f83 1757146 0 2024-02-16T19:24:18.426 INFO:tasks.workunit.client.0.smithi040.stdout:7/536: sync 2024-02-16T19:24:18.427 INFO:tasks.workunit.client.0.smithi040.stdout:7/537: creat d21/d4d/d77/d87/d9b/fa7 x:0 0 0 2024-02-16T19:24:18.429 INFO:tasks.workunit.client.0.smithi040.stdout:3/712: dwrite d1/d7/d30/f7f [0,4194304] 0 2024-02-16T19:24:18.431 INFO:tasks.workunit.client.0.smithi040.stdout:3/713: dread - d1/d4/d6e/dc2/dce/fd8 zero size 2024-02-16T19:24:18.432 INFO:tasks.workunit.client.1.smithi163.stdout:3/41: rmdir d0/da 0 2024-02-16T19:24:18.432 INFO:tasks.workunit.client.1.smithi163.stdout:3/42: creat d0/fb x:0 0 0 2024-02-16T19:24:18.432 INFO:tasks.workunit.client.1.smithi163.stdout:3/43: fsync d0/f6 0 2024-02-16T19:24:18.435 INFO:tasks.workunit.client.0.smithi040.stdout:4/533: rename dd/de/d10/d36/d9e/f90 to dd/de/d10/fa9 0 2024-02-16T19:24:18.438 INFO:tasks.workunit.client.1.smithi163.stdout:6/25: truncate d0/f2 2358387 0 2024-02-16T19:24:18.438 INFO:tasks.workunit.client.1.smithi163.stdout:0/51: dread f6 [0,4194304] 0 2024-02-16T19:24:18.438 INFO:tasks.workunit.client.1.smithi163.stdout:0/52: fsync ff 0 2024-02-16T19:24:18.439 INFO:tasks.workunit.client.1.smithi163.stdout:2/155: mkdir d2/d38 0 2024-02-16T19:24:18.439 INFO:tasks.workunit.client.1.smithi163.stdout:2/156: dread - d2/f37 zero size 2024-02-16T19:24:18.439 INFO:tasks.workunit.client.1.smithi163.stdout:2/157: readlink d2/l14 0 2024-02-16T19:24:18.440 INFO:tasks.workunit.client.0.smithi040.stdout:6/708: dwrite d0/d1/d12/d3b/d48/f9e [0,4194304] 0 2024-02-16T19:24:18.442 INFO:tasks.workunit.client.0.smithi040.stdout:8/717: truncate d6/d12/d93/fda 75373 0 2024-02-16T19:24:18.443 INFO:tasks.workunit.client.0.smithi040.stdout:3/714: dread d1/d4/f19 [0,4194304] 0 2024-02-16T19:24:18.448 INFO:tasks.workunit.client.1.smithi163.stdout:3/44: mkdir d0/dc 0 2024-02-16T19:24:18.450 INFO:tasks.workunit.client.0.smithi040.stdout:1/607: creat d17/d26/d5f/d78/dac/fbe x:0 0 0 2024-02-16T19:24:18.452 INFO:tasks.workunit.client.1.smithi163.stdout:0/53: mkdir d11 0 2024-02-16T19:24:18.455 INFO:tasks.workunit.client.1.smithi163.stdout:2/158: mkdir d2/d2a/d39 0 2024-02-16T19:24:18.455 INFO:tasks.workunit.client.1.smithi163.stdout:4/37: fsync f9 0 2024-02-16T19:24:18.459 INFO:tasks.workunit.client.1.smithi163.stdout:3/45: creat d0/dc/fd x:0 0 0 2024-02-16T19:24:18.459 INFO:tasks.workunit.client.1.smithi163.stdout:3/46: chown d0 11 1 2024-02-16T19:24:18.460 INFO:tasks.workunit.client.1.smithi163.stdout:3/47: truncate d0/dc/fd 373301 0 2024-02-16T19:24:18.460 INFO:tasks.workunit.client.1.smithi163.stdout:3/48: dread d0/dc/fd [0,4194304] 0 2024-02-16T19:24:18.461 INFO:tasks.workunit.client.1.smithi163.stdout:0/54: mkdir d11/d12 0 2024-02-16T19:24:18.461 INFO:tasks.workunit.client.0.smithi040.stdout:4/534: mkdir dd/de/daa 0 2024-02-16T19:24:18.461 INFO:tasks.workunit.client.0.smithi040.stdout:4/535: stat dd/d58/d8c/d99/d9d 0 2024-02-16T19:24:18.461 INFO:tasks.workunit.client.0.smithi040.stdout:4/536: chown dd/l78 9703 1 2024-02-16T19:24:18.461 INFO:tasks.workunit.client.0.smithi040.stdout:4/537: write dd/f1d [5235216,61516] 0 2024-02-16T19:24:18.465 INFO:tasks.workunit.client.1.smithi163.stdout:3/49: symlink d0/dc/le 0 2024-02-16T19:24:18.465 INFO:tasks.workunit.client.1.smithi163.stdout:3/50: dread - d0/f8 zero size 2024-02-16T19:24:18.465 INFO:tasks.workunit.client.1.smithi163.stdout:3/51: creat d0/dc/ff x:0 0 0 2024-02-16T19:24:18.467 INFO:tasks.workunit.client.0.smithi040.stdout:0/732: dwrite db/d2f/f66 [0,4194304] 0 2024-02-16T19:24:18.468 INFO:tasks.workunit.client.1.smithi163.stdout:0/55: symlink d11/d12/l13 0 2024-02-16T19:24:18.468 INFO:tasks.workunit.client.0.smithi040.stdout:3/715: creat d1/d4/dc4/fe0 x:0 0 0 2024-02-16T19:24:18.471 INFO:tasks.workunit.client.1.smithi163.stdout:3/52: mknod d0/dc/c10 0 2024-02-16T19:24:18.474 INFO:tasks.workunit.client.0.smithi040.stdout:4/538: chown dd/de/d45/c31 9 1 2024-02-16T19:24:18.478 INFO:tasks.workunit.client.0.smithi040.stdout:5/629: dwrite d1/d3c/f43 [0,4194304] 0 2024-02-16T19:24:18.478 INFO:tasks.workunit.client.0.smithi040.stdout:9/692: dwrite d3/fdc [0,4194304] 0 2024-02-16T19:24:18.478 INFO:tasks.workunit.client.0.smithi040.stdout:9/693: chown d3/d4/d11/d29/d38/d46/d6b/c92 70 1 2024-02-16T19:24:18.478 INFO:tasks.workunit.client.0.smithi040.stdout:7/538: dwrite d21/d29/d79/f7f [0,4194304] 0 2024-02-16T19:24:18.480 INFO:tasks.workunit.client.0.smithi040.stdout:0/733: write f7 [607029,25310] 0 2024-02-16T19:24:18.481 INFO:tasks.workunit.client.0.smithi040.stdout:8/718: dwrite d6/d12/d51/d6e/fc3 [0,4194304] 0 2024-02-16T19:24:18.481 INFO:tasks.workunit.client.0.smithi040.stdout:3/716: mkdir d1/dc/d35/d53/d64/de1 0 2024-02-16T19:24:18.482 INFO:tasks.workunit.client.0.smithi040.stdout:3/717: write d1/d7/d36/f7b [4941668,3487] 0 2024-02-16T19:24:18.483 INFO:tasks.workunit.client.1.smithi163.stdout:3/53: symlink d0/dc/l11 0 2024-02-16T19:24:18.489 INFO:tasks.workunit.client.1.smithi163.stdout:3/54: unlink d0/fb 0 2024-02-16T19:24:18.491 INFO:tasks.workunit.client.0.smithi040.stdout:9/694: truncate d3/d4/d55/d8a/d99/f9d 7193776 0 2024-02-16T19:24:18.495 INFO:tasks.workunit.client.1.smithi163.stdout:3/55: rename d0/dc/l11 to d0/l12 0 2024-02-16T19:24:18.497 INFO:tasks.workunit.client.0.smithi040.stdout:7/539: creat d21/d4d/d77/d7a/d99/fa8 x:0 0 0 2024-02-16T19:24:18.498 INFO:tasks.workunit.client.0.smithi040.stdout:0/734: creat db/d25/d2c/d52/ffe x:0 0 0 2024-02-16T19:24:18.509 INFO:tasks.workunit.client.1.smithi163.stdout:4/38: dwrite f9 [0,4194304] 0 2024-02-16T19:24:18.510 INFO:tasks.workunit.client.0.smithi040.stdout:8/719: rename d6/d12/d22/f68 to d6/d12/d51/d6e/fe2 0 2024-02-16T19:24:18.510 INFO:tasks.workunit.client.0.smithi040.stdout:8/720: readlink d6/d7/le 0 2024-02-16T19:24:18.511 INFO:tasks.workunit.client.0.smithi040.stdout:3/718: creat d1/dc/ddf/fe2 x:0 0 0 2024-02-16T19:24:18.512 INFO:tasks.workunit.client.0.smithi040.stdout:9/695: link d3/d4/d4a/f8c d3/d4/d55/d6d/daf/fe1 0 2024-02-16T19:24:18.515 INFO:tasks.workunit.client.0.smithi040.stdout:0/735: creat db/d2f/d1d/d21/d2e/fff x:0 0 0 2024-02-16T19:24:18.515 INFO:tasks.workunit.client.0.smithi040.stdout:1/608: dwrite f1 [0,4194304] 0 2024-02-16T19:24:18.518 INFO:tasks.workunit.client.0.smithi040.stdout:3/719: mkdir d1/dc/d42/d4a/d57/d79/de3 0 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:8/22: sync 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:5/15: sync 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/29: sync 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/30: fsync f1 0 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/31: creat f7 x:0 0 0 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:8/23: write d0/f6 [212399,93155] 0 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:8/24: fsync d0/f4 0 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:8/25: dread - d0/f5 zero size 2024-02-16T19:24:18.520 INFO:tasks.workunit.client.1.smithi163.stdout:8/26: dread - d0/f5 zero size 2024-02-16T19:24:18.521 INFO:tasks.workunit.client.1.smithi163.stdout:8/27: write d0/f6 [155044,88984] 0 2024-02-16T19:24:18.521 INFO:tasks.workunit.client.1.smithi163.stdout:8/28: fsync d0/f4 0 2024-02-16T19:24:18.521 INFO:tasks.workunit.client.1.smithi163.stdout:8/29: write d0/f4 [1004114,18648] 0 2024-02-16T19:24:18.521 INFO:tasks.workunit.client.1.smithi163.stdout:8/30: readlink d0/d7/l8 0 2024-02-16T19:24:18.521 INFO:tasks.workunit.client.0.smithi040.stdout:0/736: symlink db/de/d6a/l100 0 2024-02-16T19:24:18.521 INFO:tasks.workunit.client.0.smithi040.stdout:0/737: chown db/de/dc0 4 1 2024-02-16T19:24:18.521 INFO:tasks.workunit.client.0.smithi040.stdout:1/609: mknod d17/d43/d38/cbf 0 2024-02-16T19:24:18.524 INFO:tasks.workunit.client.1.smithi163.stdout:6/26: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:18.524 INFO:tasks.workunit.client.1.smithi163.stdout:5/16: mkdir d0/d2 0 2024-02-16T19:24:18.524 INFO:tasks.workunit.client.1.smithi163.stdout:7/32: dread f2 [0,4194304] 0 2024-02-16T19:24:18.524 INFO:tasks.workunit.client.1.smithi163.stdout:5/17: chown d0/f1 4203411 1 2024-02-16T19:24:18.524 INFO:tasks.workunit.client.1.smithi163.stdout:7/33: chown f7 3870 1 2024-02-16T19:24:18.524 INFO:tasks.workunit.client.1.smithi163.stdout:7/34: dread - f3 zero size 2024-02-16T19:24:18.524 INFO:tasks.workunit.client.1.smithi163.stdout:7/35: creat f8 x:0 0 0 2024-02-16T19:24:18.524 INFO:tasks.workunit.client.1.smithi163.stdout:7/36: dread - f5 zero size 2024-02-16T19:24:18.525 INFO:tasks.workunit.client.1.smithi163.stdout:7/37: write f3 [1046860,100566] 0 2024-02-16T19:24:18.525 INFO:tasks.workunit.client.1.smithi163.stdout:7/38: chown f8 16 1 2024-02-16T19:24:18.525 INFO:tasks.workunit.client.0.smithi040.stdout:8/721: dread f5 [0,4194304] 0 2024-02-16T19:24:18.525 INFO:tasks.workunit.client.0.smithi040.stdout:1/610: readlink d17/d26/d3d/l45 0 2024-02-16T19:24:18.526 INFO:tasks.workunit.client.0.smithi040.stdout:4/539: dwrite dd/de/f1a [0,4194304] 0 2024-02-16T19:24:18.528 INFO:tasks.workunit.client.0.smithi040.stdout:8/722: rename d6/d2b/d36/c6c to d6/d12/d51/d6e/db7/dc6/ce3 0 2024-02-16T19:24:18.528 INFO:tasks.workunit.client.0.smithi040.stdout:8/723: write d6/d7/f27 [4474266,87062] 0 2024-02-16T19:24:18.529 INFO:tasks.workunit.client.1.smithi163.stdout:9/230: read d5/f15 [2643,108890] 0 2024-02-16T19:24:18.529 INFO:tasks.workunit.client.1.smithi163.stdout:1/19: sync 2024-02-16T19:24:18.529 INFO:tasks.workunit.client.1.smithi163.stdout:6/27: mkdir d0/d7 0 2024-02-16T19:24:18.530 INFO:tasks.workunit.client.1.smithi163.stdout:5/18: unlink d0/f1 0 2024-02-16T19:24:18.530 INFO:tasks.workunit.client.0.smithi040.stdout:1/611: symlink d17/d3c/d74/d97/lc0 0 2024-02-16T19:24:18.530 INFO:tasks.workunit.client.0.smithi040.stdout:1/612: rename d17 to d17/d43/d92/daa/dc1 22 2024-02-16T19:24:18.531 INFO:tasks.workunit.client.1.smithi163.stdout:7/39: dread f1 [0,4194304] 0 2024-02-16T19:24:18.532 INFO:tasks.workunit.client.1.smithi163.stdout:4/39: dread f9 [0,4194304] 0 2024-02-16T19:24:18.533 INFO:tasks.workunit.client.1.smithi163.stdout:4/40: fdatasync f8 0 2024-02-16T19:24:18.533 INFO:tasks.workunit.client.1.smithi163.stdout:5/19: creat d0/d2/f3 x:0 0 0 2024-02-16T19:24:18.534 INFO:tasks.workunit.client.0.smithi040.stdout:4/540: creat dd/de/daa/fab x:0 0 0 2024-02-16T19:24:18.535 INFO:tasks.workunit.client.1.smithi163.stdout:3/56: dwrite d0/f1 [0,4194304] 0 2024-02-16T19:24:18.535 INFO:tasks.workunit.client.1.smithi163.stdout:0/56: dwrite f8 [0,4194304] 0 2024-02-16T19:24:18.536 INFO:tasks.workunit.client.1.smithi163.stdout:3/57: dread - d0/f6 zero size 2024-02-16T19:24:18.536 INFO:tasks.workunit.client.0.smithi040.stdout:7/540: dwrite d21/d29/d79/f7c [0,4194304] 0 2024-02-16T19:24:18.536 INFO:tasks.workunit.client.1.smithi163.stdout:2/159: dwrite d2/f9 [0,4194304] 0 2024-02-16T19:24:18.537 INFO:tasks.workunit.client.1.smithi163.stdout:2/160: dread - d2/f1e zero size 2024-02-16T19:24:18.537 INFO:tasks.workunit.client.1.smithi163.stdout:2/161: chown d2/c22 242 1 2024-02-16T19:24:18.539 INFO:tasks.workunit.client.0.smithi040.stdout:5/630: dwrite d1/da/da4/fa6 [0,4194304] 0 2024-02-16T19:24:18.539 INFO:tasks.workunit.client.1.smithi163.stdout:7/40: unlink f1 0 2024-02-16T19:24:18.539 INFO:tasks.workunit.client.1.smithi163.stdout:7/41: creat f9 x:0 0 0 2024-02-16T19:24:18.539 INFO:tasks.workunit.client.1.smithi163.stdout:7/42: write f5 [1047829,100857] 0 2024-02-16T19:24:18.541 INFO:tasks.workunit.client.0.smithi040.stdout:8/724: mknod d6/d2b/da3/ce4 0 2024-02-16T19:24:18.542 INFO:tasks.workunit.client.0.smithi040.stdout:3/720: dwrite d1/dc/d42/d4a/f61 [0,4194304] 0 2024-02-16T19:24:18.542 INFO:tasks.workunit.client.0.smithi040.stdout:3/721: readlink d1/d4/d6e/d8a/l94 0 2024-02-16T19:24:18.543 INFO:tasks.workunit.client.0.smithi040.stdout:3/722: dread d1/dc/d35/d48/fc7 [0,4194304] 0 2024-02-16T19:24:18.548 INFO:tasks.workunit.client.1.smithi163.stdout:9/231: dread d5/de/d30/f36 [0,4194304] 0 2024-02-16T19:24:18.548 INFO:tasks.workunit.client.1.smithi163.stdout:9/232: creat d5/de/d30/d31/f54 x:0 0 0 2024-02-16T19:24:18.548 INFO:tasks.workunit.client.1.smithi163.stdout:9/233: fdatasync d5/f1e 0 2024-02-16T19:24:18.550 INFO:tasks.workunit.client.1.smithi163.stdout:0/57: rename lb to d11/l14 0 2024-02-16T19:24:18.550 INFO:tasks.workunit.client.1.smithi163.stdout:0/58: stat f3 0 2024-02-16T19:24:18.550 INFO:tasks.workunit.client.1.smithi163.stdout:0/59: truncate f8 4912779 0 2024-02-16T19:24:18.550 INFO:tasks.workunit.client.1.smithi163.stdout:0/60: chown d11 19312645 1 2024-02-16T19:24:18.552 INFO:tasks.workunit.client.0.smithi040.stdout:4/541: getdents dd/de 0 2024-02-16T19:24:18.552 INFO:tasks.workunit.client.0.smithi040.stdout:4/542: fsync dd/f19 0 2024-02-16T19:24:18.552 INFO:tasks.workunit.client.0.smithi040.stdout:4/543: stat dd/d58/d8c/d99 0 2024-02-16T19:24:18.552 INFO:tasks.workunit.client.0.smithi040.stdout:4/544: readlink dd/d58/d68/la5 0 2024-02-16T19:24:18.556 INFO:tasks.workunit.client.1.smithi163.stdout:2/162: mkdir d2/d2a/d39/d3a 0 2024-02-16T19:24:18.556 INFO:tasks.workunit.client.1.smithi163.stdout:2/163: fdatasync d2/f15 0 2024-02-16T19:24:18.556 INFO:tasks.workunit.client.1.smithi163.stdout:2/164: write d2/f18 [5000686,36938] 0 2024-02-16T19:24:18.556 INFO:tasks.workunit.client.1.smithi163.stdout:2/165: read - d2/d26/f2f zero size 2024-02-16T19:24:18.556 INFO:tasks.workunit.client.1.smithi163.stdout:2/166: chown d2/d2a/c2c 983014613 1 2024-02-16T19:24:18.557 INFO:tasks.workunit.client.0.smithi040.stdout:3/723: write d1/dc/d42/d4a/f54 [1220158,91672] 0 2024-02-16T19:24:18.558 INFO:tasks.workunit.client.0.smithi040.stdout:7/541: creat d21/d4d/d77/d87/fa9 x:0 0 0 2024-02-16T19:24:18.559 INFO:tasks.workunit.client.0.smithi040.stdout:8/725: dread d6/d2b/f34 [0,4194304] 0 2024-02-16T19:24:18.559 INFO:tasks.workunit.client.0.smithi040.stdout:8/726: fdatasync d6/d2b/d44/fb4 0 2024-02-16T19:24:18.562 INFO:tasks.workunit.client.1.smithi163.stdout:7/43: link f7 fa 0 2024-02-16T19:24:18.565 INFO:tasks.workunit.client.0.smithi040.stdout:5/631: creat d1/fc3 x:0 0 0 2024-02-16T19:24:18.567 INFO:tasks.workunit.client.1.smithi163.stdout:9/234: creat d5/de/d30/d43/f55 x:0 0 0 2024-02-16T19:24:18.570 INFO:tasks.workunit.client.1.smithi163.stdout:5/20: dwrite d0/d2/f3 [0,4194304] 0 2024-02-16T19:24:18.570 INFO:tasks.workunit.client.1.smithi163.stdout:5/21: creat d0/d2/f4 x:0 0 0 2024-02-16T19:24:18.570 INFO:tasks.workunit.client.1.smithi163.stdout:5/22: chown d0/d2 387114 1 2024-02-16T19:24:18.570 INFO:tasks.workunit.client.1.smithi163.stdout:6/28: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:24:18.570 INFO:tasks.workunit.client.1.smithi163.stdout:6/29: chown d0 0 1 2024-02-16T19:24:18.570 INFO:tasks.workunit.client.1.smithi163.stdout:1/20: dwrite f0 [0,4194304] 0 2024-02-16T19:24:18.571 INFO:tasks.workunit.client.1.smithi163.stdout:8/31: dwrite d0/f4 [0,4194304] 0 2024-02-16T19:24:18.571 INFO:tasks.workunit.client.0.smithi040.stdout:0/738: dwrite db/d2f/d23/d56/fb6 [0,4194304] 0 2024-02-16T19:24:18.571 INFO:tasks.workunit.client.1.smithi163.stdout:3/58: rename d0/dc/ff to d0/f13 0 2024-02-16T19:24:18.571 INFO:tasks.workunit.client.1.smithi163.stdout:3/59: creat d0/f14 x:0 0 0 2024-02-16T19:24:18.573 INFO:tasks.workunit.client.0.smithi040.stdout:4/545: unlink dd/de/d10/f62 0 2024-02-16T19:24:18.575 INFO:tasks.workunit.client.1.smithi163.stdout:8/32: dread d0/f6 [0,4194304] 0 2024-02-16T19:24:18.575 INFO:tasks.workunit.client.1.smithi163.stdout:2/167: creat d2/d2a/d39/f3b x:0 0 0 2024-02-16T19:24:18.578 INFO:tasks.workunit.client.0.smithi040.stdout:3/724: unlink d1/d4/d6e/d8a/da5/fbd 0 2024-02-16T19:24:18.581 INFO:tasks.workunit.client.1.smithi163.stdout:2/168: dread d2/d26/f28 [0,4194304] 0 2024-02-16T19:24:18.581 INFO:tasks.workunit.client.1.smithi163.stdout:7/44: rename f8 to fb 0 2024-02-16T19:24:18.585 INFO:tasks.workunit.client.1.smithi163.stdout:9/235: mknod d5/de/d30/d31/c56 0 2024-02-16T19:24:18.587 INFO:tasks.workunit.client.0.smithi040.stdout:8/727: unlink d6/d7/le 0 2024-02-16T19:24:18.587 INFO:tasks.workunit.client.0.smithi040.stdout:8/728: truncate d6/d2b/d36/f3e 124195 0 2024-02-16T19:24:18.591 INFO:tasks.workunit.client.0.smithi040.stdout:5/632: unlink d1/d62/c89 0 2024-02-16T19:24:18.603 INFO:tasks.workunit.client.1.smithi163.stdout:5/23: mkdir d0/d5 0 2024-02-16T19:24:18.603 INFO:tasks.workunit.client.1.smithi163.stdout:5/24: fdatasync d0/d2/f4 0 2024-02-16T19:24:18.606 INFO:tasks.workunit.client.0.smithi040.stdout:1/613: rmdir d17/d3c/d74/d97 39 2024-02-16T19:24:18.607 INFO:tasks.workunit.client.0.smithi040.stdout:3/725: mknod d1/d4/d6e/dc2/dce/ce4 0 2024-02-16T19:24:18.610 INFO:tasks.workunit.client.0.smithi040.stdout:7/542: rename d21/d29/f59 to d21/d7e/faa 0 2024-02-16T19:24:18.610 INFO:tasks.workunit.client.0.smithi040.stdout:7/543: stat d21/f5d 0 2024-02-16T19:24:18.612 INFO:tasks.workunit.client.0.smithi040.stdout:1/614: creat d17/d43/d92/fc2 x:0 0 0 2024-02-16T19:24:18.612 INFO:tasks.workunit.client.1.smithi163.stdout:3/60: dwrite d0/f13 [0,4194304] 0 2024-02-16T19:24:18.614 INFO:tasks.workunit.client.0.smithi040.stdout:7/544: symlink d21/d8b/d9c/lab 0 2024-02-16T19:24:18.617 INFO:tasks.workunit.client.0.smithi040.stdout:1/615: creat d17/d26/d5f/fc3 x:0 0 0 2024-02-16T19:24:18.617 INFO:tasks.workunit.client.0.smithi040.stdout:1/616: fsync d17/d29/f32 0 2024-02-16T19:24:18.617 INFO:tasks.workunit.client.0.smithi040.stdout:1/617: readlink d17/d26/d3d/l45 0 2024-02-16T19:24:18.618 INFO:tasks.workunit.client.0.smithi040.stdout:3/726: read d1/dc/d42/d4a/f50 [689458,968] 0 2024-02-16T19:24:18.618 INFO:tasks.workunit.client.0.smithi040.stdout:7/545: write f1f [305962,122103] 0 2024-02-16T19:24:18.620 INFO:tasks.workunit.client.0.smithi040.stdout:1/618: getdents d17/d43 0 2024-02-16T19:24:18.620 INFO:tasks.workunit.client.0.smithi040.stdout:1/619: write d17/d26/fb9 [612487,30428] 0 2024-02-16T19:24:18.624 INFO:tasks.workunit.client.1.smithi163.stdout:2/169: rmdir d2/d38 0 2024-02-16T19:24:18.626 INFO:tasks.workunit.client.1.smithi163.stdout:9/236: dwrite d5/de/d16/f3c [0,4194304] 0 2024-02-16T19:24:18.627 INFO:tasks.workunit.client.0.smithi040.stdout:4/546: dwrite dd/de/f12 [0,4194304] 0 2024-02-16T19:24:18.627 INFO:tasks.workunit.client.0.smithi040.stdout:4/547: dread - dd/de/d71/f9a zero size 2024-02-16T19:24:18.627 INFO:tasks.workunit.client.0.smithi040.stdout:4/548: write dd/de/d10/d36/f86 [17251,60916] 0 2024-02-16T19:24:18.627 INFO:tasks.workunit.client.0.smithi040.stdout:4/549: getdents dd/de 0 2024-02-16T19:24:18.627 INFO:tasks.workunit.client.0.smithi040.stdout:0/739: dwrite db/d25/d2c/d52/ffe [0,4194304] 0 2024-02-16T19:24:18.627 INFO:tasks.workunit.client.0.smithi040.stdout:8/729: dwrite d6/d2b/da3/d8b/f90 [0,4194304] 0 2024-02-16T19:24:18.627 INFO:tasks.workunit.client.0.smithi040.stdout:7/546: truncate f11 3638453 0 2024-02-16T19:24:18.627 INFO:tasks.workunit.client.1.smithi163.stdout:8/33: dwrite d0/f4 [0,4194304] 0 2024-02-16T19:24:18.628 INFO:tasks.workunit.client.0.smithi040.stdout:5/633: dwrite d1/da/d1c/d21/f36 [0,4194304] 0 2024-02-16T19:24:18.629 INFO:tasks.workunit.client.0.smithi040.stdout:1/620: symlink d17/d26/d5f/da6/lc4 0 2024-02-16T19:24:18.629 INFO:tasks.workunit.client.0.smithi040.stdout:1/621: fsync d17/d26/d5f/d78/d7e/fbc 0 2024-02-16T19:24:18.631 INFO:tasks.workunit.client.0.smithi040.stdout:3/727: truncate d1/d7/d33/f5e 1037741 0 2024-02-16T19:24:18.632 INFO:tasks.workunit.client.0.smithi040.stdout:3/728: chown d1/d4/ld7 14775848 1 2024-02-16T19:24:18.635 INFO:tasks.workunit.client.0.smithi040.stdout:4/550: link dd/de/d10/d36/d9e/c4b dd/de/d71/cac 0 2024-02-16T19:24:18.637 INFO:tasks.workunit.client.1.smithi163.stdout:3/61: symlink d0/l15 0 2024-02-16T19:24:18.640 INFO:tasks.workunit.client.0.smithi040.stdout:7/547: dread d21/f48 [0,4194304] 0 2024-02-16T19:24:18.641 INFO:tasks.workunit.client.1.smithi163.stdout:2/170: mkdir d2/d2a/d2d/d3c 0 2024-02-16T19:24:18.644 INFO:tasks.workunit.client.0.smithi040.stdout:0/740: creat db/d2f/f101 x:0 0 0 2024-02-16T19:24:18.644 INFO:tasks.workunit.client.0.smithi040.stdout:0/741: readlink db/d2f/d23/l48 0 2024-02-16T19:24:18.647 INFO:tasks.workunit.client.0.smithi040.stdout:1/622: dwrite d17/d43/d38/f7b [0,4194304] 0 2024-02-16T19:24:18.649 INFO:tasks.workunit.client.1.smithi163.stdout:3/62: write d0/f1 [603173,1456] 0 2024-02-16T19:24:18.651 INFO:tasks.workunit.client.0.smithi040.stdout:8/730: unlink d6/d64/d50/f7d 0 2024-02-16T19:24:18.655 INFO:tasks.workunit.client.0.smithi040.stdout:5/634: dwrite d1/f1d [0,4194304] 0 2024-02-16T19:24:18.655 INFO:tasks.workunit.client.0.smithi040.stdout:5/635: write d1/da/d51/f85 [891716,117551] 0 2024-02-16T19:24:18.658 INFO:tasks.workunit.client.1.smithi163.stdout:9/237: mknod d5/de/d16/c57 0 2024-02-16T19:24:18.660 INFO:tasks.workunit.client.1.smithi163.stdout:7/45: unlink f7 0 2024-02-16T19:24:18.660 INFO:tasks.workunit.client.1.smithi163.stdout:7/46: chown f2 724312 1 2024-02-16T19:24:18.663 INFO:tasks.workunit.client.1.smithi163.stdout:6/30: truncate d0/f5 2186696 0 2024-02-16T19:24:18.663 INFO:tasks.workunit.client.1.smithi163.stdout:6/31: fsync d0/f2 0 2024-02-16T19:24:18.663 INFO:tasks.workunit.client.1.smithi163.stdout:6/32: getdents d0/d7 0 2024-02-16T19:24:18.665 INFO:tasks.workunit.client.1.smithi163.stdout:5/25: getdents d0/d2 0 2024-02-16T19:24:18.668 INFO:tasks.workunit.client.0.smithi040.stdout:3/729: symlink d1/d8c/le5 0 2024-02-16T19:24:18.668 INFO:tasks.workunit.client.0.smithi040.stdout:3/730: dread - d1/d4/d6e/d8a/da5/fd6 zero size 2024-02-16T19:24:18.669 INFO:tasks.workunit.client.1.smithi163.stdout:3/63: symlink d0/l16 0 2024-02-16T19:24:18.670 INFO:tasks.workunit.client.1.smithi163.stdout:3/64: creat d0/dc/f17 x:0 0 0 2024-02-16T19:24:18.670 INFO:tasks.workunit.client.1.smithi163.stdout:3/65: stat d0/dc 0 2024-02-16T19:24:18.670 INFO:tasks.workunit.client.0.smithi040.stdout:4/551: mkdir dd/de/d10/d36/dad 0 2024-02-16T19:24:18.672 INFO:tasks.workunit.client.0.smithi040.stdout:7/548: symlink d21/d8b/d9c/lac 0 2024-02-16T19:24:18.674 INFO:tasks.workunit.client.0.smithi040.stdout:0/742: symlink db/d25/d34/d5e/l102 0 2024-02-16T19:24:18.678 INFO:tasks.workunit.client.1.smithi163.stdout:9/238: dread d5/f18 [0,4194304] 0 2024-02-16T19:24:18.678 INFO:tasks.workunit.client.1.smithi163.stdout:7/47: dwrite f5 [0,4194304] 0 2024-02-16T19:24:18.678 INFO:tasks.workunit.client.1.smithi163.stdout:7/48: chown fb 3 1 2024-02-16T19:24:18.678 INFO:tasks.workunit.client.1.smithi163.stdout:7/49: creat fc x:0 0 0 2024-02-16T19:24:18.678 INFO:tasks.workunit.client.1.smithi163.stdout:7/50: truncate fb 17274 0 2024-02-16T19:24:18.678 INFO:tasks.workunit.client.1.smithi163.stdout:7/51: stat f9 0 2024-02-16T19:24:18.679 INFO:tasks.workunit.client.1.smithi163.stdout:5/26: link d0/d2/f4 d0/f6 0 2024-02-16T19:24:18.680 INFO:tasks.workunit.client.1.smithi163.stdout:5/27: chown d0/d2/f4 3399 1 2024-02-16T19:24:18.681 INFO:tasks.workunit.client.0.smithi040.stdout:5/636: truncate d1/da/d51/fbf 986429 0 2024-02-16T19:24:18.685 INFO:tasks.workunit.client.0.smithi040.stdout:3/731: rmdir d1/d8c/d8e 39 2024-02-16T19:24:18.685 INFO:tasks.workunit.client.0.smithi040.stdout:3/732: write d1/dc/d35/d53/fa9 [478468,94830] 0 2024-02-16T19:24:18.685 INFO:tasks.workunit.client.0.smithi040.stdout:3/733: truncate d1/d7/d30/f81 997008 0 2024-02-16T19:24:18.686 INFO:tasks.workunit.client.1.smithi163.stdout:3/66: mknod d0/c18 0 2024-02-16T19:24:18.686 INFO:tasks.workunit.client.1.smithi163.stdout:3/67: write d0/dc/f17 [233102,126128] 0 2024-02-16T19:24:18.687 INFO:tasks.workunit.client.1.smithi163.stdout:3/68: write d0/f8 [490288,55455] 0 2024-02-16T19:24:18.687 INFO:tasks.workunit.client.0.smithi040.stdout:1/623: rmdir d17/d3c/d74 39 2024-02-16T19:24:18.688 INFO:tasks.workunit.client.1.smithi163.stdout:3/69: fsync d0/f8 0 2024-02-16T19:24:18.688 INFO:tasks.workunit.client.1.smithi163.stdout:3/70: creat d0/f19 x:0 0 0 2024-02-16T19:24:18.689 INFO:tasks.workunit.client.0.smithi040.stdout:4/552: link fb dd/d58/d8c/fae 0 2024-02-16T19:24:18.691 INFO:tasks.workunit.client.1.smithi163.stdout:9/239: link d5/f9 d5/de/d30/d3d/d49/f58 0 2024-02-16T19:24:18.691 INFO:tasks.workunit.client.1.smithi163.stdout:9/240: fdatasync d5/f2f 0 2024-02-16T19:24:18.691 INFO:tasks.workunit.client.1.smithi163.stdout:9/241: truncate d5/de/f28 1043682 0 2024-02-16T19:24:18.692 INFO:tasks.workunit.client.0.smithi040.stdout:7/549: mkdir d21/d29/dad 0 2024-02-16T19:24:18.693 INFO:tasks.workunit.client.0.smithi040.stdout:0/743: mkdir db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/d103 0 2024-02-16T19:24:18.694 INFO:tasks.workunit.client.0.smithi040.stdout:0/744: readlink db/d2f/d1d/d21/lf9 0 2024-02-16T19:24:18.696 INFO:tasks.workunit.client.0.smithi040.stdout:1/624: dread f1 [0,4194304] 0 2024-02-16T19:24:18.699 INFO:tasks.workunit.client.0.smithi040.stdout:3/734: creat d1/dc/d35/d53/d64/fe6 x:0 0 0 2024-02-16T19:24:18.699 INFO:tasks.workunit.client.0.smithi040.stdout:3/735: readlink d1/d7/d30/ld4 0 2024-02-16T19:24:18.700 INFO:tasks.workunit.client.1.smithi163.stdout:7/52: mknod cd 0 2024-02-16T19:24:18.701 INFO:tasks.workunit.client.1.smithi163.stdout:5/28: creat d0/d2/f7 x:0 0 0 2024-02-16T19:24:18.703 INFO:tasks.workunit.client.0.smithi040.stdout:4/553: mknod dd/caf 0 2024-02-16T19:24:18.703 INFO:tasks.workunit.client.0.smithi040.stdout:4/554: truncate dd/de/d10/d36/d9e/d77/f7b 1697802 0 2024-02-16T19:24:18.703 INFO:tasks.workunit.client.0.smithi040.stdout:4/555: write dd/de/d10/d36/d9e/d32/f64 [363913,127605] 0 2024-02-16T19:24:18.705 INFO:tasks.workunit.client.0.smithi040.stdout:9/696: sync 2024-02-16T19:24:18.705 INFO:tasks.workunit.client.0.smithi040.stdout:2/581: sync 2024-02-16T19:24:18.705 INFO:tasks.workunit.client.0.smithi040.stdout:6/709: sync 2024-02-16T19:24:18.705 INFO:tasks.workunit.client.0.smithi040.stdout:9/697: chown d3/d4/d11/d29/d38/d46/d6b/d85/fbb 20 1 2024-02-16T19:24:18.706 INFO:tasks.workunit.client.0.smithi040.stdout:7/550: dwrite d21/d29/f37 [4194304,4194304] 0 2024-02-16T19:24:18.707 INFO:tasks.workunit.client.1.smithi163.stdout:8/34: truncate d0/f4 1071401 0 2024-02-16T19:24:18.707 INFO:tasks.workunit.client.1.smithi163.stdout:8/35: fdatasync d0/f5 0 2024-02-16T19:24:18.708 INFO:tasks.workunit.client.1.smithi163.stdout:6/33: fsync d0/f5 0 2024-02-16T19:24:18.710 INFO:tasks.workunit.client.0.smithi040.stdout:1/625: creat d17/d3c/d74/d97/fc5 x:0 0 0 2024-02-16T19:24:18.710 INFO:tasks.workunit.client.0.smithi040.stdout:1/626: write d17/d29/d8b/d93/f96 [139614,58188] 0 2024-02-16T19:24:18.710 INFO:tasks.workunit.client.1.smithi163.stdout:6/34: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:18.714 INFO:tasks.workunit.client.0.smithi040.stdout:2/582: write d5/d7/d1e/d4c/d52/f9c [3154084,106945] 0 2024-02-16T19:24:18.714 INFO:tasks.workunit.client.0.smithi040.stdout:2/583: creat d5/d5c/d65/d9d/fc0 x:0 0 0 2024-02-16T19:24:18.715 INFO:tasks.workunit.client.1.smithi163.stdout:9/242: getdents d5 0 2024-02-16T19:24:18.715 INFO:tasks.workunit.client.1.smithi163.stdout:9/243: stat d5/de/d16/d47/l48 0 2024-02-16T19:24:18.716 INFO:tasks.workunit.client.0.smithi040.stdout:2/584: dread d5/d5c/d65/d9d/fbd [0,4194304] 0 2024-02-16T19:24:18.716 INFO:tasks.workunit.client.0.smithi040.stdout:2/585: chown d5/d7/d89/cad 65 1 2024-02-16T19:24:18.721 INFO:tasks.workunit.client.0.smithi040.stdout:3/736: getdents d1/dc/d42/d4a/d57 0 2024-02-16T19:24:18.721 INFO:tasks.workunit.client.1.smithi163.stdout:2/171: write d2/f1c [2060027,469] 0 2024-02-16T19:24:18.722 INFO:tasks.workunit.client.0.smithi040.stdout:1/627: write d17/d2d/f2e [1053564,69237] 0 2024-02-16T19:24:18.728 INFO:tasks.workunit.client.0.smithi040.stdout:4/556: unlink dd/d58/c9b 0 2024-02-16T19:24:18.728 INFO:tasks.workunit.client.0.smithi040.stdout:4/557: dread - dd/de/d71/fa3 zero size 2024-02-16T19:24:18.728 INFO:tasks.workunit.client.0.smithi040.stdout:4/558: fsync dd/de/d71/f9a 0 2024-02-16T19:24:18.731 INFO:tasks.workunit.client.1.smithi163.stdout:6/35: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:18.742 INFO:tasks.workunit.client.1.smithi163.stdout:2/172: dwrite d2/f5 [0,4194304] 0 2024-02-16T19:24:18.742 INFO:tasks.workunit.client.1.smithi163.stdout:2/173: stat d2/f19 0 2024-02-16T19:24:18.742 INFO:tasks.workunit.client.1.smithi163.stdout:2/174: stat d2/f23 0 2024-02-16T19:24:18.742 INFO:tasks.workunit.client.1.smithi163.stdout:2/175: creat d2/d26/f3d x:0 0 0 2024-02-16T19:24:18.742 INFO:tasks.workunit.client.1.smithi163.stdout:2/176: write d2/d26/f2f [903935,16873] 0 2024-02-16T19:24:18.742 INFO:tasks.workunit.client.0.smithi040.stdout:5/637: dwrite d1/da/d51/fbf [0,4194304] 0 2024-02-16T19:24:18.752 INFO:tasks.workunit.client.0.smithi040.stdout:5/638: dread d1/fd [0,4194304] 0 2024-02-16T19:24:18.752 INFO:tasks.workunit.client.0.smithi040.stdout:5/639: creat d1/d45/dab/fc4 x:0 0 0 2024-02-16T19:24:18.758 INFO:tasks.workunit.client.1.smithi163.stdout:2/177: dwrite d2/f9 [0,4194304] 0 2024-02-16T19:24:18.761 INFO:tasks.workunit.client.1.smithi163.stdout:2/178: dread d2/d26/f27 [0,4194304] 0 2024-02-16T19:24:18.761 INFO:tasks.workunit.client.1.smithi163.stdout:2/179: readlink d2/l14 0 2024-02-16T19:24:18.762 INFO:tasks.workunit.client.1.smithi163.stdout:2/180: dread d2/d26/f31 [4194304,4194304] 0 2024-02-16T19:24:18.796 INFO:tasks.workunit.client.0.smithi040.stdout:6/710: link d0/d37/de1/dac/fb3 d0/d30/d32/dd2/dbc/fe9 0 2024-02-16T19:24:18.798 INFO:tasks.workunit.client.0.smithi040.stdout:9/698: mkdir d3/d4/d11/d71/da2/de2 0 2024-02-16T19:24:18.798 INFO:tasks.workunit.client.0.smithi040.stdout:9/699: chown d3/d4/d55/d70/dcc 1741 1 2024-02-16T19:24:18.876 INFO:tasks.workunit.client.1.smithi163.stdout:5/29: rmdir d0/d5 0 2024-02-16T19:24:18.878 INFO:tasks.workunit.client.1.smithi163.stdout:8/36: getdents d0 0 2024-02-16T19:24:18.880 INFO:tasks.workunit.client.0.smithi040.stdout:7/551: symlink d21/d4d/d77/d87/lae 0 2024-02-16T19:24:18.883 INFO:tasks.workunit.client.1.smithi163.stdout:7/53: link l6 le 0 2024-02-16T19:24:18.895 INFO:tasks.workunit.client.1.smithi163.stdout:6/36: creat d0/d6/f8 x:0 0 0 2024-02-16T19:24:18.895 INFO:tasks.workunit.client.0.smithi040.stdout:2/586: truncate d5/d5c/d65/d9d/fbd 658849 0 2024-02-16T19:24:18.897 INFO:tasks.workunit.client.0.smithi040.stdout:2/587: dread d5/d8/d42/f7a [0,4194304] 0 2024-02-16T19:24:18.902 INFO:tasks.workunit.client.1.smithi163.stdout:0/61: sync 2024-02-16T19:24:18.902 INFO:tasks.workunit.client.1.smithi163.stdout:1/21: sync 2024-02-16T19:24:18.902 INFO:tasks.workunit.client.1.smithi163.stdout:4/41: sync 2024-02-16T19:24:18.904 INFO:tasks.workunit.client.0.smithi040.stdout:7/552: dwrite d21/f76 [4194304,4194304] 0 2024-02-16T19:24:18.906 INFO:tasks.workunit.client.0.smithi040.stdout:0/745: dwrite db/d2f/d1d/d21/d2e/d49/f87 [0,4194304] 0 2024-02-16T19:24:18.912 INFO:tasks.workunit.client.1.smithi163.stdout:4/42: dread f2 [0,4194304] 0 2024-02-16T19:24:18.912 INFO:tasks.workunit.client.1.smithi163.stdout:4/43: dread - f6 zero size 2024-02-16T19:24:18.915 INFO:tasks.workunit.client.1.smithi163.stdout:1/22: dread f2 [0,4194304] 0 2024-02-16T19:24:18.917 INFO:tasks.workunit.client.0.smithi040.stdout:5/640: mkdir d1/d62/d9a/dc5 0 2024-02-16T19:24:18.917 INFO:tasks.workunit.client.0.smithi040.stdout:5/641: getdents d1/d62/dbb 0 2024-02-16T19:24:18.917 INFO:tasks.workunit.client.0.smithi040.stdout:5/642: chown d1/d62/d9a/fa2 6675750 1 2024-02-16T19:24:18.919 INFO:tasks.workunit.client.1.smithi163.stdout:2/181: symlink d2/l3e 0 2024-02-16T19:24:18.919 INFO:tasks.workunit.client.0.smithi040.stdout:7/553: write d21/d29/d79/f7c [3246294,45094] 0 2024-02-16T19:24:18.920 INFO:tasks.workunit.client.0.smithi040.stdout:7/554: fsync d21/d29/f2b 0 2024-02-16T19:24:18.920 INFO:tasks.workunit.client.0.smithi040.stdout:7/555: chown d21/d7e 37163 1 2024-02-16T19:24:18.922 INFO:tasks.workunit.client.0.smithi040.stdout:6/711: mknod d0/d37/de1/cea 0 2024-02-16T19:24:18.922 INFO:tasks.workunit.client.0.smithi040.stdout:6/712: write d0/d30/d4b/f93 [660176,60672] 0 2024-02-16T19:24:18.922 INFO:tasks.workunit.client.1.smithi163.stdout:2/182: dread d2/f1c [0,4194304] 0 2024-02-16T19:24:18.927 INFO:tasks.workunit.client.0.smithi040.stdout:9/700: unlink d3/d4/d55/c7b 0 2024-02-16T19:24:18.929 INFO:tasks.workunit.client.1.smithi163.stdout:5/30: symlink d0/l8 0 2024-02-16T19:24:18.932 INFO:tasks.workunit.client.1.smithi163.stdout:8/37: rename d0/f5 to d0/f9 0 2024-02-16T19:24:18.934 INFO:tasks.workunit.client.0.smithi040.stdout:2/588: mkdir d5/d7/d1e/d4c/d52/dc1 0 2024-02-16T19:24:18.935 INFO:tasks.workunit.client.1.smithi163.stdout:7/54: creat ff x:0 0 0 2024-02-16T19:24:18.937 INFO:tasks.workunit.client.1.smithi163.stdout:9/244: getdents d5/de/d30/d31 0 2024-02-16T19:24:18.937 INFO:tasks.workunit.client.1.smithi163.stdout:9/245: chown d5/l27 2567 1 2024-02-16T19:24:18.939 INFO:tasks.workunit.client.1.smithi163.stdout:6/37: rename d0/d7 to d0/d6/d9 0 2024-02-16T19:24:18.939 INFO:tasks.workunit.client.1.smithi163.stdout:9/246: rename d5 to d5/d59 22 2024-02-16T19:24:18.939 INFO:tasks.workunit.client.1.smithi163.stdout:6/38: rename d0/d6 to d0/d6/da 22 2024-02-16T19:24:18.939 INFO:tasks.workunit.client.1.smithi163.stdout:6/39: chown d0/l3 606 1 2024-02-16T19:24:18.941 INFO:tasks.workunit.client.1.smithi163.stdout:0/62: unlink fc 0 2024-02-16T19:24:18.943 INFO:tasks.workunit.client.1.smithi163.stdout:4/44: mknod ca 0 2024-02-16T19:24:18.945 INFO:tasks.workunit.client.1.smithi163.stdout:1/23: unlink f3 0 2024-02-16T19:24:18.946 INFO:tasks.workunit.client.0.smithi040.stdout:0/746: dwrite db/d2f/d1d/d21/d2e/d83/fe4 [0,4194304] 0 2024-02-16T19:24:18.946 INFO:tasks.workunit.client.0.smithi040.stdout:0/747: write db/de/f44 [4958711,43744] 0 2024-02-16T19:24:18.946 INFO:tasks.workunit.client.0.smithi040.stdout:0/748: chown db/d2f/d23/d3b/dd5 7093 1 2024-02-16T19:24:18.947 INFO:tasks.workunit.client.0.smithi040.stdout:6/713: rename d0/d1/d12/d3b/f40 to d0/d30/d32/dd2/dbc/feb 0 2024-02-16T19:24:18.947 INFO:tasks.workunit.client.0.smithi040.stdout:6/714: fdatasync d0/d30/d32/f56 0 2024-02-16T19:24:18.947 INFO:tasks.workunit.client.0.smithi040.stdout:6/715: creat d0/d1/d12/d3b/d79/fec x:0 0 0 2024-02-16T19:24:18.949 INFO:tasks.workunit.client.0.smithi040.stdout:7/556: dwrite d21/d4d/d77/d87/d9b/fa7 [0,4194304] 0 2024-02-16T19:24:18.950 INFO:tasks.workunit.client.1.smithi163.stdout:2/183: dwrite d2/d26/f3d [0,4194304] 0 2024-02-16T19:24:18.950 INFO:tasks.workunit.client.1.smithi163.stdout:5/31: symlink d0/l9 0 2024-02-16T19:24:18.950 INFO:tasks.workunit.client.0.smithi040.stdout:5/643: dwrite d1/da/d7e/f92 [0,4194304] 0 2024-02-16T19:24:18.951 INFO:tasks.workunit.client.0.smithi040.stdout:0/749: dread db/d2f/d1d/f46 [0,4194304] 0 2024-02-16T19:24:18.951 INFO:tasks.workunit.client.0.smithi040.stdout:2/589: unlink d5/d5c/d65/d9d/fbd 0 2024-02-16T19:24:18.954 INFO:tasks.workunit.client.1.smithi163.stdout:1/24: write f2 [1410563,102283] 0 2024-02-16T19:24:18.955 INFO:tasks.workunit.client.1.smithi163.stdout:1/25: write f2 [194808,125608] 0 2024-02-16T19:24:18.957 INFO:tasks.workunit.client.1.smithi163.stdout:3/71: sync 2024-02-16T19:24:18.958 INFO:tasks.workunit.client.1.smithi163.stdout:9/247: mknod d5/de/d16/d47/c5a 0 2024-02-16T19:24:18.958 INFO:tasks.workunit.client.1.smithi163.stdout:9/248: dread - d5/de/d30/d31/f51 zero size 2024-02-16T19:24:18.958 INFO:tasks.workunit.client.1.smithi163.stdout:9/249: getdents d5/de/d30/d31/d52 0 2024-02-16T19:24:18.959 INFO:tasks.workunit.client.1.smithi163.stdout:0/63: creat d11/d12/f15 x:0 0 0 2024-02-16T19:24:18.960 INFO:tasks.workunit.client.1.smithi163.stdout:3/72: dread d0/dc/fd [0,4194304] 0 2024-02-16T19:24:18.960 INFO:tasks.workunit.client.1.smithi163.stdout:3/73: creat d0/f1a x:0 0 0 2024-02-16T19:24:18.960 INFO:tasks.workunit.client.1.smithi163.stdout:3/74: write d0/f1 [5686665,4701] 0 2024-02-16T19:24:18.961 INFO:tasks.workunit.client.1.smithi163.stdout:4/45: dwrite f2 [0,4194304] 0 2024-02-16T19:24:18.961 INFO:tasks.workunit.client.0.smithi040.stdout:6/716: truncate d0/d1/d12/d3b/d48/f9e 2860337 0 2024-02-16T19:24:18.961 INFO:tasks.workunit.client.0.smithi040.stdout:6/717: read d0/f1d [120646,100964] 0 2024-02-16T19:24:18.963 INFO:tasks.workunit.client.0.smithi040.stdout:0/750: dread db/de/f59 [0,4194304] 0 2024-02-16T19:24:18.964 INFO:tasks.workunit.client.0.smithi040.stdout:0/751: write db/d2f/d1d/fd3 [172782,19128] 0 2024-02-16T19:24:18.965 INFO:tasks.workunit.client.1.smithi163.stdout:4/46: dread f9 [0,4194304] 0 2024-02-16T19:24:18.970 INFO:tasks.workunit.client.0.smithi040.stdout:7/557: rename d21/d4d/d77/d7a/d99/fa8 to d21/faf 0 2024-02-16T19:24:18.973 INFO:tasks.workunit.client.1.smithi163.stdout:8/38: symlink d0/d7/la 0 2024-02-16T19:24:18.973 INFO:tasks.workunit.client.1.smithi163.stdout:8/39: fdatasync d0/f9 0 2024-02-16T19:24:18.974 INFO:tasks.workunit.client.1.smithi163.stdout:7/55: symlink l10 0 2024-02-16T19:24:18.976 INFO:tasks.workunit.client.0.smithi040.stdout:6/718: dread d0/d1/d12/d3b/d48/f7f [8388608,4194304] 0 2024-02-16T19:24:18.976 INFO:tasks.workunit.client.0.smithi040.stdout:6/719: stat d0/d30/d32/dd2/f99 0 2024-02-16T19:24:18.977 INFO:tasks.workunit.client.0.smithi040.stdout:2/590: getdents d5 0 2024-02-16T19:24:18.977 INFO:tasks.workunit.client.0.smithi040.stdout:2/591: write d5/d7/d12/f6a [742872,44297] 0 2024-02-16T19:24:18.978 INFO:tasks.workunit.client.1.smithi163.stdout:6/40: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:24:18.978 INFO:tasks.workunit.client.1.smithi163.stdout:6/41: getdents d0/d6/d9 0 2024-02-16T19:24:18.978 INFO:tasks.workunit.client.1.smithi163.stdout:0/64: link ca d11/c16 0 2024-02-16T19:24:18.984 INFO:tasks.workunit.client.1.smithi163.stdout:2/184: rmdir d2/d2a/d2d 39 2024-02-16T19:24:18.984 INFO:tasks.workunit.client.1.smithi163.stdout:2/185: creat d2/d26/f3f x:0 0 0 2024-02-16T19:24:18.985 INFO:tasks.workunit.client.0.smithi040.stdout:7/558: creat d21/d8b/fb0 x:0 0 0 2024-02-16T19:24:18.985 INFO:tasks.workunit.client.0.smithi040.stdout:7/559: readlink d21/l5f 0 2024-02-16T19:24:18.987 INFO:tasks.workunit.client.1.smithi163.stdout:3/75: fdatasync d0/f1 0 2024-02-16T19:24:18.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/32: dwrite d0/d2/f3 [0,4194304] 0 2024-02-16T19:24:18.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/33: fsync d0/d2/f7 0 2024-02-16T19:24:18.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/34: write d0/f6 [821197,21880] 0 2024-02-16T19:24:18.987 INFO:tasks.workunit.client.1.smithi163.stdout:3/76: write d0/f6 [910540,81808] 0 2024-02-16T19:24:18.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/35: truncate d0/d2/f3 4452119 0 2024-02-16T19:24:18.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/36: truncate d0/d2/f4 905714 0 2024-02-16T19:24:18.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/37: dread - d0/d2/f7 zero size 2024-02-16T19:24:18.988 INFO:tasks.workunit.client.1.smithi163.stdout:3/77: creat d0/f1b x:0 0 0 2024-02-16T19:24:18.988 INFO:tasks.workunit.client.1.smithi163.stdout:5/38: creat d0/d2/fa x:0 0 0 2024-02-16T19:24:18.988 INFO:tasks.workunit.client.1.smithi163.stdout:5/39: dread - d0/d2/f7 zero size 2024-02-16T19:24:18.988 INFO:tasks.workunit.client.1.smithi163.stdout:3/78: read d0/f1 [1470168,86513] 0 2024-02-16T19:24:18.988 INFO:tasks.workunit.client.1.smithi163.stdout:9/250: dwrite d5/f2c [0,4194304] 0 2024-02-16T19:24:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:18 smithi163 ceph-mon[36452]: Upgrade: Updating mgr.smithi163.wjfclz 2024-02-16T19:24:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:18 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:18 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:24:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:18 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:24:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:18 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:18 smithi163 ceph-mon[36452]: Deploying daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:24:18.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:18 smithi163 ceph-mon[36452]: pgmap v248: 65 pgs: 65 active+clean; 1.4 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 128 MiB/s rd, 162 MiB/s wr, 234 op/s 2024-02-16T19:24:18.989 INFO:tasks.workunit.client.1.smithi163.stdout:8/40: creat d0/d7/fb x:0 0 0 2024-02-16T19:24:18.989 INFO:tasks.workunit.client.1.smithi163.stdout:8/41: chown d0/f9 726909 1 2024-02-16T19:24:18.989 INFO:tasks.workunit.client.0.smithi040.stdout:6/720: rename d0/d1/f1f to d0/d30/d5b/dbe/fed 0 2024-02-16T19:24:18.991 INFO:tasks.workunit.client.0.smithi040.stdout:0/752: dwrite db/d2f/d23/f33 [0,4194304] 0 2024-02-16T19:24:18.991 INFO:tasks.workunit.client.0.smithi040.stdout:0/753: write db/d2f/d1d/f46 [932860,23968] 0 2024-02-16T19:24:18.991 INFO:tasks.workunit.client.0.smithi040.stdout:0/754: write db/d2f/d1d/d21/d2e/d85/fe8 [559605,62868] 0 2024-02-16T19:24:18.991 INFO:tasks.workunit.client.1.smithi163.stdout:5/40: fdatasync d0/f6 0 2024-02-16T19:24:18.992 INFO:tasks.workunit.client.1.smithi163.stdout:5/41: creat d0/d2/fb x:0 0 0 2024-02-16T19:24:18.992 INFO:tasks.workunit.client.0.smithi040.stdout:2/592: creat d5/d8/fc2 x:0 0 0 2024-02-16T19:24:18.993 INFO:tasks.workunit.client.1.smithi163.stdout:4/47: dwrite f8 [0,4194304] 0 2024-02-16T19:24:18.993 INFO:tasks.workunit.client.1.smithi163.stdout:1/26: getdents . 0 2024-02-16T19:24:18.994 INFO:tasks.workunit.client.1.smithi163.stdout:1/27: truncate f0 4218036 0 2024-02-16T19:24:18.994 INFO:tasks.workunit.client.1.smithi163.stdout:1/28: write f2 [4043755,83188] 0 2024-02-16T19:24:18.994 INFO:tasks.workunit.client.1.smithi163.stdout:1/29: chown f2 507483 1 2024-02-16T19:24:18.995 INFO:tasks.workunit.client.0.smithi040.stdout:7/560: creat d21/d4d/d77/d7a/fb1 x:0 0 0 2024-02-16T19:24:18.997 INFO:tasks.workunit.client.1.smithi163.stdout:6/42: mknod d0/d6/d9/cb 0 2024-02-16T19:24:18.997 INFO:tasks.workunit.client.1.smithi163.stdout:6/43: write d0/d6/f8 [995304,59129] 0 2024-02-16T19:24:18.997 INFO:tasks.workunit.client.0.smithi040.stdout:6/721: mknod d0/d1/d12/d3b/d48/d73/d76/cee 0 2024-02-16T19:24:18.998 INFO:tasks.workunit.client.1.smithi163.stdout:0/65: symlink d11/l17 0 2024-02-16T19:24:19.003 INFO:tasks.workunit.client.0.smithi040.stdout:2/593: mkdir d5/d7/d1e/d31/dc3 0 2024-02-16T19:24:19.003 INFO:tasks.workunit.client.0.smithi040.stdout:2/594: readlink d5/d7/d1e/d4c/d52/l72 0 2024-02-16T19:24:19.004 INFO:tasks.workunit.client.0.smithi040.stdout:8/731: sync 2024-02-16T19:24:19.006 INFO:tasks.workunit.client.0.smithi040.stdout:7/561: symlink d21/d4d/d77/d7a/d99/lb2 0 2024-02-16T19:24:19.006 INFO:tasks.workunit.client.0.smithi040.stdout:7/562: creat d21/d4d/d77/d7a/d99/fb3 x:0 0 0 2024-02-16T19:24:19.006 INFO:tasks.workunit.client.0.smithi040.stdout:7/563: fsync d21/faf 0 2024-02-16T19:24:19.006 INFO:tasks.workunit.client.0.smithi040.stdout:7/564: fsync d21/f46 0 2024-02-16T19:24:19.008 INFO:tasks.workunit.client.0.smithi040.stdout:6/722: creat d0/d30/fef x:0 0 0 2024-02-16T19:24:19.009 INFO:tasks.workunit.client.0.smithi040.stdout:6/723: readlink d0/d30/d32/dd2/dbc/le6 0 2024-02-16T19:24:19.012 INFO:tasks.workunit.client.0.smithi040.stdout:6/724: getdents d0/d1/d12/d3b/d48/dc0 0 2024-02-16T19:24:19.012 INFO:tasks.workunit.client.0.smithi040.stdout:6/725: write d0/d37/d4a/f5e [3985148,32210] 0 2024-02-16T19:24:19.012 INFO:tasks.workunit.client.0.smithi040.stdout:6/726: dread - d0/d1/d12/d3b/d48/dba/dde/dbd/fd1 zero size 2024-02-16T19:24:19.013 INFO:tasks.workunit.client.1.smithi163.stdout:9/251: truncate d5/f14 2223979 0 2024-02-16T19:24:19.013 INFO:tasks.workunit.client.0.smithi040.stdout:8/732: mkdir d6/d7/d15/de5 0 2024-02-16T19:24:19.014 INFO:tasks.workunit.client.1.smithi163.stdout:7/56: mkdir d11 0 2024-02-16T19:24:19.022 INFO:tasks.workunit.client.1.smithi163.stdout:9/252: write d5/de/d16/f3c [2416232,81394] 0 2024-02-16T19:24:19.022 INFO:tasks.workunit.client.1.smithi163.stdout:9/253: stat d5/de/ff 0 2024-02-16T19:24:19.026 INFO:tasks.workunit.client.1.smithi163.stdout:3/79: dwrite d0/dc/fd [0,4194304] 0 2024-02-16T19:24:19.027 INFO:tasks.workunit.client.0.smithi040.stdout:0/755: dwrite db/d25/f61 [4194304,4194304] 0 2024-02-16T19:24:19.027 INFO:tasks.workunit.client.0.smithi040.stdout:0/756: creat db/d2f/d1d/d21/f104 x:0 0 0 2024-02-16T19:24:19.027 INFO:tasks.workunit.client.0.smithi040.stdout:8/733: dread d6/d7/d15/f18 [0,4194304] 0 2024-02-16T19:24:19.027 INFO:tasks.workunit.client.0.smithi040.stdout:8/734: getdents d6/d12/d51/d6e/db7/ddf 0 2024-02-16T19:24:19.027 INFO:tasks.workunit.client.0.smithi040.stdout:7/565: dwrite d21/d29/f45 [0,4194304] 0 2024-02-16T19:24:19.055 INFO:tasks.workunit.client.1.smithi163.stdout:1/30: dwrite f0 [0,4194304] 0 2024-02-16T19:24:19.055 INFO:tasks.workunit.client.1.smithi163.stdout:6/44: dwrite d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:19.055 INFO:tasks.workunit.client.1.smithi163.stdout:6/45: fsync d0/d6/f8 0 2024-02-16T19:24:19.056 INFO:tasks.workunit.client.0.smithi040.stdout:0/757: dwrite db/d2f/f66 [0,4194304] 0 2024-02-16T19:24:19.091 INFO:tasks.workunit.client.0.smithi040.stdout:6/727: mkdir d0/df0 0 2024-02-16T19:24:19.091 INFO:tasks.workunit.client.0.smithi040.stdout:6/728: creat d0/d1/d6f/ff1 x:0 0 0 2024-02-16T19:24:19.093 INFO:tasks.workunit.client.0.smithi040.stdout:6/729: dread d0/f3f [0,4194304] 0 2024-02-16T19:24:19.093 INFO:tasks.workunit.client.0.smithi040.stdout:6/730: write d0/d30/d4b/fca [869303,89242] 0 2024-02-16T19:24:19.093 INFO:tasks.workunit.client.0.smithi040.stdout:6/731: creat d0/d1/d12/d3b/d48/dba/ff2 x:0 0 0 2024-02-16T19:24:19.093 INFO:tasks.workunit.client.0.smithi040.stdout:6/732: dread - d0/d30/d5b/f8a zero size 2024-02-16T19:24:19.095 INFO:tasks.workunit.client.0.smithi040.stdout:8/735: symlink d6/d12/d51/le6 0 2024-02-16T19:24:19.099 INFO:tasks.workunit.client.1.smithi163.stdout:4/48: rename c1 to cb 0 2024-02-16T19:24:19.099 INFO:tasks.workunit.client.1.smithi163.stdout:0/66: mknod d11/c18 0 2024-02-16T19:24:19.101 INFO:tasks.workunit.client.1.smithi163.stdout:8/42: mkdir d0/dc 0 2024-02-16T19:24:19.102 INFO:tasks.workunit.client.0.smithi040.stdout:7/566: creat d21/d4d/d7d/fb4 x:0 0 0 2024-02-16T19:24:19.102 INFO:tasks.workunit.client.0.smithi040.stdout:7/567: dread - d21/f22 zero size 2024-02-16T19:24:19.103 INFO:tasks.workunit.client.1.smithi163.stdout:7/57: truncate fb 733803 0 2024-02-16T19:24:19.109 INFO:tasks.workunit.client.1.smithi163.stdout:9/254: rename d5/f15 to d5/de/f5b 0 2024-02-16T19:24:19.112 INFO:tasks.workunit.client.0.smithi040.stdout:0/758: link db/d2f/d1d/d21/d2e/d85/d89/ce5 db/de/dbe/c105 0 2024-02-16T19:24:19.112 INFO:tasks.workunit.client.0.smithi040.stdout:0/759: fdatasync db/d2f/f70 0 2024-02-16T19:24:19.114 INFO:tasks.workunit.client.1.smithi163.stdout:3/80: mkdir d0/dc/d1c 0 2024-02-16T19:24:19.118 INFO:tasks.workunit.client.0.smithi040.stdout:8/736: mknod d6/d12/d51/d6e/db7/ddf/ce7 0 2024-02-16T19:24:19.120 INFO:tasks.workunit.client.0.smithi040.stdout:0/760: mkdir db/de/d6a/df0/d106 0 2024-02-16T19:24:19.121 INFO:tasks.workunit.client.1.smithi163.stdout:8/43: dwrite d0/f9 [0,4194304] 0 2024-02-16T19:24:19.122 INFO:tasks.workunit.client.0.smithi040.stdout:7/568: dwrite f9 [4194304,4194304] 0 2024-02-16T19:24:19.122 INFO:tasks.workunit.client.0.smithi040.stdout:7/569: creat d21/d8b/fb5 x:0 0 0 2024-02-16T19:24:19.122 INFO:tasks.workunit.client.0.smithi040.stdout:7/570: truncate d21/d29/f45 4891589 0 2024-02-16T19:24:19.123 INFO:tasks.workunit.client.1.smithi163.stdout:1/31: mknod c4 0 2024-02-16T19:24:19.124 INFO:tasks.workunit.client.0.smithi040.stdout:6/733: mkdir d0/d30/d32/dd2/d87/d8f/df3 0 2024-02-16T19:24:19.124 INFO:tasks.workunit.client.0.smithi040.stdout:6/734: chown d0/d1/d12/d3b 544666 1 2024-02-16T19:24:19.130 INFO:tasks.workunit.client.1.smithi163.stdout:6/46: symlink d0/lc 0 2024-02-16T19:24:19.133 INFO:tasks.workunit.client.1.smithi163.stdout:5/42: link d0/d2/fa d0/d2/fc 0 2024-02-16T19:24:19.133 INFO:tasks.workunit.client.1.smithi163.stdout:5/43: chown d0 9 1 2024-02-16T19:24:19.134 INFO:tasks.workunit.client.0.smithi040.stdout:6/735: link d0/d1/d12/d3b/d48/d73/l60 d0/d1/d12/d3b/d48/dba/lf4 0 2024-02-16T19:24:19.135 INFO:tasks.workunit.client.0.smithi040.stdout:6/736: chown d0/d30/d4b/f93 145035376 1 2024-02-16T19:24:19.136 INFO:tasks.workunit.client.1.smithi163.stdout:4/49: symlink lc 0 2024-02-16T19:24:19.139 INFO:tasks.workunit.client.0.smithi040.stdout:7/571: read d21/f4a [181041,98058] 0 2024-02-16T19:24:19.139 INFO:tasks.workunit.client.0.smithi040.stdout:7/572: fdatasync d21/f62 0 2024-02-16T19:24:19.139 INFO:tasks.workunit.client.0.smithi040.stdout:7/573: creat d21/d8b/fb6 x:0 0 0 2024-02-16T19:24:19.141 INFO:tasks.workunit.client.0.smithi040.stdout:6/737: unlink d0/d1/d12/d3b/d48/d4f/f69 0 2024-02-16T19:24:19.141 INFO:tasks.workunit.client.0.smithi040.stdout:6/738: write d0/d1/d12/d3b/d48/dba/dde/dbd/fd1 [648428,80963] 0 2024-02-16T19:24:19.141 INFO:tasks.workunit.client.0.smithi040.stdout:6/739: fdatasync d0/d1/d12/d3b/d79/fa2 0 2024-02-16T19:24:19.142 INFO:tasks.workunit.client.1.smithi163.stdout:7/58: fsync f5 0 2024-02-16T19:24:19.142 INFO:tasks.workunit.client.0.smithi040.stdout:6/740: dread d0/d1/d12/d3b/f97 [0,4194304] 0 2024-02-16T19:24:19.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:18 smithi040 ceph-mon[29201]: Upgrade: Updating mgr.smithi163.wjfclz 2024-02-16T19:24:19.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:18 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:19.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:18 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:24:19.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:18 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:24:19.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:18 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:19.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:18 smithi040 ceph-mon[29201]: Deploying daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:24:19.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:18 smithi040 ceph-mon[29201]: pgmap v248: 65 pgs: 65 active+clean; 1.4 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 128 MiB/s rd, 162 MiB/s wr, 234 op/s 2024-02-16T19:24:19.145 INFO:tasks.workunit.client.1.smithi163.stdout:9/255: rename d5/de/d30/d43/f55 to d5/de/d30/d43/f5c 0 2024-02-16T19:24:19.145 INFO:tasks.workunit.client.1.smithi163.stdout:9/256: chown d5 124 1 2024-02-16T19:24:19.145 INFO:tasks.workunit.client.0.smithi040.stdout:6/741: mknod d0/d30/d4b/cf5 0 2024-02-16T19:24:19.145 INFO:tasks.workunit.client.0.smithi040.stdout:6/742: write d0/d1/d12/d3b/d48/dba/fc5 [189700,61868] 0 2024-02-16T19:24:19.146 INFO:tasks.workunit.client.1.smithi163.stdout:6/47: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:19.146 INFO:tasks.workunit.client.0.smithi040.stdout:6/743: write d0/d1/d12/f2c [4305828,114882] 0 2024-02-16T19:24:19.147 INFO:tasks.workunit.client.0.smithi040.stdout:6/744: readlink d0/d30/d32/la9 0 2024-02-16T19:24:19.147 INFO:tasks.workunit.client.0.smithi040.stdout:6/745: chown d0/d37/de1/dac/ld7 2928 1 2024-02-16T19:24:19.147 INFO:tasks.workunit.client.1.smithi163.stdout:3/81: mkdir d0/dc/d1d 0 2024-02-16T19:24:19.147 INFO:tasks.workunit.client.1.smithi163.stdout:3/82: write d0/f14 [74306,7559] 0 2024-02-16T19:24:19.149 INFO:tasks.workunit.client.0.smithi040.stdout:6/746: creat d0/d37/de1/ff6 x:0 0 0 2024-02-16T19:24:19.149 INFO:tasks.workunit.client.0.smithi040.stdout:6/747: write d0/d37/de1/ff6 [353118,113180] 0 2024-02-16T19:24:19.149 INFO:tasks.workunit.client.0.smithi040.stdout:6/748: read - d0/d1/d12/d3b/fd6 zero size 2024-02-16T19:24:19.149 INFO:tasks.workunit.client.1.smithi163.stdout:6/48: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:19.149 INFO:tasks.workunit.client.1.smithi163.stdout:6/49: fdatasync d0/d6/f8 0 2024-02-16T19:24:19.149 INFO:tasks.workunit.client.1.smithi163.stdout:6/50: write d0/f2 [5013481,81637] 0 2024-02-16T19:24:19.151 INFO:tasks.workunit.client.0.smithi040.stdout:6/749: getdents d0/d1/d6f/d90 0 2024-02-16T19:24:19.151 INFO:tasks.workunit.client.0.smithi040.stdout:6/750: fdatasync d0/d1/d12/d3b/d48/f7f 0 2024-02-16T19:24:19.152 INFO:tasks.workunit.client.0.smithi040.stdout:3/737: sync 2024-02-16T19:24:19.153 INFO:tasks.workunit.client.0.smithi040.stdout:4/559: sync 2024-02-16T19:24:19.153 INFO:tasks.workunit.client.0.smithi040.stdout:1/628: sync 2024-02-16T19:24:19.153 INFO:tasks.workunit.client.0.smithi040.stdout:1/629: truncate d17/d26/d5f/fc3 898830 0 2024-02-16T19:24:19.154 INFO:tasks.workunit.client.0.smithi040.stdout:7/574: dread d21/d29/f37 [4194304,4194304] 0 2024-02-16T19:24:19.155 INFO:tasks.workunit.client.0.smithi040.stdout:7/575: dread - d21/d4d/d77/d87/fa9 zero size 2024-02-16T19:24:19.155 INFO:tasks.workunit.client.0.smithi040.stdout:7/576: dread - d21/f22 zero size 2024-02-16T19:24:19.156 INFO:tasks.workunit.client.0.smithi040.stdout:0/761: dwrite db/d2f/d1d/f80 [0,4194304] 0 2024-02-16T19:24:19.156 INFO:tasks.workunit.client.1.smithi163.stdout:1/32: dwrite f0 [4194304,4194304] 0 2024-02-16T19:24:19.157 INFO:tasks.workunit.client.0.smithi040.stdout:6/751: rename d0/d1/d6f/f75 to d0/d30/d32/dd2/dbc/ff7 0 2024-02-16T19:24:19.157 INFO:tasks.workunit.client.0.smithi040.stdout:6/752: readlink d0/d1/d12/d3b/d48/dba/lf4 0 2024-02-16T19:24:19.157 INFO:tasks.workunit.client.0.smithi040.stdout:6/753: chown d0/d1/d12/d3b 21648 1 2024-02-16T19:24:19.158 INFO:tasks.workunit.client.0.smithi040.stdout:8/737: dwrite d6/d12/d51/fa1 [0,4194304] 0 2024-02-16T19:24:19.159 INFO:tasks.workunit.client.0.smithi040.stdout:4/560: mkdir dd/de/db0 0 2024-02-16T19:24:19.159 INFO:tasks.workunit.client.0.smithi040.stdout:4/561: stat dd/de/d10/d36/d9e/d77/f67 0 2024-02-16T19:24:19.159 INFO:tasks.workunit.client.0.smithi040.stdout:4/562: chown dd/ca0 32118 1 2024-02-16T19:24:19.159 INFO:tasks.workunit.client.1.smithi163.stdout:6/51: dread d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:19.163 INFO:tasks.workunit.client.0.smithi040.stdout:1/630: creat d17/d26/d5f/da6/fc6 x:0 0 0 2024-02-16T19:24:19.163 INFO:tasks.workunit.client.0.smithi040.stdout:1/631: stat d17/d3c/d6b 0 2024-02-16T19:24:19.164 INFO:tasks.workunit.client.1.smithi163.stdout:8/44: dread d0/f4 [0,4194304] 0 2024-02-16T19:24:19.166 INFO:tasks.workunit.client.1.smithi163.stdout:6/52: dread d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:19.166 INFO:tasks.workunit.client.0.smithi040.stdout:7/577: mkdir d21/d8b/d9c/da0/db7 0 2024-02-16T19:24:19.167 INFO:tasks.workunit.client.0.smithi040.stdout:8/738: dread d6/d7/d1d/f52 [0,4194304] 0 2024-02-16T19:24:19.170 INFO:tasks.workunit.client.1.smithi163.stdout:5/44: unlink d0/f6 0 2024-02-16T19:24:19.172 INFO:tasks.workunit.client.1.smithi163.stdout:4/50: truncate f9 4102424 0 2024-02-16T19:24:19.172 INFO:tasks.workunit.client.1.smithi163.stdout:4/51: chown l0 0 1 2024-02-16T19:24:19.172 INFO:tasks.workunit.client.1.smithi163.stdout:4/52: readlink l0 0 2024-02-16T19:24:19.172 INFO:tasks.workunit.client.1.smithi163.stdout:4/53: readlink lc 0 2024-02-16T19:24:19.173 INFO:tasks.workunit.client.1.smithi163.stdout:0/67: getdents d11/d12 0 2024-02-16T19:24:19.175 INFO:tasks.workunit.client.1.smithi163.stdout:7/59: fdatasync f9 0 2024-02-16T19:24:19.178 INFO:tasks.workunit.client.1.smithi163.stdout:3/83: symlink d0/dc/d1d/l1e 0 2024-02-16T19:24:19.181 INFO:tasks.workunit.client.1.smithi163.stdout:1/33: rename l1 to l5 0 2024-02-16T19:24:19.182 INFO:tasks.workunit.client.1.smithi163.stdout:8/45: rename d0/f4 to d0/d7/fd 0 2024-02-16T19:24:19.183 INFO:tasks.workunit.client.0.smithi040.stdout:6/754: dwrite d0/d1/d12/d3b/f97 [0,4194304] 0 2024-02-16T19:24:19.185 INFO:tasks.workunit.client.1.smithi163.stdout:6/53: mkdir d0/dd 0 2024-02-16T19:24:19.187 INFO:tasks.workunit.client.1.smithi163.stdout:0/68: rename d11/c16 to d11/c19 0 2024-02-16T19:24:19.190 INFO:tasks.workunit.client.0.smithi040.stdout:3/738: dwrite d1/dc/d35/d53/d64/d65/f98 [4194304,4194304] 0 2024-02-16T19:24:19.191 INFO:tasks.workunit.client.1.smithi163.stdout:7/60: write f3 [361276,9028] 0 2024-02-16T19:24:19.191 INFO:tasks.workunit.client.1.smithi163.stdout:9/257: mknod d5/de/d30/d3d/c5d 0 2024-02-16T19:24:19.192 INFO:tasks.workunit.client.1.smithi163.stdout:3/84: unlink d0/c18 0 2024-02-16T19:24:19.195 INFO:tasks.workunit.client.0.smithi040.stdout:4/563: rename dd/de/d10/d36/d9e/c2b to dd/de/d10/d36/d9e/d77/cb1 0 2024-02-16T19:24:19.195 INFO:tasks.workunit.client.0.smithi040.stdout:0/762: dwrite db/d2f/d23/d6f/ff5 [0,4194304] 0 2024-02-16T19:24:19.197 INFO:tasks.workunit.client.1.smithi163.stdout:8/46: creat d0/fe x:0 0 0 2024-02-16T19:24:19.198 INFO:tasks.workunit.client.0.smithi040.stdout:1/632: creat d17/fc7 x:0 0 0 2024-02-16T19:24:19.200 INFO:tasks.workunit.client.1.smithi163.stdout:6/54: symlink d0/d6/le 0 2024-02-16T19:24:19.201 INFO:tasks.workunit.client.0.smithi040.stdout:7/578: mknod d21/d8b/d9c/da0/db7/cb8 0 2024-02-16T19:24:19.203 INFO:tasks.workunit.client.1.smithi163.stdout:0/69: creat d11/f1a x:0 0 0 2024-02-16T19:24:19.206 INFO:tasks.workunit.client.0.smithi040.stdout:8/739: unlink d6/d2b/d44/fbd 0 2024-02-16T19:24:19.211 INFO:tasks.workunit.client.1.smithi163.stdout:7/61: creat d11/f12 x:0 0 0 2024-02-16T19:24:19.213 INFO:tasks.workunit.client.0.smithi040.stdout:6/755: symlink d0/d1/d12/d3b/d48/lf8 0 2024-02-16T19:24:19.213 INFO:tasks.workunit.client.0.smithi040.stdout:6/756: getdents d0/df0 0 2024-02-16T19:24:19.213 INFO:tasks.workunit.client.0.smithi040.stdout:6/757: write d0/d1/d12/d3b/f50 [1620798,23853] 0 2024-02-16T19:24:19.213 INFO:tasks.workunit.client.0.smithi040.stdout:6/758: truncate d0/d30/d32/dd2/dbc/ff7 1732989 0 2024-02-16T19:24:19.215 INFO:tasks.workunit.client.1.smithi163.stdout:9/258: dwrite d5/f2e [0,4194304] 0 2024-02-16T19:24:19.215 INFO:tasks.workunit.client.1.smithi163.stdout:9/259: stat d5/de/d30/d31 0 2024-02-16T19:24:19.215 INFO:tasks.workunit.client.1.smithi163.stdout:3/85: link d0/f1b d0/dc/d1d/f1f 0 2024-02-16T19:24:19.216 INFO:tasks.workunit.client.0.smithi040.stdout:3/739: creat d1/fe7 x:0 0 0 2024-02-16T19:24:19.218 INFO:tasks.workunit.client.1.smithi163.stdout:8/47: creat d0/ff x:0 0 0 2024-02-16T19:24:19.218 INFO:tasks.workunit.client.1.smithi163.stdout:8/48: write d0/d7/fb [963230,83674] 0 2024-02-16T19:24:19.218 INFO:tasks.workunit.client.1.smithi163.stdout:8/49: truncate d0/fe 267511 0 2024-02-16T19:24:19.218 INFO:tasks.workunit.client.0.smithi040.stdout:4/564: creat dd/de/db0/fb2 x:0 0 0 2024-02-16T19:24:19.221 INFO:tasks.workunit.client.1.smithi163.stdout:7/62: symlink d11/l13 0 2024-02-16T19:24:19.224 INFO:tasks.workunit.client.1.smithi163.stdout:6/55: dwrite d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:19.226 INFO:tasks.workunit.client.0.smithi040.stdout:1/633: symlink d17/d26/d5f/da6/lc8 0 2024-02-16T19:24:19.229 INFO:tasks.workunit.client.0.smithi040.stdout:7/579: link d21/l40 d21/d4d/d77/d87/lb9 0 2024-02-16T19:24:19.229 INFO:tasks.workunit.client.0.smithi040.stdout:7/580: read - d21/d4d/d77/d7a/fb1 zero size 2024-02-16T19:24:19.229 INFO:tasks.workunit.client.0.smithi040.stdout:7/581: creat d21/d4d/d77/fba x:0 0 0 2024-02-16T19:24:19.231 INFO:tasks.workunit.client.0.smithi040.stdout:8/740: mkdir d6/d12/d51/d6e/db7/dc6/dcc/de8 0 2024-02-16T19:24:19.232 INFO:tasks.workunit.client.1.smithi163.stdout:8/50: mkdir d0/dc/d10 0 2024-02-16T19:24:19.235 INFO:tasks.workunit.client.1.smithi163.stdout:0/70: getdents d11 0 2024-02-16T19:24:19.235 INFO:tasks.workunit.client.1.smithi163.stdout:0/71: chown d11/l17 55626 1 2024-02-16T19:24:19.236 INFO:tasks.workunit.client.0.smithi040.stdout:3/740: truncate d1/d4/d6e/fb5 2210155 0 2024-02-16T19:24:19.237 INFO:tasks.workunit.client.1.smithi163.stdout:9/260: read d5/de/d16/f53 [926790,77704] 0 2024-02-16T19:24:19.238 INFO:tasks.workunit.client.1.smithi163.stdout:7/63: symlink d11/l14 0 2024-02-16T19:24:19.238 INFO:tasks.workunit.client.1.smithi163.stdout:7/64: dread - d11/f12 zero size 2024-02-16T19:24:19.238 INFO:tasks.workunit.client.1.smithi163.stdout:7/65: truncate f3 1678701 0 2024-02-16T19:24:19.238 INFO:tasks.workunit.client.1.smithi163.stdout:7/66: creat d11/f15 x:0 0 0 2024-02-16T19:24:19.239 INFO:tasks.workunit.client.0.smithi040.stdout:4/565: creat dd/d58/d8c/d99/fb3 x:0 0 0 2024-02-16T19:24:19.240 INFO:tasks.workunit.client.0.smithi040.stdout:1/634: creat d17/d43/fc9 x:0 0 0 2024-02-16T19:24:19.244 INFO:tasks.workunit.client.1.smithi163.stdout:3/86: getdents d0/dc 0 2024-02-16T19:24:19.244 INFO:tasks.workunit.client.1.smithi163.stdout:3/87: stat d0/f1b 0 2024-02-16T19:24:19.244 INFO:tasks.workunit.client.1.smithi163.stdout:3/88: getdents d0/dc 0 2024-02-16T19:24:19.244 INFO:tasks.workunit.client.1.smithi163.stdout:3/89: chown d0/f6 17 1 2024-02-16T19:24:19.246 INFO:tasks.workunit.client.0.smithi040.stdout:7/582: rename d21/d4d/d77/d7a/c96 to d21/d8b/cbb 0 2024-02-16T19:24:19.246 INFO:tasks.workunit.client.0.smithi040.stdout:3/741: write d1/d4/f43 [2398386,129179] 0 2024-02-16T19:24:19.248 INFO:tasks.workunit.client.0.smithi040.stdout:8/741: unlink d6/d2b/d36/fba 0 2024-02-16T19:24:19.248 INFO:tasks.workunit.client.0.smithi040.stdout:8/742: chown d6/d12/d22/c39 12916004 1 2024-02-16T19:24:19.250 INFO:tasks.workunit.client.1.smithi163.stdout:0/72: rmdir d11 39 2024-02-16T19:24:19.250 INFO:tasks.workunit.client.1.smithi163.stdout:0/73: dread - d11/f1a zero size 2024-02-16T19:24:19.255 INFO:tasks.workunit.client.0.smithi040.stdout:7/583: mkdir d21/d29/d79/dbc 0 2024-02-16T19:24:19.256 INFO:tasks.workunit.client.1.smithi163.stdout:6/56: truncate d0/f2 3932060 0 2024-02-16T19:24:19.257 INFO:tasks.workunit.client.0.smithi040.stdout:4/566: rmdir dd/de/d10/d36/d9e/d32 39 2024-02-16T19:24:19.258 INFO:tasks.workunit.client.0.smithi040.stdout:9/701: sync 2024-02-16T19:24:19.258 INFO:tasks.workunit.client.0.smithi040.stdout:2/595: sync 2024-02-16T19:24:19.258 INFO:tasks.workunit.client.0.smithi040.stdout:5/644: sync 2024-02-16T19:24:19.259 INFO:tasks.workunit.client.0.smithi040.stdout:3/742: rename d1/dc/d35/d53/dd1 to d1/dc/d42/d58/de8 0 2024-02-16T19:24:19.260 INFO:tasks.workunit.client.0.smithi040.stdout:8/743: creat d6/d12/d51/db6/fe9 x:0 0 0 2024-02-16T19:24:19.262 INFO:tasks.workunit.client.0.smithi040.stdout:4/567: creat dd/de/d10/d36/dad/fb4 x:0 0 0 2024-02-16T19:24:19.262 INFO:tasks.workunit.client.0.smithi040.stdout:7/584: dread d21/d4d/f56 [0,4194304] 0 2024-02-16T19:24:19.262 INFO:tasks.workunit.client.0.smithi040.stdout:9/702: creat d3/d4/fe3 x:0 0 0 2024-02-16T19:24:19.263 INFO:tasks.workunit.client.0.smithi040.stdout:9/703: creat d3/d4/d55/fe4 x:0 0 0 2024-02-16T19:24:19.263 INFO:tasks.workunit.client.0.smithi040.stdout:5/645: symlink d1/d62/d9a/dc5/lc6 0 2024-02-16T19:24:19.264 INFO:tasks.workunit.client.0.smithi040.stdout:3/743: mknod d1/dc/d35/d53/d64/d65/ce9 0 2024-02-16T19:24:19.264 INFO:tasks.workunit.client.0.smithi040.stdout:3/744: creat d1/d4/d6e/fea x:0 0 0 2024-02-16T19:24:19.265 INFO:tasks.workunit.client.0.smithi040.stdout:8/744: getdents d6/d12/d51/d6e/db7/ddf 0 2024-02-16T19:24:19.265 INFO:tasks.workunit.client.0.smithi040.stdout:8/745: stat d6/d7/f7b 0 2024-02-16T19:24:19.265 INFO:tasks.workunit.client.0.smithi040.stdout:8/746: fdatasync d6/d12/d22/f4e 0 2024-02-16T19:24:19.265 INFO:tasks.workunit.client.0.smithi040.stdout:4/568: stat dd/de/c16 0 2024-02-16T19:24:19.265 INFO:tasks.workunit.client.0.smithi040.stdout:7/585: creat d21/d4d/fbd x:0 0 0 2024-02-16T19:24:19.267 INFO:tasks.workunit.client.0.smithi040.stdout:9/704: mkdir d3/de5 0 2024-02-16T19:24:19.267 INFO:tasks.workunit.client.1.smithi163.stdout:6/57: dread d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:19.269 INFO:tasks.workunit.client.0.smithi040.stdout:5/646: rename d1/da/db0/lb9 to d1/da/lc7 0 2024-02-16T19:24:19.269 INFO:tasks.workunit.client.0.smithi040.stdout:5/647: stat d1/da/d51/fbf 0 2024-02-16T19:24:19.269 INFO:tasks.workunit.client.0.smithi040.stdout:5/648: chown d1/d3c/f5d 9724273 1 2024-02-16T19:24:19.270 INFO:tasks.workunit.client.0.smithi040.stdout:3/745: dread d1/dc/d42/d4a/f54 [0,4194304] 0 2024-02-16T19:24:19.273 INFO:tasks.workunit.client.1.smithi163.stdout:3/90: dwrite d0/dc/d1d/f1f [0,4194304] 0 2024-02-16T19:24:19.275 INFO:tasks.workunit.client.1.smithi163.stdout:0/74: dwrite d11/f1a [0,4194304] 0 2024-02-16T19:24:19.275 INFO:tasks.workunit.client.0.smithi040.stdout:8/747: unlink d6/f46 0 2024-02-16T19:24:19.275 INFO:tasks.workunit.client.0.smithi040.stdout:8/748: readlink d6/d2b/d36/lc0 0 2024-02-16T19:24:19.275 INFO:tasks.workunit.client.0.smithi040.stdout:8/749: chown d6/d64/d50 11090 1 2024-02-16T19:24:19.277 INFO:tasks.workunit.client.0.smithi040.stdout:9/705: creat d3/d4/d55/d70/fe6 x:0 0 0 2024-02-16T19:24:19.277 INFO:tasks.workunit.client.0.smithi040.stdout:2/596: write d5/f79 [2112736,79897] 0 2024-02-16T19:24:19.281 INFO:tasks.workunit.client.0.smithi040.stdout:5/649: rename d1/da/d1c/f65 to d1/da/fc8 0 2024-02-16T19:24:19.281 INFO:tasks.workunit.client.0.smithi040.stdout:5/650: chown d1/da/d1c/dba/db1/l61 32639 1 2024-02-16T19:24:19.287 INFO:tasks.workunit.client.1.smithi163.stdout:0/75: dread f6 [0,4194304] 0 2024-02-16T19:24:19.288 INFO:tasks.workunit.client.0.smithi040.stdout:9/706: mknod d3/d4/d55/d70/de0/ce7 0 2024-02-16T19:24:19.288 INFO:tasks.workunit.client.1.smithi163.stdout:2/186: sync 2024-02-16T19:24:19.288 INFO:tasks.workunit.client.0.smithi040.stdout:1/635: write d17/d26/d4c/f98 [6086880,73140] 0 2024-02-16T19:24:19.289 INFO:tasks.workunit.client.0.smithi040.stdout:1/636: write d17/d3c/d74/f7f [4506665,81065] 0 2024-02-16T19:24:19.289 INFO:tasks.workunit.client.0.smithi040.stdout:0/763: dwrite db/d2f/d23/d3b/f47 [0,4194304] 0 2024-02-16T19:24:19.290 INFO:tasks.workunit.client.0.smithi040.stdout:8/750: dread d6/d2b/f63 [0,4194304] 0 2024-02-16T19:24:19.290 INFO:tasks.workunit.client.1.smithi163.stdout:3/91: dread d0/dc/d1d/f1f [0,4194304] 0 2024-02-16T19:24:19.291 INFO:tasks.workunit.client.1.smithi163.stdout:0/76: mknod d11/d12/c1b 0 2024-02-16T19:24:19.293 INFO:tasks.workunit.client.1.smithi163.stdout:6/58: truncate d0/f2 4414150 0 2024-02-16T19:24:19.293 INFO:tasks.workunit.client.0.smithi040.stdout:7/586: write f17 [1314572,72884] 0 2024-02-16T19:24:19.294 INFO:tasks.workunit.client.0.smithi040.stdout:3/746: dwrite d1/d4/d6e/fc9 [0,4194304] 0 2024-02-16T19:24:19.294 INFO:tasks.workunit.client.0.smithi040.stdout:2/597: creat d5/d7/d12/d1b/dae/fc4 x:0 0 0 2024-02-16T19:24:19.295 INFO:tasks.workunit.client.1.smithi163.stdout:3/92: dread d0/dc/d1d/f1f [0,4194304] 0 2024-02-16T19:24:19.295 INFO:tasks.workunit.client.1.smithi163.stdout:2/187: symlink d2/d26/l40 0 2024-02-16T19:24:19.296 INFO:tasks.workunit.client.1.smithi163.stdout:2/188: write d2/f1c [3857187,39854] 0 2024-02-16T19:24:19.298 INFO:tasks.workunit.client.1.smithi163.stdout:0/77: symlink d11/l1c 0 2024-02-16T19:24:19.300 INFO:tasks.workunit.client.0.smithi040.stdout:1/637: mkdir d17/d43/d92/dca 0 2024-02-16T19:24:19.300 INFO:tasks.workunit.client.0.smithi040.stdout:1/638: stat d17/d26/d5f/d78/dac/fbe 0 2024-02-16T19:24:19.301 INFO:tasks.workunit.client.0.smithi040.stdout:6/759: dwrite d0/d1/d12/d3b/d48/d73/d70/fa6 [0,4194304] 0 2024-02-16T19:24:19.307 INFO:tasks.workunit.client.1.smithi163.stdout:6/59: mkdir d0/d6/df 0 2024-02-16T19:24:19.308 INFO:tasks.workunit.client.0.smithi040.stdout:4/569: dwrite dd/de/d10/f1b [0,4194304] 0 2024-02-16T19:24:19.308 INFO:tasks.workunit.client.0.smithi040.stdout:2/598: dread d5/d7/d1e/d31/f34 [0,4194304] 0 2024-02-16T19:24:19.308 INFO:tasks.workunit.client.0.smithi040.stdout:0/764: link db/d2f/d1d/d21/d2e/d85/d89/d94/fb2 db/de/d6a/df0/f107 0 2024-02-16T19:24:19.308 INFO:tasks.workunit.client.1.smithi163.stdout:3/93: creat d0/dc/d1c/f20 x:0 0 0 2024-02-16T19:24:19.308 INFO:tasks.workunit.client.1.smithi163.stdout:3/94: chown d0/f1 792 1 2024-02-16T19:24:19.309 INFO:tasks.workunit.client.1.smithi163.stdout:2/189: write d2/d26/f31 [2814651,95797] 0 2024-02-16T19:24:19.309 INFO:tasks.workunit.client.1.smithi163.stdout:2/190: write d2/d26/f35 [932958,60563] 0 2024-02-16T19:24:19.309 INFO:tasks.workunit.client.0.smithi040.stdout:8/751: getdents d6/d12/d51/d6e/db7/dc6/dcc 0 2024-02-16T19:24:19.309 INFO:tasks.workunit.client.1.smithi163.stdout:6/60: write d0/d6/f8 [1714611,105837] 0 2024-02-16T19:24:19.309 INFO:tasks.workunit.client.1.smithi163.stdout:6/61: readlink d0/lc 0 2024-02-16T19:24:19.311 INFO:tasks.workunit.client.1.smithi163.stdout:0/78: creat d11/f1d x:0 0 0 2024-02-16T19:24:19.312 INFO:tasks.workunit.client.0.smithi040.stdout:0/765: dread db/d2f/d1d/d21/d2e/d49/d5b/f79 [0,4194304] 0 2024-02-16T19:24:19.312 INFO:tasks.workunit.client.0.smithi040.stdout:0/766: write db/de/dbe/fe1 [1037768,10344] 0 2024-02-16T19:24:19.312 INFO:tasks.workunit.client.0.smithi040.stdout:7/587: mkdir d21/d4d/d77/d7a/d99/dbe 0 2024-02-16T19:24:19.315 INFO:tasks.workunit.client.1.smithi163.stdout:3/95: mkdir d0/dc/d1c/d21 0 2024-02-16T19:24:19.317 INFO:tasks.workunit.client.0.smithi040.stdout:7/588: dread d21/d4d/d77/d87/d9b/f9e [0,4194304] 0 2024-02-16T19:24:19.317 INFO:tasks.workunit.client.0.smithi040.stdout:7/589: creat d21/d4d/d77/d7a/fbf x:0 0 0 2024-02-16T19:24:19.318 INFO:tasks.workunit.client.1.smithi163.stdout:2/191: mknod d2/d2a/c41 0 2024-02-16T19:24:19.324 INFO:tasks.workunit.client.1.smithi163.stdout:0/79: rmdir d11/d12 39 2024-02-16T19:24:19.326 INFO:tasks.workunit.client.0.smithi040.stdout:3/747: rename d1/dc/d35/d48/d85/f90 to d1/dc/d42/d4a/d57/d79/feb 0 2024-02-16T19:24:19.329 INFO:tasks.workunit.client.1.smithi163.stdout:2/192: link d2/f19 d2/d2a/d39/f42 0 2024-02-16T19:24:19.329 INFO:tasks.workunit.client.1.smithi163.stdout:2/193: write d2/f37 [162349,19822] 0 2024-02-16T19:24:19.330 INFO:tasks.workunit.client.1.smithi163.stdout:0/80: symlink d11/d12/l1e 0 2024-02-16T19:24:19.331 INFO:tasks.workunit.client.1.smithi163.stdout:0/81: fdatasync f8 0 2024-02-16T19:24:19.331 INFO:tasks.workunit.client.1.smithi163.stdout:0/82: write f8 [5674165,57233] 0 2024-02-16T19:24:19.332 INFO:tasks.workunit.client.0.smithi040.stdout:4/570: getdents dd 0 2024-02-16T19:24:19.333 INFO:tasks.workunit.client.1.smithi163.stdout:2/194: mknod d2/d2a/d39/d3a/c43 0 2024-02-16T19:24:19.335 INFO:tasks.workunit.client.0.smithi040.stdout:2/599: symlink d5/d7/d1e/d55/da3/lc5 0 2024-02-16T19:24:19.336 INFO:tasks.workunit.client.1.smithi163.stdout:0/83: unlink ce 0 2024-02-16T19:24:19.339 INFO:tasks.workunit.client.1.smithi163.stdout:6/62: dwrite d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:19.339 INFO:tasks.workunit.client.0.smithi040.stdout:0/767: dwrite db/d2f/d1d/d21/d2e/d49/fa3 [0,4194304] 0 2024-02-16T19:24:19.339 INFO:tasks.workunit.client.0.smithi040.stdout:8/752: mkdir d6/d12/d51/d6e/db7/dc6/dea 0 2024-02-16T19:24:19.339 INFO:tasks.workunit.client.0.smithi040.stdout:8/753: chown d6/d7/l37 204370673 1 2024-02-16T19:24:19.340 INFO:tasks.workunit.client.1.smithi163.stdout:3/96: dwrite d0/f1a [0,4194304] 0 2024-02-16T19:24:19.340 INFO:tasks.workunit.client.1.smithi163.stdout:3/97: getdents d0/dc/d1c/d21 0 2024-02-16T19:24:19.340 INFO:tasks.workunit.client.1.smithi163.stdout:3/98: fdatasync d0/f19 0 2024-02-16T19:24:19.340 INFO:tasks.workunit.client.1.smithi163.stdout:3/99: truncate d0/dc/d1c/f20 113940 0 2024-02-16T19:24:19.342 INFO:tasks.workunit.client.0.smithi040.stdout:7/590: creat d21/d29/d79/dbc/fc0 x:0 0 0 2024-02-16T19:24:19.342 INFO:tasks.workunit.client.0.smithi040.stdout:7/591: readlink d21/d4d/l9a 0 2024-02-16T19:24:19.345 INFO:tasks.workunit.client.0.smithi040.stdout:4/571: write dd/de/d45/f74 [3425435,47229] 0 2024-02-16T19:24:19.346 INFO:tasks.workunit.client.1.smithi163.stdout:6/63: symlink d0/d6/l10 0 2024-02-16T19:24:19.349 INFO:tasks.workunit.client.0.smithi040.stdout:6/760: rename d0/d30/d32/l53 to d0/d1/d12/d3b/d48/d4f/lf9 0 2024-02-16T19:24:19.349 INFO:tasks.workunit.client.0.smithi040.stdout:6/761: fsync d0/d30/d5b/f9f 0 2024-02-16T19:24:19.350 INFO:tasks.workunit.client.0.smithi040.stdout:3/748: mknod d1/d7/d36/cec 0 2024-02-16T19:24:19.354 INFO:tasks.workunit.client.0.smithi040.stdout:1/639: getdents d17/d3c/d71 0 2024-02-16T19:24:19.354 INFO:tasks.workunit.client.0.smithi040.stdout:1/640: creat d17/d43/fcb x:0 0 0 2024-02-16T19:24:19.356 INFO:tasks.workunit.client.0.smithi040.stdout:2/600: truncate d5/d8/d42/f7a 273570 0 2024-02-16T19:24:19.363 INFO:tasks.workunit.client.0.smithi040.stdout:7/592: mknod d21/d4d/d77/d7a/d99/dbe/cc1 0 2024-02-16T19:24:19.365 INFO:tasks.workunit.client.1.smithi163.stdout:3/100: mknod d0/c22 0 2024-02-16T19:24:19.365 INFO:tasks.workunit.client.1.smithi163.stdout:3/101: creat d0/dc/d1c/f23 x:0 0 0 2024-02-16T19:24:19.366 INFO:tasks.workunit.client.0.smithi040.stdout:3/749: write d1/f3 [4285305,104558] 0 2024-02-16T19:24:19.366 INFO:tasks.workunit.client.0.smithi040.stdout:4/572: truncate dd/d58/d5e/f69 1194071 0 2024-02-16T19:24:19.366 INFO:tasks.workunit.client.0.smithi040.stdout:4/573: write fb [2964679,119139] 0 2024-02-16T19:24:19.367 INFO:tasks.workunit.client.0.smithi040.stdout:4/574: chown dd/de/d10/fa9 148743 1 2024-02-16T19:24:19.368 INFO:tasks.workunit.client.1.smithi163.stdout:0/84: dwrite f6 [0,4194304] 0 2024-02-16T19:24:19.368 INFO:tasks.workunit.client.0.smithi040.stdout:6/762: dwrite d0/d30/d32/dd2/d87/f8e [0,4194304] 0 2024-02-16T19:24:19.368 INFO:tasks.workunit.client.0.smithi040.stdout:6/763: write d0/d30/d5b/dbe/fd9 [776794,95015] 0 2024-02-16T19:24:19.369 INFO:tasks.workunit.client.0.smithi040.stdout:6/764: fsync d0/d1/d12/d3b/d79/f88 0 2024-02-16T19:24:19.369 INFO:tasks.workunit.client.0.smithi040.stdout:6/765: write d0/d1/d12/d3b/d48/d73/d70/fc9 [212727,3016] 0 2024-02-16T19:24:19.371 INFO:tasks.workunit.client.0.smithi040.stdout:0/768: rename db/d25/d2c/c8d to db/c108 0 2024-02-16T19:24:19.371 INFO:tasks.workunit.client.0.smithi040.stdout:0/769: chown db/d25/d2c/fad 49460664 1 2024-02-16T19:24:19.371 INFO:tasks.workunit.client.0.smithi040.stdout:0/770: stat db/d2f/d1d/d21/fe3 0 2024-02-16T19:24:19.371 INFO:tasks.workunit.client.0.smithi040.stdout:0/771: chown db/d2f/d23/d3b/dd5 795800 1 2024-02-16T19:24:19.373 INFO:tasks.workunit.client.1.smithi163.stdout:6/64: mkdir d0/d6/d11 0 2024-02-16T19:24:19.376 INFO:tasks.workunit.client.0.smithi040.stdout:1/641: symlink d17/d3c/lcc 0 2024-02-16T19:24:19.378 INFO:tasks.workunit.client.1.smithi163.stdout:3/102: creat d0/dc/f24 x:0 0 0 2024-02-16T19:24:19.378 INFO:tasks.workunit.client.1.smithi163.stdout:3/103: read d0/f1 [2859557,66278] 0 2024-02-16T19:24:19.379 INFO:tasks.workunit.client.0.smithi040.stdout:7/593: creat d21/d29/fc2 x:0 0 0 2024-02-16T19:24:19.383 INFO:tasks.workunit.client.1.smithi163.stdout:0/85: unlink ff 0 2024-02-16T19:24:19.383 INFO:tasks.workunit.client.1.smithi163.stdout:0/86: creat d11/f1f x:0 0 0 2024-02-16T19:24:19.383 INFO:tasks.workunit.client.0.smithi040.stdout:3/750: dwrite d1/d4/d6e/fc9 [4194304,4194304] 0 2024-02-16T19:24:19.384 INFO:tasks.workunit.client.1.smithi163.stdout:6/65: creat d0/dd/f12 x:0 0 0 2024-02-16T19:24:19.384 INFO:tasks.workunit.client.1.smithi163.stdout:6/66: write d0/dd/f12 [802479,77207] 0 2024-02-16T19:24:19.384 INFO:tasks.workunit.client.1.smithi163.stdout:6/67: creat d0/dd/f13 x:0 0 0 2024-02-16T19:24:19.385 INFO:tasks.workunit.client.0.smithi040.stdout:4/575: symlink dd/de/d10/d36/d9e/d32/d7a/lb5 0 2024-02-16T19:24:19.386 INFO:tasks.workunit.client.0.smithi040.stdout:4/576: creat dd/de/d10/fb6 x:0 0 0 2024-02-16T19:24:19.387 INFO:tasks.workunit.client.1.smithi163.stdout:3/104: creat d0/dc/d1c/d21/f25 x:0 0 0 2024-02-16T19:24:19.388 INFO:tasks.workunit.client.1.smithi163.stdout:6/68: dread d0/dd/f12 [0,4194304] 0 2024-02-16T19:24:19.388 INFO:tasks.workunit.client.1.smithi163.stdout:6/69: truncate d0/f2 4570078 0 2024-02-16T19:24:19.395 INFO:tasks.workunit.client.1.smithi163.stdout:6/70: read d0/d6/f8 [888892,90127] 0 2024-02-16T19:24:19.395 INFO:tasks.workunit.client.1.smithi163.stdout:6/71: stat d0/c4 0 2024-02-16T19:24:19.395 INFO:tasks.workunit.client.1.smithi163.stdout:0/87: dread f10 [0,4194304] 0 2024-02-16T19:24:19.396 INFO:tasks.workunit.client.0.smithi040.stdout:6/766: mkdir d0/d30/d32/dd2/dbc/dfa 0 2024-02-16T19:24:19.399 INFO:tasks.workunit.client.1.smithi163.stdout:6/72: symlink d0/l14 0 2024-02-16T19:24:19.403 INFO:tasks.workunit.client.0.smithi040.stdout:6/767: dread d0/d1/f16 [0,4194304] 0 2024-02-16T19:24:19.403 INFO:tasks.workunit.client.0.smithi040.stdout:6/768: truncate d0/d30/d4b/f93 1702933 0 2024-02-16T19:24:19.403 INFO:tasks.workunit.client.0.smithi040.stdout:4/577: dwrite dd/d58/fa1 [0,4194304] 0 2024-02-16T19:24:19.403 INFO:tasks.workunit.client.0.smithi040.stdout:4/578: fdatasync dd/de/d10/d36/dad/fb4 0 2024-02-16T19:24:19.403 INFO:tasks.workunit.client.0.smithi040.stdout:4/579: write dd/de/d10/f20 [5146585,86237] 0 2024-02-16T19:24:19.404 INFO:tasks.workunit.client.0.smithi040.stdout:4/580: write dd/de/f80 [438790,7473] 0 2024-02-16T19:24:19.408 INFO:tasks.workunit.client.1.smithi163.stdout:0/88: write f8 [1005171,46121] 0 2024-02-16T19:24:19.408 INFO:tasks.workunit.client.1.smithi163.stdout:0/89: readlink d11/d12/l13 0 2024-02-16T19:24:19.408 INFO:tasks.workunit.client.1.smithi163.stdout:0/90: chown d11/d12/l13 49 1 2024-02-16T19:24:19.410 INFO:tasks.workunit.client.0.smithi040.stdout:0/772: mknod db/d2f/d1d/d21/d2e/c109 0 2024-02-16T19:24:19.412 INFO:tasks.workunit.client.0.smithi040.stdout:1/642: creat d17/d26/d4c/fcd x:0 0 0 2024-02-16T19:24:19.413 INFO:tasks.workunit.client.0.smithi040.stdout:7/594: mknod d21/d8b/cc3 0 2024-02-16T19:24:19.413 INFO:tasks.workunit.client.0.smithi040.stdout:7/595: write d21/d4d/f61 [1357247,59527] 0 2024-02-16T19:24:19.413 INFO:tasks.workunit.client.1.smithi163.stdout:3/105: rmdir d0/dc/d1c/d21 39 2024-02-16T19:24:19.413 INFO:tasks.workunit.client.1.smithi163.stdout:3/106: write d0/f1 [3736542,117015] 0 2024-02-16T19:24:19.416 INFO:tasks.workunit.client.0.smithi040.stdout:3/751: symlink d1/d4/d6e/dc2/led 0 2024-02-16T19:24:19.420 INFO:tasks.workunit.client.1.smithi163.stdout:0/91: dread f7 [0,4194304] 0 2024-02-16T19:24:19.420 INFO:tasks.workunit.client.1.smithi163.stdout:0/92: chown d11/f1a 357 1 2024-02-16T19:24:19.420 INFO:tasks.workunit.client.0.smithi040.stdout:6/769: creat d0/d1/d12/d3b/d48/dba/dde/dcd/ffb x:0 0 0 2024-02-16T19:24:19.422 INFO:tasks.workunit.client.1.smithi163.stdout:6/73: rmdir d0/dd 39 2024-02-16T19:24:19.422 INFO:tasks.workunit.client.1.smithi163.stdout:6/74: fsync d0/d6/f8 0 2024-02-16T19:24:19.422 INFO:tasks.workunit.client.1.smithi163.stdout:6/75: readlink d0/d6/le 0 2024-02-16T19:24:19.425 INFO:tasks.workunit.client.0.smithi040.stdout:0/773: truncate db/d25/d34/d5e/fd0 548954 0 2024-02-16T19:24:19.429 INFO:tasks.workunit.client.0.smithi040.stdout:1/643: rmdir d17/d3c/d6b 39 2024-02-16T19:24:19.429 INFO:tasks.workunit.client.0.smithi040.stdout:8/754: rename d6/d2b/l55 to d6/d7/d15/leb 0 2024-02-16T19:24:19.429 INFO:tasks.workunit.client.0.smithi040.stdout:1/644: creat d17/d43/d92/fce x:0 0 0 2024-02-16T19:24:19.431 INFO:tasks.workunit.client.1.smithi163.stdout:0/93: read fd [190936,57275] 0 2024-02-16T19:24:19.431 INFO:tasks.workunit.client.0.smithi040.stdout:3/752: unlink d1/dc/d35/d53/d64/d65/c8d 0 2024-02-16T19:24:19.432 INFO:tasks.workunit.client.0.smithi040.stdout:6/770: creat d0/d30/d32/dd2/ffc x:0 0 0 2024-02-16T19:24:19.432 INFO:tasks.workunit.client.0.smithi040.stdout:6/771: dread - d0/d1/d12/d3b/d48/d73/d76/fd5 zero size 2024-02-16T19:24:19.433 INFO:tasks.workunit.client.0.smithi040.stdout:6/772: write d0/d1/d6f/fda [923594,70615] 0 2024-02-16T19:24:19.433 INFO:tasks.workunit.client.0.smithi040.stdout:7/596: dwrite d21/d4d/d77/d7a/d99/fb3 [0,4194304] 0 2024-02-16T19:24:19.433 INFO:tasks.workunit.client.0.smithi040.stdout:7/597: creat d21/d29/d79/fc4 x:0 0 0 2024-02-16T19:24:19.433 INFO:tasks.workunit.client.0.smithi040.stdout:7/598: dread - d21/d4d/d77/fa3 zero size 2024-02-16T19:24:19.433 INFO:tasks.workunit.client.0.smithi040.stdout:7/599: fdatasync d21/d29/f2b 0 2024-02-16T19:24:19.436 INFO:tasks.workunit.client.1.smithi163.stdout:0/94: mkdir d11/d12/d20 0 2024-02-16T19:24:19.439 INFO:tasks.workunit.client.0.smithi040.stdout:4/581: rename dd/d58/d8c/d99/fb3 to dd/d58/fb7 0 2024-02-16T19:24:19.442 INFO:tasks.workunit.client.0.smithi040.stdout:6/773: getdents d0/d1/d12 0 2024-02-16T19:24:19.442 INFO:tasks.workunit.client.0.smithi040.stdout:6/774: chown d0/d30/d32/dd2/dbc 88355767 1 2024-02-16T19:24:19.443 INFO:tasks.workunit.client.1.smithi163.stdout:0/95: mkdir d11/d21 0 2024-02-16T19:24:19.443 INFO:tasks.workunit.client.1.smithi163.stdout:0/96: write f8 [1453188,34508] 0 2024-02-16T19:24:19.443 INFO:tasks.workunit.client.0.smithi040.stdout:8/755: dread d6/d12/f4b [0,4194304] 0 2024-02-16T19:24:19.445 INFO:tasks.workunit.client.1.smithi163.stdout:0/97: mkdir d11/d22 0 2024-02-16T19:24:19.447 INFO:tasks.workunit.client.0.smithi040.stdout:7/600: getdents d21/d4d/d7b 0 2024-02-16T19:24:19.450 INFO:tasks.workunit.client.1.smithi163.stdout:0/98: rename d11/l1c to d11/d22/l23 0 2024-02-16T19:24:19.450 INFO:tasks.workunit.client.1.smithi163.stdout:3/107: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:19.450 INFO:tasks.workunit.client.1.smithi163.stdout:6/76: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:19.452 INFO:tasks.workunit.client.0.smithi040.stdout:4/582: symlink dd/lb8 0 2024-02-16T19:24:19.453 INFO:tasks.workunit.client.0.smithi040.stdout:1/645: dwrite d17/d26/f7d [0,4194304] 0 2024-02-16T19:24:19.454 INFO:tasks.workunit.client.1.smithi163.stdout:0/99: creat d11/d22/f24 x:0 0 0 2024-02-16T19:24:19.455 INFO:tasks.workunit.client.0.smithi040.stdout:6/775: mknod d0/d30/d32/dd2/d87/d8f/df3/cfd 0 2024-02-16T19:24:19.456 INFO:tasks.workunit.client.0.smithi040.stdout:0/774: dwrite db/d25/d34/d5e/fb4 [0,4194304] 0 2024-02-16T19:24:19.459 INFO:tasks.workunit.client.0.smithi040.stdout:7/601: symlink d21/d29/d79/lc5 0 2024-02-16T19:24:19.459 INFO:tasks.workunit.client.0.smithi040.stdout:7/602: truncate d21/d29/d79/f80 9076 0 2024-02-16T19:24:19.465 INFO:tasks.workunit.client.0.smithi040.stdout:6/776: mknod d0/d1/d12/d3b/d48/dba/cfe 0 2024-02-16T19:24:19.465 INFO:tasks.workunit.client.0.smithi040.stdout:6/777: readlink d0/l19 0 2024-02-16T19:24:19.466 INFO:tasks.workunit.client.1.smithi163.stdout:0/100: dread d11/f1a [0,4194304] 0 2024-02-16T19:24:19.466 INFO:tasks.workunit.client.0.smithi040.stdout:7/603: dread f17 [0,4194304] 0 2024-02-16T19:24:19.470 INFO:tasks.workunit.client.0.smithi040.stdout:0/775: symlink db/d2f/d1d/d21/d2e/d49/d5b/d67/l10a 0 2024-02-16T19:24:19.470 INFO:tasks.workunit.client.0.smithi040.stdout:0/776: fdatasync db/d2f/d1d/d21/d2e/d49/d5b/fb3 0 2024-02-16T19:24:19.470 INFO:tasks.workunit.client.0.smithi040.stdout:0/777: write db/d2f/d23/d6f/fc4 [1082309,30907] 0 2024-02-16T19:24:19.470 INFO:tasks.workunit.client.0.smithi040.stdout:0/778: dread - db/d25/d34/d5e/fd9 zero size 2024-02-16T19:24:19.473 INFO:tasks.workunit.client.1.smithi163.stdout:6/77: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:19.474 INFO:tasks.workunit.client.0.smithi040.stdout:1/646: dwrite d17/f36 [0,4194304] 0 2024-02-16T19:24:19.475 INFO:tasks.workunit.client.1.smithi163.stdout:3/108: mknod d0/dc/d1c/c26 0 2024-02-16T19:24:19.476 INFO:tasks.workunit.client.0.smithi040.stdout:3/753: rename d1/dc/d35/d53/d64/d65/daa to d1/dc/d42/dee 0 2024-02-16T19:24:19.478 INFO:tasks.workunit.client.0.smithi040.stdout:6/778: mkdir d0/d1/d12/d3b/d48/d73/dff 0 2024-02-16T19:24:19.480 INFO:tasks.workunit.client.0.smithi040.stdout:7/604: mkdir d21/d4d/d77/d7a/dc6 0 2024-02-16T19:24:19.480 INFO:tasks.workunit.client.0.smithi040.stdout:7/605: creat d21/d4d/d77/d87/fc7 x:0 0 0 2024-02-16T19:24:19.482 INFO:tasks.workunit.client.0.smithi040.stdout:0/779: truncate db/d2f/d1d/d21/d2e/d49/d5b/d67/f8f 3626173 0 2024-02-16T19:24:19.482 INFO:tasks.workunit.client.0.smithi040.stdout:0/780: write db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/fcc [417788,13867] 0 2024-02-16T19:24:19.482 INFO:tasks.workunit.client.0.smithi040.stdout:0/781: chown db/lab 10467661 1 2024-02-16T19:24:19.485 INFO:tasks.workunit.client.1.smithi163.stdout:0/101: rename d11/l17 to d11/l25 0 2024-02-16T19:24:19.486 INFO:tasks.workunit.client.1.smithi163.stdout:0/102: dread - d11/f1d zero size 2024-02-16T19:24:19.486 INFO:tasks.workunit.client.1.smithi163.stdout:0/103: dread - d11/d12/f15 zero size 2024-02-16T19:24:19.486 INFO:tasks.workunit.client.0.smithi040.stdout:1/647: mkdir d17/d29/d8b/d93/dcf 0 2024-02-16T19:24:19.487 INFO:tasks.workunit.client.0.smithi040.stdout:1/648: truncate d17/d26/d3d/fb6 887840 0 2024-02-16T19:24:19.488 INFO:tasks.workunit.client.1.smithi163.stdout:6/78: mkdir d0/d6/d9/d15 0 2024-02-16T19:24:19.490 INFO:tasks.workunit.client.0.smithi040.stdout:6/779: write d0/d30/d32/f56 [3811559,99945] 0 2024-02-16T19:24:19.490 INFO:tasks.workunit.client.1.smithi163.stdout:0/104: dread d11/f1a [0,4194304] 0 2024-02-16T19:24:19.492 INFO:tasks.workunit.client.0.smithi040.stdout:8/756: rename d6/d2b/da3/d8b/fb0 to d6/d12/d51/d6e/db9/dcf/fec 0 2024-02-16T19:24:19.492 INFO:tasks.workunit.client.0.smithi040.stdout:8/757: chown d6/d12/d93/l9f 870370 1 2024-02-16T19:24:19.494 INFO:tasks.workunit.client.1.smithi163.stdout:3/109: rename d0/c22 to d0/dc/d1d/c27 0 2024-02-16T19:24:19.494 INFO:tasks.workunit.client.1.smithi163.stdout:3/110: creat d0/dc/f28 x:0 0 0 2024-02-16T19:24:19.494 INFO:tasks.workunit.client.1.smithi163.stdout:3/111: chown d0/dc/le 20 1 2024-02-16T19:24:19.494 INFO:tasks.workunit.client.1.smithi163.stdout:3/112: chown d0/dc/d1d/c27 11976222 1 2024-02-16T19:24:19.497 INFO:tasks.workunit.client.1.smithi163.stdout:8/51: sync 2024-02-16T19:24:19.497 INFO:tasks.workunit.client.1.smithi163.stdout:4/54: sync 2024-02-16T19:24:19.497 INFO:tasks.workunit.client.1.smithi163.stdout:1/34: sync 2024-02-16T19:24:19.497 INFO:tasks.workunit.client.1.smithi163.stdout:5/45: sync 2024-02-16T19:24:19.497 INFO:tasks.workunit.client.1.smithi163.stdout:8/52: chown d0/d7 1232835 1 2024-02-16T19:24:19.497 INFO:tasks.workunit.client.1.smithi163.stdout:5/46: fsync d0/d2/fa 0 2024-02-16T19:24:19.497 INFO:tasks.workunit.client.1.smithi163.stdout:5/47: creat d0/fd x:0 0 0 2024-02-16T19:24:19.497 INFO:tasks.workunit.client.1.smithi163.stdout:5/48: write d0/d2/fb [339391,47230] 0 2024-02-16T19:24:19.499 INFO:tasks.workunit.client.1.smithi163.stdout:0/105: symlink d11/d12/l26 0 2024-02-16T19:24:19.499 INFO:tasks.workunit.client.1.smithi163.stdout:0/106: dread - d11/f1d zero size 2024-02-16T19:24:19.500 INFO:tasks.workunit.client.0.smithi040.stdout:3/754: mknod d1/dc/d42/d4a/d57/cef 0 2024-02-16T19:24:19.500 INFO:tasks.workunit.client.0.smithi040.stdout:3/755: creat d1/ff0 x:0 0 0 2024-02-16T19:24:19.500 INFO:tasks.workunit.client.0.smithi040.stdout:3/756: chown d1/d7/d33/c3e 1060 1 2024-02-16T19:24:19.500 INFO:tasks.workunit.client.0.smithi040.stdout:3/757: readlink d1/dc/d42/d5c/l9d 0 2024-02-16T19:24:19.501 INFO:tasks.workunit.client.1.smithi163.stdout:9/261: sync 2024-02-16T19:24:19.501 INFO:tasks.workunit.client.1.smithi163.stdout:2/195: sync 2024-02-16T19:24:19.501 INFO:tasks.workunit.client.1.smithi163.stdout:7/67: sync 2024-02-16T19:24:19.501 INFO:tasks.workunit.client.1.smithi163.stdout:7/68: write d11/f12 [702852,10399] 0 2024-02-16T19:24:19.501 INFO:tasks.workunit.client.1.smithi163.stdout:7/69: readlink l10 0 2024-02-16T19:24:19.503 INFO:tasks.workunit.client.0.smithi040.stdout:7/606: symlink d21/d4d/d77/lc8 0 2024-02-16T19:24:19.503 INFO:tasks.workunit.client.0.smithi040.stdout:7/607: dread - d21/d4d/d77/d87/fc7 zero size 2024-02-16T19:24:19.503 INFO:tasks.workunit.client.1.smithi163.stdout:2/196: dread d2/d26/f27 [0,4194304] 0 2024-02-16T19:24:19.504 INFO:tasks.workunit.client.1.smithi163.stdout:3/113: mkdir d0/dc/d1d/d29 0 2024-02-16T19:24:19.504 INFO:tasks.workunit.client.1.smithi163.stdout:0/107: dread f10 [0,4194304] 0 2024-02-16T19:24:19.504 INFO:tasks.workunit.client.1.smithi163.stdout:0/108: write d11/f1f [861978,15959] 0 2024-02-16T19:24:19.506 INFO:tasks.workunit.client.0.smithi040.stdout:6/780: link d0/d1/d12/d3b/d48/d73/d70/fa5 d0/d37/de1/f100 0 2024-02-16T19:24:19.507 INFO:tasks.workunit.client.1.smithi163.stdout:8/53: mknod d0/d7/c11 0 2024-02-16T19:24:19.508 INFO:tasks.workunit.client.0.smithi040.stdout:6/781: dread d0/d1/d12/d3b/d48/dba/dde/fab [0,4194304] 0 2024-02-16T19:24:19.509 INFO:tasks.workunit.client.0.smithi040.stdout:8/758: mknod d6/d12/d51/d6e/db9/ced 0 2024-02-16T19:24:19.509 INFO:tasks.workunit.client.0.smithi040.stdout:8/759: chown d6/d2b/d36/c66 513577914 1 2024-02-16T19:24:19.509 INFO:tasks.workunit.client.0.smithi040.stdout:3/758: write d1/dc/fd [1673871,96420] 0 2024-02-16T19:24:19.510 INFO:tasks.workunit.client.1.smithi163.stdout:1/35: read f2 [4506340,77510] 0 2024-02-16T19:24:19.510 INFO:tasks.workunit.client.1.smithi163.stdout:1/36: fdatasync f2 0 2024-02-16T19:24:19.512 INFO:tasks.workunit.client.0.smithi040.stdout:7/608: mkdir d21/d4d/d77/d87/dc9 0 2024-02-16T19:24:19.513 INFO:tasks.workunit.client.1.smithi163.stdout:9/262: rmdir d5/de/d4e 0 2024-02-16T19:24:19.514 INFO:tasks.workunit.client.0.smithi040.stdout:6/782: creat d0/df0/f101 x:0 0 0 2024-02-16T19:24:19.515 INFO:tasks.workunit.client.1.smithi163.stdout:2/197: rmdir d2/d2a/d39/d3a 39 2024-02-16T19:24:19.515 INFO:tasks.workunit.client.1.smithi163.stdout:2/198: truncate d2/d26/f2f 984748 0 2024-02-16T19:24:19.515 INFO:tasks.workunit.client.1.smithi163.stdout:2/199: dread - d2/f23 zero size 2024-02-16T19:24:19.516 INFO:tasks.workunit.client.1.smithi163.stdout:3/114: symlink d0/dc/d1c/d21/l2a 0 2024-02-16T19:24:19.517 INFO:tasks.workunit.client.0.smithi040.stdout:6/783: write d0/d30/d4b/fca [611224,62248] 0 2024-02-16T19:24:19.517 INFO:tasks.workunit.client.1.smithi163.stdout:3/115: write d0/f6 [22023,94583] 0 2024-02-16T19:24:19.517 INFO:tasks.workunit.client.1.smithi163.stdout:0/109: creat d11/d12/f27 x:0 0 0 2024-02-16T19:24:19.517 INFO:tasks.workunit.client.1.smithi163.stdout:0/110: write f3 [5630928,126866] 0 2024-02-16T19:24:19.519 INFO:tasks.workunit.client.0.smithi040.stdout:1/649: dwrite d17/f61 [0,4194304] 0 2024-02-16T19:24:19.519 INFO:tasks.workunit.client.0.smithi040.stdout:1/650: getdents d17/d43/d92/dca 0 2024-02-16T19:24:19.519 INFO:tasks.workunit.client.1.smithi163.stdout:7/70: fdatasync d11/f12 0 2024-02-16T19:24:19.520 INFO:tasks.workunit.client.1.smithi163.stdout:6/79: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:24:19.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/71: chown le 261 1 2024-02-16T19:24:19.521 INFO:tasks.workunit.client.0.smithi040.stdout:0/782: dwrite db/d2f/f70 [0,4194304] 0 2024-02-16T19:24:19.522 INFO:tasks.workunit.client.0.smithi040.stdout:7/609: symlink d21/d7e/lca 0 2024-02-16T19:24:19.522 INFO:tasks.workunit.client.1.smithi163.stdout:4/55: dread f8 [0,4194304] 0 2024-02-16T19:24:19.525 INFO:tasks.workunit.client.1.smithi163.stdout:1/37: creat f6 x:0 0 0 2024-02-16T19:24:19.528 INFO:tasks.workunit.client.1.smithi163.stdout:4/56: write f9 [2646704,106840] 0 2024-02-16T19:24:19.528 INFO:tasks.workunit.client.1.smithi163.stdout:4/57: stat l0 0 2024-02-16T19:24:19.529 INFO:tasks.workunit.client.1.smithi163.stdout:9/263: creat d5/de/d30/f5e x:0 0 0 2024-02-16T19:24:19.530 INFO:tasks.workunit.client.0.smithi040.stdout:6/784: rename d0/d1/d6f/d90/fb6 to d0/d30/d32/dd2/dbc/de7/f102 0 2024-02-16T19:24:19.531 INFO:tasks.workunit.client.1.smithi163.stdout:2/200: mkdir d2/d44 0 2024-02-16T19:24:19.533 INFO:tasks.workunit.client.1.smithi163.stdout:3/116: symlink d0/dc/d1d/d29/l2b 0 2024-02-16T19:24:19.535 INFO:tasks.workunit.client.1.smithi163.stdout:0/111: mknod d11/d22/c28 0 2024-02-16T19:24:19.535 INFO:tasks.workunit.client.1.smithi163.stdout:0/112: chown f10 657388665 1 2024-02-16T19:24:19.536 INFO:tasks.workunit.client.0.smithi040.stdout:1/651: link d17/d3c/d74/d97/da9/lbb d17/d3c/d74/d97/da9/ld0 0 2024-02-16T19:24:19.536 INFO:tasks.workunit.client.0.smithi040.stdout:1/652: getdents d17/d26/d9a 0 2024-02-16T19:24:19.536 INFO:tasks.workunit.client.0.smithi040.stdout:1/653: stat d17/d2d/c31 0 2024-02-16T19:24:19.536 INFO:tasks.workunit.client.1.smithi163.stdout:7/72: symlink d11/l16 0 2024-02-16T19:24:19.538 INFO:tasks.workunit.client.1.smithi163.stdout:0/113: write d11/f1a [188463,100971] 0 2024-02-16T19:24:19.538 INFO:tasks.workunit.client.1.smithi163.stdout:6/80: symlink d0/d6/l16 0 2024-02-16T19:24:19.543 INFO:tasks.workunit.client.1.smithi163.stdout:1/38: mkdir d7 0 2024-02-16T19:24:19.546 INFO:tasks.workunit.client.1.smithi163.stdout:4/58: rename f8 to fd 0 2024-02-16T19:24:19.548 INFO:tasks.workunit.client.1.smithi163.stdout:3/117: write d0/f13 [3973989,9741] 0 2024-02-16T19:24:19.548 INFO:tasks.workunit.client.1.smithi163.stdout:3/118: readlink d0/dc/d1d/d29/l2b 0 2024-02-16T19:24:19.548 INFO:tasks.workunit.client.1.smithi163.stdout:3/119: creat d0/dc/d1c/f2c x:0 0 0 2024-02-16T19:24:19.548 INFO:tasks.workunit.client.1.smithi163.stdout:3/120: dread - d0/dc/d1c/d21/f25 zero size 2024-02-16T19:24:19.549 INFO:tasks.workunit.client.0.smithi040.stdout:1/654: write d17/d26/f7d [12020860,49107] 0 2024-02-16T19:24:19.550 INFO:tasks.workunit.client.1.smithi163.stdout:5/49: dwrite d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:19.550 INFO:tasks.workunit.client.1.smithi163.stdout:9/264: dread d5/de/f28 [0,4194304] 0 2024-02-16T19:24:19.550 INFO:tasks.workunit.client.1.smithi163.stdout:9/265: write d5/f1e [415877,62495] 0 2024-02-16T19:24:19.558 INFO:tasks.workunit.client.0.smithi040.stdout:8/760: dwrite d6/d64/d50/f74 [4194304,4194304] 0 2024-02-16T19:24:19.558 INFO:tasks.workunit.client.0.smithi040.stdout:7/610: dwrite d21/d4d/f6f [0,4194304] 0 2024-02-16T19:24:19.558 INFO:tasks.workunit.client.0.smithi040.stdout:3/759: dwrite d1/dc/d42/dee/fd9 [0,4194304] 0 2024-02-16T19:24:19.559 INFO:tasks.workunit.client.1.smithi163.stdout:0/114: symlink d11/d21/l29 0 2024-02-16T19:24:19.559 INFO:tasks.workunit.client.1.smithi163.stdout:0/115: chown d11/f1f 1 1 2024-02-16T19:24:19.559 INFO:tasks.workunit.client.0.smithi040.stdout:8/761: write d6/d2b/d36/f4f [19277,42407] 0 2024-02-16T19:24:19.559 INFO:tasks.workunit.client.0.smithi040.stdout:8/762: write d6/fdc [141275,3480] 0 2024-02-16T19:24:19.561 INFO:tasks.workunit.client.1.smithi163.stdout:7/73: dwrite f3 [0,4194304] 0 2024-02-16T19:24:19.561 INFO:tasks.workunit.client.1.smithi163.stdout:6/81: mkdir d0/d6/d11/d17 0 2024-02-16T19:24:19.563 INFO:tasks.workunit.client.0.smithi040.stdout:0/783: mknod db/d2f/d1d/d21/d2e/d49/d5b/c10b 0 2024-02-16T19:24:19.564 INFO:tasks.workunit.client.1.smithi163.stdout:1/39: symlink d7/l8 0 2024-02-16T19:24:19.565 INFO:tasks.workunit.client.1.smithi163.stdout:8/54: rmdir d0/d7 39 2024-02-16T19:24:19.572 INFO:tasks.workunit.client.0.smithi040.stdout:1/655: mknod d17/d26/d5f/cd1 0 2024-02-16T19:24:19.574 INFO:tasks.workunit.client.1.smithi163.stdout:3/121: dwrite d0/dc/f17 [0,4194304] 0 2024-02-16T19:24:19.577 INFO:tasks.workunit.client.1.smithi163.stdout:9/266: unlink d5/de/d30/f3e 0 2024-02-16T19:24:19.577 INFO:tasks.workunit.client.1.smithi163.stdout:9/267: read d5/de/d16/f25 [123631,107190] 0 2024-02-16T19:24:19.583 INFO:tasks.workunit.client.0.smithi040.stdout:7/611: mkdir d21/d4d/d77/d87/dc9/dcb 0 2024-02-16T19:24:19.586 INFO:tasks.workunit.client.0.smithi040.stdout:8/763: link d6/d2b/d36/f3e d6/d12/d51/d6e/db7/ddf/fee 0 2024-02-16T19:24:19.589 INFO:tasks.workunit.client.1.smithi163.stdout:9/268: write d5/f2e [3387254,93222] 0 2024-02-16T19:24:19.591 INFO:tasks.workunit.client.1.smithi163.stdout:7/74: unlink f9 0 2024-02-16T19:24:19.591 INFO:tasks.workunit.client.1.smithi163.stdout:5/50: dwrite d0/d2/fb [0,4194304] 0 2024-02-16T19:24:19.593 INFO:tasks.workunit.client.0.smithi040.stdout:1/656: unlink d17/d2d/f2e 0 2024-02-16T19:24:19.594 INFO:tasks.workunit.client.1.smithi163.stdout:0/116: dwrite d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:19.595 INFO:tasks.workunit.client.0.smithi040.stdout:7/612: mknod d21/d4d/d7b/ccc 0 2024-02-16T19:24:19.595 INFO:tasks.workunit.client.1.smithi163.stdout:3/122: dwrite d0/f6 [0,4194304] 0 2024-02-16T19:24:19.595 INFO:tasks.workunit.client.1.smithi163.stdout:3/123: truncate d0/f19 1006861 0 2024-02-16T19:24:19.596 INFO:tasks.workunit.client.1.smithi163.stdout:6/82: mkdir d0/d18 0 2024-02-16T19:24:19.597 INFO:tasks.workunit.client.0.smithi040.stdout:8/764: truncate d6/d7/d15/f67 628178 0 2024-02-16T19:24:19.597 INFO:tasks.workunit.client.0.smithi040.stdout:8/765: chown d6/cae 1 1 2024-02-16T19:24:19.599 INFO:tasks.workunit.client.0.smithi040.stdout:0/784: getdents db/de/d6a/df0 0 2024-02-16T19:24:19.603 INFO:tasks.workunit.client.0.smithi040.stdout:1/657: mkdir d17/d29/d8b/dd2 0 2024-02-16T19:24:19.606 INFO:tasks.workunit.client.0.smithi040.stdout:0/785: dread db/d2f/d1d/d21/d2e/d49/d5b/d67/f92 [0,4194304] 0 2024-02-16T19:24:19.606 INFO:tasks.workunit.client.0.smithi040.stdout:0/786: write db/d2f/d23/d56/fe6 [469262,47610] 0 2024-02-16T19:24:19.606 INFO:tasks.workunit.client.0.smithi040.stdout:0/787: read - db/d2f/d1d/d21/d2e/fff zero size 2024-02-16T19:24:19.609 INFO:tasks.workunit.client.1.smithi163.stdout:6/83: write d0/f5 [3120773,64444] 0 2024-02-16T19:24:19.609 INFO:tasks.workunit.client.1.smithi163.stdout:6/84: chown d0/d6/d11/d17 451404972 1 2024-02-16T19:24:19.610 INFO:tasks.workunit.client.1.smithi163.stdout:1/40: creat d7/f9 x:0 0 0 2024-02-16T19:24:19.612 INFO:tasks.workunit.client.1.smithi163.stdout:0/117: dwrite d11/d12/f27 [0,4194304] 0 2024-02-16T19:24:19.616 INFO:tasks.workunit.client.1.smithi163.stdout:8/55: mknod d0/d7/c12 0 2024-02-16T19:24:19.616 INFO:tasks.workunit.client.1.smithi163.stdout:8/56: chown d0/d7/c11 2020055 1 2024-02-16T19:24:19.616 INFO:tasks.workunit.client.0.smithi040.stdout:6/785: write d0/fe [195577,34474] 0 2024-02-16T19:24:19.620 INFO:tasks.workunit.client.0.smithi040.stdout:1/658: mkdir d17/d26/d4c/dab/dd3 0 2024-02-16T19:24:19.620 INFO:tasks.workunit.client.0.smithi040.stdout:1/659: write d17/d26/d5f/f62 [481249,83496] 0 2024-02-16T19:24:19.620 INFO:tasks.workunit.client.0.smithi040.stdout:1/660: creat d17/d26/d5f/fd4 x:0 0 0 2024-02-16T19:24:19.620 INFO:tasks.workunit.client.0.smithi040.stdout:1/661: fsync d17/d29/d8b/f8d 0 2024-02-16T19:24:19.622 INFO:tasks.workunit.client.1.smithi163.stdout:2/201: dwrite d2/fb [4194304,4194304] 0 2024-02-16T19:24:19.622 INFO:tasks.workunit.client.1.smithi163.stdout:2/202: chown d2/f9 13788513 1 2024-02-16T19:24:19.623 INFO:tasks.workunit.client.0.smithi040.stdout:0/788: mkdir db/d2f/d23/d3b/d10c 0 2024-02-16T19:24:19.624 INFO:tasks.workunit.client.1.smithi163.stdout:9/269: creat d5/de/f5f x:0 0 0 2024-02-16T19:24:19.626 INFO:tasks.workunit.client.1.smithi163.stdout:7/75: mknod d11/c17 0 2024-02-16T19:24:19.626 INFO:tasks.workunit.client.1.smithi163.stdout:7/76: creat d11/f18 x:0 0 0 2024-02-16T19:24:19.627 INFO:tasks.workunit.client.1.smithi163.stdout:5/51: truncate d0/d2/f3 282283 0 2024-02-16T19:24:19.628 INFO:tasks.workunit.client.1.smithi163.stdout:5/52: dread - d0/d2/f7 zero size 2024-02-16T19:24:19.635 INFO:tasks.workunit.client.1.smithi163.stdout:3/124: creat d0/dc/d1c/d21/f2d x:0 0 0 2024-02-16T19:24:19.635 INFO:tasks.workunit.client.1.smithi163.stdout:8/57: dwrite d0/f9 [0,4194304] 0 2024-02-16T19:24:19.637 INFO:tasks.workunit.client.0.smithi040.stdout:0/789: dread db/d2f/d23/f40 [0,4194304] 0 2024-02-16T19:24:19.638 INFO:tasks.workunit.client.1.smithi163.stdout:6/85: mknod d0/d6/d9/d15/c19 0 2024-02-16T19:24:19.640 INFO:tasks.workunit.client.1.smithi163.stdout:9/270: write d5/f2c [3123073,103115] 0 2024-02-16T19:24:19.641 INFO:tasks.workunit.client.1.smithi163.stdout:1/41: truncate f0 2973154 0 2024-02-16T19:24:19.641 INFO:tasks.workunit.client.1.smithi163.stdout:1/42: write d7/f9 [36605,8437] 0 2024-02-16T19:24:19.642 INFO:tasks.workunit.client.1.smithi163.stdout:1/43: fdatasync f2 0 2024-02-16T19:24:19.642 INFO:tasks.workunit.client.1.smithi163.stdout:1/44: write d7/f9 [99616,54357] 0 2024-02-16T19:24:19.648 INFO:tasks.workunit.client.1.smithi163.stdout:5/53: dread d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:19.648 INFO:tasks.workunit.client.1.smithi163.stdout:5/54: read - d0/fd zero size 2024-02-16T19:24:19.648 INFO:tasks.workunit.client.1.smithi163.stdout:0/118: mknod d11/d12/d20/c2a 0 2024-02-16T19:24:19.649 INFO:tasks.workunit.client.1.smithi163.stdout:7/77: dwrite fb [0,4194304] 0 2024-02-16T19:24:19.650 INFO:tasks.workunit.client.1.smithi163.stdout:7/78: creat d11/f19 x:0 0 0 2024-02-16T19:24:19.653 INFO:tasks.workunit.client.1.smithi163.stdout:2/203: truncate d2/f1a 4996655 0 2024-02-16T19:24:19.653 INFO:tasks.workunit.client.1.smithi163.stdout:2/204: fsync d2/fc 0 2024-02-16T19:24:19.653 INFO:tasks.workunit.client.1.smithi163.stdout:2/205: truncate d2/d26/f3f 325436 0 2024-02-16T19:24:19.655 INFO:tasks.workunit.client.0.smithi040.stdout:7/613: rmdir d21 39 2024-02-16T19:24:19.655 INFO:tasks.workunit.client.0.smithi040.stdout:7/614: truncate d21/f3c 1901280 0 2024-02-16T19:24:19.656 INFO:tasks.workunit.client.1.smithi163.stdout:3/125: dwrite d0/dc/d1c/f2c [0,4194304] 0 2024-02-16T19:24:19.658 INFO:tasks.workunit.client.0.smithi040.stdout:6/786: unlink d0/d30/d32/dd2/dbc/ce3 0 2024-02-16T19:24:19.662 INFO:tasks.workunit.client.0.smithi040.stdout:1/662: truncate d17/d26/d4c/f5b 222713 0 2024-02-16T19:24:19.666 INFO:tasks.workunit.client.1.smithi163.stdout:9/271: dwrite d5/de/d30/d3d/d49/f58 [0,4194304] 0 2024-02-16T19:24:19.666 INFO:tasks.workunit.client.1.smithi163.stdout:9/272: write d5/de/d30/d31/f54 [841444,26142] 0 2024-02-16T19:24:19.666 INFO:tasks.workunit.client.1.smithi163.stdout:9/273: creat d5/de/d30/d43/f60 x:0 0 0 2024-02-16T19:24:19.670 INFO:tasks.workunit.client.0.smithi040.stdout:7/615: mknod d21/ccd 0 2024-02-16T19:24:19.672 INFO:tasks.workunit.client.1.smithi163.stdout:8/58: rename d0 to d0/d13 22 2024-02-16T19:24:19.672 INFO:tasks.workunit.client.1.smithi163.stdout:8/59: fdatasync d0/fe 0 2024-02-16T19:24:19.673 INFO:tasks.workunit.client.0.smithi040.stdout:6/787: mknod d0/d1/d12/d3b/d48/dc0/c103 0 2024-02-16T19:24:19.676 INFO:tasks.workunit.client.0.smithi040.stdout:8/766: dwrite d6/d7/d15/f85 [0,4194304] 0 2024-02-16T19:24:19.676 INFO:tasks.workunit.client.0.smithi040.stdout:1/663: creat d17/db5/fd5 x:0 0 0 2024-02-16T19:24:19.678 INFO:tasks.workunit.client.1.smithi163.stdout:5/55: mknod d0/ce 0 2024-02-16T19:24:19.680 INFO:tasks.workunit.client.0.smithi040.stdout:7/616: rename d21/d8b/fb6 to d21/d4d/d7d/fce 0 2024-02-16T19:24:19.680 INFO:tasks.workunit.client.0.smithi040.stdout:7/617: chown d21/d4d/c6a 158 1 2024-02-16T19:24:19.680 INFO:tasks.workunit.client.0.smithi040.stdout:7/618: chown d21/d4d/d77/d7a/l8a 4 1 2024-02-16T19:24:19.680 INFO:tasks.workunit.client.0.smithi040.stdout:7/619: chown d21/d4d/d77/fba 13677925 1 2024-02-16T19:24:19.681 INFO:tasks.workunit.client.1.smithi163.stdout:0/119: mknod d11/d12/c2b 0 2024-02-16T19:24:19.681 INFO:tasks.workunit.client.1.smithi163.stdout:0/120: readlink d11/l14 0 2024-02-16T19:24:19.682 INFO:tasks.workunit.client.0.smithi040.stdout:6/788: creat d0/d1/d6f/d90/f104 x:0 0 0 2024-02-16T19:24:19.687 INFO:tasks.workunit.client.0.smithi040.stdout:8/767: creat d6/d12/d51/d6e/db9/fef x:0 0 0 2024-02-16T19:24:19.687 INFO:tasks.workunit.client.0.smithi040.stdout:7/620: dread d21/d4d/d77/d7a/d99/fb3 [0,4194304] 0 2024-02-16T19:24:19.689 INFO:tasks.workunit.client.0.smithi040.stdout:1/664: creat d17/d29/d8b/fd6 x:0 0 0 2024-02-16T19:24:19.691 INFO:tasks.workunit.client.0.smithi040.stdout:7/621: link l1a d21/d4d/d77/d7a/d99/dbe/lcf 0 2024-02-16T19:24:19.692 INFO:tasks.workunit.client.1.smithi163.stdout:4/59: sync 2024-02-16T19:24:19.692 INFO:tasks.workunit.client.1.smithi163.stdout:4/60: creat fe x:0 0 0 2024-02-16T19:24:19.692 INFO:tasks.workunit.client.1.smithi163.stdout:7/79: mknod d11/c1a 0 2024-02-16T19:24:19.693 INFO:tasks.workunit.client.1.smithi163.stdout:7/80: dread - ff zero size 2024-02-16T19:24:19.697 INFO:tasks.workunit.client.1.smithi163.stdout:2/206: unlink d2/d26/c29 0 2024-02-16T19:24:19.699 INFO:tasks.workunit.client.0.smithi040.stdout:7/622: unlink d21/d4d/d7b/l90 0 2024-02-16T19:24:19.702 INFO:tasks.workunit.client.0.smithi040.stdout:1/665: symlink d17/d26/d3d/ld7 0 2024-02-16T19:24:19.704 INFO:tasks.workunit.client.0.smithi040.stdout:1/666: creat d17/d29/d8b/d93/dcf/fd8 x:0 0 0 2024-02-16T19:24:19.705 INFO:tasks.workunit.client.1.smithi163.stdout:9/274: rmdir d5/de/d30/d31 39 2024-02-16T19:24:19.706 INFO:tasks.workunit.client.0.smithi040.stdout:1/667: write f14 [796737,129136] 0 2024-02-16T19:24:19.706 INFO:tasks.workunit.client.0.smithi040.stdout:1/668: write d17/d26/fb9 [1266256,124387] 0 2024-02-16T19:24:19.707 INFO:tasks.workunit.client.0.smithi040.stdout:6/789: dwrite d0/d30/d32/f8d [0,4194304] 0 2024-02-16T19:24:19.707 INFO:tasks.workunit.client.0.smithi040.stdout:6/790: chown d0/d1/d12/d3b/d48/d73/fa7 4310813 1 2024-02-16T19:24:19.709 INFO:tasks.workunit.client.1.smithi163.stdout:3/126: rename d0/f1a to d0/dc/f2e 0 2024-02-16T19:24:19.710 INFO:tasks.workunit.client.1.smithi163.stdout:8/60: link d0/d7/la d0/d7/l14 0 2024-02-16T19:24:19.711 INFO:tasks.workunit.client.1.smithi163.stdout:8/61: write d0/d7/fb [1108942,9580] 0 2024-02-16T19:24:19.712 INFO:tasks.workunit.client.1.smithi163.stdout:8/62: dread d0/d7/fd [0,4194304] 0 2024-02-16T19:24:19.712 INFO:tasks.workunit.client.0.smithi040.stdout:8/768: dwrite d6/d64/fc4 [4194304,4194304] 0 2024-02-16T19:24:19.713 INFO:tasks.workunit.client.0.smithi040.stdout:6/791: truncate d0/d30/d32/dd2/dbc/feb 2171187 0 2024-02-16T19:24:19.714 INFO:tasks.workunit.client.1.smithi163.stdout:1/45: truncate f0 1231935 0 2024-02-16T19:24:19.714 INFO:tasks.workunit.client.1.smithi163.stdout:2/207: dread d2/f9 [0,4194304] 0 2024-02-16T19:24:19.715 INFO:tasks.workunit.client.1.smithi163.stdout:5/56: mknod d0/d2/cf 0 2024-02-16T19:24:19.715 INFO:tasks.workunit.client.1.smithi163.stdout:5/57: fdatasync d0/d2/f4 0 2024-02-16T19:24:19.718 INFO:tasks.workunit.client.1.smithi163.stdout:1/46: dread d7/f9 [0,4194304] 0 2024-02-16T19:24:19.719 INFO:tasks.workunit.client.1.smithi163.stdout:0/121: truncate f8 1012018 0 2024-02-16T19:24:19.719 INFO:tasks.workunit.client.1.smithi163.stdout:0/122: chown d11/d12/f15 1 1 2024-02-16T19:24:19.719 INFO:tasks.workunit.client.0.smithi040.stdout:7/623: dwrite d21/d4d/d77/fba [0,4194304] 0 2024-02-16T19:24:19.719 INFO:tasks.workunit.client.0.smithi040.stdout:7/624: chown f1f 7 1 2024-02-16T19:24:19.720 INFO:tasks.workunit.client.0.smithi040.stdout:6/792: dread d0/d30/d5b/f91 [0,4194304] 0 2024-02-16T19:24:19.725 INFO:tasks.workunit.client.1.smithi163.stdout:7/81: link d11/f12 d11/f1b 0 2024-02-16T19:24:19.725 INFO:tasks.workunit.client.1.smithi163.stdout:7/82: creat d11/f1c x:0 0 0 2024-02-16T19:24:19.725 INFO:tasks.workunit.client.1.smithi163.stdout:7/83: truncate d11/f19 147963 0 2024-02-16T19:24:19.730 INFO:tasks.workunit.client.1.smithi163.stdout:9/275: mknod d5/de/d30/d31/d52/c61 0 2024-02-16T19:24:19.730 INFO:tasks.workunit.client.0.smithi040.stdout:7/625: truncate d21/d29/d79/f7f 3188954 0 2024-02-16T19:24:19.732 INFO:tasks.workunit.client.1.smithi163.stdout:3/127: symlink d0/dc/d1c/d21/l2f 0 2024-02-16T19:24:19.734 INFO:tasks.workunit.client.0.smithi040.stdout:7/626: mknod d21/d4d/d77/d87/dc9/cd0 0 2024-02-16T19:24:19.735 INFO:tasks.workunit.client.0.smithi040.stdout:6/793: dread d0/f36 [0,4194304] 0 2024-02-16T19:24:19.735 INFO:tasks.workunit.client.0.smithi040.stdout:6/794: read d0/d30/d4b/fca [219078,53974] 0 2024-02-16T19:24:19.736 INFO:tasks.workunit.client.1.smithi163.stdout:2/208: link d2/f1a d2/d44/f45 0 2024-02-16T19:24:19.736 INFO:tasks.workunit.client.1.smithi163.stdout:2/209: write d2/d2a/f33 [136416,3798] 0 2024-02-16T19:24:19.736 INFO:tasks.workunit.client.1.smithi163.stdout:2/210: creat d2/d2a/f46 x:0 0 0 2024-02-16T19:24:19.738 INFO:tasks.workunit.client.1.smithi163.stdout:0/123: creat d11/d12/d20/f2c x:0 0 0 2024-02-16T19:24:19.738 INFO:tasks.workunit.client.1.smithi163.stdout:0/124: creat d11/d22/f2d x:0 0 0 2024-02-16T19:24:19.738 INFO:tasks.workunit.client.1.smithi163.stdout:0/125: stat d11/f1f 0 2024-02-16T19:24:19.739 INFO:tasks.workunit.client.0.smithi040.stdout:8/769: dwrite d6/d2b/f48 [4194304,4194304] 0 2024-02-16T19:24:19.739 INFO:tasks.workunit.client.0.smithi040.stdout:8/770: write d6/d12/d51/db6/fe9 [238093,86343] 0 2024-02-16T19:24:19.740 INFO:tasks.workunit.client.1.smithi163.stdout:3/128: read d0/f6 [2934411,114198] 0 2024-02-16T19:24:19.740 INFO:tasks.workunit.client.0.smithi040.stdout:6/795: rename d0/d1/d12/d3b/d48/f38 to d0/d30/d32/dd2/dbc/dfa/f105 0 2024-02-16T19:24:19.741 INFO:tasks.workunit.client.0.smithi040.stdout:7/627: mknod d21/d7e/cd1 0 2024-02-16T19:24:19.741 INFO:tasks.workunit.client.0.smithi040.stdout:7/628: fsync f9 0 2024-02-16T19:24:19.742 INFO:tasks.workunit.client.0.smithi040.stdout:7/629: fdatasync d21/f9f 0 2024-02-16T19:24:19.745 INFO:tasks.workunit.client.0.smithi040.stdout:1/669: dwrite d17/fc7 [0,4194304] 0 2024-02-16T19:24:19.745 INFO:tasks.workunit.client.1.smithi163.stdout:5/58: dwrite d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:19.746 INFO:tasks.workunit.client.1.smithi163.stdout:9/276: truncate d5/de/f28 975145 0 2024-02-16T19:24:19.747 INFO:tasks.workunit.client.1.smithi163.stdout:4/61: getdents . 0 2024-02-16T19:24:19.749 INFO:tasks.workunit.client.0.smithi040.stdout:8/771: unlink d6/d12/d51/d6e/f96 0 2024-02-16T19:24:19.752 INFO:tasks.workunit.client.0.smithi040.stdout:6/796: rename d0/d30/d32/dd2/ffc to d0/d30/d32/dd2/dbc/f106 0 2024-02-16T19:24:19.752 INFO:tasks.workunit.client.1.smithi163.stdout:4/62: write fd [1585987,129134] 0 2024-02-16T19:24:19.752 INFO:tasks.workunit.client.1.smithi163.stdout:4/63: dread - fe zero size 2024-02-16T19:24:19.752 INFO:tasks.workunit.client.1.smithi163.stdout:4/64: read fd [560917,56833] 0 2024-02-16T19:24:19.754 INFO:tasks.workunit.client.0.smithi040.stdout:7/630: creat d21/d4d/d77/d87/dc9/fd2 x:0 0 0 2024-02-16T19:24:19.758 INFO:tasks.workunit.client.1.smithi163.stdout:2/211: symlink d2/d2a/d39/d3a/l47 0 2024-02-16T19:24:19.758 INFO:tasks.workunit.client.1.smithi163.stdout:2/212: fsync d2/fb 0 2024-02-16T19:24:19.758 INFO:tasks.workunit.client.1.smithi163.stdout:2/213: chown d2/l12 28215 1 2024-02-16T19:24:19.763 INFO:tasks.workunit.client.0.smithi040.stdout:7/631: dread d21/d29/f73 [0,4194304] 0 2024-02-16T19:24:19.763 INFO:tasks.workunit.client.0.smithi040.stdout:7/632: stat d21/d4d/d7b/fa6 0 2024-02-16T19:24:19.763 INFO:tasks.workunit.client.0.smithi040.stdout:7/633: write d21/d29/f38 [1552721,107495] 0 2024-02-16T19:24:19.763 INFO:tasks.workunit.client.1.smithi163.stdout:0/126: symlink d11/d12/d20/l2e 0 2024-02-16T19:24:19.763 INFO:tasks.workunit.client.1.smithi163.stdout:0/127: dread - d11/d22/f2d zero size 2024-02-16T19:24:19.764 INFO:tasks.workunit.client.1.smithi163.stdout:0/128: stat d11/d12/f15 0 2024-02-16T19:24:19.764 INFO:tasks.workunit.client.1.smithi163.stdout:0/129: dread - d11/d22/f2d zero size 2024-02-16T19:24:19.764 INFO:tasks.workunit.client.1.smithi163.stdout:0/130: fdatasync f8 0 2024-02-16T19:24:19.764 INFO:tasks.workunit.client.1.smithi163.stdout:0/131: creat d11/d12/d20/f2f x:0 0 0 2024-02-16T19:24:19.765 INFO:tasks.workunit.client.1.smithi163.stdout:3/129: link d0/dc/d1c/f20 d0/dc/d1d/d29/f30 0 2024-02-16T19:24:19.766 INFO:tasks.workunit.client.1.smithi163.stdout:3/130: chown d0/dc/f28 1 1 2024-02-16T19:24:19.766 INFO:tasks.workunit.client.1.smithi163.stdout:3/131: write d0/f14 [321835,9426] 0 2024-02-16T19:24:19.768 INFO:tasks.workunit.client.1.smithi163.stdout:7/84: rename cd to d11/c1d 0 2024-02-16T19:24:19.772 INFO:tasks.workunit.client.1.smithi163.stdout:8/63: truncate d0/d7/fb 115541 0 2024-02-16T19:24:19.774 INFO:tasks.workunit.client.1.smithi163.stdout:5/59: mkdir d0/d2/d10 0 2024-02-16T19:24:19.777 INFO:tasks.workunit.client.1.smithi163.stdout:3/132: dread d0/dc/fd [0,4194304] 0 2024-02-16T19:24:19.788 INFO:tasks.workunit.client.1.smithi163.stdout:4/65: mknod cf 0 2024-02-16T19:24:19.788 INFO:tasks.workunit.client.1.smithi163.stdout:4/66: chown cf 3177576 1 2024-02-16T19:24:19.791 INFO:tasks.workunit.client.0.smithi040.stdout:2/601: sync 2024-02-16T19:24:19.791 INFO:tasks.workunit.client.0.smithi040.stdout:2/602: stat d5/d5c 0 2024-02-16T19:24:19.791 INFO:tasks.workunit.client.0.smithi040.stdout:3/760: sync 2024-02-16T19:24:19.791 INFO:tasks.workunit.client.0.smithi040.stdout:0/790: sync 2024-02-16T19:24:19.792 INFO:tasks.workunit.client.0.smithi040.stdout:4/583: sync 2024-02-16T19:24:19.792 INFO:tasks.workunit.client.0.smithi040.stdout:4/584: chown dd/f1d 313868 1 2024-02-16T19:24:19.792 INFO:tasks.workunit.client.0.smithi040.stdout:4/585: creat dd/de/d10/fb9 x:0 0 0 2024-02-16T19:24:19.792 INFO:tasks.workunit.client.0.smithi040.stdout:4/586: readlink dd/d58/d68/la5 0 2024-02-16T19:24:19.792 INFO:tasks.workunit.client.0.smithi040.stdout:9/707: sync 2024-02-16T19:24:19.792 INFO:tasks.workunit.client.0.smithi040.stdout:7/634: dwrite d21/d4d/f56 [0,4194304] 0 2024-02-16T19:24:19.792 INFO:tasks.workunit.client.1.smithi163.stdout:0/132: dwrite d11/f1a [0,4194304] 0 2024-02-16T19:24:19.793 INFO:tasks.workunit.client.0.smithi040.stdout:1/670: symlink d17/d26/d3d/ld9 0 2024-02-16T19:24:19.797 INFO:tasks.workunit.client.0.smithi040.stdout:6/797: mknod d0/d30/d32/dd2/d87/d8f/c107 0 2024-02-16T19:24:19.797 INFO:tasks.workunit.client.1.smithi163.stdout:5/60: write d0/d2/fb [3037397,62236] 0 2024-02-16T19:24:19.800 INFO:tasks.workunit.client.1.smithi163.stdout:7/85: write fc [590149,45809] 0 2024-02-16T19:24:19.800 INFO:tasks.workunit.client.1.smithi163.stdout:7/86: dread - ff zero size 2024-02-16T19:24:19.802 INFO:tasks.workunit.client.0.smithi040.stdout:0/791: unlink db/de/f59 0 2024-02-16T19:24:19.802 INFO:tasks.workunit.client.0.smithi040.stdout:8/772: write d6/d2b/da3/d8b/f9d [1056400,93173] 0 2024-02-16T19:24:19.803 INFO:tasks.workunit.client.1.smithi163.stdout:2/214: read d2/f1a [361663,10234] 0 2024-02-16T19:24:19.803 INFO:tasks.workunit.client.1.smithi163.stdout:2/215: fsync d2/d26/f3d 0 2024-02-16T19:24:19.803 INFO:tasks.workunit.client.1.smithi163.stdout:0/133: dread d11/f1a [0,4194304] 0 2024-02-16T19:24:19.806 INFO:tasks.workunit.client.1.smithi163.stdout:9/277: link d5/de/f28 d5/de/d30/d3d/d49/f62 0 2024-02-16T19:24:19.806 INFO:tasks.workunit.client.1.smithi163.stdout:3/133: dwrite d0/dc/f24 [0,4194304] 0 2024-02-16T19:24:19.806 INFO:tasks.workunit.client.1.smithi163.stdout:3/134: creat d0/dc/f31 x:0 0 0 2024-02-16T19:24:19.807 INFO:tasks.workunit.client.0.smithi040.stdout:4/587: link dd/d58/l88 dd/de/daa/lba 0 2024-02-16T19:24:19.808 INFO:tasks.workunit.client.1.smithi163.stdout:1/47: truncate f0 459916 0 2024-02-16T19:24:19.808 INFO:tasks.workunit.client.1.smithi163.stdout:1/48: creat d7/fa x:0 0 0 2024-02-16T19:24:19.809 INFO:tasks.workunit.client.1.smithi163.stdout:1/49: read - f6 zero size 2024-02-16T19:24:19.809 INFO:tasks.workunit.client.1.smithi163.stdout:1/50: creat d7/fb x:0 0 0 2024-02-16T19:24:19.810 INFO:tasks.workunit.client.1.smithi163.stdout:4/67: creat f10 x:0 0 0 2024-02-16T19:24:19.811 INFO:tasks.workunit.client.1.smithi163.stdout:4/68: chown l0 1801804 1 2024-02-16T19:24:19.811 INFO:tasks.workunit.client.1.smithi163.stdout:4/69: fsync f6 0 2024-02-16T19:24:19.811 INFO:tasks.workunit.client.1.smithi163.stdout:4/70: chown cb 14 1 2024-02-16T19:24:19.813 INFO:tasks.workunit.client.0.smithi040.stdout:7/635: creat d21/d4d/d77/fd3 x:0 0 0 2024-02-16T19:24:19.814 INFO:tasks.workunit.client.0.smithi040.stdout:6/798: mknod d0/d1/d6f/d90/c108 0 2024-02-16T19:24:19.814 INFO:tasks.workunit.client.0.smithi040.stdout:6/799: fsync d0/f2e 0 2024-02-16T19:24:19.815 INFO:tasks.workunit.client.1.smithi163.stdout:5/61: rename d0/d2/cf to d0/d2/c11 0 2024-02-16T19:24:19.816 INFO:tasks.workunit.client.1.smithi163.stdout:2/216: write d2/f5 [445881,44178] 0 2024-02-16T19:24:19.816 INFO:tasks.workunit.client.0.smithi040.stdout:5/651: sync 2024-02-16T19:24:19.816 INFO:tasks.workunit.client.0.smithi040.stdout:5/652: write d1/d62/fb6 [949132,89509] 0 2024-02-16T19:24:19.816 INFO:tasks.workunit.client.0.smithi040.stdout:8/773: mknod d6/d12/d51/d6e/db7/dc6/cf0 0 2024-02-16T19:24:19.817 INFO:tasks.workunit.client.1.smithi163.stdout:7/87: mknod d11/c1e 0 2024-02-16T19:24:19.817 INFO:tasks.workunit.client.1.smithi163.stdout:7/88: creat d11/f1f x:0 0 0 2024-02-16T19:24:19.818 INFO:tasks.workunit.client.0.smithi040.stdout:1/671: dwrite d17/f28 [0,4194304] 0 2024-02-16T19:24:19.818 INFO:tasks.workunit.client.0.smithi040.stdout:4/588: unlink dd/de/d45/c31 0 2024-02-16T19:24:19.819 INFO:tasks.workunit.client.1.smithi163.stdout:8/64: truncate d0/d7/fb 806999 0 2024-02-16T19:24:19.820 INFO:tasks.workunit.client.0.smithi040.stdout:9/708: getdents d3/d4/d11/d71/da8 0 2024-02-16T19:24:19.820 INFO:tasks.workunit.client.0.smithi040.stdout:9/709: getdents d3/d4/d55/d70/dcc 0 2024-02-16T19:24:19.822 INFO:tasks.workunit.client.1.smithi163.stdout:0/134: rename d11/f1d to d11/d12/d20/f30 0 2024-02-16T19:24:19.823 INFO:tasks.workunit.client.0.smithi040.stdout:7/636: getdents d21/d4d 0 2024-02-16T19:24:19.823 INFO:tasks.workunit.client.0.smithi040.stdout:2/603: dwrite d5/d8/d42/f50 [0,4194304] 0 2024-02-16T19:24:19.823 INFO:tasks.workunit.client.0.smithi040.stdout:2/604: chown d5/d8/d42/ca5 4815357 1 2024-02-16T19:24:19.823 INFO:tasks.workunit.client.0.smithi040.stdout:3/761: dwrite d1/d4/d6e/fb5 [0,4194304] 0 2024-02-16T19:24:19.824 INFO:tasks.workunit.client.0.smithi040.stdout:3/762: write d1/dc/d35/d48/fc7 [186076,50414] 0 2024-02-16T19:24:19.824 INFO:tasks.workunit.client.0.smithi040.stdout:3/763: write d1/dc/d35/d48/d85/f9e [470957,7353] 0 2024-02-16T19:24:19.829 INFO:tasks.workunit.client.0.smithi040.stdout:5/653: mknod d1/da/d1c/da7/cc9 0 2024-02-16T19:24:19.832 INFO:tasks.workunit.client.1.smithi163.stdout:5/62: dread d0/d2/fb [0,4194304] 0 2024-02-16T19:24:19.832 INFO:tasks.workunit.client.0.smithi040.stdout:8/774: unlink d6/d7/fca 0 2024-02-16T19:24:19.832 INFO:tasks.workunit.client.0.smithi040.stdout:8/775: write d6/d64/d50/d69/fa4 [376243,81370] 0 2024-02-16T19:24:19.834 INFO:tasks.workunit.client.1.smithi163.stdout:3/135: mkdir d0/dc/d1c/d32 0 2024-02-16T19:24:19.835 INFO:tasks.workunit.client.1.smithi163.stdout:1/51: fsync f0 0 2024-02-16T19:24:19.836 INFO:tasks.workunit.client.1.smithi163.stdout:1/52: truncate d7/fa 435059 0 2024-02-16T19:24:19.838 INFO:tasks.workunit.client.0.smithi040.stdout:1/672: dread d17/d2d/fad [0,4194304] 0 2024-02-16T19:24:19.839 INFO:tasks.workunit.client.0.smithi040.stdout:2/605: dwrite d5/d7/d12/d1b/dae/fc4 [0,4194304] 0 2024-02-16T19:24:19.839 INFO:tasks.workunit.client.0.smithi040.stdout:9/710: rmdir d3/d4/d11/d71/da8 39 2024-02-16T19:24:19.839 INFO:tasks.workunit.client.0.smithi040.stdout:5/654: dread d1/da/d1c/dba/db1/fa3 [0,4194304] 0 2024-02-16T19:24:19.840 INFO:tasks.workunit.client.0.smithi040.stdout:1/673: dread d17/d26/d5f/f84 [0,4194304] 0 2024-02-16T19:24:19.840 INFO:tasks.workunit.client.0.smithi040.stdout:1/674: stat d17/d29/d8b/d93 0 2024-02-16T19:24:19.842 INFO:tasks.workunit.client.1.smithi163.stdout:4/71: mkdir d11 0 2024-02-16T19:24:19.842 INFO:tasks.workunit.client.1.smithi163.stdout:4/72: chown f9 17619 1 2024-02-16T19:24:19.842 INFO:tasks.workunit.client.1.smithi163.stdout:4/73: chown ca 324904 1 2024-02-16T19:24:19.849 INFO:tasks.workunit.client.0.smithi040.stdout:7/637: symlink d21/d4d/d77/d7a/ld4 0 2024-02-16T19:24:19.849 INFO:tasks.workunit.client.1.smithi163.stdout:6/86: sync 2024-02-16T19:24:19.851 INFO:tasks.workunit.client.1.smithi163.stdout:8/65: creat d0/d7/f15 x:0 0 0 2024-02-16T19:24:19.853 INFO:tasks.workunit.client.1.smithi163.stdout:9/278: rename d5/de/ff to d5/de/d30/d3d/d49/d50/f63 0 2024-02-16T19:24:19.854 INFO:tasks.workunit.client.1.smithi163.stdout:0/135: rmdir d11/d21 39 2024-02-16T19:24:19.857 INFO:tasks.workunit.client.1.smithi163.stdout:5/63: symlink d0/l12 0 2024-02-16T19:24:19.857 INFO:tasks.workunit.client.1.smithi163.stdout:5/64: chown d0/d2/f7 734916 1 2024-02-16T19:24:19.858 INFO:tasks.workunit.client.1.smithi163.stdout:4/74: dwrite fe [0,4194304] 0 2024-02-16T19:24:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:3/764: creat d1/d8c/d8e/ff1 x:0 0 0 2024-02-16T19:24:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:5/655: dwrite d1/da/d1c/f25 [0,4194304] 0 2024-02-16T19:24:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:5/656: fdatasync d1/d45/dab/fc4 0 2024-02-16T19:24:19.859 INFO:tasks.workunit.client.0.smithi040.stdout:5/657: fsync d1/da/d51/f81 0 2024-02-16T19:24:19.862 INFO:tasks.workunit.client.0.smithi040.stdout:6/800: getdents d0/d1/d12/d3b 0 2024-02-16T19:24:19.862 INFO:tasks.workunit.client.0.smithi040.stdout:6/801: readlink d0/d37/d4a/l84 0 2024-02-16T19:24:19.863 INFO:tasks.workunit.client.1.smithi163.stdout:3/136: creat d0/dc/d1d/d29/f33 x:0 0 0 2024-02-16T19:24:19.867 INFO:tasks.workunit.client.0.smithi040.stdout:8/776: truncate f3 7368555 0 2024-02-16T19:24:19.871 INFO:tasks.workunit.client.1.smithi163.stdout:7/89: getdents d11 0 2024-02-16T19:24:19.872 INFO:tasks.workunit.client.1.smithi163.stdout:6/87: link d0/dd/f13 d0/d6/f1a 0 2024-02-16T19:24:19.874 INFO:tasks.workunit.client.1.smithi163.stdout:8/66: creat d0/dc/f16 x:0 0 0 2024-02-16T19:24:19.899 INFO:tasks.workunit.client.0.smithi040.stdout:2/606: creat d5/d7/d1e/d4c/d52/dc1/fc6 x:0 0 0 2024-02-16T19:24:19.899 INFO:tasks.workunit.client.0.smithi040.stdout:2/607: creat d5/d5c/d65/d9d/fc7 x:0 0 0 2024-02-16T19:24:19.900 INFO:tasks.workunit.client.0.smithi040.stdout:9/711: getdents d3/d4/dbe 0 2024-02-16T19:24:19.903 INFO:tasks.workunit.client.1.smithi163.stdout:2/217: rename d2/l24 to d2/d26/l48 0 2024-02-16T19:24:19.903 INFO:tasks.workunit.client.0.smithi040.stdout:5/658: dwrite d1/da/d1c/d21/f8f [0,4194304] 0 2024-02-16T19:24:19.904 INFO:tasks.workunit.client.0.smithi040.stdout:5/659: write d1/da/f35 [739994,32893] 0 2024-02-16T19:24:19.904 INFO:tasks.workunit.client.0.smithi040.stdout:6/802: dwrite d0/d1/d6f/d90/fd8 [0,4194304] 0 2024-02-16T19:24:19.905 INFO:tasks.workunit.client.0.smithi040.stdout:1/675: creat d17/d43/d92/dca/fda x:0 0 0 2024-02-16T19:24:19.905 INFO:tasks.workunit.client.0.smithi040.stdout:1/676: chown d17/d3c/d74/d97 29 1 2024-02-16T19:24:19.906 INFO:tasks.workunit.client.1.smithi163.stdout:9/279: mknod d5/de/d16/d47/c64 0 2024-02-16T19:24:19.908 INFO:tasks.workunit.client.1.smithi163.stdout:0/136: symlink d11/d12/d20/l31 0 2024-02-16T19:24:19.909 INFO:tasks.workunit.client.0.smithi040.stdout:3/765: dwrite d1/d7/d30/f40 [0,4194304] 0 2024-02-16T19:24:19.909 INFO:tasks.workunit.client.0.smithi040.stdout:3/766: fsync d1/dc/d42/d4a/f50 0 2024-02-16T19:24:19.911 INFO:tasks.workunit.client.0.smithi040.stdout:3/767: dread d1/dc/d42/d4a/f7d [0,4194304] 0 2024-02-16T19:24:19.912 INFO:tasks.workunit.client.0.smithi040.stdout:7/638: rmdir d21/d7e 39 2024-02-16T19:24:19.916 INFO:tasks.workunit.client.1.smithi163.stdout:1/53: rmdir d7 39 2024-02-16T19:24:19.916 INFO:tasks.workunit.client.1.smithi163.stdout:6/88: dwrite d0/dd/f13 [0,4194304] 0 2024-02-16T19:24:19.918 INFO:tasks.workunit.client.1.smithi163.stdout:3/137: dwrite d0/f19 [0,4194304] 0 2024-02-16T19:24:19.918 INFO:tasks.workunit.client.1.smithi163.stdout:3/138: fsync d0/dc/d1c/f2c 0 2024-02-16T19:24:19.918 INFO:tasks.workunit.client.0.smithi040.stdout:9/712: mkdir d3/d4/d11/d71/de8 0 2024-02-16T19:24:19.919 INFO:tasks.workunit.client.0.smithi040.stdout:5/660: write d1/fd [453549,95209] 0 2024-02-16T19:24:19.919 INFO:tasks.workunit.client.0.smithi040.stdout:2/608: dwrite d5/d8/f38 [0,4194304] 0 2024-02-16T19:24:19.919 INFO:tasks.workunit.client.1.smithi163.stdout:8/67: dwrite d0/dc/f16 [0,4194304] 0 2024-02-16T19:24:19.924 INFO:tasks.workunit.client.0.smithi040.stdout:1/677: dread d17/d29/d8b/f8d [0,4194304] 0 2024-02-16T19:24:19.924 INFO:tasks.workunit.client.0.smithi040.stdout:9/713: dread d3/d4/d11/f67 [0,4194304] 0 2024-02-16T19:24:19.924 INFO:tasks.workunit.client.0.smithi040.stdout:9/714: read - d3/d4/d11/fc6 zero size 2024-02-16T19:24:19.924 INFO:tasks.workunit.client.1.smithi163.stdout:7/90: creat d11/f20 x:0 0 0 2024-02-16T19:24:19.924 INFO:tasks.workunit.client.1.smithi163.stdout:5/65: rename d0/d2/f3 to d0/f13 0 2024-02-16T19:24:19.924 INFO:tasks.workunit.client.1.smithi163.stdout:5/66: write d0/d2/fc [864085,69570] 0 2024-02-16T19:24:19.924 INFO:tasks.workunit.client.1.smithi163.stdout:9/280: read d5/de/d30/d3d/f3b [139985,102644] 0 2024-02-16T19:24:19.925 INFO:tasks.workunit.client.1.smithi163.stdout:0/137: mkdir d11/d12/d32 0 2024-02-16T19:24:19.926 INFO:tasks.workunit.client.0.smithi040.stdout:6/803: symlink d0/d30/d32/dd2/dbc/dfa/l109 0 2024-02-16T19:24:19.928 INFO:tasks.workunit.client.1.smithi163.stdout:6/89: write d0/dd/f13 [3676558,80073] 0 2024-02-16T19:24:19.928 INFO:tasks.workunit.client.1.smithi163.stdout:1/54: mkdir d7/dc 0 2024-02-16T19:24:19.928 INFO:tasks.workunit.client.1.smithi163.stdout:1/55: write d7/fa [1266548,127171] 0 2024-02-16T19:24:19.928 INFO:tasks.workunit.client.1.smithi163.stdout:1/56: fdatasync f0 0 2024-02-16T19:24:19.931 INFO:tasks.workunit.client.0.smithi040.stdout:7/639: dwrite d21/d7e/faa [0,4194304] 0 2024-02-16T19:24:19.938 INFO:tasks.workunit.client.1.smithi163.stdout:3/139: mknod d0/dc/d1c/d21/c34 0 2024-02-16T19:24:19.940 INFO:tasks.workunit.client.0.smithi040.stdout:9/715: dread d3/f14 [0,4194304] 0 2024-02-16T19:24:19.940 INFO:tasks.workunit.client.0.smithi040.stdout:9/716: chown d3/d4/d55/dc3/cdb 6527965 1 2024-02-16T19:24:19.941 INFO:tasks.workunit.client.0.smithi040.stdout:3/768: mknod d1/d4/d6e/dc2/dce/cf2 0 2024-02-16T19:24:19.947 INFO:tasks.workunit.client.1.smithi163.stdout:7/91: mknod d11/c21 0 2024-02-16T19:24:19.947 INFO:tasks.workunit.client.1.smithi163.stdout:7/92: dread - d11/f18 zero size 2024-02-16T19:24:19.950 INFO:tasks.workunit.client.1.smithi163.stdout:2/218: rename d2/d26/l40 to d2/l49 0 2024-02-16T19:24:19.950 INFO:tasks.workunit.client.1.smithi163.stdout:2/219: write d2/f5 [3569721,2312] 0 2024-02-16T19:24:19.953 INFO:tasks.workunit.client.0.smithi040.stdout:3/769: write d1/dc/d42/d4a/f61 [1450861,48963] 0 2024-02-16T19:24:19.954 INFO:tasks.workunit.client.1.smithi163.stdout:0/138: truncate d11/d12/f15 522590 0 2024-02-16T19:24:19.955 INFO:tasks.workunit.client.1.smithi163.stdout:0/139: creat d11/f33 x:0 0 0 2024-02-16T19:24:19.955 INFO:tasks.workunit.client.1.smithi163.stdout:0/140: truncate d11/d22/f24 844456 0 2024-02-16T19:24:19.958 INFO:tasks.workunit.client.1.smithi163.stdout:1/57: symlink d7/ld 0 2024-02-16T19:24:19.961 INFO:tasks.workunit.client.0.smithi040.stdout:5/661: creat d1/da/d1c/da7/fca x:0 0 0 2024-02-16T19:24:19.964 INFO:tasks.workunit.client.1.smithi163.stdout:8/68: rename d0/dc/d10 to d0/d7/d17 0 2024-02-16T19:24:19.964 INFO:tasks.workunit.client.1.smithi163.stdout:8/69: write d0/d7/fd [1387823,13458] 0 2024-02-16T19:24:19.964 INFO:tasks.workunit.client.1.smithi163.stdout:1/58: dread f0 [0,4194304] 0 2024-02-16T19:24:19.964 INFO:tasks.workunit.client.1.smithi163.stdout:1/59: dread - f6 zero size 2024-02-16T19:24:19.965 INFO:tasks.workunit.client.1.smithi163.stdout:0/141: dread f7 [0,4194304] 0 2024-02-16T19:24:19.965 INFO:tasks.workunit.client.1.smithi163.stdout:0/142: dread - d11/f33 zero size 2024-02-16T19:24:19.966 INFO:tasks.workunit.client.1.smithi163.stdout:5/67: dwrite d0/d2/fc [0,4194304] 0 2024-02-16T19:24:19.966 INFO:tasks.workunit.client.0.smithi040.stdout:1/678: mkdir d17/d26/d5f/d78/d7e/ddb 0 2024-02-16T19:24:19.966 INFO:tasks.workunit.client.0.smithi040.stdout:1/679: creat d17/d43/fdc x:0 0 0 2024-02-16T19:24:19.968 INFO:tasks.workunit.client.0.smithi040.stdout:6/804: getdents d0/d1/d12/d3b/d48 0 2024-02-16T19:24:19.969 INFO:tasks.workunit.client.0.smithi040.stdout:6/805: chown d0/d30/d5b/f85 166 1 2024-02-16T19:24:19.969 INFO:tasks.workunit.client.0.smithi040.stdout:6/806: truncate d0/d1/d6f/d90/f9b 877331 0 2024-02-16T19:24:19.969 INFO:tasks.workunit.client.0.smithi040.stdout:6/807: chown d0/d1/d12/d3b/ldb 215364 1 2024-02-16T19:24:19.969 INFO:tasks.workunit.client.1.smithi163.stdout:9/281: dwrite d5/f9 [0,4194304] 0 2024-02-16T19:24:19.969 INFO:tasks.workunit.client.1.smithi163.stdout:9/282: fdatasync d5/f40 0 2024-02-16T19:24:19.973 INFO:tasks.workunit.client.1.smithi163.stdout:6/90: rename d0/lc to d0/d6/l1b 0 2024-02-16T19:24:19.976 INFO:tasks.workunit.client.1.smithi163.stdout:8/70: mkdir d0/d7/d18 0 2024-02-16T19:24:19.976 INFO:tasks.workunit.client.1.smithi163.stdout:8/71: stat d0/d7/c12 0 2024-02-16T19:24:19.978 INFO:tasks.workunit.client.1.smithi163.stdout:1/60: rename d7/ld to d7/le 0 2024-02-16T19:24:19.982 INFO:tasks.workunit.client.1.smithi163.stdout:5/68: mknod d0/c14 0 2024-02-16T19:24:19.982 INFO:tasks.workunit.client.1.smithi163.stdout:6/91: write d0/dd/f13 [4564935,100069] 0 2024-02-16T19:24:19.982 INFO:tasks.workunit.client.1.smithi163.stdout:6/92: truncate d0/f2 4806274 0 2024-02-16T19:24:19.982 INFO:tasks.workunit.client.0.smithi040.stdout:2/609: rmdir d5/d7/d10 39 2024-02-16T19:24:19.983 INFO:tasks.workunit.client.0.smithi040.stdout:9/717: dwrite d3/d4/d55/d31/fad [0,4194304] 0 2024-02-16T19:24:19.984 INFO:tasks.workunit.client.1.smithi163.stdout:3/140: dwrite d0/dc/d1d/d29/f33 [0,4194304] 0 2024-02-16T19:24:19.985 INFO:tasks.workunit.client.1.smithi163.stdout:9/283: mknod d5/de/d30/d3d/d49/c65 0 2024-02-16T19:24:19.986 INFO:tasks.workunit.client.0.smithi040.stdout:1/680: stat d17/d2d/l3b 0 2024-02-16T19:24:19.990 INFO:tasks.workunit.client.1.smithi163.stdout:8/72: mkdir d0/d7/d19 0 2024-02-16T19:24:19.990 INFO:tasks.workunit.client.1.smithi163.stdout:8/73: write d0/d7/fd [1435583,6210] 0 2024-02-16T19:24:19.990 INFO:tasks.workunit.client.1.smithi163.stdout:8/74: readlink d0/d7/l14 0 2024-02-16T19:24:19.992 INFO:tasks.workunit.client.1.smithi163.stdout:1/61: creat d7/dc/ff x:0 0 0 2024-02-16T19:24:19.994 INFO:tasks.workunit.client.0.smithi040.stdout:2/610: write d5/d8/d42/f67 [690820,17000] 0 2024-02-16T19:24:19.995 INFO:tasks.workunit.client.1.smithi163.stdout:6/93: mkdir d0/d6/d9/d15/d1c 0 2024-02-16T19:24:19.995 INFO:tasks.workunit.client.1.smithi163.stdout:6/94: read d0/d6/f1a [2492009,83630] 0 2024-02-16T19:24:19.997 INFO:tasks.workunit.client.1.smithi163.stdout:7/93: dwrite d11/f1c [0,4194304] 0 2024-02-16T19:24:19.998 INFO:tasks.workunit.client.0.smithi040.stdout:5/662: dwrite d1/f13 [0,4194304] 0 2024-02-16T19:24:19.998 INFO:tasks.workunit.client.0.smithi040.stdout:9/718: write d3/d4/d4a/f8c [559587,23674] 0 2024-02-16T19:24:19.998 INFO:tasks.workunit.client.1.smithi163.stdout:2/220: dwrite d2/d26/f31 [0,4194304] 0 2024-02-16T19:24:19.999 INFO:tasks.workunit.client.0.smithi040.stdout:6/808: dwrite d0/d1/d12/d3b/d48/d4f/f68 [0,4194304] 0 2024-02-16T19:24:19.999 INFO:tasks.workunit.client.0.smithi040.stdout:6/809: creat d0/d1/d6f/d90/f10a x:0 0 0 2024-02-16T19:24:19.999 INFO:tasks.workunit.client.1.smithi163.stdout:8/75: dread d0/f9 [0,4194304] 0 2024-02-16T19:24:20.001 INFO:tasks.workunit.client.0.smithi040.stdout:3/770: rename d1/d8c/dae to d1/dc/d42/d5c/df3 0 2024-02-16T19:24:20.002 INFO:tasks.workunit.client.1.smithi163.stdout:3/141: symlink d0/dc/l35 0 2024-02-16T19:24:20.006 INFO:tasks.workunit.client.1.smithi163.stdout:9/284: creat d5/de/d30/d3d/f66 x:0 0 0 2024-02-16T19:24:20.007 INFO:tasks.workunit.client.0.smithi040.stdout:5/663: symlink d1/lcb 0 2024-02-16T19:24:20.008 INFO:tasks.workunit.client.1.smithi163.stdout:3/142: dread d0/f1b [0,4194304] 0 2024-02-16T19:24:20.008 INFO:tasks.workunit.client.1.smithi163.stdout:1/62: mknod d7/dc/c10 0 2024-02-16T19:24:20.009 INFO:tasks.workunit.client.1.smithi163.stdout:5/69: dwrite d0/fd [0,4194304] 0 2024-02-16T19:24:20.009 INFO:tasks.workunit.client.1.smithi163.stdout:5/70: chown d0/ce 0 1 2024-02-16T19:24:20.009 INFO:tasks.workunit.client.1.smithi163.stdout:5/71: creat d0/f15 x:0 0 0 2024-02-16T19:24:20.012 INFO:tasks.workunit.client.1.smithi163.stdout:6/95: symlink d0/d6/l1d 0 2024-02-16T19:24:20.012 INFO:tasks.workunit.client.1.smithi163.stdout:6/96: chown d0/d6/l10 227185 1 2024-02-16T19:24:20.012 INFO:tasks.workunit.client.1.smithi163.stdout:0/143: read d11/d12/f15 [174042,23388] 0 2024-02-16T19:24:20.013 INFO:tasks.workunit.client.1.smithi163.stdout:0/144: truncate d11/d22/f24 1715291 0 2024-02-16T19:24:20.014 INFO:tasks.workunit.client.0.smithi040.stdout:9/719: dread d3/d4/d11/d71/da8/f58 [0,4194304] 0 2024-02-16T19:24:20.018 INFO:tasks.workunit.client.1.smithi163.stdout:9/285: write d5/f9 [2083809,99839] 0 2024-02-16T19:24:20.018 INFO:tasks.workunit.client.1.smithi163.stdout:9/286: chown d5 813636 1 2024-02-16T19:24:20.018 INFO:tasks.workunit.client.1.smithi163.stdout:9/287: chown d5 433452 1 2024-02-16T19:24:20.018 INFO:tasks.workunit.client.1.smithi163.stdout:9/288: chown d5/de/d16/c3f 8 1 2024-02-16T19:24:20.018 INFO:tasks.workunit.client.1.smithi163.stdout:9/289: write d5/de/d30/d3d/d49/d50/f63 [8494315,31537] 0 2024-02-16T19:24:20.019 INFO:tasks.workunit.client.1.smithi163.stdout:9/290: dread - d5/f1a zero size 2024-02-16T19:24:20.019 INFO:tasks.workunit.client.1.smithi163.stdout:7/94: mknod d11/c22 0 2024-02-16T19:24:20.021 INFO:tasks.workunit.client.0.smithi040.stdout:6/810: dwrite d0/fa0 [0,4194304] 0 2024-02-16T19:24:20.021 INFO:tasks.workunit.client.0.smithi040.stdout:6/811: truncate d0/d30/d32/dd2/dbc/feb 2644551 0 2024-02-16T19:24:20.021 INFO:tasks.workunit.client.0.smithi040.stdout:6/812: creat d0/d30/d5b/dbe/f10b x:0 0 0 2024-02-16T19:24:20.021 INFO:tasks.workunit.client.0.smithi040.stdout:6/813: stat d0/df0 0 2024-02-16T19:24:20.021 INFO:tasks.workunit.client.0.smithi040.stdout:6/814: chown d0/df0 4016 1 2024-02-16T19:24:20.022 INFO:tasks.workunit.client.0.smithi040.stdout:1/681: getdents d17/d3c/d74/d97 0 2024-02-16T19:24:20.023 INFO:tasks.workunit.client.1.smithi163.stdout:6/97: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:20.025 INFO:tasks.workunit.client.0.smithi040.stdout:3/771: mknod d1/d8c/d8e/dde/cf4 0 2024-02-16T19:24:20.028 INFO:tasks.workunit.client.0.smithi040.stdout:5/664: creat d1/d62/d9a/dc5/fcc x:0 0 0 2024-02-16T19:24:20.031 INFO:tasks.workunit.client.0.smithi040.stdout:9/720: unlink d3/d4/d55/f66 0 2024-02-16T19:24:20.031 INFO:tasks.workunit.client.0.smithi040.stdout:9/721: dread - d3/d1c/f9e zero size 2024-02-16T19:24:20.031 INFO:tasks.workunit.client.0.smithi040.stdout:9/722: chown d3/d1c/f3f 0 1 2024-02-16T19:24:20.036 INFO:tasks.workunit.client.0.smithi040.stdout:6/815: creat d0/d1/d12/d3b/d48/d73/f10c x:0 0 0 2024-02-16T19:24:20.036 INFO:tasks.workunit.client.0.smithi040.stdout:6/816: fdatasync d0/d1/d12/d3b/d48/d4f/f68 0 2024-02-16T19:24:20.036 INFO:tasks.workunit.client.0.smithi040.stdout:6/817: dread - d0/d30/d32/dd2/fb8 zero size 2024-02-16T19:24:20.038 INFO:tasks.workunit.client.1.smithi163.stdout:0/145: dwrite d11/d12/f27 [0,4194304] 0 2024-02-16T19:24:20.038 INFO:tasks.workunit.client.1.smithi163.stdout:0/146: fsync f6 0 2024-02-16T19:24:20.039 INFO:tasks.workunit.client.0.smithi040.stdout:1/682: link d17/f37 d17/d29/d8b/d93/fdd 0 2024-02-16T19:24:20.039 INFO:tasks.workunit.client.0.smithi040.stdout:1/683: dread - d17/d43/fc9 zero size 2024-02-16T19:24:20.040 INFO:tasks.workunit.client.0.smithi040.stdout:3/772: symlink d1/d7/d33/lf5 0 2024-02-16T19:24:20.042 INFO:tasks.workunit.client.0.smithi040.stdout:7/640: dwrite d21/f3c [0,4194304] 0 2024-02-16T19:24:20.045 INFO:tasks.workunit.client.0.smithi040.stdout:6/818: mknod d0/d1/d12/d3b/d48/d4d/c10d 0 2024-02-16T19:24:20.045 INFO:tasks.workunit.client.0.smithi040.stdout:9/723: dwrite d3/d4/fe3 [0,4194304] 0 2024-02-16T19:24:20.045 INFO:tasks.workunit.client.0.smithi040.stdout:9/724: readlink d3/d4/l1b 0 2024-02-16T19:24:20.047 INFO:tasks.workunit.client.0.smithi040.stdout:6/819: dread d0/d30/d5b/f91 [0,4194304] 0 2024-02-16T19:24:20.047 INFO:tasks.workunit.client.0.smithi040.stdout:6/820: fsync d0/f33 0 2024-02-16T19:24:20.047 INFO:tasks.workunit.client.0.smithi040.stdout:6/821: fdatasync d0/d1/d12/d3b/d48/fc8 0 2024-02-16T19:24:20.047 INFO:tasks.workunit.client.0.smithi040.stdout:6/822: chown d0/d1/d12/f66 15 1 2024-02-16T19:24:20.047 INFO:tasks.workunit.client.0.smithi040.stdout:6/823: chown d0/d30/fb5 403651 1 2024-02-16T19:24:20.047 INFO:tasks.workunit.client.0.smithi040.stdout:1/684: mkdir d17/d3c/dde 0 2024-02-16T19:24:20.047 INFO:tasks.workunit.client.0.smithi040.stdout:1/685: chown d17/d26/f7a 10932072 1 2024-02-16T19:24:20.048 INFO:tasks.workunit.client.0.smithi040.stdout:1/686: truncate d17/d26/d5f/fd4 273560 0 2024-02-16T19:24:20.049 INFO:tasks.workunit.client.0.smithi040.stdout:1/687: dread d17/d3c/d71/fa0 [0,4194304] 0 2024-02-16T19:24:20.049 INFO:tasks.workunit.client.0.smithi040.stdout:3/773: symlink d1/dc/d42/d58/de8/lf6 0 2024-02-16T19:24:20.050 INFO:tasks.workunit.client.1.smithi163.stdout:8/76: mknod d0/d7/d18/c1a 0 2024-02-16T19:24:20.055 INFO:tasks.workunit.client.1.smithi163.stdout:3/143: rename d0/dc/d1c/d21/f25 to d0/dc/d1c/d32/f36 0 2024-02-16T19:24:20.057 INFO:tasks.workunit.client.1.smithi163.stdout:1/63: symlink d7/dc/l11 0 2024-02-16T19:24:20.062 INFO:tasks.workunit.client.1.smithi163.stdout:1/64: dread f2 [0,4194304] 0 2024-02-16T19:24:20.066 INFO:tasks.workunit.client.1.smithi163.stdout:9/291: unlink d5/cb 0 2024-02-16T19:24:20.067 INFO:tasks.workunit.client.0.smithi040.stdout:7/641: mknod d21/d29/d79/cd5 0 2024-02-16T19:24:20.067 INFO:tasks.workunit.client.0.smithi040.stdout:7/642: chown d21/d8b/d9c 21883 1 2024-02-16T19:24:20.068 INFO:tasks.workunit.client.1.smithi163.stdout:7/95: readlink l0 0 2024-02-16T19:24:20.068 INFO:tasks.workunit.client.1.smithi163.stdout:7/96: dread - ff zero size 2024-02-16T19:24:20.069 INFO:tasks.workunit.client.0.smithi040.stdout:5/665: getdents d1/d45/dab 0 2024-02-16T19:24:20.070 INFO:tasks.workunit.client.1.smithi163.stdout:6/98: mkdir d0/d6/d11/d1e 0 2024-02-16T19:24:20.073 INFO:tasks.workunit.client.1.smithi163.stdout:2/221: getdents d2/d2a 0 2024-02-16T19:24:20.074 INFO:tasks.workunit.client.1.smithi163.stdout:2/222: dread d2/d2a/f33 [0,4194304] 0 2024-02-16T19:24:20.074 INFO:tasks.workunit.client.1.smithi163.stdout:6/99: fdatasync d0/dd/f13 0 2024-02-16T19:24:20.075 INFO:tasks.workunit.client.0.smithi040.stdout:6/824: unlink d0/d1/f52 0 2024-02-16T19:24:20.076 INFO:tasks.workunit.client.0.smithi040.stdout:1/688: rename d17/d3c/d6b/c8e to d17/d3c/d6b/cdf 0 2024-02-16T19:24:20.078 INFO:tasks.workunit.client.1.smithi163.stdout:0/147: rename d11/f1a to d11/d22/f34 0 2024-02-16T19:24:20.078 INFO:tasks.workunit.client.1.smithi163.stdout:0/148: truncate d11/f33 891583 0 2024-02-16T19:24:20.078 INFO:tasks.workunit.client.1.smithi163.stdout:0/149: write f8 [6834,92930] 0 2024-02-16T19:24:20.078 INFO:tasks.workunit.client.1.smithi163.stdout:3/144: symlink d0/l37 0 2024-02-16T19:24:20.079 INFO:tasks.workunit.client.0.smithi040.stdout:5/666: truncate d1/d3c/f72 2380060 0 2024-02-16T19:24:20.079 INFO:tasks.workunit.client.0.smithi040.stdout:5/667: write d1/da/d51/f8b [4010260,70089] 0 2024-02-16T19:24:20.080 INFO:tasks.workunit.client.1.smithi163.stdout:1/65: link d7/l8 d7/dc/l12 0 2024-02-16T19:24:20.080 INFO:tasks.workunit.client.1.smithi163.stdout:1/66: chown d7/dc 13 1 2024-02-16T19:24:20.081 INFO:tasks.workunit.client.1.smithi163.stdout:1/67: chown f2 29685 1 2024-02-16T19:24:20.081 INFO:tasks.workunit.client.0.smithi040.stdout:6/825: mknod d0/d1/d6f/d90/db1/c10e 0 2024-02-16T19:24:20.082 INFO:tasks.workunit.client.0.smithi040.stdout:6/826: fdatasync d0/d30/d32/dd2/dbc/f106 0 2024-02-16T19:24:20.082 INFO:tasks.workunit.client.0.smithi040.stdout:3/774: dwrite d1/fe7 [0,4194304] 0 2024-02-16T19:24:20.082 INFO:tasks.workunit.client.1.smithi163.stdout:8/77: dwrite d0/d7/fb [0,4194304] 0 2024-02-16T19:24:20.083 INFO:tasks.workunit.client.1.smithi163.stdout:1/68: fdatasync d7/fa 0 2024-02-16T19:24:20.083 INFO:tasks.workunit.client.1.smithi163.stdout:4/75: sync 2024-02-16T19:24:20.083 INFO:tasks.workunit.client.1.smithi163.stdout:4/76: stat l0 0 2024-02-16T19:24:20.084 INFO:tasks.workunit.client.1.smithi163.stdout:5/72: unlink d0/f15 0 2024-02-16T19:24:20.086 INFO:tasks.workunit.client.0.smithi040.stdout:1/689: unlink d17/d26/l40 0 2024-02-16T19:24:20.087 INFO:tasks.workunit.client.1.smithi163.stdout:9/292: symlink d5/de/d30/d43/l67 0 2024-02-16T19:24:20.087 INFO:tasks.workunit.client.1.smithi163.stdout:9/293: truncate d5/de/d30/d31/f51 928820 0 2024-02-16T19:24:20.092 INFO:tasks.workunit.client.0.smithi040.stdout:5/668: unlink d1/da/d1c/dba/db1/fa3 0 2024-02-16T19:24:20.093 INFO:tasks.workunit.client.0.smithi040.stdout:1/690: dread d17/d26/f7a [4194304,4194304] 0 2024-02-16T19:24:20.093 INFO:tasks.workunit.client.1.smithi163.stdout:2/223: mknod d2/d2a/d39/d3a/c4a 0 2024-02-16T19:24:20.095 INFO:tasks.workunit.client.1.smithi163.stdout:6/100: mknod d0/d6/d9/d15/c1f 0 2024-02-16T19:24:20.096 INFO:tasks.workunit.client.1.smithi163.stdout:9/294: dread d5/f2f [0,4194304] 0 2024-02-16T19:24:20.097 INFO:tasks.workunit.client.1.smithi163.stdout:6/101: dread d0/dd/f12 [0,4194304] 0 2024-02-16T19:24:20.097 INFO:tasks.workunit.client.1.smithi163.stdout:6/102: chown d0/d6/l1b 81 1 2024-02-16T19:24:20.097 INFO:tasks.workunit.client.1.smithi163.stdout:6/103: fdatasync d0/d6/f8 0 2024-02-16T19:24:20.100 INFO:tasks.workunit.client.1.smithi163.stdout:0/150: link f3 d11/f35 0 2024-02-16T19:24:20.102 INFO:tasks.workunit.client.0.smithi040.stdout:5/669: creat d1/da/fcd x:0 0 0 2024-02-16T19:24:20.102 INFO:tasks.workunit.client.0.smithi040.stdout:5/670: write d1/da/da4/fc1 [295373,99195] 0 2024-02-16T19:24:20.102 INFO:tasks.workunit.client.0.smithi040.stdout:5/671: fsync d1/da/d1c/d73/fbd 0 2024-02-16T19:24:20.102 INFO:tasks.workunit.client.0.smithi040.stdout:5/672: write d1/da/d1c/d73/fbd [409221,10173] 0 2024-02-16T19:24:20.102 INFO:tasks.workunit.client.0.smithi040.stdout:3/775: dwrite d1/d4/f25 [4194304,4194304] 0 2024-02-16T19:24:20.102 INFO:tasks.workunit.client.0.smithi040.stdout:1/691: dread d17/d29/f2b [0,4194304] 0 2024-02-16T19:24:20.105 INFO:tasks.workunit.client.1.smithi163.stdout:3/145: mkdir d0/dc/d1d/d29/d38 0 2024-02-16T19:24:20.105 INFO:tasks.workunit.client.1.smithi163.stdout:3/146: creat d0/f39 x:0 0 0 2024-02-16T19:24:20.105 INFO:tasks.workunit.client.1.smithi163.stdout:3/147: read d0/f13 [558875,2364] 0 2024-02-16T19:24:20.108 INFO:tasks.workunit.client.0.smithi040.stdout:3/776: symlink d1/lf7 0 2024-02-16T19:24:20.109 INFO:tasks.workunit.client.1.smithi163.stdout:8/78: creat d0/d7/d19/f1b x:0 0 0 2024-02-16T19:24:20.110 INFO:tasks.workunit.client.1.smithi163.stdout:8/79: readlink d0/d7/la 0 2024-02-16T19:24:20.111 INFO:tasks.workunit.client.0.smithi040.stdout:1/692: symlink d17/d26/d9a/le0 0 2024-02-16T19:24:20.111 INFO:tasks.workunit.client.0.smithi040.stdout:1/693: truncate d17/d26/d5f/da6/fc6 266141 0 2024-02-16T19:24:20.111 INFO:tasks.workunit.client.0.smithi040.stdout:1/694: creat d17/d26/d5f/fe1 x:0 0 0 2024-02-16T19:24:20.112 INFO:tasks.workunit.client.1.smithi163.stdout:1/69: symlink d7/dc/l13 0 2024-02-16T19:24:20.114 INFO:tasks.workunit.client.1.smithi163.stdout:2/224: dwrite d2/f23 [0,4194304] 0 2024-02-16T19:24:20.114 INFO:tasks.workunit.client.1.smithi163.stdout:2/225: chown d2/d26/l48 27031324 1 2024-02-16T19:24:20.115 INFO:tasks.workunit.client.1.smithi163.stdout:4/77: rename fe to d11/f12 0 2024-02-16T19:24:20.115 INFO:tasks.workunit.client.1.smithi163.stdout:4/78: chown ca 1017627922 1 2024-02-16T19:24:20.118 INFO:tasks.workunit.client.1.smithi163.stdout:7/97: getdents d11 0 2024-02-16T19:24:20.118 INFO:tasks.workunit.client.1.smithi163.stdout:7/98: read d11/f19 [51645,5435] 0 2024-02-16T19:24:20.118 INFO:tasks.workunit.client.1.smithi163.stdout:7/99: fdatasync d11/f1c 0 2024-02-16T19:24:20.118 INFO:tasks.workunit.client.1.smithi163.stdout:7/100: write d11/f12 [1558027,49037] 0 2024-02-16T19:24:20.120 INFO:tasks.workunit.client.0.smithi040.stdout:3/777: creat d1/dc/d42/d4a/d57/d79/de3/ff8 x:0 0 0 2024-02-16T19:24:20.121 INFO:tasks.workunit.client.1.smithi163.stdout:6/104: truncate d0/f2 4622971 0 2024-02-16T19:24:20.122 INFO:tasks.workunit.client.0.smithi040.stdout:3/778: dread d1/d7/d30/f81 [0,4194304] 0 2024-02-16T19:24:20.122 INFO:tasks.workunit.client.0.smithi040.stdout:3/779: chown d1/d4/d6e/d8a/lc5 27 1 2024-02-16T19:24:20.124 INFO:tasks.workunit.client.1.smithi163.stdout:0/151: symlink d11/d21/l36 0 2024-02-16T19:24:20.127 INFO:tasks.workunit.client.0.smithi040.stdout:3/780: rename d1/dc/d35/d55/f74 to d1/d7/d36/ff9 0 2024-02-16T19:24:20.127 INFO:tasks.workunit.client.0.smithi040.stdout:3/781: chown d1/dc/d42/d58/de8/lf6 1 1 2024-02-16T19:24:20.129 INFO:tasks.workunit.client.0.smithi040.stdout:1/695: getdents d17/d26/d5f/da6 0 2024-02-16T19:24:20.129 INFO:tasks.workunit.client.0.smithi040.stdout:1/696: readlink d17/d3c/d74/d97/lb3 0 2024-02-16T19:24:20.130 INFO:tasks.workunit.client.0.smithi040.stdout:1/697: fdatasync d17/d43/fdc 0 2024-02-16T19:24:20.130 INFO:tasks.workunit.client.0.smithi040.stdout:1/698: chown d17/d43/fa2 9 1 2024-02-16T19:24:20.131 INFO:tasks.workunit.client.1.smithi163.stdout:1/70: link f2 d7/f14 0 2024-02-16T19:24:20.132 INFO:tasks.workunit.client.0.smithi040.stdout:3/782: link d1/d7/d33/c1e d1/d7/d33/cfa 0 2024-02-16T19:24:20.132 INFO:tasks.workunit.client.0.smithi040.stdout:3/783: write d1/ff0 [430011,62957] 0 2024-02-16T19:24:20.135 INFO:tasks.workunit.client.0.smithi040.stdout:1/699: rename d17/fc7 to d17/d26/d4c/fe2 0 2024-02-16T19:24:20.140 INFO:tasks.workunit.client.0.smithi040.stdout:3/784: unlink d1/dc/fc6 0 2024-02-16T19:24:20.140 INFO:tasks.workunit.client.0.smithi040.stdout:3/785: stat d1/d7/d30/f40 0 2024-02-16T19:24:20.141 INFO:tasks.workunit.client.1.smithi163.stdout:2/226: mknod d2/d2a/d2d/c4b 0 2024-02-16T19:24:20.141 INFO:tasks.workunit.client.1.smithi163.stdout:2/227: write d2/fc [1086640,25256] 0 2024-02-16T19:24:20.142 INFO:tasks.workunit.client.0.smithi040.stdout:1/700: symlink d17/d26/d5f/le3 0 2024-02-16T19:24:20.144 INFO:tasks.workunit.client.1.smithi163.stdout:5/73: rename d0/f13 to d0/d2/f16 0 2024-02-16T19:24:20.146 INFO:tasks.workunit.client.0.smithi040.stdout:1/701: mknod d17/d26/d5f/d78/ce4 0 2024-02-16T19:24:20.146 INFO:tasks.workunit.client.1.smithi163.stdout:1/71: dwrite f6 [0,4194304] 0 2024-02-16T19:24:20.147 INFO:tasks.workunit.client.1.smithi163.stdout:4/79: link cf d11/c13 0 2024-02-16T19:24:20.147 INFO:tasks.workunit.client.1.smithi163.stdout:4/80: write f10 [242535,20241] 0 2024-02-16T19:24:20.148 INFO:tasks.workunit.client.1.smithi163.stdout:4/81: chown fd 54720 1 2024-02-16T19:24:20.148 INFO:tasks.workunit.client.0.smithi040.stdout:3/786: getdents d1/d4/d6e/d8a/da5 0 2024-02-16T19:24:20.150 INFO:tasks.workunit.client.1.smithi163.stdout:6/105: mkdir d0/d6/df/d20 0 2024-02-16T19:24:20.150 INFO:tasks.workunit.client.1.smithi163.stdout:6/106: read d0/dd/f13 [4598452,105261] 0 2024-02-16T19:24:20.150 INFO:tasks.workunit.client.1.smithi163.stdout:6/107: write d0/f5 [4951208,69111] 0 2024-02-16T19:24:20.150 INFO:tasks.workunit.client.1.smithi163.stdout:6/108: write d0/d6/f8 [379326,48020] 0 2024-02-16T19:24:20.150 INFO:tasks.workunit.client.0.smithi040.stdout:1/702: stat d17/d3c/d74/d97/da9/lbb 0 2024-02-16T19:24:20.151 INFO:tasks.workunit.client.1.smithi163.stdout:0/152: link d11/d12/d20/l2e d11/l37 0 2024-02-16T19:24:20.153 INFO:tasks.workunit.client.0.smithi040.stdout:3/787: mkdir d1/d7/d33/dfb 0 2024-02-16T19:24:20.154 INFO:tasks.workunit.client.1.smithi163.stdout:8/80: symlink d0/l1c 0 2024-02-16T19:24:20.155 INFO:tasks.workunit.client.0.smithi040.stdout:1/703: creat d17/d3c/d74/d97/fe5 x:0 0 0 2024-02-16T19:24:20.156 INFO:tasks.workunit.client.1.smithi163.stdout:2/228: symlink d2/d2a/d2d/l4c 0 2024-02-16T19:24:20.157 INFO:tasks.workunit.client.1.smithi163.stdout:5/74: dwrite d0/d2/f16 [0,4194304] 0 2024-02-16T19:24:20.158 INFO:tasks.workunit.client.0.smithi040.stdout:3/788: link d1/d4/d6e/d8a/caf d1/d8c/d8e/cfc 0 2024-02-16T19:24:20.160 INFO:tasks.workunit.client.1.smithi163.stdout:9/295: rename d5/de/d30/l4f to d5/l68 0 2024-02-16T19:24:20.160 INFO:tasks.workunit.client.1.smithi163.stdout:9/296: fsync d5/de/d16/f53 0 2024-02-16T19:24:20.160 INFO:tasks.workunit.client.1.smithi163.stdout:9/297: write d5/f2e [1583817,47190] 0 2024-02-16T19:24:20.167 INFO:tasks.workunit.client.1.smithi163.stdout:4/82: creat d11/f14 x:0 0 0 2024-02-16T19:24:20.168 INFO:tasks.workunit.client.1.smithi163.stdout:4/83: write f6 [700950,63266] 0 2024-02-16T19:24:20.168 INFO:tasks.workunit.client.1.smithi163.stdout:2/229: read d2/f15 [3945111,130706] 0 2024-02-16T19:24:20.169 INFO:tasks.workunit.client.1.smithi163.stdout:6/109: mknod d0/d6/d11/c21 0 2024-02-16T19:24:20.170 INFO:tasks.workunit.client.1.smithi163.stdout:0/153: symlink d11/d12/d32/l38 0 2024-02-16T19:24:20.171 INFO:tasks.workunit.client.1.smithi163.stdout:0/154: creat d11/d12/d20/f39 x:0 0 0 2024-02-16T19:24:20.171 INFO:tasks.workunit.client.1.smithi163.stdout:0/155: creat d11/d12/d20/f3a x:0 0 0 2024-02-16T19:24:20.171 INFO:tasks.workunit.client.1.smithi163.stdout:8/81: symlink d0/d7/d17/l1d 0 2024-02-16T19:24:20.171 INFO:tasks.workunit.client.1.smithi163.stdout:8/82: dread - d0/d7/d19/f1b zero size 2024-02-16T19:24:20.172 INFO:tasks.workunit.client.1.smithi163.stdout:6/110: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:20.173 INFO:tasks.workunit.client.1.smithi163.stdout:5/75: mknod d0/d2/c17 0 2024-02-16T19:24:20.174 INFO:tasks.workunit.client.1.smithi163.stdout:3/148: rename d0/dc/d1d/l1e to d0/dc/d1d/d29/d38/l3a 0 2024-02-16T19:24:20.177 INFO:tasks.workunit.client.0.smithi040.stdout:3/789: dwrite d1/d7/d30/f63 [0,4194304] 0 2024-02-16T19:24:20.178 INFO:tasks.workunit.client.1.smithi163.stdout:9/298: creat d5/de/d30/d31/f69 x:0 0 0 2024-02-16T19:24:20.178 INFO:tasks.workunit.client.1.smithi163.stdout:2/230: mknod d2/d2a/c4d 0 2024-02-16T19:24:20.179 INFO:tasks.workunit.client.1.smithi163.stdout:2/231: chown d2/d26/f3f 185 1 2024-02-16T19:24:20.183 INFO:tasks.workunit.client.1.smithi163.stdout:2/232: dread d2/f9 [0,4194304] 0 2024-02-16T19:24:20.186 INFO:tasks.workunit.client.0.smithi040.stdout:3/790: link d1/l95 d1/d7/d33/lfd 0 2024-02-16T19:24:20.188 INFO:tasks.workunit.client.1.smithi163.stdout:8/83: dread d0/dc/f16 [0,4194304] 0 2024-02-16T19:24:20.188 INFO:tasks.workunit.client.1.smithi163.stdout:8/84: dread - d0/ff zero size 2024-02-16T19:24:20.188 INFO:tasks.workunit.client.1.smithi163.stdout:0/156: link d11/d12/l26 d11/d12/l3b 0 2024-02-16T19:24:20.190 INFO:tasks.workunit.client.1.smithi163.stdout:5/76: rmdir d0/d2/d10 0 2024-02-16T19:24:20.190 INFO:tasks.workunit.client.1.smithi163.stdout:6/111: dread d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:20.192 INFO:tasks.workunit.client.0.smithi040.stdout:3/791: link d1/d7/d30/l31 d1/dc/d42/lfe 0 2024-02-16T19:24:20.195 INFO:tasks.workunit.client.1.smithi163.stdout:7/101: rename le to d11/l23 0 2024-02-16T19:24:20.197 INFO:tasks.workunit.client.1.smithi163.stdout:9/299: link d5/de/d16/c1d d5/de/d16/d47/c6a 0 2024-02-16T19:24:20.198 INFO:tasks.workunit.client.1.smithi163.stdout:0/157: truncate d11/d22/f34 4129755 0 2024-02-16T19:24:20.199 INFO:tasks.workunit.client.1.smithi163.stdout:6/112: creat d0/d18/f22 x:0 0 0 2024-02-16T19:24:20.201 INFO:tasks.workunit.client.1.smithi163.stdout:1/72: rename d7/dc/ff to d7/f15 0 2024-02-16T19:24:20.201 INFO:tasks.workunit.client.1.smithi163.stdout:0/158: dread fd [0,4194304] 0 2024-02-16T19:24:20.201 INFO:tasks.workunit.client.0.smithi040.stdout:3/792: dread d1/d7/f4e [0,4194304] 0 2024-02-16T19:24:20.201 INFO:tasks.workunit.client.0.smithi040.stdout:3/793: fsync d1/dc/d42/d5c/df3/fc1 0 2024-02-16T19:24:20.202 INFO:tasks.workunit.client.1.smithi163.stdout:7/102: mkdir d11/d24 0 2024-02-16T19:24:20.202 INFO:tasks.workunit.client.1.smithi163.stdout:7/103: fdatasync f3 0 2024-02-16T19:24:20.203 INFO:tasks.workunit.client.1.smithi163.stdout:9/300: rmdir d5/de/d16 39 2024-02-16T19:24:20.204 INFO:tasks.workunit.client.1.smithi163.stdout:2/233: dwrite d2/d2a/f33 [0,4194304] 0 2024-02-16T19:24:20.207 INFO:tasks.workunit.client.1.smithi163.stdout:6/113: creat d0/dd/f23 x:0 0 0 2024-02-16T19:24:20.207 INFO:tasks.workunit.client.1.smithi163.stdout:6/114: stat d0/d18 0 2024-02-16T19:24:20.207 INFO:tasks.workunit.client.0.smithi040.stdout:3/794: creat d1/d4/d6e/d8a/da5/fff x:0 0 0 2024-02-16T19:24:20.208 INFO:tasks.workunit.client.1.smithi163.stdout:5/77: write d0/fd [1913140,117498] 0 2024-02-16T19:24:20.211 INFO:tasks.workunit.client.1.smithi163.stdout:4/84: rename f6 to d11/f15 0 2024-02-16T19:24:20.211 INFO:tasks.workunit.client.1.smithi163.stdout:4/85: creat d11/f16 x:0 0 0 2024-02-16T19:24:20.215 INFO:tasks.workunit.client.1.smithi163.stdout:9/301: creat d5/de/d16/f6b x:0 0 0 2024-02-16T19:24:20.215 INFO:tasks.workunit.client.1.smithi163.stdout:9/302: dread - d5/de/d30/d3d/f66 zero size 2024-02-16T19:24:20.220 INFO:tasks.workunit.client.1.smithi163.stdout:2/234: symlink d2/d2a/d39/d3a/l4e 0 2024-02-16T19:24:20.221 INFO:tasks.workunit.client.1.smithi163.stdout:5/78: mknod d0/c18 0 2024-02-16T19:24:20.222 INFO:tasks.workunit.client.1.smithi163.stdout:5/79: write d0/d2/fa [4671154,34848] 0 2024-02-16T19:24:20.225 INFO:tasks.workunit.client.1.smithi163.stdout:3/149: rename d0/f1 to d0/dc/d1c/f3b 0 2024-02-16T19:24:20.228 INFO:tasks.workunit.client.1.smithi163.stdout:0/159: dwrite d11/d12/d20/f39 [0,4194304] 0 2024-02-16T19:24:20.233 INFO:tasks.workunit.client.1.smithi163.stdout:5/80: symlink d0/l19 0 2024-02-16T19:24:20.233 INFO:tasks.workunit.client.1.smithi163.stdout:5/81: read d0/d2/fb [1838941,25165] 0 2024-02-16T19:24:20.233 INFO:tasks.workunit.client.1.smithi163.stdout:5/82: creat d0/d2/f1a x:0 0 0 2024-02-16T19:24:20.233 INFO:tasks.workunit.client.1.smithi163.stdout:5/83: truncate d0/d2/f16 4370807 0 2024-02-16T19:24:20.234 INFO:tasks.workunit.client.1.smithi163.stdout:8/85: rename d0/d7/fb to d0/d7/d18/f1e 0 2024-02-16T19:24:20.241 INFO:tasks.workunit.client.1.smithi163.stdout:8/86: rmdir d0/dc 39 2024-02-16T19:24:20.243 INFO:tasks.workunit.client.1.smithi163.stdout:8/87: mkdir d0/d7/d19/d1f 0 2024-02-16T19:24:20.251 INFO:tasks.workunit.client.1.smithi163.stdout:0/160: dread f10 [0,4194304] 0 2024-02-16T19:24:20.253 INFO:tasks.workunit.client.1.smithi163.stdout:0/161: mkdir d11/d12/d32/d3c 0 2024-02-16T19:24:20.253 INFO:tasks.workunit.client.1.smithi163.stdout:0/162: write fd [3753616,128483] 0 2024-02-16T19:24:20.253 INFO:tasks.workunit.client.1.smithi163.stdout:0/163: fsync d11/d12/f15 0 2024-02-16T19:24:20.254 INFO:tasks.workunit.client.1.smithi163.stdout:0/164: symlink d11/d12/d32/d3c/l3d 0 2024-02-16T19:24:20.255 INFO:tasks.workunit.client.1.smithi163.stdout:5/84: read d0/d2/fa [4117892,104949] 0 2024-02-16T19:24:20.255 INFO:tasks.workunit.client.1.smithi163.stdout:5/85: write d0/d2/f16 [5342565,92289] 0 2024-02-16T19:24:20.255 INFO:tasks.workunit.client.1.smithi163.stdout:5/86: chown d0/c14 31 1 2024-02-16T19:24:20.256 INFO:tasks.workunit.client.1.smithi163.stdout:1/73: dwrite f0 [0,4194304] 0 2024-02-16T19:24:20.256 INFO:tasks.workunit.client.1.smithi163.stdout:1/74: fsync f0 0 2024-02-16T19:24:20.256 INFO:tasks.workunit.client.1.smithi163.stdout:1/75: readlink d7/dc/l12 0 2024-02-16T19:24:20.257 INFO:tasks.workunit.client.1.smithi163.stdout:1/76: symlink d7/l16 0 2024-02-16T19:24:20.267 INFO:tasks.workunit.client.1.smithi163.stdout:0/165: read d11/d12/d20/f39 [438781,22352] 0 2024-02-16T19:24:20.267 INFO:tasks.workunit.client.1.smithi163.stdout:0/166: write d11/d12/d20/f3a [292998,20708] 0 2024-02-16T19:24:20.268 INFO:tasks.workunit.client.0.smithi040.stdout:3/795: dwrite d1/d7/d36/f87 [0,4194304] 0 2024-02-16T19:24:20.270 INFO:tasks.workunit.client.1.smithi163.stdout:6/115: dwrite d0/d6/f1a [0,4194304] 0 2024-02-16T19:24:20.271 INFO:tasks.workunit.client.1.smithi163.stdout:4/86: dwrite f9 [0,4194304] 0 2024-02-16T19:24:20.271 INFO:tasks.workunit.client.1.smithi163.stdout:9/303: dwrite d5/de/d16/f53 [0,4194304] 0 2024-02-16T19:24:20.272 INFO:tasks.workunit.client.1.smithi163.stdout:6/116: creat d0/d6/d11/d1e/f24 x:0 0 0 2024-02-16T19:24:20.273 INFO:tasks.workunit.client.1.smithi163.stdout:2/235: dwrite d2/fb [0,4194304] 0 2024-02-16T19:24:20.276 INFO:tasks.workunit.client.1.smithi163.stdout:3/150: dwrite d0/dc/f31 [0,4194304] 0 2024-02-16T19:24:20.277 INFO:tasks.workunit.client.1.smithi163.stdout:3/151: dread d0/dc/d1c/f20 [0,4194304] 0 2024-02-16T19:24:20.277 INFO:tasks.workunit.client.1.smithi163.stdout:3/152: creat d0/dc/f3c x:0 0 0 2024-02-16T19:24:20.277 INFO:tasks.workunit.client.1.smithi163.stdout:3/153: fdatasync d0/dc/f28 0 2024-02-16T19:24:20.278 INFO:tasks.workunit.client.0.smithi040.stdout:8/777: sync 2024-02-16T19:24:20.278 INFO:tasks.workunit.client.0.smithi040.stdout:4/589: sync 2024-02-16T19:24:20.278 INFO:tasks.workunit.client.0.smithi040.stdout:0/792: sync 2024-02-16T19:24:20.278 INFO:tasks.workunit.client.0.smithi040.stdout:2/611: sync 2024-02-16T19:24:20.278 INFO:tasks.workunit.client.0.smithi040.stdout:0/793: dread - db/d25/fdf zero size 2024-02-16T19:24:20.278 INFO:tasks.workunit.client.0.smithi040.stdout:0/794: fsync db/d25/d34/d5e/fb4 0 2024-02-16T19:24:20.278 INFO:tasks.workunit.client.0.smithi040.stdout:2/612: write d5/d7/fa1 [4756637,18153] 0 2024-02-16T19:24:20.283 INFO:tasks.workunit.client.0.smithi040.stdout:3/796: write d1/d4/d6e/f84 [3944287,113447] 0 2024-02-16T19:24:20.283 INFO:tasks.workunit.client.0.smithi040.stdout:3/797: truncate d1/dc/d42/d4a/d57/d79/de3/ff8 906394 0 2024-02-16T19:24:20.290 INFO:tasks.workunit.client.1.smithi163.stdout:4/87: rmdir d11 39 2024-02-16T19:24:20.290 INFO:tasks.workunit.client.1.smithi163.stdout:4/88: chown d11/f15 19 1 2024-02-16T19:24:20.291 INFO:tasks.workunit.client.1.smithi163.stdout:4/89: chown f2 173516 1 2024-02-16T19:24:20.295 INFO:tasks.workunit.client.0.smithi040.stdout:2/613: dread d5/d7/f74 [4194304,4194304] 0 2024-02-16T19:24:20.295 INFO:tasks.workunit.client.0.smithi040.stdout:2/614: creat d5/d8/d42/fc8 x:0 0 0 2024-02-16T19:24:20.295 INFO:tasks.workunit.client.1.smithi163.stdout:4/90: creat d11/f17 x:0 0 0 2024-02-16T19:24:20.296 INFO:tasks.workunit.client.1.smithi163.stdout:5/87: getdents d0/d2 0 2024-02-16T19:24:20.297 INFO:tasks.workunit.client.1.smithi163.stdout:8/88: rmdir d0/d7 39 2024-02-16T19:24:20.298 INFO:tasks.workunit.client.1.smithi163.stdout:9/304: dwrite d5/f19 [4194304,4194304] 0 2024-02-16T19:24:20.301 INFO:tasks.workunit.client.0.smithi040.stdout:2/615: link d5/d7/d1e/d31/d4f/l99 d5/d7/lc9 0 2024-02-16T19:24:20.306 INFO:tasks.workunit.client.1.smithi163.stdout:0/167: rename d11/d12/d32 to d11/d22/d3e 0 2024-02-16T19:24:20.306 INFO:tasks.workunit.client.1.smithi163.stdout:0/168: write d11/d22/f2d [302298,120618] 0 2024-02-16T19:24:20.307 INFO:tasks.workunit.client.1.smithi163.stdout:6/117: rename d0/d6/l1d to d0/d6/df/l25 0 2024-02-16T19:24:20.310 INFO:tasks.workunit.client.0.smithi040.stdout:0/795: dwrite db/d25/d34/d5e/fed [0,4194304] 0 2024-02-16T19:24:20.310 INFO:tasks.workunit.client.1.smithi163.stdout:0/169: dread d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:20.310 INFO:tasks.workunit.client.1.smithi163.stdout:0/170: chown d11/d12/c1b 1957223170 1 2024-02-16T19:24:20.310 INFO:tasks.workunit.client.1.smithi163.stdout:0/171: creat d11/d22/f3f x:0 0 0 2024-02-16T19:24:20.311 INFO:tasks.workunit.client.1.smithi163.stdout:2/236: rename d2/d2a/d39/d3a/l47 to d2/d2a/d2d/d3c/l4f 0 2024-02-16T19:24:20.311 INFO:tasks.workunit.client.1.smithi163.stdout:5/88: rename d0/d2 to d0/d2/d1b 22 2024-02-16T19:24:20.312 INFO:tasks.workunit.client.1.smithi163.stdout:6/118: write d0/f2 [314461,63598] 0 2024-02-16T19:24:20.312 INFO:tasks.workunit.client.1.smithi163.stdout:6/119: chown d0/d6/l16 14614 1 2024-02-16T19:24:20.316 INFO:tasks.workunit.client.1.smithi163.stdout:0/172: unlink f7 0 2024-02-16T19:24:20.317 INFO:tasks.workunit.client.0.smithi040.stdout:0/796: unlink db/d2f/d23/d3b/l60 0 2024-02-16T19:24:20.317 INFO:tasks.workunit.client.0.smithi040.stdout:0/797: creat db/d2f/d1d/d21/f10d x:0 0 0 2024-02-16T19:24:20.317 INFO:tasks.workunit.client.0.smithi040.stdout:0/798: dread - db/d2f/d1d/d21/f10d zero size 2024-02-16T19:24:20.318 INFO:tasks.workunit.client.1.smithi163.stdout:0/173: read d11/f33 [47350,101001] 0 2024-02-16T19:24:20.318 INFO:tasks.workunit.client.1.smithi163.stdout:5/89: mkdir d0/d1c 0 2024-02-16T19:24:20.318 INFO:tasks.workunit.client.1.smithi163.stdout:5/90: creat d0/f1d x:0 0 0 2024-02-16T19:24:20.318 INFO:tasks.workunit.client.1.smithi163.stdout:5/91: chown d0/fd 58074 1 2024-02-16T19:24:20.319 INFO:tasks.workunit.client.1.smithi163.stdout:6/120: symlink d0/d6/d11/d1e/l26 0 2024-02-16T19:24:20.319 INFO:tasks.workunit.client.1.smithi163.stdout:6/121: chown d0/d6/d11/d1e/f24 0 1 2024-02-16T19:24:20.320 INFO:tasks.workunit.client.1.smithi163.stdout:2/237: dread d2/f16 [0,4194304] 0 2024-02-16T19:24:20.321 INFO:tasks.workunit.client.1.smithi163.stdout:2/238: fdatasync d2/d26/f3d 0 2024-02-16T19:24:20.321 INFO:tasks.workunit.client.1.smithi163.stdout:6/122: read d0/dd/f12 [107767,89180] 0 2024-02-16T19:24:20.321 INFO:tasks.workunit.client.1.smithi163.stdout:0/174: link d11/d21/l29 d11/d22/d3e/l40 0 2024-02-16T19:24:20.321 INFO:tasks.workunit.client.1.smithi163.stdout:0/175: chown d11/f1f 1256684 1 2024-02-16T19:24:20.323 INFO:tasks.workunit.client.0.smithi040.stdout:0/799: unlink db/de/d6a/df0/f107 0 2024-02-16T19:24:20.323 INFO:tasks.workunit.client.0.smithi040.stdout:0/800: creat db/de/dbe/f10e x:0 0 0 2024-02-16T19:24:20.324 INFO:tasks.workunit.client.1.smithi163.stdout:5/92: truncate d0/fd 924075 0 2024-02-16T19:24:20.326 INFO:tasks.workunit.client.1.smithi163.stdout:6/123: symlink d0/d18/l27 0 2024-02-16T19:24:20.327 INFO:tasks.workunit.client.0.smithi040.stdout:0/801: creat db/d2f/d1d/f10f x:0 0 0 2024-02-16T19:24:20.328 INFO:tasks.workunit.client.0.smithi040.stdout:0/802: creat db/de/d6a/f110 x:0 0 0 2024-02-16T19:24:20.329 INFO:tasks.workunit.client.1.smithi163.stdout:5/93: symlink d0/d2/l1e 0 2024-02-16T19:24:20.330 INFO:tasks.workunit.client.1.smithi163.stdout:6/124: symlink d0/d6/d9/d15/l28 0 2024-02-16T19:24:20.330 INFO:tasks.workunit.client.1.smithi163.stdout:2/239: dread d2/f15 [0,4194304] 0 2024-02-16T19:24:20.330 INFO:tasks.workunit.client.1.smithi163.stdout:6/125: write d0/dd/f23 [837269,86683] 0 2024-02-16T19:24:20.332 INFO:tasks.workunit.client.1.smithi163.stdout:0/176: getdents d11 0 2024-02-16T19:24:20.332 INFO:tasks.workunit.client.1.smithi163.stdout:0/177: chown d11/c18 9039463 1 2024-02-16T19:24:20.333 INFO:tasks.workunit.client.1.smithi163.stdout:5/94: symlink d0/d2/l1f 0 2024-02-16T19:24:20.334 INFO:tasks.workunit.client.1.smithi163.stdout:6/126: mknod d0/d18/c29 0 2024-02-16T19:24:20.335 INFO:tasks.workunit.client.1.smithi163.stdout:5/95: mknod d0/d1c/c20 0 2024-02-16T19:24:20.336 INFO:tasks.workunit.client.1.smithi163.stdout:6/127: creat d0/d6/df/f2a x:0 0 0 2024-02-16T19:24:20.336 INFO:tasks.workunit.client.1.smithi163.stdout:6/128: stat d0/d6/d9/d15/d1c 0 2024-02-16T19:24:20.336 INFO:tasks.workunit.client.0.smithi040.stdout:0/803: dread db/d2f/d1d/d21/d2e/d49/d5b/d67/f92 [0,4194304] 0 2024-02-16T19:24:20.337 INFO:tasks.workunit.client.0.smithi040.stdout:8/778: dwrite d6/d12/d51/d6e/db7/dc6/fdd [0,4194304] 0 2024-02-16T19:24:20.337 INFO:tasks.workunit.client.0.smithi040.stdout:0/804: creat db/d2f/d23/d6f/f111 x:0 0 0 2024-02-16T19:24:20.337 INFO:tasks.workunit.client.0.smithi040.stdout:0/805: chown db/d2f/d23/d3b/d10c 5 1 2024-02-16T19:24:20.337 INFO:tasks.workunit.client.1.smithi163.stdout:3/154: dwrite d0/dc/f31 [0,4194304] 0 2024-02-16T19:24:20.337 INFO:tasks.workunit.client.1.smithi163.stdout:5/96: creat d0/d2/f21 x:0 0 0 2024-02-16T19:24:20.337 INFO:tasks.workunit.client.1.smithi163.stdout:6/129: creat d0/d6/d9/d15/d1c/f2b x:0 0 0 2024-02-16T19:24:20.338 INFO:tasks.workunit.client.0.smithi040.stdout:8/779: link d6/d2b/da3/d8b/cb2 d6/d64/d50/cf1 0 2024-02-16T19:24:20.342 INFO:tasks.workunit.client.1.smithi163.stdout:9/305: dwrite d5/de/d30/f5e [0,4194304] 0 2024-02-16T19:24:20.343 INFO:tasks.workunit.client.0.smithi040.stdout:3/798: dwrite d1/ff [0,4194304] 0 2024-02-16T19:24:20.343 INFO:tasks.workunit.client.0.smithi040.stdout:0/806: symlink db/d2f/d1d/d21/d2e/d49/l112 0 2024-02-16T19:24:20.345 INFO:tasks.workunit.client.0.smithi040.stdout:8/780: unlink d6/d2b/f95 0 2024-02-16T19:24:20.347 INFO:tasks.workunit.client.0.smithi040.stdout:3/799: mknod d1/dc/d42/d58/de8/c100 0 2024-02-16T19:24:20.349 INFO:tasks.workunit.client.0.smithi040.stdout:0/807: link db/de/d6a/df0/ff8 db/de/dbe/f113 0 2024-02-16T19:24:20.349 INFO:tasks.workunit.client.0.smithi040.stdout:0/808: chown db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/fc6 17938 1 2024-02-16T19:24:20.351 INFO:tasks.workunit.client.1.smithi163.stdout:9/306: dread d5/f2c [0,4194304] 0 2024-02-16T19:24:20.351 INFO:tasks.workunit.client.0.smithi040.stdout:8/781: mkdir d6/d64/d50/d69/df2 0 2024-02-16T19:24:20.351 INFO:tasks.workunit.client.0.smithi040.stdout:8/782: getdents d6/d12/d51/d6e/db7/dc6/dcc 0 2024-02-16T19:24:20.351 INFO:tasks.workunit.client.0.smithi040.stdout:8/783: fdatasync d6/d64/d50/fb8 0 2024-02-16T19:24:20.353 INFO:tasks.workunit.client.0.smithi040.stdout:3/800: unlink d1/d4/d6e/fc8 0 2024-02-16T19:24:20.355 INFO:tasks.workunit.client.1.smithi163.stdout:8/89: dwrite d0/d7/fd [0,4194304] 0 2024-02-16T19:24:20.355 INFO:tasks.workunit.client.1.smithi163.stdout:8/90: truncate d0/d7/d19/f1b 485133 0 2024-02-16T19:24:20.356 INFO:tasks.workunit.client.0.smithi040.stdout:2/616: dwrite d5/d7/d10/f8f [0,4194304] 0 2024-02-16T19:24:20.356 INFO:tasks.workunit.client.0.smithi040.stdout:2/617: dread - d5/d7/d1e/d4c/d52/dc1/fc6 zero size 2024-02-16T19:24:20.356 INFO:tasks.workunit.client.0.smithi040.stdout:2/618: dread - d5/d8/d42/fc8 zero size 2024-02-16T19:24:20.356 INFO:tasks.workunit.client.0.smithi040.stdout:2/619: write d5/d7/d1e/d55/f6f [2014145,100008] 0 2024-02-16T19:24:20.356 INFO:tasks.workunit.client.0.smithi040.stdout:0/809: mknod db/d2f/d23/d56/dbb/c114 0 2024-02-16T19:24:20.356 INFO:tasks.workunit.client.0.smithi040.stdout:0/810: readlink db/d25/lea 0 2024-02-16T19:24:20.356 INFO:tasks.workunit.client.0.smithi040.stdout:0/811: read - db/d25/fec zero size 2024-02-16T19:24:20.357 INFO:tasks.workunit.client.1.smithi163.stdout:2/240: dwrite d2/f16 [0,4194304] 0 2024-02-16T19:24:20.357 INFO:tasks.workunit.client.1.smithi163.stdout:2/241: write d2/d2a/f46 [39918,88274] 0 2024-02-16T19:24:20.357 INFO:tasks.workunit.client.1.smithi163.stdout:5/97: dwrite d0/d2/f21 [0,4194304] 0 2024-02-16T19:24:20.357 INFO:tasks.workunit.client.1.smithi163.stdout:3/155: rmdir d0 39 2024-02-16T19:24:20.360 INFO:tasks.workunit.client.0.smithi040.stdout:3/801: rename d1/dc/d42/d5c/fad to d1/dc/d42/d5c/f101 0 2024-02-16T19:24:20.364 INFO:tasks.workunit.client.0.smithi040.stdout:3/802: read d1/d7/d36/ff9 [24569,55094] 0 2024-02-16T19:24:20.365 INFO:tasks.workunit.client.1.smithi163.stdout:9/307: link d5/de/d30/f36 d5/de/d30/f6c 0 2024-02-16T19:24:20.370 INFO:tasks.workunit.client.1.smithi163.stdout:8/91: creat d0/f20 x:0 0 0 2024-02-16T19:24:20.371 INFO:tasks.workunit.client.1.smithi163.stdout:0/178: getdents d11/d22/d3e 0 2024-02-16T19:24:20.371 INFO:tasks.workunit.client.1.smithi163.stdout:0/179: read fd [787747,83897] 0 2024-02-16T19:24:20.379 INFO:tasks.workunit.client.0.smithi040.stdout:0/812: dwrite db/d2f/d23/d6f/ff5 [0,4194304] 0 2024-02-16T19:24:20.380 INFO:tasks.workunit.client.1.smithi163.stdout:9/308: dwrite d5/de/d30/d31/f54 [0,4194304] 0 2024-02-16T19:24:20.383 INFO:tasks.workunit.client.0.smithi040.stdout:2/620: unlink d5/d7/d1e/d4c/d52/l72 0 2024-02-16T19:24:20.384 INFO:tasks.workunit.client.0.smithi040.stdout:3/803: creat d1/dc/d42/d5c/df3/f102 x:0 0 0 2024-02-16T19:24:20.387 INFO:tasks.workunit.client.1.smithi163.stdout:2/242: symlink d2/d2a/l50 0 2024-02-16T19:24:20.387 INFO:tasks.workunit.client.1.smithi163.stdout:2/243: creat d2/d2a/d39/f51 x:0 0 0 2024-02-16T19:24:20.388 INFO:tasks.workunit.client.1.smithi163.stdout:8/92: dwrite d0/d7/d19/f1b [0,4194304] 0 2024-02-16T19:24:20.392 INFO:tasks.workunit.client.1.smithi163.stdout:5/98: mkdir d0/d22 0 2024-02-16T19:24:20.394 INFO:tasks.workunit.client.0.smithi040.stdout:0/813: dread db/d2f/d1d/d21/d2e/d49/d5b/fb3 [0,4194304] 0 2024-02-16T19:24:20.394 INFO:tasks.workunit.client.0.smithi040.stdout:0/814: fsync db/fd 0 2024-02-16T19:24:20.394 INFO:tasks.workunit.client.1.smithi163.stdout:3/156: mknod d0/c3d 0 2024-02-16T19:24:20.395 INFO:tasks.workunit.client.1.smithi163.stdout:3/157: chown d0/dc/d1c/f23 390269 1 2024-02-16T19:24:20.398 INFO:tasks.workunit.client.1.smithi163.stdout:2/244: write d2/d26/f31 [4019892,112946] 0 2024-02-16T19:24:20.400 INFO:tasks.workunit.client.1.smithi163.stdout:8/93: dread d0/d7/d19/f1b [0,4194304] 0 2024-02-16T19:24:20.405 INFO:tasks.workunit.client.1.smithi163.stdout:9/309: mkdir d5/de/d30/d3d/d49/d6d 0 2024-02-16T19:24:20.405 INFO:tasks.workunit.client.1.smithi163.stdout:9/310: chown d5/de/d16/l20 6288021 1 2024-02-16T19:24:20.410 INFO:tasks.workunit.client.1.smithi163.stdout:3/158: dwrite d0/dc/fd [0,4194304] 0 2024-02-16T19:24:20.412 INFO:tasks.workunit.client.1.smithi163.stdout:5/99: truncate d0/d2/fb 1711352 0 2024-02-16T19:24:20.414 INFO:tasks.workunit.client.1.smithi163.stdout:2/245: link d2/d2a/c4d d2/d2a/d2d/c52 0 2024-02-16T19:24:20.417 INFO:tasks.workunit.client.1.smithi163.stdout:9/311: mkdir d5/de/d30/d3d/d49/d6e 0 2024-02-16T19:24:20.418 INFO:tasks.workunit.client.0.smithi040.stdout:8/784: rmdir d6 39 2024-02-16T19:24:20.418 INFO:tasks.workunit.client.0.smithi040.stdout:8/785: readlink d6/d12/d51/d6e/db7/dc6/ld3 0 2024-02-16T19:24:20.419 INFO:tasks.workunit.client.0.smithi040.stdout:8/786: write d6/d7/d15/f67 [829745,115582] 0 2024-02-16T19:24:20.419 INFO:tasks.workunit.client.0.smithi040.stdout:8/787: chown d6/fa5 24 1 2024-02-16T19:24:20.421 INFO:tasks.workunit.client.1.smithi163.stdout:5/100: dread d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:20.422 INFO:tasks.workunit.client.1.smithi163.stdout:5/101: fdatasync d0/d2/f4 0 2024-02-16T19:24:20.425 INFO:tasks.workunit.client.1.smithi163.stdout:9/312: write d5/f2b [683784,38984] 0 2024-02-16T19:24:20.426 INFO:tasks.workunit.client.1.smithi163.stdout:9/313: write d5/f40 [8852660,94752] 0 2024-02-16T19:24:20.426 INFO:tasks.workunit.client.1.smithi163.stdout:3/159: mkdir d0/d3e 0 2024-02-16T19:24:20.427 INFO:tasks.workunit.client.0.smithi040.stdout:2/621: mkdir d5/d7/d12/dca 0 2024-02-16T19:24:20.429 INFO:tasks.workunit.client.0.smithi040.stdout:3/804: rmdir d1/d8c/dda 0 2024-02-16T19:24:20.432 INFO:tasks.workunit.client.0.smithi040.stdout:0/815: mkdir db/d2f/d23/d3b/dd5/d115 0 2024-02-16T19:24:20.433 INFO:tasks.workunit.client.0.smithi040.stdout:0/816: fsync db/d2f/d23/d3b/fc7 0 2024-02-16T19:24:20.433 INFO:tasks.workunit.client.0.smithi040.stdout:0/817: chown db/d2f/d23/d6f/fc4 32386 1 2024-02-16T19:24:20.433 INFO:tasks.workunit.client.1.smithi163.stdout:8/94: getdents d0 0 2024-02-16T19:24:20.436 INFO:tasks.workunit.client.1.smithi163.stdout:5/102: creat d0/d22/f23 x:0 0 0 2024-02-16T19:24:20.439 INFO:tasks.workunit.client.0.smithi040.stdout:2/622: rename d5/d7/d1e/d31/c39 to d5/d7/d1e/d31/d4f/d63/ccb 0 2024-02-16T19:24:20.441 INFO:tasks.workunit.client.1.smithi163.stdout:9/314: creat d5/de/d30/d3d/f6f x:0 0 0 2024-02-16T19:24:20.441 INFO:tasks.workunit.client.1.smithi163.stdout:9/315: chown d5/de/d16/d47 941068870 1 2024-02-16T19:24:20.442 INFO:tasks.workunit.client.1.smithi163.stdout:3/160: creat d0/dc/d1c/d32/f3f x:0 0 0 2024-02-16T19:24:20.442 INFO:tasks.workunit.client.1.smithi163.stdout:7/104: sync 2024-02-16T19:24:20.442 INFO:tasks.workunit.client.1.smithi163.stdout:1/77: sync 2024-02-16T19:24:20.442 INFO:tasks.workunit.client.1.smithi163.stdout:1/78: read - d7/fb zero size 2024-02-16T19:24:20.444 INFO:tasks.workunit.client.0.smithi040.stdout:2/623: rename d5/d7/d1e/d55/d8d/l88 to d5/d7/d89/lcc 0 2024-02-16T19:24:20.444 INFO:tasks.workunit.client.0.smithi040.stdout:2/624: write d5/d7/d10/d40/f9a [741767,126652] 0 2024-02-16T19:24:20.446 INFO:tasks.workunit.client.1.smithi163.stdout:8/95: unlink d0/f6 0 2024-02-16T19:24:20.447 INFO:tasks.workunit.client.1.smithi163.stdout:5/103: getdents d0/d1c 0 2024-02-16T19:24:20.452 INFO:tasks.workunit.client.1.smithi163.stdout:9/316: symlink d5/de/d30/d3d/d49/d6d/l70 0 2024-02-16T19:24:20.452 INFO:tasks.workunit.client.1.smithi163.stdout:9/317: fdatasync d5/de/d30/d43/f4b 0 2024-02-16T19:24:20.455 INFO:tasks.workunit.client.1.smithi163.stdout:9/318: mknod d5/de/d30/d31/c71 0 2024-02-16T19:24:20.461 INFO:tasks.workunit.client.0.smithi040.stdout:9/725: sync 2024-02-16T19:24:20.461 INFO:tasks.workunit.client.0.smithi040.stdout:9/726: readlink d3/d4/d55/d8a/lc1 0 2024-02-16T19:24:20.461 INFO:tasks.workunit.client.0.smithi040.stdout:9/727: chown d3/d4/d11/ld0 30593 1 2024-02-16T19:24:20.462 INFO:tasks.workunit.client.1.smithi163.stdout:5/104: write d0/d2/f4 [81324,122459] 0 2024-02-16T19:24:20.463 INFO:tasks.workunit.client.0.smithi040.stdout:9/728: mkdir d3/d4/dbe/de9 0 2024-02-16T19:24:20.464 INFO:tasks.workunit.client.1.smithi163.stdout:2/246: dwrite d2/f23 [0,4194304] 0 2024-02-16T19:24:20.464 INFO:tasks.workunit.client.1.smithi163.stdout:5/105: rename d0/d2/c11 to d0/c24 0 2024-02-16T19:24:20.466 INFO:tasks.workunit.client.0.smithi040.stdout:9/729: rename d3/d4/l1b to d3/d4/d11/d71/dc9/lea 0 2024-02-16T19:24:20.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/106: symlink d0/l25 0 2024-02-16T19:24:20.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/107: fdatasync d0/d2/fb 0 2024-02-16T19:24:20.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/108: fsync d0/f1d 0 2024-02-16T19:24:20.475 INFO:tasks.workunit.client.1.smithi163.stdout:2/247: dread d2/f5 [0,4194304] 0 2024-02-16T19:24:20.476 INFO:tasks.workunit.client.1.smithi163.stdout:2/248: dread d2/d26/f3f [0,4194304] 0 2024-02-16T19:24:20.478 INFO:tasks.workunit.client.1.smithi163.stdout:2/249: mknod d2/d44/c53 0 2024-02-16T19:24:20.478 INFO:tasks.workunit.client.1.smithi163.stdout:2/250: chown d2/c34 1 1 2024-02-16T19:24:20.479 INFO:tasks.workunit.client.1.smithi163.stdout:2/251: dread d2/d26/f3f [0,4194304] 0 2024-02-16T19:24:20.479 INFO:tasks.workunit.client.1.smithi163.stdout:2/252: write d2/f1c [3953808,59825] 0 2024-02-16T19:24:20.479 INFO:tasks.workunit.client.0.smithi040.stdout:8/788: dwrite d6/d64/d50/f84 [4194304,4194304] 0 2024-02-16T19:24:20.483 INFO:tasks.workunit.client.0.smithi040.stdout:3/805: dwrite d1/d7/d36/f45 [4194304,4194304] 0 2024-02-16T19:24:20.483 INFO:tasks.workunit.client.0.smithi040.stdout:3/806: dread - d1/dc/ddf/fe2 zero size 2024-02-16T19:24:20.484 INFO:tasks.workunit.client.0.smithi040.stdout:3/807: symlink d1/dc/d35/l103 0 2024-02-16T19:24:20.484 INFO:tasks.workunit.client.0.smithi040.stdout:3/808: creat d1/dc/d35/d53/f104 x:0 0 0 2024-02-16T19:24:20.484 INFO:tasks.workunit.client.0.smithi040.stdout:3/809: creat d1/d7/f105 x:0 0 0 2024-02-16T19:24:20.484 INFO:tasks.workunit.client.0.smithi040.stdout:3/810: fsync d1/d7/f2b 0 2024-02-16T19:24:20.487 INFO:tasks.workunit.client.0.smithi040.stdout:3/811: rename d1/d7/d30/f63 to d1/dc/d42/d58/f106 0 2024-02-16T19:24:20.487 INFO:tasks.workunit.client.0.smithi040.stdout:0/818: dwrite db/d2f/d1d/d21/d2e/d83/ff2 [0,4194304] 0 2024-02-16T19:24:20.487 INFO:tasks.workunit.client.0.smithi040.stdout:2/625: dwrite d5/d7/d1e/d31/d4f/d63/f2c [4194304,4194304] 0 2024-02-16T19:24:20.489 INFO:tasks.workunit.client.0.smithi040.stdout:0/819: getdents db/d25/ddc 0 2024-02-16T19:24:20.490 INFO:tasks.workunit.client.1.smithi163.stdout:1/79: dwrite f0 [0,4194304] 0 2024-02-16T19:24:20.491 INFO:tasks.workunit.client.0.smithi040.stdout:2/626: mkdir d5/d7/d1e/d4c/d52/daa/dcd 0 2024-02-16T19:24:20.492 INFO:tasks.workunit.client.0.smithi040.stdout:8/789: write d6/d2b/d44/f49 [1219859,100086] 0 2024-02-16T19:24:20.492 INFO:tasks.workunit.client.1.smithi163.stdout:3/161: dwrite d0/dc/d1c/f3b [4194304,4194304] 0 2024-02-16T19:24:20.493 INFO:tasks.workunit.client.0.smithi040.stdout:0/820: mknod db/de/d6a/df0/c116 0 2024-02-16T19:24:20.494 INFO:tasks.workunit.client.1.smithi163.stdout:1/80: unlink f0 0 2024-02-16T19:24:20.496 INFO:tasks.workunit.client.1.smithi163.stdout:3/162: creat d0/dc/d1d/d29/d38/f40 x:0 0 0 2024-02-16T19:24:20.496 INFO:tasks.workunit.client.1.smithi163.stdout:3/163: dread - d0/f39 zero size 2024-02-16T19:24:20.496 INFO:tasks.workunit.client.1.smithi163.stdout:3/164: write d0/dc/d1d/d29/d38/f40 [977495,100697] 0 2024-02-16T19:24:20.496 INFO:tasks.workunit.client.1.smithi163.stdout:3/165: dread - d0/dc/d1c/f23 zero size 2024-02-16T19:24:20.496 INFO:tasks.workunit.client.1.smithi163.stdout:3/166: chown d0/d3e 0 1 2024-02-16T19:24:20.497 INFO:tasks.workunit.client.0.smithi040.stdout:2/627: rename d5/d8/f38 to d5/d7/d1e/d4c/fce 0 2024-02-16T19:24:20.497 INFO:tasks.workunit.client.0.smithi040.stdout:2/628: chown d5/d5c/d65/c77 4 1 2024-02-16T19:24:20.497 INFO:tasks.workunit.client.1.smithi163.stdout:1/81: truncate d7/fa 614343 0 2024-02-16T19:24:20.498 INFO:tasks.workunit.client.1.smithi163.stdout:1/82: creat d7/dc/f17 x:0 0 0 2024-02-16T19:24:20.498 INFO:tasks.workunit.client.1.smithi163.stdout:1/83: truncate d7/dc/f17 486367 0 2024-02-16T19:24:20.499 INFO:tasks.workunit.client.1.smithi163.stdout:3/167: mknod d0/dc/d1c/d32/c41 0 2024-02-16T19:24:20.501 INFO:tasks.workunit.client.1.smithi163.stdout:8/96: dwrite d0/d7/fd [0,4194304] 0 2024-02-16T19:24:20.504 INFO:tasks.workunit.client.1.smithi163.stdout:3/168: rename d0/dc/fd to d0/dc/d1c/d21/f42 0 2024-02-16T19:24:20.507 INFO:tasks.workunit.client.0.smithi040.stdout:9/730: dwrite d3/d4/d11/fc5 [0,4194304] 0 2024-02-16T19:24:20.508 INFO:tasks.workunit.client.0.smithi040.stdout:9/731: chown d3/d4/d11/d29/l28 1 1 2024-02-16T19:24:20.508 INFO:tasks.workunit.client.0.smithi040.stdout:0/821: read db/d2f/d1d/d21/d2e/d83/ff2 [1919216,84928] 0 2024-02-16T19:24:20.508 INFO:tasks.workunit.client.0.smithi040.stdout:8/790: dread d6/f8 [0,4194304] 0 2024-02-16T19:24:20.511 INFO:tasks.workunit.client.1.smithi163.stdout:9/319: dwrite d5/de/f5b [0,4194304] 0 2024-02-16T19:24:20.511 INFO:tasks.workunit.client.1.smithi163.stdout:8/97: mknod d0/c21 0 2024-02-16T19:24:20.513 INFO:tasks.workunit.client.1.smithi163.stdout:3/169: creat d0/f43 x:0 0 0 2024-02-16T19:24:20.515 INFO:tasks.workunit.client.1.smithi163.stdout:9/320: mkdir d5/de/d72 0 2024-02-16T19:24:20.518 INFO:tasks.workunit.client.0.smithi040.stdout:9/732: rename d3/d4/d55/d70/ca0 to d3/de5/ceb 0 2024-02-16T19:24:20.520 INFO:tasks.workunit.client.1.smithi163.stdout:3/170: symlink d0/dc/d1d/d29/l44 0 2024-02-16T19:24:20.521 INFO:tasks.workunit.client.0.smithi040.stdout:2/629: truncate d5/d7/d1e/f70 785664 0 2024-02-16T19:24:20.521 INFO:tasks.workunit.client.0.smithi040.stdout:2/630: write d5/d7/f74 [726104,32515] 0 2024-02-16T19:24:20.524 INFO:tasks.workunit.client.1.smithi163.stdout:9/321: creat d5/de/d30/d3d/d49/d6e/f73 x:0 0 0 2024-02-16T19:24:20.525 INFO:tasks.workunit.client.1.smithi163.stdout:5/109: dwrite d0/fd [0,4194304] 0 2024-02-16T19:24:20.527 INFO:tasks.workunit.client.0.smithi040.stdout:2/631: creat d5/d7/d89/fcf x:0 0 0 2024-02-16T19:24:20.527 INFO:tasks.workunit.client.0.smithi040.stdout:2/632: stat d5/d8 0 2024-02-16T19:24:20.528 INFO:tasks.workunit.client.1.smithi163.stdout:9/322: dread d5/f18 [0,4194304] 0 2024-02-16T19:24:20.528 INFO:tasks.workunit.client.1.smithi163.stdout:9/323: write d5/de/d30/d3d/d49/d6e/f73 [363026,115781] 0 2024-02-16T19:24:20.533 INFO:tasks.workunit.client.1.smithi163.stdout:2/253: dwrite d2/d2a/f46 [0,4194304] 0 2024-02-16T19:24:20.533 INFO:tasks.workunit.client.1.smithi163.stdout:2/254: write d2/f15 [4922308,42436] 0 2024-02-16T19:24:20.533 INFO:tasks.workunit.client.0.smithi040.stdout:9/733: write d3/d4/d11/d71/f88 [2690539,106389] 0 2024-02-16T19:24:20.533 INFO:tasks.workunit.client.0.smithi040.stdout:9/734: chown d3/d4/d55/l27 2 1 2024-02-16T19:24:20.534 INFO:tasks.workunit.client.0.smithi040.stdout:9/735: chown d3/c91 1 1 2024-02-16T19:24:20.534 INFO:tasks.workunit.client.0.smithi040.stdout:9/736: write d3/d4/d55/d8a/fcf [1012393,114269] 0 2024-02-16T19:24:20.534 INFO:tasks.workunit.client.1.smithi163.stdout:5/110: symlink d0/d1c/l26 0 2024-02-16T19:24:20.534 INFO:tasks.workunit.client.1.smithi163.stdout:5/111: chown d0/d2/f21 0 1 2024-02-16T19:24:20.534 INFO:tasks.workunit.client.1.smithi163.stdout:5/112: fsync d0/fd 0 2024-02-16T19:24:20.536 INFO:tasks.workunit.client.0.smithi040.stdout:9/737: truncate d3/d4/d11/f60 746929 0 2024-02-16T19:24:20.536 INFO:tasks.workunit.client.0.smithi040.stdout:9/738: readlink d3/d4/d55/d8a/d99/la9 0 2024-02-16T19:24:20.541 INFO:tasks.workunit.client.0.smithi040.stdout:9/739: mknod d3/d1c/d73/cec 0 2024-02-16T19:24:20.542 INFO:tasks.workunit.client.0.smithi040.stdout:9/740: fsync d3/d4/d11/d71/da8/f84 0 2024-02-16T19:24:20.544 INFO:tasks.workunit.client.1.smithi163.stdout:6/130: sync 2024-02-16T19:24:20.544 INFO:tasks.workunit.client.1.smithi163.stdout:4/91: sync 2024-02-16T19:24:20.544 INFO:tasks.workunit.client.1.smithi163.stdout:6/131: readlink d0/l14 0 2024-02-16T19:24:20.549 INFO:tasks.workunit.client.1.smithi163.stdout:4/92: truncate f2 2087514 0 2024-02-16T19:24:20.554 INFO:tasks.workunit.client.0.smithi040.stdout:3/812: dwrite d1/dc/ddf/fe2 [0,4194304] 0 2024-02-16T19:24:20.555 INFO:tasks.workunit.client.1.smithi163.stdout:1/84: dwrite f6 [0,4194304] 0 2024-02-16T19:24:20.564 INFO:tasks.workunit.client.0.smithi040.stdout:0/822: dwrite db/d25/fdf [0,4194304] 0 2024-02-16T19:24:20.565 INFO:tasks.workunit.client.1.smithi163.stdout:8/98: dwrite d0/f20 [0,4194304] 0 2024-02-16T19:24:20.565 INFO:tasks.workunit.client.0.smithi040.stdout:2/633: fdatasync d5/d7/f74 0 2024-02-16T19:24:20.565 INFO:tasks.workunit.client.0.smithi040.stdout:2/634: dread - d5/d5c/d65/d9d/fc0 zero size 2024-02-16T19:24:20.565 INFO:tasks.workunit.client.0.smithi040.stdout:2/635: dread - d5/d7/f68 zero size 2024-02-16T19:24:20.568 INFO:tasks.workunit.client.1.smithi163.stdout:3/171: dwrite d0/dc/d1c/f2c [0,4194304] 0 2024-02-16T19:24:20.569 INFO:tasks.workunit.client.1.smithi163.stdout:1/85: mknod d7/c18 0 2024-02-16T19:24:20.569 INFO:tasks.workunit.client.0.smithi040.stdout:8/791: dwrite d6/d7/f9b [0,4194304] 0 2024-02-16T19:24:20.569 INFO:tasks.workunit.client.0.smithi040.stdout:8/792: write d6/f24 [1562191,54727] 0 2024-02-16T19:24:20.576 INFO:tasks.workunit.client.0.smithi040.stdout:0/823: mknod db/d25/d34/d5e/c117 0 2024-02-16T19:24:20.578 INFO:tasks.workunit.client.1.smithi163.stdout:8/99: creat d0/dc/f22 x:0 0 0 2024-02-16T19:24:20.580 INFO:tasks.workunit.client.1.smithi163.stdout:2/255: dwrite d2/f5 [0,4194304] 0 2024-02-16T19:24:20.580 INFO:tasks.workunit.client.1.smithi163.stdout:5/113: dwrite d0/fd [4194304,4194304] 0 2024-02-16T19:24:20.582 INFO:tasks.workunit.client.1.smithi163.stdout:1/86: symlink d7/l19 0 2024-02-16T19:24:20.585 INFO:tasks.workunit.client.1.smithi163.stdout:8/100: mknod d0/d7/d17/c23 0 2024-02-16T19:24:20.585 INFO:tasks.workunit.client.1.smithi163.stdout:8/101: write d0/d7/f15 [962751,116586] 0 2024-02-16T19:24:20.588 INFO:tasks.workunit.client.1.smithi163.stdout:5/114: mknod d0/d2/c27 0 2024-02-16T19:24:20.589 INFO:tasks.workunit.client.1.smithi163.stdout:1/87: mknod d7/dc/c1a 0 2024-02-16T19:24:20.589 INFO:tasks.workunit.client.1.smithi163.stdout:1/88: fdatasync d7/fa 0 2024-02-16T19:24:20.589 INFO:tasks.workunit.client.1.smithi163.stdout:1/89: stat d7/dc 0 2024-02-16T19:24:20.590 INFO:tasks.workunit.client.1.smithi163.stdout:0/180: sync 2024-02-16T19:24:20.590 INFO:tasks.workunit.client.1.smithi163.stdout:5/115: mkdir d0/d22/d28 0 2024-02-16T19:24:20.592 INFO:tasks.workunit.client.0.smithi040.stdout:8/793: creat d6/d64/d50/dd5/ff3 x:0 0 0 2024-02-16T19:24:20.592 INFO:tasks.workunit.client.1.smithi163.stdout:8/102: write d0/dc/f16 [2339061,43089] 0 2024-02-16T19:24:20.592 INFO:tasks.workunit.client.1.smithi163.stdout:8/103: chown d0/f20 0 1 2024-02-16T19:24:20.592 INFO:tasks.workunit.client.1.smithi163.stdout:8/104: dread d0/fe [0,4194304] 0 2024-02-16T19:24:20.593 INFO:tasks.workunit.client.1.smithi163.stdout:6/132: dwrite d0/d6/d9/d15/d1c/f2b [0,4194304] 0 2024-02-16T19:24:20.594 INFO:tasks.workunit.client.0.smithi040.stdout:0/824: rename db/f7f to db/d25/f118 0 2024-02-16T19:24:20.596 INFO:tasks.workunit.client.1.smithi163.stdout:0/181: symlink d11/d22/d3e/d3c/l41 0 2024-02-16T19:24:20.599 INFO:tasks.workunit.client.0.smithi040.stdout:2/636: dwrite d5/d7/d10/f86 [0,4194304] 0 2024-02-16T19:24:20.599 INFO:tasks.workunit.client.1.smithi163.stdout:8/105: mkdir d0/d24 0 2024-02-16T19:24:20.599 INFO:tasks.workunit.client.1.smithi163.stdout:6/133: dread d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:20.601 INFO:tasks.workunit.client.0.smithi040.stdout:0/825: link db/d2f/d23/f33 db/de/f119 0 2024-02-16T19:24:20.604 INFO:tasks.workunit.client.0.smithi040.stdout:8/794: write d6/d7/d15/f61 [1915753,59545] 0 2024-02-16T19:24:20.604 INFO:tasks.workunit.client.0.smithi040.stdout:8/795: chown d6/d12/d22/f86 1388354 1 2024-02-16T19:24:20.604 INFO:tasks.workunit.client.0.smithi040.stdout:3/813: dwrite d1/dc/ddf/fe2 [0,4194304] 0 2024-02-16T19:24:20.607 INFO:tasks.workunit.client.1.smithi163.stdout:4/93: dwrite d11/f16 [0,4194304] 0 2024-02-16T19:24:20.607 INFO:tasks.workunit.client.1.smithi163.stdout:3/172: dwrite d0/f14 [0,4194304] 0 2024-02-16T19:24:20.609 INFO:tasks.workunit.client.1.smithi163.stdout:0/182: truncate d11/f35 2333886 0 2024-02-16T19:24:20.610 INFO:tasks.workunit.client.1.smithi163.stdout:0/183: write d11/d12/d20/f3a [871397,73237] 0 2024-02-16T19:24:20.610 INFO:tasks.workunit.client.1.smithi163.stdout:0/184: read d11/d22/f2d [66662,127291] 0 2024-02-16T19:24:20.610 INFO:tasks.workunit.client.0.smithi040.stdout:2/637: mknod d5/cd0 0 2024-02-16T19:24:20.612 INFO:tasks.workunit.client.0.smithi040.stdout:8/796: dread d6/d12/f5a [0,4194304] 0 2024-02-16T19:24:20.612 INFO:tasks.workunit.client.0.smithi040.stdout:2/638: dread d5/f3a [0,4194304] 0 2024-02-16T19:24:20.618 INFO:tasks.workunit.client.0.smithi040.stdout:0/826: dread db/d2f/d23/f40 [0,4194304] 0 2024-02-16T19:24:20.624 INFO:tasks.workunit.client.1.smithi163.stdout:3/173: write d0/dc/f31 [4019583,7620] 0 2024-02-16T19:24:20.624 INFO:tasks.workunit.client.0.smithi040.stdout:3/814: link d1/d4/d6e/fb5 d1/dc/d42/d4a/d57/f107 0 2024-02-16T19:24:20.626 INFO:tasks.workunit.client.0.smithi040.stdout:8/797: mknod d6/d2b/da3/dcb/cf4 0 2024-02-16T19:24:20.628 INFO:tasks.workunit.client.0.smithi040.stdout:2/639: rename d5/d5c/d65/d9d/fc0 to d5/d7/d12/dca/fd1 0 2024-02-16T19:24:20.630 INFO:tasks.workunit.client.0.smithi040.stdout:7/643: sync 2024-02-16T19:24:20.632 INFO:tasks.workunit.client.0.smithi040.stdout:0/827: mkdir db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/d11a 0 2024-02-16T19:24:20.632 INFO:tasks.workunit.client.0.smithi040.stdout:0/828: creat db/d25/f11b x:0 0 0 2024-02-16T19:24:20.632 INFO:tasks.workunit.client.0.smithi040.stdout:0/829: chown db/d2f/d1d/d21/d2e/d85/cbd 6845235 1 2024-02-16T19:24:20.634 INFO:tasks.workunit.client.0.smithi040.stdout:3/815: symlink d1/dc/d42/d4a/d57/d79/de3/l108 0 2024-02-16T19:24:20.637 INFO:tasks.workunit.client.1.smithi163.stdout:8/106: dwrite d0/dc/f22 [0,4194304] 0 2024-02-16T19:24:20.638 INFO:tasks.workunit.client.0.smithi040.stdout:8/798: rename d6/d2b/d36 to d6/d12/d51/d6e/db7/dc6/df5 0 2024-02-16T19:24:20.639 INFO:tasks.workunit.client.1.smithi163.stdout:4/94: mkdir d11/d18 0 2024-02-16T19:24:20.640 INFO:tasks.workunit.client.0.smithi040.stdout:3/816: dread d1/dc/d35/d55/f86 [0,4194304] 0 2024-02-16T19:24:20.640 INFO:tasks.workunit.client.0.smithi040.stdout:2/640: link d5/d7/d10/d40/f9a d5/d7/d1e/d4c/d52/daa/fd2 0 2024-02-16T19:24:20.642 INFO:tasks.workunit.client.1.smithi163.stdout:0/185: dwrite d11/d12/d20/f39 [0,4194304] 0 2024-02-16T19:24:20.642 INFO:tasks.workunit.client.1.smithi163.stdout:0/186: creat d11/d22/f42 x:0 0 0 2024-02-16T19:24:20.642 INFO:tasks.workunit.client.1.smithi163.stdout:1/90: dwrite d7/f14 [0,4194304] 0 2024-02-16T19:24:20.642 INFO:tasks.workunit.client.1.smithi163.stdout:1/91: readlink d7/l16 0 2024-02-16T19:24:20.643 INFO:tasks.workunit.client.1.smithi163.stdout:1/92: fsync d7/f9 0 2024-02-16T19:24:20.643 INFO:tasks.workunit.client.1.smithi163.stdout:1/93: write d7/fa [777309,62605] 0 2024-02-16T19:24:20.643 INFO:tasks.workunit.client.1.smithi163.stdout:1/94: read - d7/f15 zero size 2024-02-16T19:24:20.643 INFO:tasks.workunit.client.1.smithi163.stdout:5/116: dwrite d0/d2/fb [0,4194304] 0 2024-02-16T19:24:20.643 INFO:tasks.workunit.client.1.smithi163.stdout:8/107: fsync d0/dc/f16 0 2024-02-16T19:24:20.643 INFO:tasks.workunit.client.1.smithi163.stdout:8/108: chown d0/d7/d18/c1a 1559832267 1 2024-02-16T19:24:20.643 INFO:tasks.workunit.client.1.smithi163.stdout:6/134: dwrite d0/d6/f1a [0,4194304] 0 2024-02-16T19:24:20.643 INFO:tasks.workunit.client.1.smithi163.stdout:3/174: dwrite d0/dc/f3c [0,4194304] 0 2024-02-16T19:24:20.652 INFO:tasks.workunit.client.1.smithi163.stdout:6/135: dread d0/f5 [0,4194304] 0 2024-02-16T19:24:20.652 INFO:tasks.workunit.client.1.smithi163.stdout:6/136: chown d0/f2 37619 1 2024-02-16T19:24:20.653 INFO:tasks.workunit.client.0.smithi040.stdout:2/641: write d5/d8/d42/f50 [3241404,14220] 0 2024-02-16T19:24:20.653 INFO:tasks.workunit.client.0.smithi040.stdout:0/830: symlink db/d2f/d23/d56/dbb/l11c 0 2024-02-16T19:24:20.654 INFO:tasks.workunit.client.0.smithi040.stdout:2/642: write d5/d8/f69 [166376,27337] 0 2024-02-16T19:24:20.654 INFO:tasks.workunit.client.0.smithi040.stdout:2/643: fsync d5/d7/d12/fb1 0 2024-02-16T19:24:20.654 INFO:tasks.workunit.client.0.smithi040.stdout:2/644: fsync d5/d7/d1e/d31/f43 0 2024-02-16T19:24:20.654 INFO:tasks.workunit.client.0.smithi040.stdout:2/645: fsync d5/d7/d1e/d4c/fce 0 2024-02-16T19:24:20.654 INFO:tasks.workunit.client.0.smithi040.stdout:2/646: fdatasync d5/d7/d1e/d4c/d52/f6b 0 2024-02-16T19:24:20.655 INFO:tasks.workunit.client.1.smithi163.stdout:0/187: rename d11/d12/d20/c2a to d11/d22/d3e/c43 0 2024-02-16T19:24:20.655 INFO:tasks.workunit.client.1.smithi163.stdout:0/188: chown f10 33 1 2024-02-16T19:24:20.655 INFO:tasks.workunit.client.1.smithi163.stdout:0/189: stat d11/d22/f2d 0 2024-02-16T19:24:20.657 INFO:tasks.workunit.client.0.smithi040.stdout:7/644: rename d21/f46 to d21/d4d/d7b/fd6 0 2024-02-16T19:24:20.658 INFO:tasks.workunit.client.1.smithi163.stdout:5/117: dwrite d0/d2/f16 [4194304,4194304] 0 2024-02-16T19:24:20.658 INFO:tasks.workunit.client.1.smithi163.stdout:5/118: dread - d0/d2/f7 zero size 2024-02-16T19:24:20.658 INFO:tasks.workunit.client.1.smithi163.stdout:5/119: fsync d0/d2/f21 0 2024-02-16T19:24:20.660 INFO:tasks.workunit.client.0.smithi040.stdout:3/817: link d1/d7/d30/fa8 d1/dc/d35/f109 0 2024-02-16T19:24:20.662 INFO:tasks.workunit.client.1.smithi163.stdout:1/95: link d7/dc/l13 d7/l1b 0 2024-02-16T19:24:20.666 INFO:tasks.workunit.client.1.smithi163.stdout:7/105: sync 2024-02-16T19:24:20.668 INFO:tasks.workunit.client.0.smithi040.stdout:0/831: symlink db/d2f/d23/d3b/dd5/d115/l11d 0 2024-02-16T19:24:20.670 INFO:tasks.workunit.client.1.smithi163.stdout:3/175: creat d0/dc/d1d/d29/f45 x:0 0 0 2024-02-16T19:24:20.670 INFO:tasks.workunit.client.1.smithi163.stdout:3/176: readlink d0/l37 0 2024-02-16T19:24:20.670 INFO:tasks.workunit.client.1.smithi163.stdout:4/95: write d11/f16 [5171503,15658] 0 2024-02-16T19:24:20.670 INFO:tasks.workunit.client.1.smithi163.stdout:4/96: chown cb 372 1 2024-02-16T19:24:20.673 INFO:tasks.workunit.client.0.smithi040.stdout:2/647: dwrite d5/f3a [0,4194304] 0 2024-02-16T19:24:20.675 INFO:tasks.workunit.client.0.smithi040.stdout:8/799: rename d6/d64/d50/dd5/le1 to d6/d64/d50/d69/df2/lf6 0 2024-02-16T19:24:20.675 INFO:tasks.workunit.client.0.smithi040.stdout:8/800: fdatasync d6/d2b/f9a 0 2024-02-16T19:24:20.675 INFO:tasks.workunit.client.0.smithi040.stdout:7/645: write d21/d4d/d7b/f8f [1968311,55227] 0 2024-02-16T19:24:20.677 INFO:tasks.workunit.client.0.smithi040.stdout:0/832: symlink db/d2f/d1d/d21/d2e/d83/l11e 0 2024-02-16T19:24:20.678 INFO:tasks.workunit.client.1.smithi163.stdout:6/137: rename d0/d6/d11/d1e/f24 to d0/d6/d11/d1e/f2c 0 2024-02-16T19:24:20.681 INFO:tasks.workunit.client.1.smithi163.stdout:0/190: link d11/d12/d20/l31 d11/d22/l44 0 2024-02-16T19:24:20.682 INFO:tasks.workunit.client.1.smithi163.stdout:0/191: readlink d11/d21/l29 0 2024-02-16T19:24:20.682 INFO:tasks.workunit.client.0.smithi040.stdout:3/818: dwrite d1/dc/d42/d4a/f50 [0,4194304] 0 2024-02-16T19:24:20.684 INFO:tasks.workunit.client.0.smithi040.stdout:4/590: sync 2024-02-16T19:24:20.684 INFO:tasks.workunit.client.0.smithi040.stdout:6/827: sync 2024-02-16T19:24:20.684 INFO:tasks.workunit.client.0.smithi040.stdout:5/673: sync 2024-02-16T19:24:20.684 INFO:tasks.workunit.client.0.smithi040.stdout:4/591: dread - dd/f6e zero size 2024-02-16T19:24:20.685 INFO:tasks.workunit.client.0.smithi040.stdout:5/674: getdents d1/da/d1c/da9 0 2024-02-16T19:24:20.685 INFO:tasks.workunit.client.0.smithi040.stdout:1/704: sync 2024-02-16T19:24:20.685 INFO:tasks.workunit.client.0.smithi040.stdout:4/592: fsync dd/de/d10/f30 0 2024-02-16T19:24:20.685 INFO:tasks.workunit.client.0.smithi040.stdout:5/675: fdatasync d1/da/d1c/f91 0 2024-02-16T19:24:20.685 INFO:tasks.workunit.client.0.smithi040.stdout:4/593: creat dd/d58/fbb x:0 0 0 2024-02-16T19:24:20.685 INFO:tasks.workunit.client.0.smithi040.stdout:4/594: chown dd/de/c16 12642 1 2024-02-16T19:24:20.685 INFO:tasks.workunit.client.1.smithi163.stdout:5/120: truncate d0/d2/fa 3579607 0 2024-02-16T19:24:20.685 INFO:tasks.workunit.client.1.smithi163.stdout:5/121: write d0/d22/f23 [20701,108126] 0 2024-02-16T19:24:20.688 INFO:tasks.workunit.client.0.smithi040.stdout:2/648: rename d5/d7/d12/fb1 to d5/d7/d1e/d31/d4f/fd3 0 2024-02-16T19:24:20.690 INFO:tasks.workunit.client.1.smithi163.stdout:7/106: getdents d11 0 2024-02-16T19:24:20.690 INFO:tasks.workunit.client.1.smithi163.stdout:7/107: readlink d11/l16 0 2024-02-16T19:24:20.692 INFO:tasks.workunit.client.0.smithi040.stdout:8/801: mknod d6/d64/d50/d69/df2/cf7 0 2024-02-16T19:24:20.692 INFO:tasks.workunit.client.0.smithi040.stdout:8/802: fdatasync d6/d2b/f34 0 2024-02-16T19:24:20.696 INFO:tasks.workunit.client.1.smithi163.stdout:4/97: mknod d11/c19 0 2024-02-16T19:24:20.696 INFO:tasks.workunit.client.1.smithi163.stdout:4/98: dread - d11/f17 zero size 2024-02-16T19:24:20.696 INFO:tasks.workunit.client.1.smithi163.stdout:0/192: dread f6 [0,4194304] 0 2024-02-16T19:24:20.696 INFO:tasks.workunit.client.1.smithi163.stdout:4/99: read f10 [2146,24515] 0 2024-02-16T19:24:20.698 INFO:tasks.workunit.client.0.smithi040.stdout:7/646: creat d21/d29/d79/dbc/fd7 x:0 0 0 2024-02-16T19:24:20.698 INFO:tasks.workunit.client.1.smithi163.stdout:6/138: creat d0/d6/d9/f2d x:0 0 0 2024-02-16T19:24:20.699 INFO:tasks.workunit.client.1.smithi163.stdout:6/139: chown d0/dd/f12 3540201 1 2024-02-16T19:24:20.699 INFO:tasks.workunit.client.0.smithi040.stdout:9/741: sync 2024-02-16T19:24:20.699 INFO:tasks.workunit.client.0.smithi040.stdout:9/742: creat d3/d4/d11/d71/fed x:0 0 0 2024-02-16T19:24:20.700 INFO:tasks.workunit.client.1.smithi163.stdout:1/96: dread f2 [0,4194304] 0 2024-02-16T19:24:20.700 INFO:tasks.workunit.client.1.smithi163.stdout:3/177: dwrite d0/dc/d1c/d32/f3f [0,4194304] 0 2024-02-16T19:24:20.700 INFO:tasks.workunit.client.1.smithi163.stdout:3/178: write d0/f39 [748172,15685] 0 2024-02-16T19:24:20.700 INFO:tasks.workunit.client.1.smithi163.stdout:5/122: dwrite d0/d2/f7 [0,4194304] 0 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/833: symlink db/d2f/d1d/d21/d2e/d85/l11f 0 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/834: fsync db/d2f/d1d/d21/d2e/d83/fcd 0 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/835: truncate db/d2f/d1d/d21/d2e/d83/faa 40567 0 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/836: truncate db/de/d6a/df0/ff8 489481 0 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/837: read - db/d2f/d1d/f10f zero size 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/838: dread - db/d2f/d1d/d21/d2e/fff zero size 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/839: chown db/d2f/d1d/d21/l6c 3579 1 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/840: read - db/de/dc0/fd8 zero size 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/841: readlink db/lab 0 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/842: creat db/de/d6a/f120 x:0 0 0 2024-02-16T19:24:20.702 INFO:tasks.workunit.client.0.smithi040.stdout:0/843: truncate db/d25/f11b 303501 0 2024-02-16T19:24:20.703 INFO:tasks.workunit.client.1.smithi163.stdout:7/108: mkdir d11/d24/d25 0 2024-02-16T19:24:20.703 INFO:tasks.workunit.client.1.smithi163.stdout:7/109: readlink l10 0 2024-02-16T19:24:20.703 INFO:tasks.workunit.client.0.smithi040.stdout:3/819: getdents d1/dc/d35/d48/d85 0 2024-02-16T19:24:20.704 INFO:tasks.workunit.client.1.smithi163.stdout:0/193: mknod d11/d22/c45 0 2024-02-16T19:24:20.706 INFO:tasks.workunit.client.0.smithi040.stdout:5/676: creat d1/d45/fce x:0 0 0 2024-02-16T19:24:20.706 INFO:tasks.workunit.client.0.smithi040.stdout:5/677: stat d1/da/d1c/dba/db1 0 2024-02-16T19:24:20.707 INFO:tasks.workunit.client.1.smithi163.stdout:4/100: symlink d11/l1a 0 2024-02-16T19:24:20.710 INFO:tasks.workunit.client.1.smithi163.stdout:6/140: mkdir d0/d6/d9/d2e 0 2024-02-16T19:24:20.710 INFO:tasks.workunit.client.1.smithi163.stdout:6/141: read d0/f2 [3071420,33875] 0 2024-02-16T19:24:20.713 INFO:tasks.workunit.client.1.smithi163.stdout:1/97: creat d7/f1c x:0 0 0 2024-02-16T19:24:20.714 INFO:tasks.workunit.client.1.smithi163.stdout:1/98: creat d7/f1d x:0 0 0 2024-02-16T19:24:20.714 INFO:tasks.workunit.client.1.smithi163.stdout:1/99: fdatasync d7/f1d 0 2024-02-16T19:24:20.714 INFO:tasks.workunit.client.0.smithi040.stdout:2/649: mkdir d5/d7/d1e/d55/dd4 0 2024-02-16T19:24:20.719 INFO:tasks.workunit.client.0.smithi040.stdout:4/595: rmdir dd/de 39 2024-02-16T19:24:20.721 INFO:tasks.workunit.client.1.smithi163.stdout:5/123: dwrite d0/f1d [0,4194304] 0 2024-02-16T19:24:20.721 INFO:tasks.workunit.client.0.smithi040.stdout:7/647: link d21/l40 d21/d29/ld8 0 2024-02-16T19:24:20.723 INFO:tasks.workunit.client.1.smithi163.stdout:3/179: mkdir d0/dc/d1d/d29/d38/d46 0 2024-02-16T19:24:20.727 INFO:tasks.workunit.client.1.smithi163.stdout:7/110: dwrite f3 [0,4194304] 0 2024-02-16T19:24:20.727 INFO:tasks.workunit.client.1.smithi163.stdout:7/111: write d11/f19 [902468,85604] 0 2024-02-16T19:24:20.731 INFO:tasks.workunit.client.0.smithi040.stdout:4/596: dread dd/f52 [0,4194304] 0 2024-02-16T19:24:20.734 INFO:tasks.workunit.client.1.smithi163.stdout:7/112: dread d11/f19 [0,4194304] 0 2024-02-16T19:24:20.734 INFO:tasks.workunit.client.1.smithi163.stdout:7/113: chown d11/c17 30825 1 2024-02-16T19:24:20.734 INFO:tasks.workunit.client.1.smithi163.stdout:0/194: creat d11/d21/f46 x:0 0 0 2024-02-16T19:24:20.734 INFO:tasks.workunit.client.1.smithi163.stdout:0/195: creat d11/d21/f47 x:0 0 0 2024-02-16T19:24:20.734 INFO:tasks.workunit.client.1.smithi163.stdout:0/196: write d11/d12/f15 [254827,12079] 0 2024-02-16T19:24:20.736 INFO:tasks.workunit.client.1.smithi163.stdout:4/101: link cb d11/d18/c1b 0 2024-02-16T19:24:20.737 INFO:tasks.workunit.client.0.smithi040.stdout:8/803: rename d6/d7 to d6/d12/d51/d6e/db7/ddf/df8 0 2024-02-16T19:24:20.738 INFO:tasks.workunit.client.0.smithi040.stdout:3/820: mkdir d1/dc/d35/d10a 0 2024-02-16T19:24:20.740 INFO:tasks.workunit.client.1.smithi163.stdout:6/142: unlink d0/d18/l27 0 2024-02-16T19:24:20.740 INFO:tasks.workunit.client.1.smithi163.stdout:6/143: readlink d0/d6/l16 0 2024-02-16T19:24:20.740 INFO:tasks.workunit.client.1.smithi163.stdout:6/144: chown d0/d6/d11/d1e 42779 1 2024-02-16T19:24:20.741 INFO:tasks.workunit.client.0.smithi040.stdout:5/678: write d1/f17 [5045209,53654] 0 2024-02-16T19:24:20.741 INFO:tasks.workunit.client.0.smithi040.stdout:5/679: write d1/da/d1c/dba/fc2 [251133,65887] 0 2024-02-16T19:24:20.750 INFO:tasks.workunit.client.0.smithi040.stdout:7/648: symlink d21/d4d/d77/ld9 0 2024-02-16T19:24:20.752 INFO:tasks.workunit.client.0.smithi040.stdout:0/844: rename db/d2f/d1d/d21/d2e/d49/d5b/f5d to db/d2f/d1d/d21/d2e/d85/d89/d94/f121 0 2024-02-16T19:24:20.754 INFO:tasks.workunit.client.1.smithi163.stdout:5/124: truncate d0/d2/fc 2102558 0 2024-02-16T19:24:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:5/680: write d1/da/fa1 [400534,46624] 0 2024-02-16T19:24:20.755 INFO:tasks.workunit.client.0.smithi040.stdout:5/681: fdatasync d1/da/d1c/d73/fbd 0 2024-02-16T19:24:20.756 INFO:tasks.workunit.client.0.smithi040.stdout:5/682: dread - d1/d3c/f5d zero size 2024-02-16T19:24:20.757 INFO:tasks.workunit.client.0.smithi040.stdout:7/649: rename d21/d4d/d7d/f83 to d21/d8b/fda 0 2024-02-16T19:24:20.760 INFO:tasks.workunit.client.0.smithi040.stdout:0/845: symlink db/d2f/d23/d56/l122 0 2024-02-16T19:24:20.764 INFO:tasks.workunit.client.0.smithi040.stdout:7/650: rename d21/d4d/c95 to d21/cdb 0 2024-02-16T19:24:20.766 INFO:tasks.workunit.client.1.smithi163.stdout:3/180: mknod d0/d3e/c47 0 2024-02-16T19:24:20.766 INFO:tasks.workunit.client.1.smithi163.stdout:3/181: chown d0/dc/d1c/d21/l2f 194656 1 2024-02-16T19:24:20.768 INFO:tasks.workunit.client.0.smithi040.stdout:0/846: creat db/de/d6a/df0/d106/f123 x:0 0 0 2024-02-16T19:24:20.768 INFO:tasks.workunit.client.0.smithi040.stdout:0/847: write db/d25/d34/d5e/fdb [6070,46862] 0 2024-02-16T19:24:20.768 INFO:tasks.workunit.client.0.smithi040.stdout:0/848: readlink db/d25/ddc/le2 0 2024-02-16T19:24:20.771 INFO:tasks.workunit.client.1.smithi163.stdout:7/114: fsync fa 0 2024-02-16T19:24:20.771 INFO:tasks.workunit.client.1.smithi163.stdout:7/115: dread - fa zero size 2024-02-16T19:24:20.773 INFO:tasks.workunit.client.0.smithi040.stdout:8/804: dwrite d6/d12/d51/d6e/db7/ddf/df8/d15/f67 [0,4194304] 0 2024-02-16T19:24:20.774 INFO:tasks.workunit.client.0.smithi040.stdout:7/651: rmdir d21/d7e 39 2024-02-16T19:24:20.776 INFO:tasks.workunit.client.0.smithi040.stdout:3/821: dwrite d1/dc/d35/d53/d64/d65/f98 [4194304,4194304] 0 2024-02-16T19:24:20.776 INFO:tasks.workunit.client.0.smithi040.stdout:5/683: dwrite d1/da/d1c/d21/f42 [0,4194304] 0 2024-02-16T19:24:20.778 INFO:tasks.workunit.client.1.smithi163.stdout:0/197: mknod d11/c48 0 2024-02-16T19:24:20.780 INFO:tasks.workunit.client.0.smithi040.stdout:5/684: dread d1/d62/fb6 [0,4194304] 0 2024-02-16T19:24:20.780 INFO:tasks.workunit.client.0.smithi040.stdout:5/685: fsync d1/da/da4/fc1 0 2024-02-16T19:24:20.781 INFO:tasks.workunit.client.0.smithi040.stdout:8/805: mknod d6/d12/d51/d6e/db7/dc6/dcc/de8/cf9 0 2024-02-16T19:24:20.781 INFO:tasks.workunit.client.0.smithi040.stdout:8/806: stat d6/d12/d51/d6e/db7/ddf/df8/l37 0 2024-02-16T19:24:20.782 INFO:tasks.workunit.client.1.smithi163.stdout:0/198: write d11/d22/f34 [2574205,91715] 0 2024-02-16T19:24:20.782 INFO:tasks.workunit.client.1.smithi163.stdout:5/125: dread d0/d2/fa [0,4194304] 0 2024-02-16T19:24:20.782 INFO:tasks.workunit.client.1.smithi163.stdout:0/199: creat d11/d21/f49 x:0 0 0 2024-02-16T19:24:20.782 INFO:tasks.workunit.client.1.smithi163.stdout:0/200: chown fd 5325 1 2024-02-16T19:24:20.782 INFO:tasks.workunit.client.1.smithi163.stdout:0/201: write d11/d21/f49 [577817,9392] 0 2024-02-16T19:24:20.782 INFO:tasks.workunit.client.1.smithi163.stdout:4/102: mknod d11/c1c 0 2024-02-16T19:24:20.784 INFO:tasks.workunit.client.1.smithi163.stdout:1/100: rmdir d7/dc 39 2024-02-16T19:24:20.785 INFO:tasks.workunit.client.0.smithi040.stdout:0/849: rmdir db/d2f/d23/d56 39 2024-02-16T19:24:20.785 INFO:tasks.workunit.client.0.smithi040.stdout:0/850: readlink db/d2f/d1d/d21/d2e/d49/l112 0 2024-02-16T19:24:20.785 INFO:tasks.workunit.client.0.smithi040.stdout:0/851: write db/d2f/d23/d3b/fc7 [463783,97670] 0 2024-02-16T19:24:20.786 INFO:tasks.workunit.client.0.smithi040.stdout:3/822: link d1/d4/d6e/d8a/da5/ccc d1/dc/d42/d4a/d57/d79/de3/c10b 0 2024-02-16T19:24:20.788 INFO:tasks.workunit.client.1.smithi163.stdout:9/324: sync 2024-02-16T19:24:20.790 INFO:tasks.workunit.client.1.smithi163.stdout:3/182: link d0/f1b d0/dc/d1d/d29/f48 0 2024-02-16T19:24:20.792 INFO:tasks.workunit.client.1.smithi163.stdout:7/116: mkdir d11/d26 0 2024-02-16T19:24:20.793 INFO:tasks.workunit.client.0.smithi040.stdout:5/686: rename d1/da/c8e to d1/ccf 0 2024-02-16T19:24:20.796 INFO:tasks.workunit.client.0.smithi040.stdout:8/807: unlink d6/d12/d51/d6e/db7/dc6/dcc/de8/cf9 0 2024-02-16T19:24:20.796 INFO:tasks.workunit.client.0.smithi040.stdout:8/808: truncate d6/f43 1637069 0 2024-02-16T19:24:20.797 INFO:tasks.workunit.client.1.smithi163.stdout:9/325: write d5/de/d30/f36 [2070873,107454] 0 2024-02-16T19:24:20.797 INFO:tasks.workunit.client.1.smithi163.stdout:0/202: truncate f6 4977091 0 2024-02-16T19:24:20.797 INFO:tasks.workunit.client.1.smithi163.stdout:0/203: truncate d11/d12/d20/f2f 609699 0 2024-02-16T19:24:20.797 INFO:tasks.workunit.client.1.smithi163.stdout:9/326: creat d5/de/d30/d31/f74 x:0 0 0 2024-02-16T19:24:20.801 INFO:tasks.workunit.client.1.smithi163.stdout:4/103: rename d11/f12 to d11/d18/f1d 0 2024-02-16T19:24:20.802 INFO:tasks.workunit.client.1.smithi163.stdout:4/104: truncate d11/f17 143979 0 2024-02-16T19:24:20.802 INFO:tasks.workunit.client.1.smithi163.stdout:4/105: stat d11/l1a 0 2024-02-16T19:24:20.802 INFO:tasks.workunit.client.1.smithi163.stdout:3/183: dread d0/dc/d1c/d32/f3f [0,4194304] 0 2024-02-16T19:24:20.802 INFO:tasks.workunit.client.0.smithi040.stdout:5/687: dread d1/da/d1c/dba/db1/f7b [0,4194304] 0 2024-02-16T19:24:20.802 INFO:tasks.workunit.client.0.smithi040.stdout:5/688: chown d1/d3c/f68 84834017 1 2024-02-16T19:24:20.804 INFO:tasks.workunit.client.0.smithi040.stdout:3/823: rename d1/dc/d42/d4a/d57/f107 to d1/dc/ddf/f10c 0 2024-02-16T19:24:20.804 INFO:tasks.workunit.client.0.smithi040.stdout:3/824: creat d1/dc/d35/d53/d64/d65/f10d x:0 0 0 2024-02-16T19:24:20.804 INFO:tasks.workunit.client.0.smithi040.stdout:3/825: fsync d1/dc/f3c 0 2024-02-16T19:24:20.804 INFO:tasks.workunit.client.0.smithi040.stdout:3/826: fdatasync d1/d8c/d8e/fdc 0 2024-02-16T19:24:20.804 INFO:tasks.workunit.client.0.smithi040.stdout:3/827: chown d1/dc/d35/f109 15 1 2024-02-16T19:24:20.811 INFO:tasks.workunit.client.1.smithi163.stdout:0/204: write f10 [1509202,42659] 0 2024-02-16T19:24:20.811 INFO:tasks.workunit.client.0.smithi040.stdout:8/809: write d6/d64/fc4 [1806296,115540] 0 2024-02-16T19:24:20.812 INFO:tasks.workunit.client.0.smithi040.stdout:5/689: mknod d1/da/d7e/db2/cd0 0 2024-02-16T19:24:20.818 INFO:tasks.workunit.client.0.smithi040.stdout:3/828: mknod d1/dc/d42/d4a/d57/d79/de3/c10e 0 2024-02-16T19:24:20.821 INFO:tasks.workunit.client.1.smithi163.stdout:1/101: rename d7/dc/c1a to d7/c1e 0 2024-02-16T19:24:20.823 INFO:tasks.workunit.client.1.smithi163.stdout:7/117: dwrite fa [0,4194304] 0 2024-02-16T19:24:20.824 INFO:tasks.workunit.client.1.smithi163.stdout:3/184: mkdir d0/dc/d1d/d49 0 2024-02-16T19:24:20.825 INFO:tasks.workunit.client.0.smithi040.stdout:8/810: creat d6/d64/ffa x:0 0 0 2024-02-16T19:24:20.825 INFO:tasks.workunit.client.0.smithi040.stdout:8/811: read d6/d64/fc4 [978056,43878] 0 2024-02-16T19:24:20.826 INFO:tasks.workunit.client.1.smithi163.stdout:3/185: dread d0/f8 [0,4194304] 0 2024-02-16T19:24:20.826 INFO:tasks.workunit.client.1.smithi163.stdout:3/186: getdents d0/dc/d1d/d29/d38/d46 0 2024-02-16T19:24:20.826 INFO:tasks.workunit.client.1.smithi163.stdout:9/327: getdents d5/de/d30/d43 0 2024-02-16T19:24:20.829 INFO:tasks.workunit.client.0.smithi040.stdout:5/690: mknod d1/d45/dab/cd1 0 2024-02-16T19:24:20.829 INFO:tasks.workunit.client.0.smithi040.stdout:5/691: creat d1/d45/fd2 x:0 0 0 2024-02-16T19:24:20.829 INFO:tasks.workunit.client.0.smithi040.stdout:5/692: write d1/da/fa1 [2422365,76119] 0 2024-02-16T19:24:20.829 INFO:tasks.workunit.client.1.smithi163.stdout:1/102: truncate d7/f14 1739578 0 2024-02-16T19:24:20.829 INFO:tasks.workunit.client.1.smithi163.stdout:1/103: write d7/f1c [384968,46043] 0 2024-02-16T19:24:20.833 INFO:tasks.workunit.client.1.smithi163.stdout:4/106: dwrite fd [0,4194304] 0 2024-02-16T19:24:20.833 INFO:tasks.workunit.client.1.smithi163.stdout:4/107: chown d11/d18 273873 1 2024-02-16T19:24:20.833 INFO:tasks.workunit.client.1.smithi163.stdout:4/108: fdatasync d11/d18/f1d 0 2024-02-16T19:24:20.833 INFO:tasks.workunit.client.0.smithi040.stdout:3/829: truncate d1/d7/f4e 6814926 0 2024-02-16T19:24:20.833 INFO:tasks.workunit.client.0.smithi040.stdout:3/830: fsync d1/d7/d30/f92 0 2024-02-16T19:24:20.834 INFO:tasks.workunit.client.0.smithi040.stdout:3/831: dread - d1/d4/d6e/fcd zero size 2024-02-16T19:24:20.834 INFO:tasks.workunit.client.0.smithi040.stdout:3/832: readlink d1/d7/d30/l31 0 2024-02-16T19:24:20.834 INFO:tasks.workunit.client.1.smithi163.stdout:7/118: symlink d11/l27 0 2024-02-16T19:24:20.835 INFO:tasks.workunit.client.1.smithi163.stdout:3/187: mknod d0/dc/d1d/d29/d38/d46/c4a 0 2024-02-16T19:24:20.837 INFO:tasks.workunit.client.1.smithi163.stdout:0/205: dwrite d11/d12/d20/f3a [0,4194304] 0 2024-02-16T19:24:20.838 INFO:tasks.workunit.client.1.smithi163.stdout:9/328: rename d5/de/d16/l20 to d5/de/d30/l75 0 2024-02-16T19:24:20.847 INFO:tasks.workunit.client.1.smithi163.stdout:1/104: getdents d7 0 2024-02-16T19:24:20.848 INFO:tasks.workunit.client.0.smithi040.stdout:5/693: write d1/da/da4/fa6 [124659,25298] 0 2024-02-16T19:24:20.848 INFO:tasks.workunit.client.0.smithi040.stdout:5/694: fdatasync d1/f4c 0 2024-02-16T19:24:20.849 INFO:tasks.workunit.client.1.smithi163.stdout:9/329: mknod d5/de/d30/d3d/d49/d50/c76 0 2024-02-16T19:24:20.850 INFO:tasks.workunit.client.1.smithi163.stdout:9/330: truncate d5/de/d16/d47/f4c 14051 0 2024-02-16T19:24:20.851 INFO:tasks.workunit.client.1.smithi163.stdout:1/105: mknod d7/dc/c1f 0 2024-02-16T19:24:20.851 INFO:tasks.workunit.client.1.smithi163.stdout:1/106: chown d7/dc/c10 117632386 1 2024-02-16T19:24:20.852 INFO:tasks.workunit.client.1.smithi163.stdout:3/188: dread d0/dc/f3c [0,4194304] 0 2024-02-16T19:24:20.854 INFO:tasks.workunit.client.1.smithi163.stdout:1/107: mkdir d7/d20 0 2024-02-16T19:24:20.854 INFO:tasks.workunit.client.1.smithi163.stdout:1/108: write d7/fa [1465835,113586] 0 2024-02-16T19:24:20.855 INFO:tasks.workunit.client.1.smithi163.stdout:9/331: getdents d5/de/d30/d3d/d49/d6e 0 2024-02-16T19:24:20.855 INFO:tasks.workunit.client.1.smithi163.stdout:9/332: fsync d5/de/f28 0 2024-02-16T19:24:20.855 INFO:tasks.workunit.client.1.smithi163.stdout:9/333: chown d5/de/d30/d3d/d49/d6e/f73 42801 1 2024-02-16T19:24:20.856 INFO:tasks.workunit.client.1.smithi163.stdout:1/109: unlink d7/f9 0 2024-02-16T19:24:20.857 INFO:tasks.workunit.client.1.smithi163.stdout:1/110: write d7/fb [1016832,76473] 0 2024-02-16T19:24:20.859 INFO:tasks.workunit.client.1.smithi163.stdout:9/334: creat d5/f77 x:0 0 0 2024-02-16T19:24:20.859 INFO:tasks.workunit.client.1.smithi163.stdout:9/335: readlink d5/de/d16/d47/l48 0 2024-02-16T19:24:20.859 INFO:tasks.workunit.client.1.smithi163.stdout:9/336: truncate d5/de/d30/d43/f4b 779958 0 2024-02-16T19:24:20.859 INFO:tasks.workunit.client.1.smithi163.stdout:9/337: write d5/de/d16/d47/f4c [500129,116703] 0 2024-02-16T19:24:20.859 INFO:tasks.workunit.client.1.smithi163.stdout:9/338: fdatasync d5/f2e 0 2024-02-16T19:24:20.859 INFO:tasks.workunit.client.1.smithi163.stdout:4/109: dwrite f10 [0,4194304] 0 2024-02-16T19:24:20.859 INFO:tasks.workunit.client.1.smithi163.stdout:4/110: fdatasync d11/f17 0 2024-02-16T19:24:20.860 INFO:tasks.workunit.client.1.smithi163.stdout:1/111: unlink d7/dc/l12 0 2024-02-16T19:24:20.860 INFO:tasks.workunit.client.1.smithi163.stdout:9/339: symlink d5/de/d30/d3d/d49/d6e/l78 0 2024-02-16T19:24:20.861 INFO:tasks.workunit.client.1.smithi163.stdout:9/340: chown d5/de/l22 77 1 2024-02-16T19:24:20.863 INFO:tasks.workunit.client.1.smithi163.stdout:3/189: dread d0/dc/f17 [0,4194304] 0 2024-02-16T19:24:20.863 INFO:tasks.workunit.client.1.smithi163.stdout:3/190: truncate d0/f6 4916580 0 2024-02-16T19:24:20.863 INFO:tasks.workunit.client.1.smithi163.stdout:4/111: mknod d11/d18/c1e 0 2024-02-16T19:24:20.864 INFO:tasks.workunit.client.1.smithi163.stdout:1/112: mkdir d7/d20/d21 0 2024-02-16T19:24:20.864 INFO:tasks.workunit.client.1.smithi163.stdout:1/113: read - d7/f1d zero size 2024-02-16T19:24:20.864 INFO:tasks.workunit.client.1.smithi163.stdout:9/341: mknod d5/de/d72/c79 0 2024-02-16T19:24:20.865 INFO:tasks.workunit.client.1.smithi163.stdout:4/112: mkdir d11/d18/d1f 0 2024-02-16T19:24:20.865 INFO:tasks.workunit.client.1.smithi163.stdout:4/113: read d11/f17 [60715,9543] 0 2024-02-16T19:24:20.867 INFO:tasks.workunit.client.1.smithi163.stdout:1/114: creat d7/f22 x:0 0 0 2024-02-16T19:24:20.867 INFO:tasks.workunit.client.1.smithi163.stdout:7/119: dwrite d11/f19 [0,4194304] 0 2024-02-16T19:24:20.867 INFO:tasks.workunit.client.1.smithi163.stdout:7/120: readlink d11/l14 0 2024-02-16T19:24:20.868 INFO:tasks.workunit.client.1.smithi163.stdout:0/206: dwrite d11/d12/d20/f39 [0,4194304] 0 2024-02-16T19:24:20.871 INFO:tasks.workunit.client.0.smithi040.stdout:6/828: sync 2024-02-16T19:24:20.872 INFO:tasks.workunit.client.1.smithi163.stdout:1/115: dread d7/f1c [0,4194304] 0 2024-02-16T19:24:20.876 INFO:tasks.workunit.client.0.smithi040.stdout:6/829: symlink d0/d1/d12/d3b/d48/d73/d70/l10f 0 2024-02-16T19:24:20.876 INFO:tasks.workunit.client.1.smithi163.stdout:7/121: symlink d11/d26/l28 0 2024-02-16T19:24:20.876 INFO:tasks.workunit.client.1.smithi163.stdout:9/342: dread d5/de/f5b [0,4194304] 0 2024-02-16T19:24:20.876 INFO:tasks.workunit.client.1.smithi163.stdout:9/343: fsync d5/de/d30/f5e 0 2024-02-16T19:24:20.879 INFO:tasks.workunit.client.0.smithi040.stdout:6/830: symlink d0/d1/d6f/l110 0 2024-02-16T19:24:20.883 INFO:tasks.workunit.client.0.smithi040.stdout:6/831: mkdir d0/d1/d12/d3b/d48/d73/dff/d111 0 2024-02-16T19:24:20.886 INFO:tasks.workunit.client.0.smithi040.stdout:6/832: mkdir d0/d1/d12/d3b/d48/d73/dff/d111/d112 0 2024-02-16T19:24:20.889 INFO:tasks.workunit.client.0.smithi040.stdout:6/833: truncate d0/d30/d4b/f57 1302430 0 2024-02-16T19:24:20.889 INFO:tasks.workunit.client.0.smithi040.stdout:6/834: read - d0/d1/d12/d3b/d48/dba/ff2 zero size 2024-02-16T19:24:20.890 INFO:tasks.workunit.client.1.smithi163.stdout:4/114: dwrite d11/f17 [0,4194304] 0 2024-02-16T19:24:20.891 INFO:tasks.workunit.client.1.smithi163.stdout:0/207: getdents d11 0 2024-02-16T19:24:20.891 INFO:tasks.workunit.client.1.smithi163.stdout:0/208: dread - d11/d22/f42 zero size 2024-02-16T19:24:20.891 INFO:tasks.workunit.client.1.smithi163.stdout:0/209: write d11/d12/d20/f30 [807712,117531] 0 2024-02-16T19:24:20.892 INFO:tasks.workunit.client.1.smithi163.stdout:9/344: creat d5/de/d72/f7a x:0 0 0 2024-02-16T19:24:20.892 INFO:tasks.workunit.client.1.smithi163.stdout:9/345: fdatasync d5/de/d30/d3d/d49/f62 0 2024-02-16T19:24:20.895 INFO:tasks.workunit.client.1.smithi163.stdout:0/210: mknod d11/d22/d3e/c4a 0 2024-02-16T19:24:20.895 INFO:tasks.workunit.client.1.smithi163.stdout:0/211: write d11/f1f [907702,125099] 0 2024-02-16T19:24:20.896 INFO:tasks.workunit.client.1.smithi163.stdout:9/346: rename d5/de/d30/d31/f51 to d5/de/d30/f7b 0 2024-02-16T19:24:20.897 INFO:tasks.workunit.client.1.smithi163.stdout:0/212: unlink d11/d12/f27 0 2024-02-16T19:24:20.898 INFO:tasks.workunit.client.1.smithi163.stdout:9/347: mkdir d5/de/d30/d7c 0 2024-02-16T19:24:20.899 INFO:tasks.workunit.client.1.smithi163.stdout:0/213: symlink d11/d22/d3e/l4b 0 2024-02-16T19:24:20.899 INFO:tasks.workunit.client.1.smithi163.stdout:0/214: chown f10 81 1 2024-02-16T19:24:20.899 INFO:tasks.workunit.client.1.smithi163.stdout:0/215: stat f6 0 2024-02-16T19:24:20.899 INFO:tasks.workunit.client.1.smithi163.stdout:7/122: dwrite d11/f20 [0,4194304] 0 2024-02-16T19:24:20.899 INFO:tasks.workunit.client.1.smithi163.stdout:9/348: symlink d5/de/d30/d31/d52/l7d 0 2024-02-16T19:24:20.900 INFO:tasks.workunit.client.1.smithi163.stdout:3/191: dwrite d0/dc/d1d/d29/f30 [0,4194304] 0 2024-02-16T19:24:20.901 INFO:tasks.workunit.client.1.smithi163.stdout:0/216: symlink d11/d22/l4c 0 2024-02-16T19:24:20.901 INFO:tasks.workunit.client.1.smithi163.stdout:0/217: stat d11/d12/c1b 0 2024-02-16T19:24:20.901 INFO:tasks.workunit.client.1.smithi163.stdout:0/218: chown d11/d12/l1e 46474 1 2024-02-16T19:24:20.901 INFO:tasks.workunit.client.1.smithi163.stdout:0/219: write f6 [5395560,26367] 0 2024-02-16T19:24:20.905 INFO:tasks.workunit.client.1.smithi163.stdout:4/115: write f10 [3944159,67613] 0 2024-02-16T19:24:20.906 INFO:tasks.workunit.client.1.smithi163.stdout:4/116: stat c7 0 2024-02-16T19:24:20.909 INFO:tasks.workunit.client.1.smithi163.stdout:0/220: mknod d11/d12/c4d 0 2024-02-16T19:24:20.909 INFO:tasks.workunit.client.1.smithi163.stdout:0/221: creat d11/d12/d20/f4e x:0 0 0 2024-02-16T19:24:20.909 INFO:tasks.workunit.client.1.smithi163.stdout:9/349: symlink d5/de/d30/d3d/l7e 0 2024-02-16T19:24:20.910 INFO:tasks.workunit.client.1.smithi163.stdout:7/123: dread d11/f1c [0,4194304] 0 2024-02-16T19:24:20.914 INFO:tasks.workunit.client.1.smithi163.stdout:7/124: creat d11/d26/f29 x:0 0 0 2024-02-16T19:24:20.917 INFO:tasks.workunit.client.1.smithi163.stdout:0/222: dread d11/f35 [0,4194304] 0 2024-02-16T19:24:20.918 INFO:tasks.workunit.client.1.smithi163.stdout:4/117: write d11/f17 [1830769,100862] 0 2024-02-16T19:24:20.918 INFO:tasks.workunit.client.1.smithi163.stdout:7/125: write d11/f1b [1239788,60839] 0 2024-02-16T19:24:20.919 INFO:tasks.workunit.client.1.smithi163.stdout:8/109: sync 2024-02-16T19:24:20.921 INFO:tasks.workunit.client.1.smithi163.stdout:2/256: sync 2024-02-16T19:24:20.921 INFO:tasks.workunit.client.1.smithi163.stdout:2/257: chown d2/d2a/d39/f42 59 1 2024-02-16T19:24:20.921 INFO:tasks.workunit.client.1.smithi163.stdout:6/145: sync 2024-02-16T19:24:20.921 INFO:tasks.workunit.client.1.smithi163.stdout:6/146: getdents d0/d6/d11/d17 0 2024-02-16T19:24:20.921 INFO:tasks.workunit.client.1.smithi163.stdout:5/126: sync 2024-02-16T19:24:20.923 INFO:tasks.workunit.client.1.smithi163.stdout:0/223: mkdir d11/d4f 0 2024-02-16T19:24:20.923 INFO:tasks.workunit.client.1.smithi163.stdout:0/224: fdatasync f10 0 2024-02-16T19:24:20.923 INFO:tasks.workunit.client.1.smithi163.stdout:0/225: write d11/f35 [1865905,113349] 0 2024-02-16T19:24:20.923 INFO:tasks.workunit.client.1.smithi163.stdout:7/126: mknod d11/d24/d25/c2a 0 2024-02-16T19:24:20.926 INFO:tasks.workunit.client.0.smithi040.stdout:6/835: dwrite d0/d1/d6f/d90/f9b [0,4194304] 0 2024-02-16T19:24:20.926 INFO:tasks.workunit.client.0.smithi040.stdout:6/836: dread - d0/d1/d6f/d90/f104 zero size 2024-02-16T19:24:20.928 INFO:tasks.workunit.client.1.smithi163.stdout:2/258: truncate d2/f18 4505546 0 2024-02-16T19:24:20.928 INFO:tasks.workunit.client.1.smithi163.stdout:2/259: write d2/d26/f28 [2362221,39512] 0 2024-02-16T19:24:20.928 INFO:tasks.workunit.client.1.smithi163.stdout:2/260: chown d2/f23 137721495 1 2024-02-16T19:24:20.929 INFO:tasks.workunit.client.1.smithi163.stdout:6/147: creat d0/d6/d11/d17/f2f x:0 0 0 2024-02-16T19:24:20.929 INFO:tasks.workunit.client.1.smithi163.stdout:9/350: dread d5/f40 [0,4194304] 0 2024-02-16T19:24:20.929 INFO:tasks.workunit.client.1.smithi163.stdout:0/226: mkdir d11/d21/d50 0 2024-02-16T19:24:20.929 INFO:tasks.workunit.client.1.smithi163.stdout:0/227: chown d11/f33 13412003 1 2024-02-16T19:24:20.929 INFO:tasks.workunit.client.1.smithi163.stdout:0/228: read - d11/d21/f47 zero size 2024-02-16T19:24:20.929 INFO:tasks.workunit.client.1.smithi163.stdout:2/261: mkdir d2/d26/d54 0 2024-02-16T19:24:20.929 INFO:tasks.workunit.client.1.smithi163.stdout:6/148: write d0/d6/f1a [3727360,94413] 0 2024-02-16T19:24:20.929 INFO:tasks.workunit.client.1.smithi163.stdout:6/149: stat d0/d6/df 0 2024-02-16T19:24:20.930 INFO:tasks.workunit.client.1.smithi163.stdout:9/351: rmdir d5/de/d30/d7c 0 2024-02-16T19:24:20.930 INFO:tasks.workunit.client.1.smithi163.stdout:9/352: creat d5/de/d30/d31/f7f x:0 0 0 2024-02-16T19:24:20.930 INFO:tasks.workunit.client.1.smithi163.stdout:9/353: write d5/de/d30/d31/f69 [1012257,40997] 0 2024-02-16T19:24:20.934 INFO:tasks.workunit.client.1.smithi163.stdout:0/229: rename d11/d22/d3e/c4a to d11/c51 0 2024-02-16T19:24:20.936 INFO:tasks.workunit.client.1.smithi163.stdout:5/127: write d0/d2/f21 [354592,9650] 0 2024-02-16T19:24:20.936 INFO:tasks.workunit.client.1.smithi163.stdout:3/192: dwrite d0/dc/d1c/d32/f3f [0,4194304] 0 2024-02-16T19:24:20.939 INFO:tasks.workunit.client.1.smithi163.stdout:8/110: dwrite d0/d7/f15 [0,4194304] 0 2024-02-16T19:24:20.939 INFO:tasks.workunit.client.1.smithi163.stdout:0/230: creat d11/d21/d50/f52 x:0 0 0 2024-02-16T19:24:20.942 INFO:tasks.workunit.client.1.smithi163.stdout:2/262: rename d2/l1d to d2/d2a/d2d/l55 0 2024-02-16T19:24:20.942 INFO:tasks.workunit.client.1.smithi163.stdout:2/263: readlink d2/d26/l48 0 2024-02-16T19:24:20.945 INFO:tasks.workunit.client.0.smithi040.stdout:6/837: rmdir d0/d1/d12/d3b 39 2024-02-16T19:24:20.945 INFO:tasks.workunit.client.0.smithi040.stdout:6/838: write d0/d30/d5b/f91 [1883098,100338] 0 2024-02-16T19:24:20.945 INFO:tasks.workunit.client.0.smithi040.stdout:6/839: truncate d0/d1/d12/d3b/d48/d73/d70/fa6 4855264 0 2024-02-16T19:24:20.945 INFO:tasks.workunit.client.0.smithi040.stdout:6/840: chown d0/d1/d12/d3b/d48/dba/fc5 65 1 2024-02-16T19:24:20.947 INFO:tasks.workunit.client.1.smithi163.stdout:5/128: rename d0/d2/f1a to d0/d2/f29 0 2024-02-16T19:24:20.947 INFO:tasks.workunit.client.1.smithi163.stdout:5/129: chown d0/d2/c27 28 1 2024-02-16T19:24:20.950 INFO:tasks.workunit.client.0.smithi040.stdout:6/841: creat d0/d1/d6f/d90/db1/f113 x:0 0 0 2024-02-16T19:24:20.950 INFO:tasks.workunit.client.0.smithi040.stdout:6/842: readlink d0/d1/d12/d3b/l81 0 2024-02-16T19:24:20.950 INFO:tasks.workunit.client.0.smithi040.stdout:6/843: write d0/d30/fef [813594,57909] 0 2024-02-16T19:24:20.951 INFO:tasks.workunit.client.1.smithi163.stdout:6/150: dwrite d0/dd/f13 [4194304,4194304] 0 2024-02-16T19:24:20.952 INFO:tasks.workunit.client.1.smithi163.stdout:4/118: truncate f10 1672138 0 2024-02-16T19:24:20.955 INFO:tasks.workunit.client.0.smithi040.stdout:6/844: rmdir d0/d1/d12/d3b/d48/d73/d70 39 2024-02-16T19:24:20.956 INFO:tasks.workunit.client.0.smithi040.stdout:6/845: write d0/d1/d12/d3b/fd6 [752451,35818] 0 2024-02-16T19:24:20.956 INFO:tasks.workunit.client.0.smithi040.stdout:6/846: write d0/d30/d5b/f8a [431819,109510] 0 2024-02-16T19:24:20.959 INFO:tasks.workunit.client.1.smithi163.stdout:8/111: symlink d0/l25 0 2024-02-16T19:24:20.959 INFO:tasks.workunit.client.1.smithi163.stdout:8/112: readlink d0/d7/l8 0 2024-02-16T19:24:20.959 INFO:tasks.workunit.client.1.smithi163.stdout:0/231: mknod d11/d4f/c53 0 2024-02-16T19:24:20.960 INFO:tasks.workunit.client.1.smithi163.stdout:2/264: symlink d2/d2a/d2d/d3c/l56 0 2024-02-16T19:24:20.963 INFO:tasks.workunit.client.1.smithi163.stdout:5/130: symlink d0/d2/l2a 0 2024-02-16T19:24:20.964 INFO:tasks.workunit.client.1.smithi163.stdout:6/151: mkdir d0/d6/df/d30 0 2024-02-16T19:24:20.965 INFO:tasks.workunit.client.0.smithi040.stdout:6/847: dread d0/d30/d32/dd2/f71 [4194304,4194304] 0 2024-02-16T19:24:20.965 INFO:tasks.workunit.client.0.smithi040.stdout:6/848: creat d0/d1/d6f/d90/db1/f114 x:0 0 0 2024-02-16T19:24:20.965 INFO:tasks.workunit.client.0.smithi040.stdout:6/849: creat d0/d37/f115 x:0 0 0 2024-02-16T19:24:20.966 INFO:tasks.workunit.client.1.smithi163.stdout:8/113: unlink d0/l25 0 2024-02-16T19:24:20.967 INFO:tasks.workunit.client.1.smithi163.stdout:0/232: symlink d11/d4f/l54 0 2024-02-16T19:24:20.971 INFO:tasks.workunit.client.0.smithi040.stdout:6/850: creat d0/d30/d4b/dcc/f116 x:0 0 0 2024-02-16T19:24:20.973 INFO:tasks.workunit.client.1.smithi163.stdout:2/265: rename d2/d26/f3f to d2/d2a/d39/f57 0 2024-02-16T19:24:20.974 INFO:tasks.workunit.client.1.smithi163.stdout:5/131: truncate d0/f1d 1103946 0 2024-02-16T19:24:20.975 INFO:tasks.workunit.client.1.smithi163.stdout:5/132: write d0/d2/fc [2927631,34118] 0 2024-02-16T19:24:20.976 INFO:tasks.workunit.client.0.smithi040.stdout:6/851: unlink d0/d30/d32/l4c 0 2024-02-16T19:24:20.976 INFO:tasks.workunit.client.1.smithi163.stdout:8/114: truncate d0/dc/f16 3724693 0 2024-02-16T19:24:20.979 INFO:tasks.workunit.client.1.smithi163.stdout:0/233: dwrite fd [4194304,4194304] 0 2024-02-16T19:24:20.982 INFO:tasks.workunit.client.1.smithi163.stdout:0/234: dread d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:20.983 INFO:tasks.workunit.client.1.smithi163.stdout:2/266: dread d2/f1a [0,4194304] 0 2024-02-16T19:24:20.983 INFO:tasks.workunit.client.1.smithi163.stdout:2/267: fsync d2/d2a/f33 0 2024-02-16T19:24:20.983 INFO:tasks.workunit.client.1.smithi163.stdout:8/115: rename d0/ff to d0/d7/d19/f26 0 2024-02-16T19:24:20.988 INFO:tasks.workunit.client.1.smithi163.stdout:8/116: dread d0/d7/d19/f1b [0,4194304] 0 2024-02-16T19:24:20.990 INFO:tasks.workunit.client.1.smithi163.stdout:5/133: getdents d0/d2 0 2024-02-16T19:24:20.991 INFO:tasks.workunit.client.1.smithi163.stdout:2/268: mkdir d2/d26/d54/d58 0 2024-02-16T19:24:20.993 INFO:tasks.workunit.client.1.smithi163.stdout:8/117: mkdir d0/d24/d27 0 2024-02-16T19:24:20.995 INFO:tasks.workunit.client.1.smithi163.stdout:0/235: rmdir d11/d21/d50 39 2024-02-16T19:24:20.995 INFO:tasks.workunit.client.1.smithi163.stdout:0/236: chown d11/d12/l26 16428427 1 2024-02-16T19:24:20.995 INFO:tasks.workunit.client.1.smithi163.stdout:0/237: write d11/d12/d20/f30 [968946,61155] 0 2024-02-16T19:24:20.995 INFO:tasks.workunit.client.1.smithi163.stdout:0/238: write d11/d22/f3f [915436,30424] 0 2024-02-16T19:24:20.995 INFO:tasks.workunit.client.1.smithi163.stdout:0/239: write d11/d12/d20/f2c [22708,4585] 0 2024-02-16T19:24:20.995 INFO:tasks.workunit.client.1.smithi163.stdout:1/116: sync 2024-02-16T19:24:20.996 INFO:tasks.workunit.client.0.smithi040.stdout:6/852: symlink d0/d30/d32/dd2/d87/d8f/l117 0 2024-02-16T19:24:20.996 INFO:tasks.workunit.client.0.smithi040.stdout:6/853: creat d0/df0/f118 x:0 0 0 2024-02-16T19:24:20.996 INFO:tasks.workunit.client.0.smithi040.stdout:6/854: write d0/d30/d5b/dbe/fed [8761529,71828] 0 2024-02-16T19:24:20.998 INFO:tasks.workunit.client.1.smithi163.stdout:5/134: symlink d0/d22/l2b 0 2024-02-16T19:24:21.002 INFO:tasks.workunit.client.1.smithi163.stdout:0/240: truncate d11/d12/d20/f39 3981541 0 2024-02-16T19:24:21.002 INFO:tasks.workunit.client.1.smithi163.stdout:0/241: write f3 [2087174,54924] 0 2024-02-16T19:24:21.004 INFO:tasks.workunit.client.0.smithi040.stdout:6/855: symlink d0/d30/d32/dd2/dbc/l119 0 2024-02-16T19:24:21.014 INFO:tasks.workunit.client.1.smithi163.stdout:5/135: dread d0/d2/f16 [4194304,4194304] 0 2024-02-16T19:24:21.023 INFO:tasks.workunit.client.1.smithi163.stdout:8/118: dwrite d0/fe [0,4194304] 0 2024-02-16T19:24:21.023 INFO:tasks.workunit.client.1.smithi163.stdout:2/269: dwrite d2/ff [8388608,4194304] 0 2024-02-16T19:24:21.023 INFO:tasks.workunit.client.1.smithi163.stdout:0/242: dwrite d11/d21/f46 [0,4194304] 0 2024-02-16T19:24:21.027 INFO:tasks.workunit.client.1.smithi163.stdout:8/119: symlink d0/l28 0 2024-02-16T19:24:21.027 INFO:tasks.workunit.client.1.smithi163.stdout:8/120: read - d0/d7/d19/f26 zero size 2024-02-16T19:24:21.028 INFO:tasks.workunit.client.1.smithi163.stdout:2/270: symlink d2/d2a/d39/l59 0 2024-02-16T19:24:21.029 INFO:tasks.workunit.client.0.smithi040.stdout:6/856: getdents d0/df0 0 2024-02-16T19:24:21.029 INFO:tasks.workunit.client.0.smithi040.stdout:6/857: truncate d0/d1/d12/d3b/d48/d73/d76/fd5 904316 0 2024-02-16T19:24:21.029 INFO:tasks.workunit.client.0.smithi040.stdout:6/858: fdatasync d0/d30/d32/dd2/dbc/dfa/f105 0 2024-02-16T19:24:21.029 INFO:tasks.workunit.client.0.smithi040.stdout:6/859: write d0/f3f [1137515,85575] 0 2024-02-16T19:24:21.029 INFO:tasks.workunit.client.0.smithi040.stdout:6/860: chown d0/d1/d12/d3b/d48/d73/d70/fa5 480076 1 2024-02-16T19:24:21.029 INFO:tasks.workunit.client.0.smithi040.stdout:6/861: truncate d0/d30/d32/dd2/dbc/f106 919049 0 2024-02-16T19:24:21.033 INFO:tasks.workunit.client.1.smithi163.stdout:0/243: symlink d11/d4f/l55 0 2024-02-16T19:24:21.033 INFO:tasks.workunit.client.1.smithi163.stdout:0/244: fsync d11/d22/f2d 0 2024-02-16T19:24:21.033 INFO:tasks.workunit.client.1.smithi163.stdout:0/245: write d11/d12/f15 [191195,25718] 0 2024-02-16T19:24:21.033 INFO:tasks.workunit.client.0.smithi040.stdout:6/862: symlink d0/d1/d12/d3b/d79/l11a 0 2024-02-16T19:24:21.034 INFO:tasks.workunit.client.1.smithi163.stdout:2/271: link d2/d26/f2b d2/d2a/d39/f5a 0 2024-02-16T19:24:21.037 INFO:tasks.workunit.client.0.smithi040.stdout:6/863: mknod d0/d1/d12/d3b/d48/dba/dde/c11b 0 2024-02-16T19:24:21.039 INFO:tasks.workunit.client.1.smithi163.stdout:2/272: creat d2/d2a/d39/d3a/f5b x:0 0 0 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:1/705: sync 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:9/743: sync 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/650: sync 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:9/744: chown d3/d4/d55/d8a/fcf 774 1 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/651: dread - d5/d8/d42/f87 zero size 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/652: chown d5/d7/d10/d40 3039 1 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:9/745: dread - d3/d4/d11/d71/fed zero size 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/653: readlink d5/d5c/l92 0 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/654: getdents d5/d7/d1e/d31/dc3 0 2024-02-16T19:24:21.043 INFO:tasks.workunit.client.0.smithi040.stdout:9/746: getdents d3/dd3 0 2024-02-16T19:24:21.044 INFO:tasks.workunit.client.0.smithi040.stdout:1/706: creat d17/d26/d54/fe6 x:0 0 0 2024-02-16T19:24:21.045 INFO:tasks.workunit.client.1.smithi163.stdout:0/246: read d11/d12/d20/f3a [3308391,42747] 0 2024-02-16T19:24:21.046 INFO:tasks.workunit.client.0.smithi040.stdout:9/747: link d3/d4/d11/d29/d38/d46/d6b/cd8 d3/d4/d11/d29/d38/d86/cee 0 2024-02-16T19:24:21.049 INFO:tasks.workunit.client.0.smithi040.stdout:1/707: rename d17/d3c/d74/d97/f9e to d17/d29/d8b/dd2/fe7 0 2024-02-16T19:24:21.052 INFO:tasks.workunit.client.1.smithi163.stdout:2/273: write d2/f23 [3973718,12526] 0 2024-02-16T19:24:21.052 INFO:tasks.workunit.client.1.smithi163.stdout:2/274: write d2/f9 [5048384,52146] 0 2024-02-16T19:24:21.052 INFO:tasks.workunit.client.1.smithi163.stdout:2/275: write d2/f18 [2303755,56244] 0 2024-02-16T19:24:21.056 INFO:tasks.workunit.client.0.smithi040.stdout:2/655: dread d5/d7/d1e/d31/f43 [0,4194304] 0 2024-02-16T19:24:21.057 INFO:tasks.workunit.client.0.smithi040.stdout:9/748: dread d3/d4/d11/d29/f24 [4194304,4194304] 0 2024-02-16T19:24:21.057 INFO:tasks.workunit.client.0.smithi040.stdout:2/656: rmdir d5/d7/d1e/d31/d4f/d63 39 2024-02-16T19:24:21.058 INFO:tasks.workunit.client.0.smithi040.stdout:2/657: dread d5/d7/d1e/d4c/d52/f96 [0,4194304] 0 2024-02-16T19:24:21.059 INFO:tasks.workunit.client.0.smithi040.stdout:2/658: creat d5/d7/d1e/d4c/d52/daa/fd5 x:0 0 0 2024-02-16T19:24:21.059 INFO:tasks.workunit.client.0.smithi040.stdout:2/659: fdatasync d5/f3a 0 2024-02-16T19:24:21.062 INFO:tasks.workunit.client.0.smithi040.stdout:9/749: dread d3/d4/d11/d29/d38/f54 [0,4194304] 0 2024-02-16T19:24:21.063 INFO:tasks.workunit.client.0.smithi040.stdout:9/750: stat d3/d4/d11/d29/l68 0 2024-02-16T19:24:21.064 INFO:tasks.workunit.client.0.smithi040.stdout:9/751: mkdir d3/d4/d55/d31/def 0 2024-02-16T19:24:21.064 INFO:tasks.workunit.client.0.smithi040.stdout:9/752: truncate d3/d4/d55/d8a/f9f 26690 0 2024-02-16T19:24:21.064 INFO:tasks.workunit.client.0.smithi040.stdout:9/753: chown d3/d4/d4a/c4e 702366523 1 2024-02-16T19:24:21.064 INFO:tasks.workunit.client.0.smithi040.stdout:9/754: truncate d3/d4/d11/d71/da8/f43 2084565 0 2024-02-16T19:24:21.064 INFO:tasks.workunit.client.0.smithi040.stdout:9/755: write d3/d4/d55/d63/fa5 [652986,3162] 0 2024-02-16T19:24:21.065 INFO:tasks.workunit.client.0.smithi040.stdout:9/756: truncate d3/d4/d11/d29/d38/d46/d6b/f6f 2449171 0 2024-02-16T19:24:21.066 INFO:tasks.workunit.client.0.smithi040.stdout:9/757: rename d3/d1c/d73/ca7 to d3/d4/d11/d29/d38/d46/d6b/dac/cf0 0 2024-02-16T19:24:21.067 INFO:tasks.workunit.client.0.smithi040.stdout:9/758: truncate d3/d4/d11/d29/d38/d46/d6b/f97 794553 0 2024-02-16T19:24:21.067 INFO:tasks.workunit.client.0.smithi040.stdout:9/759: write d3/d1c/fb3 [1987163,9123] 0 2024-02-16T19:24:21.067 INFO:tasks.workunit.client.0.smithi040.stdout:9/760: fdatasync d3/d4/d11/d29/d38/d46/f6a 0 2024-02-16T19:24:21.067 INFO:tasks.workunit.client.0.smithi040.stdout:9/761: chown d3/d4/d11/d29/d38/d46/d6b/dac/cf0 21360946 1 2024-02-16T19:24:21.071 INFO:tasks.workunit.client.0.smithi040.stdout:6/864: dwrite d0/d30/fef [0,4194304] 0 2024-02-16T19:24:21.072 INFO:tasks.workunit.client.0.smithi040.stdout:6/865: dread - d0/d30/d32/dd2/d87/d8f/fb9 zero size 2024-02-16T19:24:21.072 INFO:tasks.workunit.client.1.smithi163.stdout:8/121: dwrite d0/d7/d19/f26 [0,4194304] 0 2024-02-16T19:24:21.072 INFO:tasks.workunit.client.1.smithi163.stdout:8/122: fdatasync d0/fe 0 2024-02-16T19:24:21.072 INFO:tasks.workunit.client.1.smithi163.stdout:0/247: dwrite d11/d21/d50/f52 [0,4194304] 0 2024-02-16T19:24:21.080 INFO:tasks.workunit.client.0.smithi040.stdout:6/866: dread d0/d30/d32/dd2/dbc/dfa/f105 [0,4194304] 0 2024-02-16T19:24:21.083 INFO:tasks.workunit.client.0.smithi040.stdout:2/660: dwrite d5/d7/d1e/d31/fb8 [0,4194304] 0 2024-02-16T19:24:21.083 INFO:tasks.workunit.client.1.smithi163.stdout:0/248: rename d11/c48 to d11/d4f/c56 0 2024-02-16T19:24:21.083 INFO:tasks.workunit.client.1.smithi163.stdout:0/249: chown d11/d21/l36 123956116 1 2024-02-16T19:24:21.084 INFO:tasks.workunit.client.0.smithi040.stdout:9/762: dwrite d3/d4/d55/d31/f4c [0,4194304] 0 2024-02-16T19:24:21.084 INFO:tasks.workunit.client.0.smithi040.stdout:9/763: write d3/d4/d11/d29/d38/d46/d6b/d85/fbb [259879,78771] 0 2024-02-16T19:24:21.085 INFO:tasks.workunit.client.1.smithi163.stdout:0/250: link d11/f1f d11/d22/d3e/f57 0 2024-02-16T19:24:21.086 INFO:tasks.workunit.client.0.smithi040.stdout:6/867: mknod d0/d1/d12/d3b/d48/d73/dff/c11c 0 2024-02-16T19:24:21.088 INFO:tasks.workunit.client.1.smithi163.stdout:0/251: mkdir d11/d21/d50/d58 0 2024-02-16T19:24:21.091 INFO:tasks.workunit.client.1.smithi163.stdout:0/252: mkdir d11/d22/d3e/d3c/d59 0 2024-02-16T19:24:21.091 INFO:tasks.workunit.client.0.smithi040.stdout:2/661: dread d5/d7/d1e/d31/d4f/d63/f2c [4194304,4194304] 0 2024-02-16T19:24:21.092 INFO:tasks.workunit.client.0.smithi040.stdout:2/662: creat d5/d5c/d65/d9d/fd6 x:0 0 0 2024-02-16T19:24:21.093 INFO:tasks.workunit.client.0.smithi040.stdout:9/764: rename d3/d4/d11/d71/da2/de2 to d3/d4/d11/d29/df1 0 2024-02-16T19:24:21.093 INFO:tasks.workunit.client.0.smithi040.stdout:9/765: chown d3/d4/d55/d31/l47 22633462 1 2024-02-16T19:24:21.096 INFO:tasks.workunit.client.0.smithi040.stdout:2/663: mknod d5/d7/d1e/d55/cd7 0 2024-02-16T19:24:21.096 INFO:tasks.workunit.client.0.smithi040.stdout:2/664: creat d5/d7/d1e/d31/fd8 x:0 0 0 2024-02-16T19:24:21.096 INFO:tasks.workunit.client.0.smithi040.stdout:2/665: creat d5/d8/d42/fd9 x:0 0 0 2024-02-16T19:24:21.097 INFO:tasks.workunit.client.0.smithi040.stdout:9/766: symlink d3/d4/d11/d29/d38/d46/d6b/lf2 0 2024-02-16T19:24:21.098 INFO:tasks.workunit.client.1.smithi163.stdout:7/127: sync 2024-02-16T19:24:21.098 INFO:tasks.workunit.client.1.smithi163.stdout:7/128: creat d11/d26/f2b x:0 0 0 2024-02-16T19:24:21.098 INFO:tasks.workunit.client.1.smithi163.stdout:7/129: write fa [4442527,102984] 0 2024-02-16T19:24:21.098 INFO:tasks.workunit.client.1.smithi163.stdout:7/130: dread - d11/d26/f2b zero size 2024-02-16T19:24:21.101 INFO:tasks.workunit.client.0.smithi040.stdout:4/597: sync 2024-02-16T19:24:21.104 INFO:tasks.workunit.client.0.smithi040.stdout:2/666: symlink d5/d7/d1e/d55/lda 0 2024-02-16T19:24:21.105 INFO:tasks.workunit.client.1.smithi163.stdout:7/131: write f2 [1588120,253] 0 2024-02-16T19:24:21.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/767: rename d3/d4/d11/d71/f88 to d3/d4/d11/d29/d38/d46/d6b/dac/ff3 0 2024-02-16T19:24:21.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/768: creat d3/d4/d55/ff4 x:0 0 0 2024-02-16T19:24:21.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/769: fsync d3/d4/d11/d29/d38/d46/f5b 0 2024-02-16T19:24:21.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/770: write d3/d4/d4a/f8c [3594073,79950] 0 2024-02-16T19:24:21.108 INFO:tasks.workunit.client.0.smithi040.stdout:4/598: link dd/de/d71/f84 dd/d58/fbc 0 2024-02-16T19:24:21.109 INFO:tasks.workunit.client.0.smithi040.stdout:4/599: write dd/de/d10/fa9 [1407409,90916] 0 2024-02-16T19:24:21.110 INFO:tasks.workunit.client.1.smithi163.stdout:7/132: unlink d11/d26/f29 0 2024-02-16T19:24:21.113 INFO:tasks.workunit.client.0.smithi040.stdout:6/868: dwrite d0/d1/d12/d3b/d48/dba/dde/fab [0,4194304] 0 2024-02-16T19:24:21.113 INFO:tasks.workunit.client.0.smithi040.stdout:6/869: creat d0/d1/d12/d3b/d48/dba/f11d x:0 0 0 2024-02-16T19:24:21.113 INFO:tasks.workunit.client.0.smithi040.stdout:6/870: write d0/d1/d6f/d90/db1/f113 [840304,105391] 0 2024-02-16T19:24:21.113 INFO:tasks.workunit.client.0.smithi040.stdout:6/871: write d0/d30/d4b/f57 [1747115,54449] 0 2024-02-16T19:24:21.113 INFO:tasks.workunit.client.0.smithi040.stdout:2/667: truncate d5/f3c 2556591 0 2024-02-16T19:24:21.113 INFO:tasks.workunit.client.0.smithi040.stdout:2/668: getdents d5/d8/dbe 0 2024-02-16T19:24:21.113 INFO:tasks.workunit.client.0.smithi040.stdout:2/669: chown d5/d8 11937 1 2024-02-16T19:24:21.114 INFO:tasks.workunit.client.1.smithi163.stdout:0/253: dwrite f8 [0,4194304] 0 2024-02-16T19:24:21.114 INFO:tasks.workunit.client.1.smithi163.stdout:0/254: dread d11/d22/f24 [0,4194304] 0 2024-02-16T19:24:21.114 INFO:tasks.workunit.client.1.smithi163.stdout:0/255: chown d11/d12/l13 890479 1 2024-02-16T19:24:21.115 INFO:tasks.workunit.client.0.smithi040.stdout:9/771: mknod d3/d4/d55/d6d/daf/cf5 0 2024-02-16T19:24:21.117 INFO:tasks.workunit.client.0.smithi040.stdout:4/600: symlink dd/de/d10/d36/dad/lbd 0 2024-02-16T19:24:21.120 INFO:tasks.workunit.client.1.smithi163.stdout:0/256: link d11/c51 d11/c5a 0 2024-02-16T19:24:21.120 INFO:tasks.workunit.client.1.smithi163.stdout:0/257: truncate d11/d21/f47 486093 0 2024-02-16T19:24:21.125 INFO:tasks.workunit.client.0.smithi040.stdout:6/872: creat d0/d1/d12/d3b/d48/d73/d70/d83/f11e x:0 0 0 2024-02-16T19:24:21.126 INFO:tasks.workunit.client.0.smithi040.stdout:9/772: write d3/d1c/fb9 [3485394,46398] 0 2024-02-16T19:24:21.126 INFO:tasks.workunit.client.0.smithi040.stdout:9/773: write d3/d4/d11/d71/fc2 [164542,62795] 0 2024-02-16T19:24:21.126 INFO:tasks.workunit.client.0.smithi040.stdout:9/774: write d3/d4/d55/d8a/d99/fb2 [734328,104004] 0 2024-02-16T19:24:21.126 INFO:tasks.workunit.client.0.smithi040.stdout:9/775: chown d3/d4/d55/l41 37 1 2024-02-16T19:24:21.127 INFO:tasks.workunit.client.0.smithi040.stdout:2/670: symlink d5/d7/d1e/d4c/d52/daa/dcd/ldb 0 2024-02-16T19:24:21.127 INFO:tasks.workunit.client.0.smithi040.stdout:2/671: readlink d5/d7/d1e/d55/d8d/la8 0 2024-02-16T19:24:21.127 INFO:tasks.workunit.client.0.smithi040.stdout:2/672: readlink d5/d7/d12/lb3 0 2024-02-16T19:24:21.128 INFO:tasks.workunit.client.0.smithi040.stdout:6/873: creat d0/d1/d12/d3b/d48/d73/dff/d111/f11f x:0 0 0 2024-02-16T19:24:21.131 INFO:tasks.workunit.client.0.smithi040.stdout:9/776: rmdir d3/d4/d55/d31/def 0 2024-02-16T19:24:21.132 INFO:tasks.workunit.client.1.smithi163.stdout:0/258: dread f8 [0,4194304] 0 2024-02-16T19:24:21.138 INFO:tasks.workunit.client.0.smithi040.stdout:9/777: symlink d3/d4/d11/d29/lf6 0 2024-02-16T19:24:21.139 INFO:tasks.workunit.client.1.smithi163.stdout:0/259: mkdir d11/d5b 0 2024-02-16T19:24:21.139 INFO:tasks.workunit.client.1.smithi163.stdout:0/260: chown d11/d4f 141626278 1 2024-02-16T19:24:21.141 INFO:tasks.workunit.client.0.smithi040.stdout:2/673: getdents d5/d7/d1e/d4c/d52 0 2024-02-16T19:24:21.141 INFO:tasks.workunit.client.0.smithi040.stdout:2/674: chown d5/d7/d1e/d55/l83 10640636 1 2024-02-16T19:24:21.141 INFO:tasks.workunit.client.0.smithi040.stdout:2/675: truncate d5/d7/d1e/d31/f43 4519537 0 2024-02-16T19:24:21.142 INFO:tasks.workunit.client.0.smithi040.stdout:4/601: dwrite dd/de/d10/f4a [4194304,4194304] 0 2024-02-16T19:24:21.143 INFO:tasks.workunit.client.0.smithi040.stdout:4/602: readlink dd/de/l8f 0 2024-02-16T19:24:21.143 INFO:tasks.workunit.client.0.smithi040.stdout:6/874: dwrite d0/d1/d12/f18 [0,4194304] 0 2024-02-16T19:24:21.146 INFO:tasks.workunit.client.1.smithi163.stdout:0/261: truncate d11/d22/f24 404012 0 2024-02-16T19:24:21.146 INFO:tasks.workunit.client.1.smithi163.stdout:0/262: fdatasync d11/d22/f42 0 2024-02-16T19:24:21.146 INFO:tasks.workunit.client.1.smithi163.stdout:0/263: creat d11/d12/d20/f5c x:0 0 0 2024-02-16T19:24:21.148 INFO:tasks.workunit.client.0.smithi040.stdout:2/676: mknod d5/d7/d1e/d55/dd4/cdc 0 2024-02-16T19:24:21.148 INFO:tasks.workunit.client.0.smithi040.stdout:2/677: read - d5/d7/d12/dca/fd1 zero size 2024-02-16T19:24:21.152 INFO:tasks.workunit.client.0.smithi040.stdout:4/603: creat dd/d58/d8c/fbe x:0 0 0 2024-02-16T19:24:21.158 INFO:tasks.workunit.client.0.smithi040.stdout:6/875: mkdir d0/d1/d12/d3b/d48/d4f/d120 0 2024-02-16T19:24:21.158 INFO:tasks.workunit.client.0.smithi040.stdout:6/876: truncate d0/d1/d12/d3b/f50 2224894 0 2024-02-16T19:24:21.162 INFO:tasks.workunit.client.0.smithi040.stdout:9/778: rename d3/d4/d55/d6d to d3/d4/d4a/db4/df7 0 2024-02-16T19:24:21.164 INFO:tasks.workunit.client.1.smithi163.stdout:0/264: unlink d11/d12/l13 0 2024-02-16T19:24:21.165 INFO:tasks.workunit.client.1.smithi163.stdout:0/265: fsync d11/f1f 0 2024-02-16T19:24:21.165 INFO:tasks.workunit.client.1.smithi163.stdout:0/266: fdatasync d11/d21/f47 0 2024-02-16T19:24:21.166 INFO:tasks.workunit.client.0.smithi040.stdout:2/678: creat d5/d7/d10/fdd x:0 0 0 2024-02-16T19:24:21.167 INFO:tasks.workunit.client.1.smithi163.stdout:0/267: symlink d11/d22/d3e/l5d 0 2024-02-16T19:24:21.167 INFO:tasks.workunit.client.1.smithi163.stdout:0/268: truncate d11/d22/f42 761509 0 2024-02-16T19:24:21.168 INFO:tasks.workunit.client.1.smithi163.stdout:0/269: dread d11/f33 [0,4194304] 0 2024-02-16T19:24:21.170 INFO:tasks.workunit.client.1.smithi163.stdout:0/270: dread d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:21.175 INFO:tasks.workunit.client.0.smithi040.stdout:4/604: symlink dd/de/d10/d36/d9e/d32/lbf 0 2024-02-16T19:24:21.177 INFO:tasks.workunit.client.0.smithi040.stdout:9/779: symlink d3/d4/d11/d29/df1/lf8 0 2024-02-16T19:24:21.177 INFO:tasks.workunit.client.0.smithi040.stdout:9/780: readlink d3/d1c/d73/lbf 0 2024-02-16T19:24:21.178 INFO:tasks.workunit.client.0.smithi040.stdout:2/679: creat d5/d5c/d65/fde x:0 0 0 2024-02-16T19:24:21.179 INFO:tasks.workunit.client.0.smithi040.stdout:2/680: write d5/d7/d12/f6a [774671,39661] 0 2024-02-16T19:24:21.182 INFO:tasks.workunit.client.0.smithi040.stdout:4/605: unlink dd/de/f2a 0 2024-02-16T19:24:21.183 INFO:tasks.workunit.client.0.smithi040.stdout:6/877: getdents d0/d30 0 2024-02-16T19:24:21.184 INFO:tasks.workunit.client.0.smithi040.stdout:9/781: creat d3/d4/d11/d29/d36/ff9 x:0 0 0 2024-02-16T19:24:21.184 INFO:tasks.workunit.client.0.smithi040.stdout:9/782: chown d3/d1c/d73/fb1 1262982 1 2024-02-16T19:24:21.184 INFO:tasks.workunit.client.0.smithi040.stdout:9/783: fdatasync d3/d4/d11/d71/da8/f58 0 2024-02-16T19:24:21.185 INFO:tasks.workunit.client.0.smithi040.stdout:6/878: fsync d0/d1/d6f/d90/db1/f113 0 2024-02-16T19:24:21.185 INFO:tasks.workunit.client.1.smithi163.stdout:0/271: dwrite fd [0,4194304] 0 2024-02-16T19:24:21.185 INFO:tasks.workunit.client.0.smithi040.stdout:2/681: unlink d5/d7/d12/d1b/f9b 0 2024-02-16T19:24:21.187 INFO:tasks.workunit.client.1.smithi163.stdout:0/272: symlink d11/d22/d3e/d3c/l5e 0 2024-02-16T19:24:21.189 INFO:tasks.workunit.client.1.smithi163.stdout:9/354: sync 2024-02-16T19:24:21.189 INFO:tasks.workunit.client.1.smithi163.stdout:3/193: sync 2024-02-16T19:24:21.189 INFO:tasks.workunit.client.1.smithi163.stdout:6/152: sync 2024-02-16T19:24:21.189 INFO:tasks.workunit.client.1.smithi163.stdout:6/153: write d0/d6/f8 [419193,101495] 0 2024-02-16T19:24:21.189 INFO:tasks.workunit.client.1.smithi163.stdout:4/119: sync 2024-02-16T19:24:21.190 INFO:tasks.workunit.client.0.smithi040.stdout:6/879: mknod d0/d37/de1/c121 0 2024-02-16T19:24:21.193 INFO:tasks.workunit.client.0.smithi040.stdout:2/682: rename d5/d7/fa1 to d5/d5c/fdf 0 2024-02-16T19:24:21.197 INFO:tasks.workunit.client.1.smithi163.stdout:9/355: mknod d5/de/d30/d3d/d49/d6e/c80 0 2024-02-16T19:24:21.197 INFO:tasks.workunit.client.1.smithi163.stdout:9/356: chown d5/de/c38 2603033 1 2024-02-16T19:24:21.197 INFO:tasks.workunit.client.1.smithi163.stdout:9/357: write d5/de/d30/d43/f4b [683139,24589] 0 2024-02-16T19:24:21.199 INFO:tasks.workunit.client.0.smithi040.stdout:9/784: getdents d3/d4/d55 0 2024-02-16T19:24:21.200 INFO:tasks.workunit.client.1.smithi163.stdout:4/120: dread d11/f16 [0,4194304] 0 2024-02-16T19:24:21.201 INFO:tasks.workunit.client.1.smithi163.stdout:3/194: mknod d0/dc/d1d/d29/d38/c4b 0 2024-02-16T19:24:21.202 INFO:tasks.workunit.client.0.smithi040.stdout:6/880: mknod d0/d1/d6f/d90/db1/c122 0 2024-02-16T19:24:21.202 INFO:tasks.workunit.client.0.smithi040.stdout:6/881: chown d0/d37/d4a 177 1 2024-02-16T19:24:21.204 INFO:tasks.workunit.client.0.smithi040.stdout:2/683: dread d5/d7/d12/f6a [0,4194304] 0 2024-02-16T19:24:21.207 INFO:tasks.workunit.client.0.smithi040.stdout:9/785: link d3/d4/d55/d31/fad d3/d4/d55/d70/dcc/ffa 0 2024-02-16T19:24:21.207 INFO:tasks.workunit.client.0.smithi040.stdout:9/786: creat d3/d1c/d73/ffb x:0 0 0 2024-02-16T19:24:21.207 INFO:tasks.workunit.client.0.smithi040.stdout:9/787: dread - d3/d4/d11/d29/d36/ff9 zero size 2024-02-16T19:24:21.208 INFO:tasks.workunit.client.1.smithi163.stdout:6/154: dwrite d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:21.208 INFO:tasks.workunit.client.1.smithi163.stdout:6/155: dread - d0/d18/f22 zero size 2024-02-16T19:24:21.209 INFO:tasks.workunit.client.1.smithi163.stdout:3/195: symlink d0/dc/d1d/d49/l4c 0 2024-02-16T19:24:21.210 INFO:tasks.workunit.client.1.smithi163.stdout:9/358: getdents d5/de/d72 0 2024-02-16T19:24:21.210 INFO:tasks.workunit.client.1.smithi163.stdout:9/359: write d5/de/f5b [4242275,87643] 0 2024-02-16T19:24:21.213 INFO:tasks.workunit.client.0.smithi040.stdout:2/684: mknod d5/d7/d1e/d55/da3/ce0 0 2024-02-16T19:24:21.214 INFO:tasks.workunit.client.1.smithi163.stdout:6/156: link d0/f5 d0/d6/f31 0 2024-02-16T19:24:21.214 INFO:tasks.workunit.client.1.smithi163.stdout:6/157: creat d0/d6/df/f32 x:0 0 0 2024-02-16T19:24:21.214 INFO:tasks.workunit.client.1.smithi163.stdout:6/158: chown d0/d6/f8 0 1 2024-02-16T19:24:21.216 INFO:tasks.workunit.client.1.smithi163.stdout:3/196: symlink d0/dc/d1d/d49/l4d 0 2024-02-16T19:24:21.217 INFO:tasks.workunit.client.0.smithi040.stdout:6/882: getdents d0/d37 0 2024-02-16T19:24:21.217 INFO:tasks.workunit.client.0.smithi040.stdout:6/883: write d0/d30/d5b/dbe/fd9 [1086543,6353] 0 2024-02-16T19:24:21.218 INFO:tasks.workunit.client.0.smithi040.stdout:6/884: dread - d0/d30/d32/dd2/d87/d8f/fb9 zero size 2024-02-16T19:24:21.218 INFO:tasks.workunit.client.0.smithi040.stdout:3/833: sync 2024-02-16T19:24:21.218 INFO:tasks.workunit.client.0.smithi040.stdout:5/695: sync 2024-02-16T19:24:21.219 INFO:tasks.workunit.client.0.smithi040.stdout:3/834: write d1/d4/dc4/fe0 [776116,17021] 0 2024-02-16T19:24:21.219 INFO:tasks.workunit.client.0.smithi040.stdout:5/696: write d1/d45/fce [61887,27577] 0 2024-02-16T19:24:21.219 INFO:tasks.workunit.client.0.smithi040.stdout:5/697: write d1/d62/d9a/fa2 [711809,83558] 0 2024-02-16T19:24:21.219 INFO:tasks.workunit.client.0.smithi040.stdout:0/852: sync 2024-02-16T19:24:21.219 INFO:tasks.workunit.client.0.smithi040.stdout:8/812: sync 2024-02-16T19:24:21.219 INFO:tasks.workunit.client.0.smithi040.stdout:7/652: sync 2024-02-16T19:24:21.220 INFO:tasks.workunit.client.1.smithi163.stdout:9/360: creat d5/de/d30/d31/d52/f81 x:0 0 0 2024-02-16T19:24:21.220 INFO:tasks.workunit.client.0.smithi040.stdout:7/653: dread d21/d29/d79/f80 [0,4194304] 0 2024-02-16T19:24:21.221 INFO:tasks.workunit.client.0.smithi040.stdout:7/654: write d21/d29/d79/f80 [464129,6935] 0 2024-02-16T19:24:21.221 INFO:tasks.workunit.client.0.smithi040.stdout:7/655: write d21/d4d/f5b [1470842,25253] 0 2024-02-16T19:24:21.222 INFO:tasks.workunit.client.0.smithi040.stdout:3/835: dread d1/d7/d30/f81 [0,4194304] 0 2024-02-16T19:24:21.224 INFO:tasks.workunit.client.1.smithi163.stdout:3/197: write d0/dc/d1c/d32/f3f [156451,110478] 0 2024-02-16T19:24:21.224 INFO:tasks.workunit.client.1.smithi163.stdout:3/198: creat d0/dc/d1d/d29/d38/f4e x:0 0 0 2024-02-16T19:24:21.225 INFO:tasks.workunit.client.0.smithi040.stdout:8/813: dread d6/d12/d51/d6e/db7/ddf/df8/d15/f61 [4194304,4194304] 0 2024-02-16T19:24:21.226 INFO:tasks.workunit.client.0.smithi040.stdout:6/885: creat d0/d1/d12/d3b/d48/dba/dde/dbd/f123 x:0 0 0 2024-02-16T19:24:21.228 INFO:tasks.workunit.client.0.smithi040.stdout:5/698: link d1/c38 d1/cd3 0 2024-02-16T19:24:21.228 INFO:tasks.workunit.client.0.smithi040.stdout:5/699: chown d1/da/d7e/db2/cd0 80 1 2024-02-16T19:24:21.228 INFO:tasks.workunit.client.1.smithi163.stdout:4/121: dwrite d11/f15 [0,4194304] 0 2024-02-16T19:24:21.230 INFO:tasks.workunit.client.0.smithi040.stdout:0/853: creat db/d2f/d1d/d21/d2e/d49/f124 x:0 0 0 2024-02-16T19:24:21.230 INFO:tasks.workunit.client.0.smithi040.stdout:0/854: fdatasync db/d2f/d1d/d21/d2e/d49/d5b/fd7 0 2024-02-16T19:24:21.236 INFO:tasks.workunit.client.1.smithi163.stdout:4/122: creat d11/d18/d1f/f20 x:0 0 0 2024-02-16T19:24:21.236 INFO:tasks.workunit.client.1.smithi163.stdout:4/123: readlink lc 0 2024-02-16T19:24:21.236 INFO:tasks.workunit.client.1.smithi163.stdout:4/124: write d11/d18/d1f/f20 [552435,47634] 0 2024-02-16T19:24:21.237 INFO:tasks.workunit.client.0.smithi040.stdout:1/708: sync 2024-02-16T19:24:21.237 INFO:tasks.workunit.client.0.smithi040.stdout:9/788: dwrite d3/d4/d11/d71/da8/fd2 [0,4194304] 0 2024-02-16T19:24:21.239 INFO:tasks.workunit.client.0.smithi040.stdout:2/685: rmdir d5/d5c/d65 39 2024-02-16T19:24:21.241 INFO:tasks.workunit.client.1.smithi163.stdout:3/199: dread d0/f1b [0,4194304] 0 2024-02-16T19:24:21.241 INFO:tasks.workunit.client.1.smithi163.stdout:6/159: dwrite d0/d6/d9/f2d [0,4194304] 0 2024-02-16T19:24:21.242 INFO:tasks.workunit.client.0.smithi040.stdout:3/836: truncate d1/d7/d36/f7b 4012622 0 2024-02-16T19:24:21.245 INFO:tasks.workunit.client.1.smithi163.stdout:4/125: write d11/d18/f1d [1037273,94005] 0 2024-02-16T19:24:21.246 INFO:tasks.workunit.client.0.smithi040.stdout:8/814: link d6/d2b/da3/d8b/f90 d6/d2b/ffb 0 2024-02-16T19:24:21.249 INFO:tasks.workunit.client.0.smithi040.stdout:6/886: mkdir d0/d1/d12/d3b/d48/dc0/d124 0 2024-02-16T19:24:21.249 INFO:tasks.workunit.client.0.smithi040.stdout:8/815: dread d6/f24 [0,4194304] 0 2024-02-16T19:24:21.250 INFO:tasks.workunit.client.0.smithi040.stdout:5/700: getdents d1/d62/d9a 0 2024-02-16T19:24:21.254 INFO:tasks.workunit.client.0.smithi040.stdout:3/837: write d1/dc/fbc [818465,66976] 0 2024-02-16T19:24:21.254 INFO:tasks.workunit.client.1.smithi163.stdout:0/273: rename d11/d22/d3e/d3c to d11/d21/d5f 0 2024-02-16T19:24:21.255 INFO:tasks.workunit.client.0.smithi040.stdout:3/838: dread d1/dc/d42/d4a/d57/d79/feb [0,4194304] 0 2024-02-16T19:24:21.256 INFO:tasks.workunit.client.1.smithi163.stdout:0/274: dread d11/d12/d20/f2f [0,4194304] 0 2024-02-16T19:24:21.257 INFO:tasks.workunit.client.0.smithi040.stdout:0/855: rename db/d2f/d23/d3b/l4c to db/d2f/d1d/d21/d2e/d85/l125 0 2024-02-16T19:24:21.257 INFO:tasks.workunit.client.0.smithi040.stdout:7/656: mknod d21/d7e/cdc 0 2024-02-16T19:24:21.257 INFO:tasks.workunit.client.0.smithi040.stdout:7/657: chown d21/d29/d79/f80 4 1 2024-02-16T19:24:21.258 INFO:tasks.workunit.client.0.smithi040.stdout:1/709: mkdir d17/d3c/d6b/de8 0 2024-02-16T19:24:21.258 INFO:tasks.workunit.client.0.smithi040.stdout:1/710: chown d17/d26/d5f/da6/fc6 1 1 2024-02-16T19:24:21.269 INFO:tasks.workunit.client.0.smithi040.stdout:9/789: mkdir d3/d4/d11/d29/d38/d46/d6b/d85/dfc 0 2024-02-16T19:24:21.270 INFO:tasks.workunit.client.0.smithi040.stdout:2/686: getdents d5/d7/d10/d40 0 2024-02-16T19:24:21.271 INFO:tasks.workunit.client.0.smithi040.stdout:4/606: write dd/de/d10/f8a [4097009,74787] 0 2024-02-16T19:24:21.271 INFO:tasks.workunit.client.0.smithi040.stdout:4/607: write dd/de/d45/f74 [1435484,87072] 0 2024-02-16T19:24:21.272 INFO:tasks.workunit.client.0.smithi040.stdout:4/608: truncate dd/d58/d8c/fbe 1011389 0 2024-02-16T19:24:21.272 INFO:tasks.workunit.client.1.smithi163.stdout:3/200: mknod d0/dc/d1c/d21/c4f 0 2024-02-16T19:24:21.277 INFO:tasks.workunit.client.0.smithi040.stdout:8/816: creat d6/d12/d51/d6e/db7/dc6/dcc/ffc x:0 0 0 2024-02-16T19:24:21.277 INFO:tasks.workunit.client.0.smithi040.stdout:8/817: truncate d6/f43 2475999 0 2024-02-16T19:24:21.280 INFO:tasks.workunit.client.0.smithi040.stdout:5/701: symlink d1/d62/ld4 0 2024-02-16T19:24:21.282 INFO:tasks.workunit.client.0.smithi040.stdout:6/887: dwrite d0/d1/d12/d3b/d48/dba/dde/dbd/f123 [0,4194304] 0 2024-02-16T19:24:21.282 INFO:tasks.workunit.client.0.smithi040.stdout:6/888: creat d0/d1/d12/d3b/d48/d73/dff/d111/f125 x:0 0 0 2024-02-16T19:24:21.282 INFO:tasks.workunit.client.1.smithi163.stdout:4/126: unlink ca 0 2024-02-16T19:24:21.283 INFO:tasks.workunit.client.1.smithi163.stdout:4/127: creat d11/f21 x:0 0 0 2024-02-16T19:24:21.286 INFO:tasks.workunit.client.1.smithi163.stdout:9/361: rename d5/de/d16/f25 to d5/f82 0 2024-02-16T19:24:21.286 INFO:tasks.workunit.client.1.smithi163.stdout:9/362: write d5/de/d30/d3d/f66 [318614,62959] 0 2024-02-16T19:24:21.286 INFO:tasks.workunit.client.1.smithi163.stdout:9/363: dread - d5/f1a zero size 2024-02-16T19:24:21.286 INFO:tasks.workunit.client.1.smithi163.stdout:9/364: write d5/de/d30/d3d/f6f [6197,47712] 0 2024-02-16T19:24:21.288 INFO:tasks.workunit.client.0.smithi040.stdout:7/658: rename d21/d4d/d77/d7a/d99 to d21/d4d/d77/d87/dc9/ddd 0 2024-02-16T19:24:21.290 INFO:tasks.workunit.client.0.smithi040.stdout:1/711: symlink d17/d3c/dde/le9 0 2024-02-16T19:24:21.290 INFO:tasks.workunit.client.0.smithi040.stdout:1/712: readlink d17/d26/d3d/l42 0 2024-02-16T19:24:21.290 INFO:tasks.workunit.client.0.smithi040.stdout:1/713: write d17/d29/d8b/dd2/fe7 [474860,43733] 0 2024-02-16T19:24:21.290 INFO:tasks.workunit.client.0.smithi040.stdout:1/714: creat d17/d26/d5f/d78/dac/fea x:0 0 0 2024-02-16T19:24:21.293 INFO:tasks.workunit.client.0.smithi040.stdout:9/790: link d3/d4/d11/d71/l78 d3/d4/d11/d71/lfd 0 2024-02-16T19:24:21.296 INFO:tasks.workunit.client.1.smithi163.stdout:4/128: symlink d11/l22 0 2024-02-16T19:24:21.296 INFO:tasks.workunit.client.0.smithi040.stdout:3/839: dwrite d1/dc/fd [0,4194304] 0 2024-02-16T19:24:21.299 INFO:tasks.workunit.client.0.smithi040.stdout:2/687: mknod d5/d7/d1e/d55/d8d/ce1 0 2024-02-16T19:24:21.299 INFO:tasks.workunit.client.0.smithi040.stdout:2/688: write d5/d8/d42/fd9 [857653,83314] 0 2024-02-16T19:24:21.299 INFO:tasks.workunit.client.0.smithi040.stdout:2/689: read d5/d7/d1e/d4c/d52/f9c [1411971,34249] 0 2024-02-16T19:24:21.301 INFO:tasks.workunit.client.0.smithi040.stdout:0/856: dwrite db/d25/d2c/d52/fc1 [0,4194304] 0 2024-02-16T19:24:21.301 INFO:tasks.workunit.client.1.smithi163.stdout:0/275: dwrite d11/d12/d20/f4e [0,4194304] 0 2024-02-16T19:24:21.301 INFO:tasks.workunit.client.1.smithi163.stdout:3/201: dwrite d0/dc/d1d/d29/f45 [0,4194304] 0 2024-02-16T19:24:21.302 INFO:tasks.workunit.client.1.smithi163.stdout:0/276: fdatasync d11/d12/f15 0 2024-02-16T19:24:21.302 INFO:tasks.workunit.client.1.smithi163.stdout:0/277: fdatasync fd 0 2024-02-16T19:24:21.302 INFO:tasks.workunit.client.1.smithi163.stdout:0/278: write d11/d21/d50/f52 [4688564,18799] 0 2024-02-16T19:24:21.302 INFO:tasks.workunit.client.1.smithi163.stdout:0/279: chown d11/d21/d5f 1625041004 1 2024-02-16T19:24:21.303 INFO:tasks.workunit.client.1.smithi163.stdout:9/365: mknod d5/de/d30/d31/d52/c83 0 2024-02-16T19:24:21.306 INFO:tasks.workunit.client.1.smithi163.stdout:4/129: write d11/d18/d1f/f20 [416929,39542] 0 2024-02-16T19:24:21.312 INFO:tasks.workunit.client.0.smithi040.stdout:8/818: creat d6/d2b/da3/ffd x:0 0 0 2024-02-16T19:24:21.312 INFO:tasks.workunit.client.0.smithi040.stdout:8/819: write d6/d12/f1f [425137,76756] 0 2024-02-16T19:24:21.320 INFO:tasks.workunit.client.1.smithi163.stdout:9/366: mkdir d5/de/d84 0 2024-02-16T19:24:21.322 INFO:tasks.workunit.client.0.smithi040.stdout:6/889: link d0/d37/de1/cea d0/d1/d12/d3b/d48/dc0/c126 0 2024-02-16T19:24:21.325 INFO:tasks.workunit.client.1.smithi163.stdout:9/367: link d5/ld d5/de/d16/d46/l85 0 2024-02-16T19:24:21.325 INFO:tasks.workunit.client.1.smithi163.stdout:9/368: write d5/de/d16/f3c [2580157,107719] 0 2024-02-16T19:24:21.325 INFO:tasks.workunit.client.1.smithi163.stdout:6/160: getdents d0/d6/df 0 2024-02-16T19:24:21.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/369: dread d5/de/d16/f2a [0,4194304] 0 2024-02-16T19:24:21.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/370: creat d5/de/d16/d47/f86 x:0 0 0 2024-02-16T19:24:21.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/371: creat d5/de/d16/d47/f87 x:0 0 0 2024-02-16T19:24:21.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/372: readlink d5/de/d16/d46/l85 0 2024-02-16T19:24:21.327 INFO:tasks.workunit.client.1.smithi163.stdout:9/373: write d5/de/d30/d3d/d49/f62 [34779,101895] 0 2024-02-16T19:24:21.328 INFO:tasks.workunit.client.1.smithi163.stdout:6/161: mkdir d0/d6/d9/d2e/d33 0 2024-02-16T19:24:21.335 INFO:tasks.workunit.client.1.smithi163.stdout:6/162: mknod d0/d6/d11/c34 0 2024-02-16T19:24:21.336 INFO:tasks.workunit.client.0.smithi040.stdout:7/659: dwrite d21/d4d/d7b/f8f [4194304,4194304] 0 2024-02-16T19:24:21.337 INFO:tasks.workunit.client.0.smithi040.stdout:1/715: dwrite d17/d43/d38/f72 [0,4194304] 0 2024-02-16T19:24:21.337 INFO:tasks.workunit.client.0.smithi040.stdout:9/791: dwrite d3/d4/d55/d8a/f8f [0,4194304] 0 2024-02-16T19:24:21.337 INFO:tasks.workunit.client.0.smithi040.stdout:9/792: chown d3/d4/d55/fc7 32534863 1 2024-02-16T19:24:21.339 INFO:tasks.workunit.client.1.smithi163.stdout:9/374: rename d5/f2e to d5/de/d30/d3d/f88 0 2024-02-16T19:24:21.340 INFO:tasks.workunit.client.1.smithi163.stdout:3/202: dwrite d0/dc/f3c [4194304,4194304] 0 2024-02-16T19:24:21.343 INFO:tasks.workunit.client.0.smithi040.stdout:2/690: dwrite d5/d7/d12/d1b/dae/fc4 [0,4194304] 0 2024-02-16T19:24:21.344 INFO:tasks.workunit.client.1.smithi163.stdout:6/163: rename d0/d6/d11/d1e/l26 to d0/d6/df/l35 0 2024-02-16T19:24:21.344 INFO:tasks.workunit.client.1.smithi163.stdout:6/164: fsync d0/d6/d11/d17/f2f 0 2024-02-16T19:24:21.345 INFO:tasks.workunit.client.1.smithi163.stdout:9/375: dread d5/f6 [0,4194304] 0 2024-02-16T19:24:21.346 INFO:tasks.workunit.client.1.smithi163.stdout:9/376: stat d5/de/d30/d3d/d49/f62 0 2024-02-16T19:24:21.346 INFO:tasks.workunit.client.1.smithi163.stdout:9/377: creat d5/de/d30/f89 x:0 0 0 2024-02-16T19:24:21.346 INFO:tasks.workunit.client.1.smithi163.stdout:0/280: dwrite f6 [0,4194304] 0 2024-02-16T19:24:21.348 INFO:tasks.workunit.client.0.smithi040.stdout:5/702: getdents d1/da/d1c 0 2024-02-16T19:24:21.348 INFO:tasks.workunit.client.0.smithi040.stdout:5/703: chown d1/da/d1c/dba/db1/f7b 5 1 2024-02-16T19:24:21.350 INFO:tasks.workunit.client.1.smithi163.stdout:6/165: truncate d0/d6/f31 2021125 0 2024-02-16T19:24:21.350 INFO:tasks.workunit.client.1.smithi163.stdout:6/166: write d0/dd/f12 [1095175,57701] 0 2024-02-16T19:24:21.351 INFO:tasks.workunit.client.0.smithi040.stdout:6/890: symlink d0/d1/d12/d3b/d48/dc0/d124/l127 0 2024-02-16T19:24:21.351 INFO:tasks.workunit.client.0.smithi040.stdout:6/891: chown d0/d1/d12/d3b/d48/dba/f11d 12359109 1 2024-02-16T19:24:21.354 INFO:tasks.workunit.client.1.smithi163.stdout:3/203: dread d0/dc/d1d/d29/f30 [0,4194304] 0 2024-02-16T19:24:21.355 INFO:tasks.workunit.client.0.smithi040.stdout:3/840: getdents d1/dc/d42/d5c 0 2024-02-16T19:24:21.355 INFO:tasks.workunit.client.0.smithi040.stdout:3/841: write d1/d7/d36/f45 [3096015,125324] 0 2024-02-16T19:24:21.358 INFO:tasks.workunit.client.1.smithi163.stdout:0/281: getdents d11/d4f 0 2024-02-16T19:24:21.358 INFO:tasks.workunit.client.1.smithi163.stdout:9/378: write d5/de/d30/d3d/d49/d50/f63 [3404219,113447] 0 2024-02-16T19:24:21.358 INFO:tasks.workunit.client.0.smithi040.stdout:8/820: dwrite d6/d12/d51/d6e/fa2 [0,4194304] 0 2024-02-16T19:24:21.358 INFO:tasks.workunit.client.0.smithi040.stdout:8/821: readlink d6/d64/d50/l8e 0 2024-02-16T19:24:21.359 INFO:tasks.workunit.client.1.smithi163.stdout:1/117: sync 2024-02-16T19:24:21.359 INFO:tasks.workunit.client.1.smithi163.stdout:7/133: sync 2024-02-16T19:24:21.359 INFO:tasks.workunit.client.1.smithi163.stdout:1/118: stat d7/f1d 0 2024-02-16T19:24:21.359 INFO:tasks.workunit.client.1.smithi163.stdout:8/123: sync 2024-02-16T19:24:21.359 INFO:tasks.workunit.client.1.smithi163.stdout:4/130: dwrite d11/d18/d1f/f20 [0,4194304] 0 2024-02-16T19:24:21.359 INFO:tasks.workunit.client.0.smithi040.stdout:7/660: getdents d21/d7e 0 2024-02-16T19:24:21.359 INFO:tasks.workunit.client.0.smithi040.stdout:7/661: dread - d21/d4d/d77/d87/dc9/fd2 zero size 2024-02-16T19:24:21.360 INFO:tasks.workunit.client.1.smithi163.stdout:2/276: sync 2024-02-16T19:24:21.362 INFO:tasks.workunit.client.1.smithi163.stdout:5/136: sync 2024-02-16T19:24:21.364 INFO:tasks.workunit.client.0.smithi040.stdout:5/704: write d1/da/d1c/d21/f36 [3987729,19945] 0 2024-02-16T19:24:21.364 INFO:tasks.workunit.client.0.smithi040.stdout:5/705: write d1/da/d1c/f26 [3732434,39509] 0 2024-02-16T19:24:21.364 INFO:tasks.workunit.client.0.smithi040.stdout:5/706: chown d1/d45/f49 1695929 1 2024-02-16T19:24:21.365 INFO:tasks.workunit.client.1.smithi163.stdout:3/204: mkdir d0/dc/d50 0 2024-02-16T19:24:21.365 INFO:tasks.workunit.client.1.smithi163.stdout:3/205: chown d0/d3e 9942497 1 2024-02-16T19:24:21.365 INFO:tasks.workunit.client.1.smithi163.stdout:3/206: stat d0/dc/f24 0 2024-02-16T19:24:21.369 INFO:tasks.workunit.client.1.smithi163.stdout:5/137: dread d0/d2/fc [0,4194304] 0 2024-02-16T19:24:21.370 INFO:tasks.workunit.client.1.smithi163.stdout:5/138: chown d0/d2/f7 951730324 1 2024-02-16T19:24:21.370 INFO:tasks.workunit.client.1.smithi163.stdout:5/139: creat d0/f2c x:0 0 0 2024-02-16T19:24:21.370 INFO:tasks.workunit.client.0.smithi040.stdout:6/892: write d0/d1/d6f/d90/fd3 [2378741,54138] 0 2024-02-16T19:24:21.370 INFO:tasks.workunit.client.0.smithi040.stdout:5/707: dread d1/da/d1c/dba/fc2 [0,4194304] 0 2024-02-16T19:24:21.370 INFO:tasks.workunit.client.0.smithi040.stdout:5/708: write d1/da/d51/f81 [1657405,96982] 0 2024-02-16T19:24:21.370 INFO:tasks.workunit.client.0.smithi040.stdout:5/709: fsync d1/da/d1c/d21/f8f 0 2024-02-16T19:24:21.370 INFO:tasks.workunit.client.0.smithi040.stdout:5/710: stat d1/da/d1c/c6a 0 2024-02-16T19:24:21.370 INFO:tasks.workunit.client.0.smithi040.stdout:5/711: chown d1/da/d1c 23 1 2024-02-16T19:24:21.372 INFO:tasks.workunit.client.1.smithi163.stdout:8/124: dread d0/d7/d19/f26 [0,4194304] 0 2024-02-16T19:24:21.372 INFO:tasks.workunit.client.1.smithi163.stdout:0/282: rmdir d11/d5b 0 2024-02-16T19:24:21.378 INFO:tasks.workunit.client.1.smithi163.stdout:9/379: mkdir d5/de/d30/d8a 0 2024-02-16T19:24:21.379 INFO:tasks.workunit.client.1.smithi163.stdout:7/134: getdents d11 0 2024-02-16T19:24:21.382 INFO:tasks.workunit.client.0.smithi040.stdout:1/716: creat d17/d43/d38/feb x:0 0 0 2024-02-16T19:24:21.384 INFO:tasks.workunit.client.1.smithi163.stdout:8/125: dread d0/f9 [0,4194304] 0 2024-02-16T19:24:21.384 INFO:tasks.workunit.client.1.smithi163.stdout:8/126: chown d0/d7/d18/f1e 67202 1 2024-02-16T19:24:21.385 INFO:tasks.workunit.client.1.smithi163.stdout:1/119: mknod d7/d20/c23 0 2024-02-16T19:24:21.385 INFO:tasks.workunit.client.1.smithi163.stdout:9/380: fsync d5/de/f5b 0 2024-02-16T19:24:21.386 INFO:tasks.workunit.client.0.smithi040.stdout:9/793: symlink d3/d4/d11/d29/d38/d46/lfe 0 2024-02-16T19:24:21.386 INFO:tasks.workunit.client.0.smithi040.stdout:9/794: write d3/d4/d11/d71/da8/f5c [582902,44806] 0 2024-02-16T19:24:21.390 INFO:tasks.workunit.client.0.smithi040.stdout:0/857: getdents db/de 0 2024-02-16T19:24:21.392 INFO:tasks.workunit.client.1.smithi163.stdout:4/131: mkdir d11/d18/d1f/d23 0 2024-02-16T19:24:21.393 INFO:tasks.workunit.client.1.smithi163.stdout:2/277: dwrite d2/f18 [0,4194304] 0 2024-02-16T19:24:21.393 INFO:tasks.workunit.client.1.smithi163.stdout:2/278: chown d2/d2a/d2d/d3c/l4f 23 1 2024-02-16T19:24:21.394 INFO:tasks.workunit.client.1.smithi163.stdout:2/279: dread d2/d2a/d39/f57 [0,4194304] 0 2024-02-16T19:24:21.394 INFO:tasks.workunit.client.1.smithi163.stdout:2/280: readlink d2/d2a/d2d/l55 0 2024-02-16T19:24:21.397 INFO:tasks.workunit.client.1.smithi163.stdout:0/283: dwrite d11/d12/d20/f5c [0,4194304] 0 2024-02-16T19:24:21.397 INFO:tasks.workunit.client.1.smithi163.stdout:0/284: stat d11/d12/c4d 0 2024-02-16T19:24:21.397 INFO:tasks.workunit.client.1.smithi163.stdout:0/285: write d11/f35 [800531,34794] 0 2024-02-16T19:24:21.397 INFO:tasks.workunit.client.1.smithi163.stdout:0/286: readlink d11/d22/d3e/l38 0 2024-02-16T19:24:21.398 INFO:tasks.workunit.client.1.smithi163.stdout:3/207: link d0/dc/l35 d0/dc/l51 0 2024-02-16T19:24:21.398 INFO:tasks.workunit.client.1.smithi163.stdout:3/208: chown d0/dc/d1c/d21/f42 0 1 2024-02-16T19:24:21.399 INFO:tasks.workunit.client.1.smithi163.stdout:3/209: dread - d0/dc/f28 zero size 2024-02-16T19:24:21.400 INFO:tasks.workunit.client.0.smithi040.stdout:2/691: mknod d5/d5c/ce2 0 2024-02-16T19:24:21.400 INFO:tasks.workunit.client.0.smithi040.stdout:2/692: write d5/d7/d89/fcf [961183,54276] 0 2024-02-16T19:24:21.402 INFO:tasks.workunit.client.1.smithi163.stdout:3/210: write d0/dc/f17 [2947375,83355] 0 2024-02-16T19:24:21.403 INFO:tasks.workunit.client.0.smithi040.stdout:3/842: link d1/d4/c8 d1/dc/d42/d58/de8/c10f 0 2024-02-16T19:24:21.403 INFO:tasks.workunit.client.0.smithi040.stdout:3/843: fsync d1/f3 0 2024-02-16T19:24:21.404 INFO:tasks.workunit.client.1.smithi163.stdout:5/140: link d0/c24 d0/c2d 0 2024-02-16T19:24:21.405 INFO:tasks.workunit.client.0.smithi040.stdout:8/822: creat d6/d12/d51/d6e/ffe x:0 0 0 2024-02-16T19:24:21.406 INFO:tasks.workunit.client.0.smithi040.stdout:8/823: stat d6/d2b/l54 0 2024-02-16T19:24:21.407 INFO:tasks.workunit.client.1.smithi163.stdout:7/135: creat d11/d24/d25/f2c x:0 0 0 2024-02-16T19:24:21.407 INFO:tasks.workunit.client.1.smithi163.stdout:7/136: creat d11/d24/d25/f2d x:0 0 0 2024-02-16T19:24:21.411 INFO:tasks.workunit.client.1.smithi163.stdout:8/127: symlink d0/dc/l29 0 2024-02-16T19:24:21.412 INFO:tasks.workunit.client.0.smithi040.stdout:1/717: dwrite d17/f25 [0,4194304] 0 2024-02-16T19:24:21.412 INFO:tasks.workunit.client.0.smithi040.stdout:0/858: dread db/d2f/d1d/d21/d2e/d49/fa3 [0,4194304] 0 2024-02-16T19:24:21.413 INFO:tasks.workunit.client.0.smithi040.stdout:1/718: stat d17/d26/d5f/d78/d7e/fbc 0 2024-02-16T19:24:21.413 INFO:tasks.workunit.client.0.smithi040.stdout:1/719: creat d17/d26/d54/fec x:0 0 0 2024-02-16T19:24:21.413 INFO:tasks.workunit.client.0.smithi040.stdout:1/720: dread - d17/d43/fdc zero size 2024-02-16T19:24:21.416 INFO:tasks.workunit.client.1.smithi163.stdout:7/137: dread d11/f1c [0,4194304] 0 2024-02-16T19:24:21.416 INFO:tasks.workunit.client.1.smithi163.stdout:5/141: dread d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:21.420 INFO:tasks.workunit.client.1.smithi163.stdout:2/281: link d2/d26/l32 d2/d2a/l5c 0 2024-02-16T19:24:21.421 INFO:tasks.workunit.client.1.smithi163.stdout:2/282: creat d2/d26/f5d x:0 0 0 2024-02-16T19:24:21.421 INFO:tasks.workunit.client.1.smithi163.stdout:2/283: dread - d2/d2a/d39/d3a/f5b zero size 2024-02-16T19:24:21.422 INFO:tasks.workunit.client.1.smithi163.stdout:6/167: getdents d0/d6/d11 0 2024-02-16T19:24:21.431 INFO:tasks.workunit.client.1.smithi163.stdout:2/284: dread d2/f23 [0,4194304] 0 2024-02-16T19:24:21.431 INFO:tasks.workunit.client.1.smithi163.stdout:2/285: chown d2/c1b 4998853 1 2024-02-16T19:24:21.432 INFO:tasks.workunit.client.1.smithi163.stdout:3/211: dwrite d0/dc/d1c/d32/f3f [0,4194304] 0 2024-02-16T19:24:21.432 INFO:tasks.workunit.client.1.smithi163.stdout:3/212: readlink d0/dc/d1d/d29/l2b 0 2024-02-16T19:24:21.433 INFO:tasks.workunit.client.0.smithi040.stdout:1/721: dwrite d17/d29/d8b/dd2/fe7 [0,4194304] 0 2024-02-16T19:24:21.434 INFO:tasks.workunit.client.0.smithi040.stdout:7/662: symlink d21/d29/dad/lde 0 2024-02-16T19:24:21.439 INFO:tasks.workunit.client.1.smithi163.stdout:6/168: dwrite d0/d6/f8 [0,4194304] 0 2024-02-16T19:24:21.444 INFO:tasks.workunit.client.0.smithi040.stdout:6/893: truncate d0/d1/d12/d3b/d48/d73/d70/fa6 4369215 0 2024-02-16T19:24:21.447 INFO:tasks.workunit.client.0.smithi040.stdout:5/712: mkdir d1/da/dd5 0 2024-02-16T19:24:21.449 INFO:tasks.workunit.client.1.smithi163.stdout:8/128: mknod d0/d7/c2a 0 2024-02-16T19:24:21.456 INFO:tasks.workunit.client.1.smithi163.stdout:9/381: getdents d5 0 2024-02-16T19:24:21.456 INFO:tasks.workunit.client.1.smithi163.stdout:9/382: chown d5/f2b 367935275 1 2024-02-16T19:24:21.456 INFO:tasks.workunit.client.1.smithi163.stdout:9/383: readlink d5/de/d30/d31/l42 0 2024-02-16T19:24:21.457 INFO:tasks.workunit.client.0.smithi040.stdout:1/722: dwrite d17/d26/d3d/f56 [4194304,4194304] 0 2024-02-16T19:24:21.457 INFO:tasks.workunit.client.0.smithi040.stdout:1/723: write d17/d29/d8b/fd6 [773990,5309] 0 2024-02-16T19:24:21.457 INFO:tasks.workunit.client.0.smithi040.stdout:1/724: dread - d17/d43/fc9 zero size 2024-02-16T19:24:21.461 INFO:tasks.workunit.client.1.smithi163.stdout:4/132: getdents d11 0 2024-02-16T19:24:21.463 INFO:tasks.workunit.client.0.smithi040.stdout:3/844: unlink d1/d4/lb7 0 2024-02-16T19:24:21.463 INFO:tasks.workunit.client.0.smithi040.stdout:3/845: write d1/d4/d6e/dc2/dce/fd8 [861651,97622] 0 2024-02-16T19:24:21.465 INFO:tasks.workunit.client.0.smithi040.stdout:8/824: mknod d6/d12/d93/cff 0 2024-02-16T19:24:21.465 INFO:tasks.workunit.client.1.smithi163.stdout:2/286: rmdir d2/d2a/d2d/d3c 39 2024-02-16T19:24:21.467 INFO:tasks.workunit.client.1.smithi163.stdout:3/213: unlink d0/dc/d1d/d29/d38/f4e 0 2024-02-16T19:24:21.468 INFO:tasks.workunit.client.0.smithi040.stdout:0/859: symlink db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/d103/l126 0 2024-02-16T19:24:21.469 INFO:tasks.workunit.client.0.smithi040.stdout:5/713: dwrite d1/f4c [0,4194304] 0 2024-02-16T19:24:21.469 INFO:tasks.workunit.client.1.smithi163.stdout:6/169: dwrite d0/dd/f13 [4194304,4194304] 0 2024-02-16T19:24:21.471 INFO:tasks.workunit.client.0.smithi040.stdout:6/894: dwrite d0/d1/d12/d3b/d48/fc8 [0,4194304] 0 2024-02-16T19:24:21.474 INFO:tasks.workunit.client.0.smithi040.stdout:7/663: rmdir d21/d8b 39 2024-02-16T19:24:21.476 INFO:tasks.workunit.client.1.smithi163.stdout:0/287: getdents d11/d21 0 2024-02-16T19:24:21.480 INFO:tasks.workunit.client.1.smithi163.stdout:8/129: rename d0/d7/l14 to d0/d7/d17/l2b 0 2024-02-16T19:24:21.481 INFO:tasks.workunit.client.1.smithi163.stdout:9/384: truncate d5/de/d30/d3d/d49/d6e/f73 282817 0 2024-02-16T19:24:21.483 INFO:tasks.workunit.client.0.smithi040.stdout:2/693: getdents d5/d7/d12/d1b 0 2024-02-16T19:24:21.483 INFO:tasks.workunit.client.0.smithi040.stdout:2/694: chown d5/d8 80701281 1 2024-02-16T19:24:21.484 INFO:tasks.workunit.client.1.smithi163.stdout:9/385: dread d5/de/d30/d3d/f3b [0,4194304] 0 2024-02-16T19:24:21.485 INFO:tasks.workunit.client.0.smithi040.stdout:3/846: creat d1/dc/d42/d4a/f110 x:0 0 0 2024-02-16T19:24:21.485 INFO:tasks.workunit.client.0.smithi040.stdout:3/847: dread - d1/d4/d6e/fcd zero size 2024-02-16T19:24:21.485 INFO:tasks.workunit.client.0.smithi040.stdout:3/848: stat d1/dc/d35 0 2024-02-16T19:24:21.487 INFO:tasks.workunit.client.1.smithi163.stdout:4/133: creat d11/d18/f24 x:0 0 0 2024-02-16T19:24:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:21 smithi163 ceph-mon[36452]: pgmap v249: 65 pgs: 65 active+clean; 1.4 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 86 MiB/s rd, 110 MiB/s wr, 138 op/s 2024-02-16T19:24:21.489 INFO:tasks.workunit.client.0.smithi040.stdout:8/825: symlink d6/d12/d51/d6e/db9/l100 0 2024-02-16T19:24:21.490 INFO:tasks.workunit.client.1.smithi163.stdout:2/287: link d2/fc d2/d2a/d2d/f5e 0 2024-02-16T19:24:21.491 INFO:tasks.workunit.client.0.smithi040.stdout:0/860: dwrite db/d25/f61 [4194304,4194304] 0 2024-02-16T19:24:21.492 INFO:tasks.workunit.client.0.smithi040.stdout:0/861: write db/d2f/d1d/d21/d2e/d49/d5b/d67/f92 [3746881,38302] 0 2024-02-16T19:24:21.493 INFO:tasks.workunit.client.1.smithi163.stdout:3/214: mknod d0/dc/d1d/d29/d38/d46/c52 0 2024-02-16T19:24:21.493 INFO:tasks.workunit.client.1.smithi163.stdout:7/138: getdents d11/d24/d25 0 2024-02-16T19:24:21.493 INFO:tasks.workunit.client.1.smithi163.stdout:7/139: fsync fc 0 2024-02-16T19:24:21.496 INFO:tasks.workunit.client.1.smithi163.stdout:6/170: mknod d0/d6/df/c36 0 2024-02-16T19:24:21.497 INFO:tasks.workunit.client.0.smithi040.stdout:5/714: mkdir d1/da/d1c/dba/dd6 0 2024-02-16T19:24:21.497 INFO:tasks.workunit.client.0.smithi040.stdout:5/715: dread - d1/d3c/f5d zero size 2024-02-16T19:24:21.498 INFO:tasks.workunit.client.1.smithi163.stdout:0/288: symlink d11/d12/d20/l60 0 2024-02-16T19:24:21.499 INFO:tasks.workunit.client.0.smithi040.stdout:6/895: creat d0/d30/d32/dd2/d87/d8f/f128 x:0 0 0 2024-02-16T19:24:21.500 INFO:tasks.workunit.client.1.smithi163.stdout:2/288: dread d2/f18 [0,4194304] 0 2024-02-16T19:24:21.500 INFO:tasks.workunit.client.1.smithi163.stdout:9/386: dwrite d5/de/d16/d47/f4c [0,4194304] 0 2024-02-16T19:24:21.501 INFO:tasks.workunit.client.0.smithi040.stdout:7/664: rmdir d21/d8b 39 2024-02-16T19:24:21.501 INFO:tasks.workunit.client.1.smithi163.stdout:9/387: write d5/f1e [43178,70717] 0 2024-02-16T19:24:21.503 INFO:tasks.workunit.client.1.smithi163.stdout:4/134: symlink d11/d18/l25 0 2024-02-16T19:24:21.504 INFO:tasks.workunit.client.0.smithi040.stdout:2/695: link d5/d7/d1e/d31/f7d d5/d7/d10/dbc/fe3 0 2024-02-16T19:24:21.507 INFO:tasks.workunit.client.1.smithi163.stdout:3/215: link d0/l3 d0/dc/d1d/d49/l53 0 2024-02-16T19:24:21.507 INFO:tasks.workunit.client.1.smithi163.stdout:6/171: dread d0/d6/d9/d15/d1c/f2b [0,4194304] 0 2024-02-16T19:24:21.508 INFO:tasks.workunit.client.1.smithi163.stdout:7/140: symlink d11/d24/l2e 0 2024-02-16T19:24:21.509 INFO:tasks.workunit.client.1.smithi163.stdout:0/289: read fd [1148963,11379] 0 2024-02-16T19:24:21.509 INFO:tasks.workunit.client.0.smithi040.stdout:1/725: rmdir d17/d29/d8b/d93 39 2024-02-16T19:24:21.512 INFO:tasks.workunit.client.0.smithi040.stdout:8/826: mknod d6/d64/d50/d69/c101 0 2024-02-16T19:24:21.512 INFO:tasks.workunit.client.0.smithi040.stdout:8/827: chown d6/d12/l2a 31391 1 2024-02-16T19:24:21.512 INFO:tasks.workunit.client.0.smithi040.stdout:8/828: chown d6/d2b/f9a 251 1 2024-02-16T19:24:21.512 INFO:tasks.workunit.client.0.smithi040.stdout:8/829: stat d6/d12/d51/d6e/db7/ddf/df8/d15/f61 0 2024-02-16T19:24:21.515 INFO:tasks.workunit.client.1.smithi163.stdout:3/216: dread d0/dc/d1c/f3b [0,4194304] 0 2024-02-16T19:24:21.517 INFO:tasks.workunit.client.1.smithi163.stdout:8/130: rmdir d0 39 2024-02-16T19:24:21.520 INFO:tasks.workunit.client.1.smithi163.stdout:4/135: link f2 d11/f26 0 2024-02-16T19:24:21.529 INFO:tasks.workunit.client.1.smithi163.stdout:7/141: symlink d11/l2f 0 2024-02-16T19:24:21.531 INFO:tasks.workunit.client.1.smithi163.stdout:0/290: mkdir d11/d22/d61 0 2024-02-16T19:24:21.532 INFO:tasks.workunit.client.0.smithi040.stdout:5/716: creat d1/da/d1c/da9/fd7 x:0 0 0 2024-02-16T19:24:21.533 INFO:tasks.workunit.client.1.smithi163.stdout:3/217: truncate d0/dc/d1c/f3b 51867 0 2024-02-16T19:24:21.533 INFO:tasks.workunit.client.1.smithi163.stdout:3/218: chown d0/dc/d1c/f23 1639979 1 2024-02-16T19:24:21.533 INFO:tasks.workunit.client.1.smithi163.stdout:3/219: chown d0/dc/l35 774 1 2024-02-16T19:24:21.537 INFO:tasks.workunit.client.1.smithi163.stdout:8/131: mkdir d0/d24/d2c 0 2024-02-16T19:24:21.539 INFO:tasks.workunit.client.0.smithi040.stdout:7/665: rmdir d21/d4d/d7d 39 2024-02-16T19:24:21.542 INFO:tasks.workunit.client.1.smithi163.stdout:3/220: write d0/dc/d1c/d21/f42 [3970084,101034] 0 2024-02-16T19:24:21.543 INFO:tasks.workunit.client.1.smithi163.stdout:4/136: rename d11/f15 to d11/d18/d1f/f27 0 2024-02-16T19:24:21.544 INFO:tasks.workunit.client.1.smithi163.stdout:0/291: truncate d11/d22/d3e/f57 1008858 0 2024-02-16T19:24:21.545 INFO:tasks.workunit.client.0.smithi040.stdout:2/696: creat d5/d7/d1e/d4c/fe4 x:0 0 0 2024-02-16T19:24:21.546 INFO:tasks.workunit.client.0.smithi040.stdout:2/697: write d5/d7/d10/d40/fa9 [5189672,74463] 0 2024-02-16T19:24:21.548 INFO:tasks.workunit.client.0.smithi040.stdout:1/726: truncate d17/d26/f53 1545015 0 2024-02-16T19:24:21.548 INFO:tasks.workunit.client.0.smithi040.stdout:1/727: chown d17/d43/d38/feb 1583800 1 2024-02-16T19:24:21.548 INFO:tasks.workunit.client.0.smithi040.stdout:1/728: fdatasync d17/f44 0 2024-02-16T19:24:21.549 INFO:tasks.workunit.client.1.smithi163.stdout:3/221: symlink d0/d3e/l54 0 2024-02-16T19:24:21.551 INFO:tasks.workunit.client.0.smithi040.stdout:8/830: unlink d6/d2b/d44/cd6 0 2024-02-16T19:24:21.551 INFO:tasks.workunit.client.0.smithi040.stdout:8/831: stat d6/d64/fb1 0 2024-02-16T19:24:21.551 INFO:tasks.workunit.client.0.smithi040.stdout:8/832: fdatasync d6/f3f 0 2024-02-16T19:24:21.551 INFO:tasks.workunit.client.0.smithi040.stdout:8/833: chown d6/d12/d93/fda 2 1 2024-02-16T19:24:21.551 INFO:tasks.workunit.client.0.smithi040.stdout:8/834: readlink d6/d2b/ld2 0 2024-02-16T19:24:21.552 INFO:tasks.workunit.client.1.smithi163.stdout:4/137: link cb d11/d18/c28 0 2024-02-16T19:24:21.553 INFO:tasks.workunit.client.1.smithi163.stdout:3/222: dread d0/f39 [0,4194304] 0 2024-02-16T19:24:21.553 INFO:tasks.workunit.client.1.smithi163.stdout:0/292: creat d11/f62 x:0 0 0 2024-02-16T19:24:21.554 INFO:tasks.workunit.client.0.smithi040.stdout:0/862: rmdir db/d2f/d23/d6f 39 2024-02-16T19:24:21.556 INFO:tasks.workunit.client.1.smithi163.stdout:0/293: mknod d11/d21/d50/c63 0 2024-02-16T19:24:21.557 INFO:tasks.workunit.client.0.smithi040.stdout:7/666: creat d21/d29/d79/fdf x:0 0 0 2024-02-16T19:24:21.559 INFO:tasks.workunit.client.1.smithi163.stdout:0/294: link d11/d22/l4c d11/d22/l64 0 2024-02-16T19:24:21.560 INFO:tasks.workunit.client.0.smithi040.stdout:4/609: sync 2024-02-16T19:24:21.560 INFO:tasks.workunit.client.1.smithi163.stdout:4/138: dread d11/f26 [0,4194304] 0 2024-02-16T19:24:21.560 INFO:tasks.workunit.client.1.smithi163.stdout:4/139: readlink lc 0 2024-02-16T19:24:21.562 INFO:tasks.workunit.client.0.smithi040.stdout:2/698: symlink d5/d7/d89/le5 0 2024-02-16T19:24:21.562 INFO:tasks.workunit.client.0.smithi040.stdout:2/699: creat d5/d7/d12/fe6 x:0 0 0 2024-02-16T19:24:21.564 INFO:tasks.workunit.client.1.smithi163.stdout:4/140: mkdir d11/d18/d1f/d29 0 2024-02-16T19:24:21.566 INFO:tasks.workunit.client.0.smithi040.stdout:1/729: dread d17/d43/d38/f7b [0,4194304] 0 2024-02-16T19:24:21.566 INFO:tasks.workunit.client.0.smithi040.stdout:8/835: mkdir d6/d64/d50/dd5/d102 0 2024-02-16T19:24:21.566 INFO:tasks.workunit.client.0.smithi040.stdout:8/836: getdents d6/d12/d51/d6e/db9/dc9 0 2024-02-16T19:24:21.566 INFO:tasks.workunit.client.0.smithi040.stdout:8/837: chown d6/d12/d51/d6e/db7/ddf/df8/d15/leb 597921306 1 2024-02-16T19:24:21.572 INFO:tasks.workunit.client.0.smithi040.stdout:6/896: write d0/d1/d12/d3b/d48/f3c [573533,70215] 0 2024-02-16T19:24:21.572 INFO:tasks.workunit.client.0.smithi040.stdout:0/863: dread db/d25/fdf [0,4194304] 0 2024-02-16T19:24:21.572 INFO:tasks.workunit.client.1.smithi163.stdout:4/141: unlink l0 0 2024-02-16T19:24:21.572 INFO:tasks.workunit.client.1.smithi163.stdout:4/142: chown d11/c13 1549156 1 2024-02-16T19:24:21.572 INFO:tasks.workunit.client.1.smithi163.stdout:6/172: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:21.577 INFO:tasks.workunit.client.0.smithi040.stdout:8/838: symlink d6/d2b/da3/d8b/l103 0 2024-02-16T19:24:21.577 INFO:tasks.workunit.client.0.smithi040.stdout:8/839: fsync d6/d12/d51/d6e/db7/ddf/fee 0 2024-02-16T19:24:21.578 INFO:tasks.workunit.client.1.smithi163.stdout:0/295: dread d11/d12/d20/f5c [0,4194304] 0 2024-02-16T19:24:21.578 INFO:tasks.workunit.client.1.smithi163.stdout:0/296: chown d11/d12/d20/f3a 0 1 2024-02-16T19:24:21.579 INFO:tasks.workunit.client.1.smithi163.stdout:8/132: dwrite d0/f20 [4194304,4194304] 0 2024-02-16T19:24:21.579 INFO:tasks.workunit.client.0.smithi040.stdout:5/717: dwrite d1/da/d51/f8b [4194304,4194304] 0 2024-02-16T19:24:21.579 INFO:tasks.workunit.client.1.smithi163.stdout:7/142: dwrite d11/f12 [0,4194304] 0 2024-02-16T19:24:21.580 INFO:tasks.workunit.client.0.smithi040.stdout:6/897: rename d0/d1/d6f/d90/c108 to d0/d1/d12/d3b/d48/dba/dde/dcd/c129 0 2024-02-16T19:24:21.580 INFO:tasks.workunit.client.0.smithi040.stdout:6/898: fsync d0/d1/d6f/fdd 0 2024-02-16T19:24:21.581 INFO:tasks.workunit.client.0.smithi040.stdout:6/899: read d0/d30/d5b/f91 [686515,15456] 0 2024-02-16T19:24:21.582 INFO:tasks.workunit.client.0.smithi040.stdout:0/864: mknod db/de/dc0/c127 0 2024-02-16T19:24:21.583 INFO:tasks.workunit.client.1.smithi163.stdout:0/297: creat d11/d21/d5f/f65 x:0 0 0 2024-02-16T19:24:21.583 INFO:tasks.workunit.client.1.smithi163.stdout:0/298: dread - d11/d21/d5f/f65 zero size 2024-02-16T19:24:21.585 INFO:tasks.workunit.client.1.smithi163.stdout:6/173: rmdir d0/dd 39 2024-02-16T19:24:21.587 INFO:tasks.workunit.client.0.smithi040.stdout:5/718: rmdir d1/da/d1c/dba/db1 39 2024-02-16T19:24:21.587 INFO:tasks.workunit.client.0.smithi040.stdout:5/719: creat d1/da/d1c/da7/fd8 x:0 0 0 2024-02-16T19:24:21.587 INFO:tasks.workunit.client.0.smithi040.stdout:5/720: fsync d1/da/d51/f6c 0 2024-02-16T19:24:21.588 INFO:tasks.workunit.client.0.smithi040.stdout:4/610: write dd/de/d10/f2e [1496819,70405] 0 2024-02-16T19:24:21.588 INFO:tasks.workunit.client.0.smithi040.stdout:4/611: fsync dd/de/f54 0 2024-02-16T19:24:21.588 INFO:tasks.workunit.client.0.smithi040.stdout:4/612: dread - dd/d58/fbb zero size 2024-02-16T19:24:21.589 INFO:tasks.workunit.client.1.smithi163.stdout:6/174: dread d0/dd/f23 [0,4194304] 0 2024-02-16T19:24:21.589 INFO:tasks.workunit.client.0.smithi040.stdout:0/865: rename db/d2f/d23/d6f/cf7 to db/d2f/d1d/d21/d2e/d49/d5b/d67/c128 0 2024-02-16T19:24:21.590 INFO:tasks.workunit.client.0.smithi040.stdout:0/866: read db/d25/f11b [192476,64761] 0 2024-02-16T19:24:21.590 INFO:tasks.workunit.client.1.smithi163.stdout:8/133: truncate d0/d7/d19/f1b 3811680 0 2024-02-16T19:24:21.590 INFO:tasks.workunit.client.1.smithi163.stdout:3/223: dwrite d0/dc/d1d/d29/f33 [0,4194304] 0 2024-02-16T19:24:21.594 INFO:tasks.workunit.client.0.smithi040.stdout:8/840: read d6/d12/d51/d6e/db7/dc6/fdd [693771,44859] 0 2024-02-16T19:24:21.594 INFO:tasks.workunit.client.0.smithi040.stdout:7/667: dwrite d21/d7e/faa [0,4194304] 0 2024-02-16T19:24:21.595 INFO:tasks.workunit.client.1.smithi163.stdout:7/143: mkdir d11/d30 0 2024-02-16T19:24:21.596 INFO:tasks.workunit.client.1.smithi163.stdout:7/144: write fc [411009,62045] 0 2024-02-16T19:24:21.597 INFO:tasks.workunit.client.1.smithi163.stdout:6/175: mknod d0/d6/c37 0 2024-02-16T19:24:21.599 INFO:tasks.workunit.client.1.smithi163.stdout:3/224: link d0/dc/d1c/f23 d0/dc/d1c/d32/f55 0 2024-02-16T19:24:21.600 INFO:tasks.workunit.client.0.smithi040.stdout:7/668: truncate d21/f48 2928702 0 2024-02-16T19:24:21.600 INFO:tasks.workunit.client.1.smithi163.stdout:7/145: mknod d11/d24/d25/c31 0 2024-02-16T19:24:21.601 INFO:tasks.workunit.client.1.smithi163.stdout:7/146: readlink d11/l2f 0 2024-02-16T19:24:21.601 INFO:tasks.workunit.client.1.smithi163.stdout:7/147: chown d11/c17 3367 1 2024-02-16T19:24:21.602 INFO:tasks.workunit.client.1.smithi163.stdout:6/176: creat d0/d6/d9/d2e/d33/f38 x:0 0 0 2024-02-16T19:24:21.603 INFO:tasks.workunit.client.1.smithi163.stdout:7/148: creat d11/d30/f32 x:0 0 0 2024-02-16T19:24:21.603 INFO:tasks.workunit.client.1.smithi163.stdout:7/149: creat d11/d26/f33 x:0 0 0 2024-02-16T19:24:21.603 INFO:tasks.workunit.client.1.smithi163.stdout:7/150: truncate d11/d24/d25/f2c 299207 0 2024-02-16T19:24:21.603 INFO:tasks.workunit.client.1.smithi163.stdout:7/151: fdatasync d11/f12 0 2024-02-16T19:24:21.603 INFO:tasks.workunit.client.1.smithi163.stdout:7/152: fsync d11/d24/d25/f2d 0 2024-02-16T19:24:21.604 INFO:tasks.workunit.client.0.smithi040.stdout:7/669: getdents d21/d4d/d77/d7a 0 2024-02-16T19:24:21.604 INFO:tasks.workunit.client.1.smithi163.stdout:8/134: read d0/dc/f22 [1876942,86898] 0 2024-02-16T19:24:21.607 INFO:tasks.workunit.client.0.smithi040.stdout:2/700: dwrite d5/d7/d12/f15 [4194304,4194304] 0 2024-02-16T19:24:21.608 INFO:tasks.workunit.client.1.smithi163.stdout:8/135: rename d0/fe to d0/dc/f2d 0 2024-02-16T19:24:21.611 INFO:tasks.workunit.client.0.smithi040.stdout:0/867: dread db/de/f38 [0,4194304] 0 2024-02-16T19:24:21.615 INFO:tasks.workunit.client.0.smithi040.stdout:8/841: write d6/d12/d22/f86 [2377067,29438] 0 2024-02-16T19:24:21.615 INFO:tasks.workunit.client.0.smithi040.stdout:8/842: fsync d6/d64/ffa 0 2024-02-16T19:24:21.615 INFO:tasks.workunit.client.0.smithi040.stdout:8/843: write d6/d2b/da3/f53 [588287,71500] 0 2024-02-16T19:24:21.616 INFO:tasks.workunit.client.0.smithi040.stdout:0/868: mknod db/de/d6a/df0/c129 0 2024-02-16T19:24:21.616 INFO:tasks.workunit.client.0.smithi040.stdout:0/869: truncate db/d25/d2c/f6d 1249928 0 2024-02-16T19:24:21.616 INFO:tasks.workunit.client.0.smithi040.stdout:0/870: fdatasync db/d25/d2c/f5a 0 2024-02-16T19:24:21.616 INFO:tasks.workunit.client.0.smithi040.stdout:0/871: fsync db/d2f/d1d/d21/d2e/d49/f124 0 2024-02-16T19:24:21.617 INFO:tasks.workunit.client.1.smithi163.stdout:4/143: getdents d11/d18/d1f 0 2024-02-16T19:24:21.623 INFO:tasks.workunit.client.0.smithi040.stdout:1/730: dwrite d17/d3c/d71/fa4 [0,4194304] 0 2024-02-16T19:24:21.626 INFO:tasks.workunit.client.1.smithi163.stdout:7/153: read fb [2842081,98480] 0 2024-02-16T19:24:21.627 INFO:tasks.workunit.client.0.smithi040.stdout:1/731: symlink d17/d26/d5f/d78/dac/led 0 2024-02-16T19:24:21.629 INFO:tasks.workunit.client.1.smithi163.stdout:7/154: mkdir d11/d24/d34 0 2024-02-16T19:24:21.631 INFO:tasks.workunit.client.0.smithi040.stdout:0/872: dread db/d2f/d1d/d21/d2e/d49/fa9 [0,4194304] 0 2024-02-16T19:24:21.633 INFO:tasks.workunit.client.0.smithi040.stdout:0/873: unlink db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/lfb 0 2024-02-16T19:24:21.634 INFO:tasks.workunit.client.0.smithi040.stdout:0/874: dread - db/de/d6a/df0/d106/f123 zero size 2024-02-16T19:24:21.635 INFO:tasks.workunit.client.0.smithi040.stdout:0/875: creat db/d2f/d23/d3b/dd5/f12a x:0 0 0 2024-02-16T19:24:21.635 INFO:tasks.workunit.client.0.smithi040.stdout:6/900: dwrite d0/d1/d12/d3b/d48/d4f/f68 [0,4194304] 0 2024-02-16T19:24:21.638 INFO:tasks.workunit.client.0.smithi040.stdout:0/876: unlink db/de/d6a/f6e 0 2024-02-16T19:24:21.638 INFO:tasks.workunit.client.0.smithi040.stdout:0/877: creat db/d2f/f12b x:0 0 0 2024-02-16T19:24:21.640 INFO:tasks.workunit.client.0.smithi040.stdout:6/901: mknod d0/d1/d12/d3b/d48/d4f/c12a 0 2024-02-16T19:24:21.640 INFO:tasks.workunit.client.0.smithi040.stdout:6/902: write d0/d37/de1/f94 [646910,1032] 0 2024-02-16T19:24:21.641 INFO:tasks.workunit.client.0.smithi040.stdout:0/878: symlink db/de/l12c 0 2024-02-16T19:24:21.643 INFO:tasks.workunit.client.0.smithi040.stdout:0/879: mkdir db/de/d12d 0 2024-02-16T19:24:21.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:21 smithi040 ceph-mon[29201]: pgmap v249: 65 pgs: 65 active+clean; 1.4 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 86 MiB/s rd, 110 MiB/s wr, 138 op/s 2024-02-16T19:24:21.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/903: dread d0/d1/d12/d3b/d48/d73/d70/fc9 [0,4194304] 0 2024-02-16T19:24:21.653 INFO:tasks.workunit.client.0.smithi040.stdout:6/904: dread - d0/d30/d32/dd2/d87/d8f/fb9 zero size 2024-02-16T19:24:21.662 INFO:tasks.workunit.client.0.smithi040.stdout:6/905: dread d0/f2e [4194304,4194304] 0 2024-02-16T19:24:21.664 INFO:tasks.workunit.client.0.smithi040.stdout:1/732: dread d17/d29/d8b/d93/fdd [0,4194304] 0 2024-02-16T19:24:21.664 INFO:tasks.workunit.client.0.smithi040.stdout:1/733: creat d17/d26/d5f/d78/fee x:0 0 0 2024-02-16T19:24:21.668 INFO:tasks.workunit.client.0.smithi040.stdout:0/880: dread db/d2f/f3e [4194304,4194304] 0 2024-02-16T19:24:21.670 INFO:tasks.workunit.client.1.smithi163.stdout:6/177: dwrite d0/d6/f1a [4194304,4194304] 0 2024-02-16T19:24:21.671 INFO:tasks.workunit.client.0.smithi040.stdout:0/881: mknod db/d2f/d23/d3b/c12e 0 2024-02-16T19:24:21.671 INFO:tasks.workunit.client.0.smithi040.stdout:0/882: dread - db/d2f/d1d/f10f zero size 2024-02-16T19:24:21.671 INFO:tasks.workunit.client.1.smithi163.stdout:0/299: dwrite d11/d12/d20/f4e [0,4194304] 0 2024-02-16T19:24:21.671 INFO:tasks.workunit.client.1.smithi163.stdout:0/300: fsync d11/d22/f2d 0 2024-02-16T19:24:21.672 INFO:tasks.workunit.client.0.smithi040.stdout:8/844: dwrite d6/d12/d51/d6e/db7/ddf/df8/d15/f85 [0,4194304] 0 2024-02-16T19:24:21.673 INFO:tasks.workunit.client.1.smithi163.stdout:8/136: dwrite d0/dc/f2d [4194304,4194304] 0 2024-02-16T19:24:21.673 INFO:tasks.workunit.client.1.smithi163.stdout:3/225: dwrite d0/dc/d1c/f20 [0,4194304] 0 2024-02-16T19:24:21.674 INFO:tasks.workunit.client.1.smithi163.stdout:7/155: fsync d11/d26/f33 0 2024-02-16T19:24:21.675 INFO:tasks.workunit.client.0.smithi040.stdout:4/613: dwrite dd/de/d10/d36/d9e/f15 [0,4194304] 0 2024-02-16T19:24:21.675 INFO:tasks.workunit.client.0.smithi040.stdout:4/614: fsync dd/de/db0/fb2 0 2024-02-16T19:24:21.677 INFO:tasks.workunit.client.0.smithi040.stdout:5/721: dwrite d1/da/d1c/d21/f8f [0,4194304] 0 2024-02-16T19:24:21.677 INFO:tasks.workunit.client.0.smithi040.stdout:7/670: dwrite d21/d4d/d7b/f94 [0,4194304] 0 2024-02-16T19:24:21.677 INFO:tasks.workunit.client.1.smithi163.stdout:4/144: dwrite d11/f21 [0,4194304] 0 2024-02-16T19:24:21.685 INFO:tasks.workunit.client.0.smithi040.stdout:8/845: getdents d6/d12/d51/d6e/db7/dc6 0 2024-02-16T19:24:21.687 INFO:tasks.workunit.client.1.smithi163.stdout:0/301: creat d11/d22/d3e/f66 x:0 0 0 2024-02-16T19:24:21.690 INFO:tasks.workunit.client.1.smithi163.stdout:6/178: getdents d0/d6/d9/d2e/d33 0 2024-02-16T19:24:21.693 INFO:tasks.workunit.client.0.smithi040.stdout:5/722: creat d1/d62/d9a/fd9 x:0 0 0 2024-02-16T19:24:21.693 INFO:tasks.workunit.client.0.smithi040.stdout:5/723: chown d1/da/d1c/d21/daa 0 1 2024-02-16T19:24:21.695 INFO:tasks.workunit.client.0.smithi040.stdout:6/906: dwrite d0/d30/d5b/dbe/fed [0,4194304] 0 2024-02-16T19:24:21.695 INFO:tasks.workunit.client.1.smithi163.stdout:8/137: mkdir d0/d7/d18/d2e 0 2024-02-16T19:24:21.696 INFO:tasks.workunit.client.0.smithi040.stdout:4/615: dwrite dd/d58/f8d [0,4194304] 0 2024-02-16T19:24:21.698 INFO:tasks.workunit.client.0.smithi040.stdout:1/734: dwrite d17/d26/f7d [0,4194304] 0 2024-02-16T19:24:21.698 INFO:tasks.workunit.client.0.smithi040.stdout:1/735: chown d17/d29/l47 37 1 2024-02-16T19:24:21.698 INFO:tasks.workunit.client.0.smithi040.stdout:1/736: dread - d17/d26/d5f/d78/dac/fbe zero size 2024-02-16T19:24:21.698 INFO:tasks.workunit.client.0.smithi040.stdout:0/883: rmdir db/d2f/d1d/d21/d2e/d49 39 2024-02-16T19:24:21.703 INFO:tasks.workunit.client.0.smithi040.stdout:1/737: dread d17/d2d/f3f [0,4194304] 0 2024-02-16T19:24:21.703 INFO:tasks.workunit.client.0.smithi040.stdout:1/738: creat d17/d43/fef x:0 0 0 2024-02-16T19:24:21.705 INFO:tasks.workunit.client.0.smithi040.stdout:7/671: link d21/d29/d79/dbc/fc0 d21/d8b/fe0 0 2024-02-16T19:24:21.706 INFO:tasks.workunit.client.0.smithi040.stdout:8/846: dwrite d6/d2b/d44/f49 [0,4194304] 0 2024-02-16T19:24:21.707 INFO:tasks.workunit.client.1.smithi163.stdout:7/156: mknod d11/d24/c35 0 2024-02-16T19:24:21.711 INFO:tasks.workunit.client.0.smithi040.stdout:6/907: mkdir d0/d1/d6f/d90/db1/d12b 0 2024-02-16T19:24:21.715 INFO:tasks.workunit.client.1.smithi163.stdout:4/145: symlink d11/d18/d1f/d23/l2a 0 2024-02-16T19:24:21.717 INFO:tasks.workunit.client.1.smithi163.stdout:0/302: mkdir d11/d12/d67 0 2024-02-16T19:24:21.718 INFO:tasks.workunit.client.1.smithi163.stdout:0/303: dread - d11/f62 zero size 2024-02-16T19:24:21.718 INFO:tasks.workunit.client.1.smithi163.stdout:0/304: rename d11/d22/d3e to d11/d22/d3e/d68 22 2024-02-16T19:24:21.719 INFO:tasks.workunit.client.1.smithi163.stdout:8/138: getdents d0/d7/d19 0 2024-02-16T19:24:21.721 INFO:tasks.workunit.client.0.smithi040.stdout:0/884: symlink db/d2f/d1d/l12f 0 2024-02-16T19:24:21.722 INFO:tasks.workunit.client.1.smithi163.stdout:4/146: link c7 d11/d18/d1f/d23/c2b 0 2024-02-16T19:24:21.724 INFO:tasks.workunit.client.1.smithi163.stdout:8/139: creat d0/d7/d19/f2f x:0 0 0 2024-02-16T19:24:21.724 INFO:tasks.workunit.client.0.smithi040.stdout:1/739: symlink d17/d26/d54/lf0 0 2024-02-16T19:24:21.726 INFO:tasks.workunit.client.1.smithi163.stdout:4/147: rename lc to d11/d18/d1f/d23/l2c 0 2024-02-16T19:24:21.726 INFO:tasks.workunit.client.1.smithi163.stdout:4/148: truncate f2 2367405 0 2024-02-16T19:24:21.731 INFO:tasks.workunit.client.1.smithi163.stdout:4/149: unlink d11/d18/d1f/d23/c2b 0 2024-02-16T19:24:21.731 INFO:tasks.workunit.client.1.smithi163.stdout:4/150: write f10 [543149,37181] 0 2024-02-16T19:24:21.731 INFO:tasks.workunit.client.1.smithi163.stdout:4/151: write d11/d18/f1d [3318480,24667] 0 2024-02-16T19:24:21.731 INFO:tasks.workunit.client.0.smithi040.stdout:5/724: dwrite d1/da/fc8 [0,4194304] 0 2024-02-16T19:24:21.731 INFO:tasks.workunit.client.0.smithi040.stdout:5/725: write d1/da/d1c/d73/fbe [395160,17557] 0 2024-02-16T19:24:21.731 INFO:tasks.workunit.client.0.smithi040.stdout:6/908: write d0/d1/f9c [1740126,107525] 0 2024-02-16T19:24:21.731 INFO:tasks.workunit.client.0.smithi040.stdout:6/909: read d0/d30/d5b/f91 [423567,122710] 0 2024-02-16T19:24:21.732 INFO:tasks.workunit.client.1.smithi163.stdout:8/140: mkdir d0/d7/d19/d1f/d30 0 2024-02-16T19:24:21.732 INFO:tasks.workunit.client.0.smithi040.stdout:0/885: rmdir db/d25/d2c/d52 39 2024-02-16T19:24:21.732 INFO:tasks.workunit.client.0.smithi040.stdout:9/795: sync 2024-02-16T19:24:21.735 INFO:tasks.workunit.client.1.smithi163.stdout:1/120: sync 2024-02-16T19:24:21.735 INFO:tasks.workunit.client.1.smithi163.stdout:5/142: sync 2024-02-16T19:24:21.735 INFO:tasks.workunit.client.1.smithi163.stdout:5/143: creat d0/d22/f2e x:0 0 0 2024-02-16T19:24:21.736 INFO:tasks.workunit.client.0.smithi040.stdout:1/740: mknod d17/d26/d3d/cf1 0 2024-02-16T19:24:21.736 INFO:tasks.workunit.client.0.smithi040.stdout:1/741: creat d17/db5/ff2 x:0 0 0 2024-02-16T19:24:21.736 INFO:tasks.workunit.client.0.smithi040.stdout:1/742: chown d17/d43/fc9 7433800 1 2024-02-16T19:24:21.736 INFO:tasks.workunit.client.0.smithi040.stdout:1/743: chown d17/d26/d3d/fb6 199 1 2024-02-16T19:24:21.737 INFO:tasks.workunit.client.1.smithi163.stdout:8/141: mkdir d0/d24/d31 0 2024-02-16T19:24:21.737 INFO:tasks.workunit.client.0.smithi040.stdout:3/849: sync 2024-02-16T19:24:21.738 INFO:tasks.workunit.client.0.smithi040.stdout:5/726: symlink d1/da/d1c/d73/lda 0 2024-02-16T19:24:21.740 INFO:tasks.workunit.client.1.smithi163.stdout:9/388: sync 2024-02-16T19:24:21.740 INFO:tasks.workunit.client.1.smithi163.stdout:2/289: sync 2024-02-16T19:24:21.744 INFO:tasks.workunit.client.1.smithi163.stdout:8/142: rename d0/d24/d31 to d0/d7/d19/d1f/d32 0 2024-02-16T19:24:21.747 INFO:tasks.workunit.client.0.smithi040.stdout:0/886: rename db/d25/d2c/fcb to db/d2f/d1d/d21/d2e/d85/f130 0 2024-02-16T19:24:21.749 INFO:tasks.workunit.client.1.smithi163.stdout:9/389: mkdir d5/de/d16/d46/d8b 0 2024-02-16T19:24:21.749 INFO:tasks.workunit.client.1.smithi163.stdout:9/390: dread - d5/f1a zero size 2024-02-16T19:24:21.751 INFO:tasks.workunit.client.0.smithi040.stdout:8/847: dwrite d6/f3f [0,4194304] 0 2024-02-16T19:24:21.751 INFO:tasks.workunit.client.0.smithi040.stdout:7/672: dwrite d21/d8b/fb0 [0,4194304] 0 2024-02-16T19:24:21.752 INFO:tasks.workunit.client.1.smithi163.stdout:6/179: dwrite d0/d6/d9/d15/d1c/f2b [0,4194304] 0 2024-02-16T19:24:21.752 INFO:tasks.workunit.client.1.smithi163.stdout:6/180: fsync d0/d6/d9/f2d 0 2024-02-16T19:24:21.752 INFO:tasks.workunit.client.1.smithi163.stdout:6/181: read d0/dd/f12 [71213,3573] 0 2024-02-16T19:24:21.752 INFO:tasks.workunit.client.1.smithi163.stdout:2/290: creat d2/d2a/f5f x:0 0 0 2024-02-16T19:24:21.753 INFO:tasks.workunit.client.0.smithi040.stdout:1/744: creat d17/d26/d4c/dab/dd3/ff3 x:0 0 0 2024-02-16T19:24:21.756 INFO:tasks.workunit.client.0.smithi040.stdout:3/850: creat d1/d8c/d8e/dde/f111 x:0 0 0 2024-02-16T19:24:21.757 INFO:tasks.workunit.client.0.smithi040.stdout:8/848: write d6/d12/d51/d6e/db9/dcf/fa7 [310847,110960] 0 2024-02-16T19:24:21.757 INFO:tasks.workunit.client.1.smithi163.stdout:8/143: mknod d0/d7/c33 0 2024-02-16T19:24:21.758 INFO:tasks.workunit.client.1.smithi163.stdout:5/144: dread d0/d2/f7 [0,4194304] 0 2024-02-16T19:24:21.758 INFO:tasks.workunit.client.1.smithi163.stdout:5/145: chown d0/ce 28366500 1 2024-02-16T19:24:21.758 INFO:tasks.workunit.client.0.smithi040.stdout:5/727: creat d1/da/d1c/dba/db1/fdb x:0 0 0 2024-02-16T19:24:21.759 INFO:tasks.workunit.client.1.smithi163.stdout:9/391: truncate d5/de/d30/d3d/d49/d50/f63 5598977 0 2024-02-16T19:24:21.762 INFO:tasks.workunit.client.0.smithi040.stdout:5/728: dread d1/d62/d9a/fa2 [0,4194304] 0 2024-02-16T19:24:21.763 INFO:tasks.workunit.client.0.smithi040.stdout:0/887: rename db/de/lba to db/d2f/d1d/d21/d2e/l131 0 2024-02-16T19:24:21.764 INFO:tasks.workunit.client.0.smithi040.stdout:7/673: dread d21/f76 [0,4194304] 0 2024-02-16T19:24:21.765 INFO:tasks.workunit.client.1.smithi163.stdout:6/182: getdents d0/d6/df 0 2024-02-16T19:24:21.765 INFO:tasks.workunit.client.1.smithi163.stdout:6/183: fdatasync d0/d6/d9/d2e/d33/f38 0 2024-02-16T19:24:21.769 INFO:tasks.workunit.client.1.smithi163.stdout:2/291: write d2/f16 [436113,5967] 0 2024-02-16T19:24:21.775 INFO:tasks.workunit.client.0.smithi040.stdout:1/745: symlink d17/d43/d92/daa/lf4 0 2024-02-16T19:24:21.776 INFO:tasks.workunit.client.1.smithi163.stdout:8/144: read d0/d7/d19/f26 [484650,48700] 0 2024-02-16T19:24:21.777 INFO:tasks.workunit.client.0.smithi040.stdout:8/849: link d6/d12/d51/d6e/db7/ddf/df8/d15/leb d6/d12/d51/db6/l104 0 2024-02-16T19:24:21.777 INFO:tasks.workunit.client.0.smithi040.stdout:8/850: write d6/d12/d51/d6e/db9/dcf/fa7 [1028113,109960] 0 2024-02-16T19:24:21.777 INFO:tasks.workunit.client.0.smithi040.stdout:8/851: chown d6/d12/d51/d6e/db7/ddf/df8/d15/l1b 70465 1 2024-02-16T19:24:21.778 INFO:tasks.workunit.client.1.smithi163.stdout:1/121: dwrite d7/f1d [0,4194304] 0 2024-02-16T19:24:21.779 INFO:tasks.workunit.client.0.smithi040.stdout:5/729: symlink d1/da/d1c/da7/ldc 0 2024-02-16T19:24:21.779 INFO:tasks.workunit.client.0.smithi040.stdout:5/730: truncate d1/da/d1c/da9/fd7 149199 0 2024-02-16T19:24:21.779 INFO:tasks.workunit.client.0.smithi040.stdout:5/731: truncate d1/da/d7e/f86 296567 0 2024-02-16T19:24:21.779 INFO:tasks.workunit.client.0.smithi040.stdout:5/732: creat d1/d62/fdd x:0 0 0 2024-02-16T19:24:21.779 INFO:tasks.workunit.client.0.smithi040.stdout:5/733: chown d1/d62/l6b 247716880 1 2024-02-16T19:24:21.780 INFO:tasks.workunit.client.0.smithi040.stdout:3/851: rmdir d1/dc/d42 39 2024-02-16T19:24:21.781 INFO:tasks.workunit.client.0.smithi040.stdout:0/888: mkdir db/d25/d34/d5e/d132 0 2024-02-16T19:24:21.781 INFO:tasks.workunit.client.0.smithi040.stdout:8/852: dread d6/d2b/f9a [0,4194304] 0 2024-02-16T19:24:21.782 INFO:tasks.workunit.client.0.smithi040.stdout:8/853: chown d6/d2b/ld2 1053378 1 2024-02-16T19:24:21.784 INFO:tasks.workunit.client.1.smithi163.stdout:6/184: rmdir d0 39 2024-02-16T19:24:21.785 INFO:tasks.workunit.client.1.smithi163.stdout:0/305: dwrite d11/d22/f2d [0,4194304] 0 2024-02-16T19:24:21.785 INFO:tasks.workunit.client.1.smithi163.stdout:4/152: dwrite d11/f16 [4194304,4194304] 0 2024-02-16T19:24:21.785 INFO:tasks.workunit.client.1.smithi163.stdout:4/153: stat d11/f17 0 2024-02-16T19:24:21.785 INFO:tasks.workunit.client.0.smithi040.stdout:5/734: mkdir d1/da/d1c/dba/db1/dde 0 2024-02-16T19:24:21.786 INFO:tasks.workunit.client.1.smithi163.stdout:8/145: rename d0/d7/fd to d0/d7/d19/d1f/d32/f34 0 2024-02-16T19:24:21.787 INFO:tasks.workunit.client.1.smithi163.stdout:4/154: read f2 [396528,115705] 0 2024-02-16T19:24:21.787 INFO:tasks.workunit.client.0.smithi040.stdout:8/854: mknod d6/d12/d51/d6e/db9/dcf/c105 0 2024-02-16T19:24:21.789 INFO:tasks.workunit.client.0.smithi040.stdout:1/746: getdents d17/d2d 0 2024-02-16T19:24:21.794 INFO:tasks.workunit.client.1.smithi163.stdout:1/122: creat d7/d20/f24 x:0 0 0 2024-02-16T19:24:21.795 INFO:tasks.workunit.client.0.smithi040.stdout:5/735: mkdir d1/da/d1c/dba/db1/dde/ddf 0 2024-02-16T19:24:21.797 INFO:tasks.workunit.client.1.smithi163.stdout:6/185: mknod d0/d6/d11/d17/c39 0 2024-02-16T19:24:21.800 INFO:tasks.workunit.client.0.smithi040.stdout:3/852: read d1/d7/f2b [1023792,91768] 0 2024-02-16T19:24:21.800 INFO:tasks.workunit.client.0.smithi040.stdout:3/853: creat d1/d8c/d8e/dde/f112 x:0 0 0 2024-02-16T19:24:21.801 INFO:tasks.workunit.client.0.smithi040.stdout:1/747: rename d17/f94 to d17/d26/d5f/d78/d7e/ddb/ff5 0 2024-02-16T19:24:21.801 INFO:tasks.workunit.client.0.smithi040.stdout:1/748: write d17/d26/d3d/fb0 [221519,87298] 0 2024-02-16T19:24:21.803 INFO:tasks.workunit.client.0.smithi040.stdout:0/889: dread db/d2f/d1d/d21/d2e/d49/f87 [0,4194304] 0 2024-02-16T19:24:21.803 INFO:tasks.workunit.client.0.smithi040.stdout:0/890: fdatasync db/d25/f61 0 2024-02-16T19:24:21.803 INFO:tasks.workunit.client.0.smithi040.stdout:0/891: creat db/d2f/d1d/d21/d2e/d83/f133 x:0 0 0 2024-02-16T19:24:21.803 INFO:tasks.workunit.client.0.smithi040.stdout:9/796: dwrite d3/d4/f2c [0,4194304] 0 2024-02-16T19:24:21.804 INFO:tasks.workunit.client.0.smithi040.stdout:6/910: dwrite d0/d1/d6f/d90/f9b [4194304,4194304] 0 2024-02-16T19:24:21.804 INFO:tasks.workunit.client.0.smithi040.stdout:6/911: fsync d0/d30/d32/f21 0 2024-02-16T19:24:21.804 INFO:tasks.workunit.client.0.smithi040.stdout:9/797: read d3/d4/d4a/db4/df7/daf/fe1 [2082002,105182] 0 2024-02-16T19:24:21.805 INFO:tasks.workunit.client.1.smithi163.stdout:5/146: dwrite d0/d2/fa [0,4194304] 0 2024-02-16T19:24:21.805 INFO:tasks.workunit.client.1.smithi163.stdout:4/155: dread fd [0,4194304] 0 2024-02-16T19:24:21.807 INFO:tasks.workunit.client.0.smithi040.stdout:8/855: dread d6/d12/d51/d6e/db7/ddf/df8/f7b [0,4194304] 0 2024-02-16T19:24:21.810 INFO:tasks.workunit.client.1.smithi163.stdout:7/157: dwrite d11/d24/d25/f2c [0,4194304] 0 2024-02-16T19:24:21.811 INFO:tasks.workunit.client.1.smithi163.stdout:0/306: creat d11/d22/d61/f69 x:0 0 0 2024-02-16T19:24:21.812 INFO:tasks.workunit.client.1.smithi163.stdout:0/307: getdents d11/d21/d5f/d59 0 2024-02-16T19:24:21.818 INFO:tasks.workunit.client.0.smithi040.stdout:0/892: write db/d25/d34/d5e/fd0 [474721,46812] 0 2024-02-16T19:24:21.819 INFO:tasks.workunit.client.0.smithi040.stdout:7/674: dwrite d21/d4d/d77/d87/d9b/fa7 [4194304,4194304] 0 2024-02-16T19:24:21.819 INFO:tasks.workunit.client.1.smithi163.stdout:9/392: dwrite d5/de/d30/d43/f60 [0,4194304] 0 2024-02-16T19:24:21.822 INFO:tasks.workunit.client.0.smithi040.stdout:3/854: mknod d1/d8c/d8e/dde/c113 0 2024-02-16T19:24:21.824 INFO:tasks.workunit.client.0.smithi040.stdout:1/749: symlink d17/d43/d38/lf6 0 2024-02-16T19:24:21.828 INFO:tasks.workunit.client.1.smithi163.stdout:5/147: mknod d0/c2f 0 2024-02-16T19:24:21.834 INFO:tasks.workunit.client.0.smithi040.stdout:3/855: dread d1/dc/d35/d53/d64/d65/f98 [0,4194304] 0 2024-02-16T19:24:21.835 INFO:tasks.workunit.client.0.smithi040.stdout:9/798: link d3/d4/d11/d29/f52 d3/d4/d4a/fff 0 2024-02-16T19:24:21.837 INFO:tasks.workunit.client.1.smithi163.stdout:0/308: mkdir d11/d12/d67/d6a 0 2024-02-16T19:24:21.837 INFO:tasks.workunit.client.1.smithi163.stdout:0/309: fsync d11/d22/f2d 0 2024-02-16T19:24:21.837 INFO:tasks.workunit.client.1.smithi163.stdout:0/310: read - d11/f62 zero size 2024-02-16T19:24:21.837 INFO:tasks.workunit.client.0.smithi040.stdout:8/856: stat d6/d12/d51/d6e/db7/ddf/df8/d1d/c2d 0 2024-02-16T19:24:21.844 INFO:tasks.workunit.client.0.smithi040.stdout:5/736: dread d1/da/d51/f85 [0,4194304] 0 2024-02-16T19:24:21.844 INFO:tasks.workunit.client.1.smithi163.stdout:8/146: dwrite d0/dc/f22 [0,4194304] 0 2024-02-16T19:24:21.846 INFO:tasks.workunit.client.1.smithi163.stdout:9/393: link d5/de/d16/f2a d5/de/d16/d46/f8c 0 2024-02-16T19:24:21.846 INFO:tasks.workunit.client.1.smithi163.stdout:1/123: dwrite d7/f1d [0,4194304] 0 2024-02-16T19:24:21.847 INFO:tasks.workunit.client.0.smithi040.stdout:6/912: dwrite d0/d30/d32/dd2/d87/d8f/f128 [0,4194304] 0 2024-02-16T19:24:21.848 INFO:tasks.workunit.client.1.smithi163.stdout:5/148: creat d0/d2/f30 x:0 0 0 2024-02-16T19:24:21.849 INFO:tasks.workunit.client.0.smithi040.stdout:6/913: read d0/d1/d6f/fda [807675,16372] 0 2024-02-16T19:24:21.849 INFO:tasks.workunit.client.0.smithi040.stdout:6/914: write d0/d1/d12/d3b/d79/fa2 [4794894,113554] 0 2024-02-16T19:24:21.850 INFO:tasks.workunit.client.1.smithi163.stdout:4/156: dwrite d11/d18/f1d [0,4194304] 0 2024-02-16T19:24:21.856 INFO:tasks.workunit.client.0.smithi040.stdout:0/893: creat db/de/d12d/f134 x:0 0 0 2024-02-16T19:24:21.859 INFO:tasks.workunit.client.1.smithi163.stdout:1/124: stat d7/l1b 0 2024-02-16T19:24:21.860 INFO:tasks.workunit.client.1.smithi163.stdout:5/149: write d0/d2/fc [264333,106004] 0 2024-02-16T19:24:21.860 INFO:tasks.workunit.client.1.smithi163.stdout:1/125: write d7/f1c [54073,113508] 0 2024-02-16T19:24:21.860 INFO:tasks.workunit.client.1.smithi163.stdout:1/126: dread - d7/d20/f24 zero size 2024-02-16T19:24:21.863 INFO:tasks.workunit.client.0.smithi040.stdout:7/675: creat d21/d7e/fe1 x:0 0 0 2024-02-16T19:24:21.864 INFO:tasks.workunit.client.1.smithi163.stdout:5/150: dread d0/d22/f23 [0,4194304] 0 2024-02-16T19:24:21.864 INFO:tasks.workunit.client.1.smithi163.stdout:5/151: dread - d0/d22/f2e zero size 2024-02-16T19:24:21.864 INFO:tasks.workunit.client.1.smithi163.stdout:4/157: mkdir d11/d2d 0 2024-02-16T19:24:21.864 INFO:tasks.workunit.client.1.smithi163.stdout:4/158: write d11/d18/f1d [5183519,27743] 0 2024-02-16T19:24:21.865 INFO:tasks.workunit.client.0.smithi040.stdout:1/750: symlink d17/d29/d8b/d93/dcf/lf7 0 2024-02-16T19:24:21.866 INFO:tasks.workunit.client.1.smithi163.stdout:8/147: truncate d0/d7/d19/f1b 3547500 0 2024-02-16T19:24:21.868 INFO:tasks.workunit.client.0.smithi040.stdout:8/857: symlink d6/d2b/l106 0 2024-02-16T19:24:21.869 INFO:tasks.workunit.client.1.smithi163.stdout:1/127: dread f2 [0,4194304] 0 2024-02-16T19:24:21.875 INFO:tasks.workunit.client.1.smithi163.stdout:5/152: truncate d0/d2/f21 2776594 0 2024-02-16T19:24:21.877 INFO:tasks.workunit.client.1.smithi163.stdout:1/128: mknod d7/d20/d21/c25 0 2024-02-16T19:24:21.877 INFO:tasks.workunit.client.1.smithi163.stdout:1/129: creat d7/f26 x:0 0 0 2024-02-16T19:24:21.878 INFO:tasks.workunit.client.0.smithi040.stdout:5/737: dwrite d1/da/f2a [0,4194304] 0 2024-02-16T19:24:21.880 INFO:tasks.workunit.client.1.smithi163.stdout:5/153: mkdir d0/d2/d31 0 2024-02-16T19:24:21.880 INFO:tasks.workunit.client.1.smithi163.stdout:5/154: chown d0/d22/d28 26 1 2024-02-16T19:24:21.880 INFO:tasks.workunit.client.1.smithi163.stdout:5/155: creat d0/f32 x:0 0 0 2024-02-16T19:24:21.882 INFO:tasks.workunit.client.1.smithi163.stdout:3/226: sync 2024-02-16T19:24:21.886 INFO:tasks.workunit.client.0.smithi040.stdout:6/915: unlink d0/d1/d12/d3b/d48/d73/d76/fd5 0 2024-02-16T19:24:21.886 INFO:tasks.workunit.client.0.smithi040.stdout:6/916: chown d0/d1/d12/d3b/d48/d4d/l6e 120312 1 2024-02-16T19:24:21.886 INFO:tasks.workunit.client.0.smithi040.stdout:6/917: getdents d0/d1/d6f/d90/db1/d12b 0 2024-02-16T19:24:21.886 INFO:tasks.workunit.client.0.smithi040.stdout:6/918: write d0/df0/f118 [267229,81304] 0 2024-02-16T19:24:21.888 INFO:tasks.workunit.client.1.smithi163.stdout:6/186: dwrite d0/f5 [0,4194304] 0 2024-02-16T19:24:21.888 INFO:tasks.workunit.client.1.smithi163.stdout:6/187: read - d0/d6/d11/d17/f2f zero size 2024-02-16T19:24:21.888 INFO:tasks.workunit.client.1.smithi163.stdout:6/188: chown d0/d6/d11/c21 20115 1 2024-02-16T19:24:21.889 INFO:tasks.workunit.client.1.smithi163.stdout:6/189: chown d0/d6/d9/d2e 1 1 2024-02-16T19:24:21.889 INFO:tasks.workunit.client.1.smithi163.stdout:6/190: stat d0/l3 0 2024-02-16T19:24:21.889 INFO:tasks.workunit.client.1.smithi163.stdout:9/394: dwrite d5/de/d16/f6b [0,4194304] 0 2024-02-16T19:24:21.889 INFO:tasks.workunit.client.1.smithi163.stdout:9/395: fsync d5/f1a 0 2024-02-16T19:24:21.889 INFO:tasks.workunit.client.1.smithi163.stdout:9/396: chown d5/f40 3283326 1 2024-02-16T19:24:21.889 INFO:tasks.workunit.client.1.smithi163.stdout:8/148: dwrite d0/d7/d19/d1f/d32/f34 [0,4194304] 0 2024-02-16T19:24:21.890 INFO:tasks.workunit.client.0.smithi040.stdout:0/894: link db/d2f/d1d/d21/f10d db/d2f/d1d/d21/d2e/d83/f135 0 2024-02-16T19:24:21.891 INFO:tasks.workunit.client.0.smithi040.stdout:8/858: dwrite d6/d12/d22/f4e [0,4194304] 0 2024-02-16T19:24:21.892 INFO:tasks.workunit.client.0.smithi040.stdout:8/859: dread - d6/d12/d51/d6e/db7/ddf/df8/d1d/f77 zero size 2024-02-16T19:24:21.892 INFO:tasks.workunit.client.0.smithi040.stdout:9/799: dwrite d3/d4/d11/d71/fed [0,4194304] 0 2024-02-16T19:24:21.892 INFO:tasks.workunit.client.0.smithi040.stdout:7/676: mkdir d21/d29/dad/de2 0 2024-02-16T19:24:21.894 INFO:tasks.workunit.client.0.smithi040.stdout:1/751: truncate d17/f2c 1972071 0 2024-02-16T19:24:21.894 INFO:tasks.workunit.client.0.smithi040.stdout:1/752: truncate d17/d43/d92/dca/fda 855381 0 2024-02-16T19:24:21.894 INFO:tasks.workunit.client.0.smithi040.stdout:2/701: sync 2024-02-16T19:24:21.894 INFO:tasks.workunit.client.1.smithi163.stdout:6/191: mknod d0/d6/d11/d17/c3a 0 2024-02-16T19:24:21.898 INFO:tasks.workunit.client.0.smithi040.stdout:5/738: creat d1/da/d1c/dba/db1/dde/ddf/fe0 x:0 0 0 2024-02-16T19:24:21.899 INFO:tasks.workunit.client.0.smithi040.stdout:5/739: readlink d1/da/l15 0 2024-02-16T19:24:21.900 INFO:tasks.workunit.client.1.smithi163.stdout:9/397: creat d5/de/d30/d3d/d49/d6e/f8d x:0 0 0 2024-02-16T19:24:21.901 INFO:tasks.workunit.client.0.smithi040.stdout:6/919: mkdir d0/d1/d12/d3b/d48/dc0/d124/d12c 0 2024-02-16T19:24:21.901 INFO:tasks.workunit.client.0.smithi040.stdout:6/920: chown d0/d37/de1/dac/fb3 3866822 1 2024-02-16T19:24:21.903 INFO:tasks.workunit.client.0.smithi040.stdout:0/895: mknod db/de/c136 0 2024-02-16T19:24:21.903 INFO:tasks.workunit.client.0.smithi040.stdout:0/896: write db/de/dbe/f10e [833242,20682] 0 2024-02-16T19:24:21.904 INFO:tasks.workunit.client.0.smithi040.stdout:0/897: write db/d25/d2c/d52/fc1 [4409297,43770] 0 2024-02-16T19:24:21.904 INFO:tasks.workunit.client.0.smithi040.stdout:0/898: creat db/d2f/d23/f137 x:0 0 0 2024-02-16T19:24:21.904 INFO:tasks.workunit.client.0.smithi040.stdout:7/677: dread d21/f23 [0,4194304] 0 2024-02-16T19:24:21.906 INFO:tasks.workunit.client.0.smithi040.stdout:7/678: dread d21/d29/f5c [0,4194304] 0 2024-02-16T19:24:21.906 INFO:tasks.workunit.client.0.smithi040.stdout:7/679: stat d21/d29/f37 0 2024-02-16T19:24:21.907 INFO:tasks.workunit.client.1.smithi163.stdout:6/192: symlink d0/d6/df/d20/l3b 0 2024-02-16T19:24:21.907 INFO:tasks.workunit.client.1.smithi163.stdout:3/227: dwrite d0/dc/f31 [0,4194304] 0 2024-02-16T19:24:21.909 INFO:tasks.workunit.client.0.smithi040.stdout:4/616: sync 2024-02-16T19:24:21.909 INFO:tasks.workunit.client.1.smithi163.stdout:9/398: mknod d5/de/d30/d3d/d49/d6e/c8e 0 2024-02-16T19:24:21.909 INFO:tasks.workunit.client.1.smithi163.stdout:9/399: write d5/f77 [517934,94346] 0 2024-02-16T19:24:21.910 INFO:tasks.workunit.client.1.smithi163.stdout:1/130: rmdir d7/d20 39 2024-02-16T19:24:21.912 INFO:tasks.workunit.client.1.smithi163.stdout:8/149: rmdir d0/d7/d19 39 2024-02-16T19:24:21.916 INFO:tasks.workunit.client.1.smithi163.stdout:3/228: creat d0/dc/d1d/d49/f56 x:0 0 0 2024-02-16T19:24:21.917 INFO:tasks.workunit.client.1.smithi163.stdout:9/400: mkdir d5/de/d30/d43/d8f 0 2024-02-16T19:24:21.917 INFO:tasks.workunit.client.1.smithi163.stdout:9/401: stat d5/de/d16/d46/l4d 0 2024-02-16T19:24:21.919 INFO:tasks.workunit.client.0.smithi040.stdout:8/860: creat d6/d64/d50/dd5/d102/f107 x:0 0 0 2024-02-16T19:24:21.920 INFO:tasks.workunit.client.1.smithi163.stdout:1/131: symlink d7/dc/l27 0 2024-02-16T19:24:21.921 INFO:tasks.workunit.client.0.smithi040.stdout:9/800: getdents d3/de5 0 2024-02-16T19:24:21.921 INFO:tasks.workunit.client.1.smithi163.stdout:8/150: mkdir d0/d7/d19/d1f/d30/d35 0 2024-02-16T19:24:21.924 INFO:tasks.workunit.client.0.smithi040.stdout:1/753: mknod d17/d43/d92/dca/cf8 0 2024-02-16T19:24:21.926 INFO:tasks.workunit.client.1.smithi163.stdout:1/132: mkdir d7/d20/d28 0 2024-02-16T19:24:21.927 INFO:tasks.workunit.client.1.smithi163.stdout:3/229: dread d0/f19 [0,4194304] 0 2024-02-16T19:24:21.928 INFO:tasks.workunit.client.1.smithi163.stdout:8/151: creat d0/d24/f36 x:0 0 0 2024-02-16T19:24:21.929 INFO:tasks.workunit.client.0.smithi040.stdout:5/740: symlink d1/da/d1c/d73/le1 0 2024-02-16T19:24:21.932 INFO:tasks.workunit.client.0.smithi040.stdout:6/921: rename d0/d1/d6f/d90/f104 to d0/d37/d4a/f12d 0 2024-02-16T19:24:21.932 INFO:tasks.workunit.client.0.smithi040.stdout:6/922: fdatasync d0/f4 0 2024-02-16T19:24:21.932 INFO:tasks.workunit.client.0.smithi040.stdout:6/923: chown d0/d30/d5b/f9f 996249 1 2024-02-16T19:24:21.933 INFO:tasks.workunit.client.0.smithi040.stdout:1/754: read d17/d26/f7d [3801062,22188] 0 2024-02-16T19:24:21.933 INFO:tasks.workunit.client.1.smithi163.stdout:5/156: dwrite d0/d2/f21 [0,4194304] 0 2024-02-16T19:24:21.934 INFO:tasks.workunit.client.1.smithi163.stdout:5/157: creat d0/d22/f33 x:0 0 0 2024-02-16T19:24:21.935 INFO:tasks.workunit.client.0.smithi040.stdout:4/617: dwrite dd/de/d10/d36/d9e/d77/f3b [4194304,4194304] 0 2024-02-16T19:24:21.938 INFO:tasks.workunit.client.0.smithi040.stdout:0/899: unlink db/d2f/d23/d56/dbb/c114 0 2024-02-16T19:24:21.938 INFO:tasks.workunit.client.0.smithi040.stdout:0/900: write db/d2f/f70 [5219801,110444] 0 2024-02-16T19:24:21.939 INFO:tasks.workunit.client.0.smithi040.stdout:4/618: dread dd/de/d10/f2e [0,4194304] 0 2024-02-16T19:24:21.941 INFO:tasks.workunit.client.0.smithi040.stdout:7/680: link d21/f23 d21/d4d/d77/d87/dc9/ddd/dbe/fe3 0 2024-02-16T19:24:21.941 INFO:tasks.workunit.client.0.smithi040.stdout:1/755: read d17/d43/d38/f72 [3629724,56002] 0 2024-02-16T19:24:21.944 INFO:tasks.workunit.client.0.smithi040.stdout:5/741: dwrite d1/da/fc8 [0,4194304] 0 2024-02-16T19:24:21.944 INFO:tasks.workunit.client.0.smithi040.stdout:5/742: creat d1/da/d1c/da7/fe2 x:0 0 0 2024-02-16T19:24:21.944 INFO:tasks.workunit.client.0.smithi040.stdout:5/743: truncate d1/da/d7e/f86 533370 0 2024-02-16T19:24:21.944 INFO:tasks.workunit.client.1.smithi163.stdout:1/133: mknod d7/d20/d28/c29 0 2024-02-16T19:24:21.945 INFO:tasks.workunit.client.0.smithi040.stdout:9/801: mknod d3/d4/d11/d71/c100 0 2024-02-16T19:24:21.946 INFO:tasks.workunit.client.1.smithi163.stdout:3/230: rename d0/dc/d1d/d29/f48 to d0/dc/d1c/d32/f57 0 2024-02-16T19:24:21.947 INFO:tasks.workunit.client.0.smithi040.stdout:2/702: truncate d5/d7/d1e/f70 1729074 0 2024-02-16T19:24:21.947 INFO:tasks.workunit.client.0.smithi040.stdout:2/703: chown d5/d5c/d65/d9d/fc7 46558 1 2024-02-16T19:24:21.948 INFO:tasks.workunit.client.1.smithi163.stdout:8/152: getdents d0 0 2024-02-16T19:24:21.950 INFO:tasks.workunit.client.0.smithi040.stdout:6/924: rename d0/d30/d32/dd2/dbc/fe9 to d0/d37/f12e 0 2024-02-16T19:24:21.953 INFO:tasks.workunit.client.1.smithi163.stdout:9/402: rmdir d5/de/d30 39 2024-02-16T19:24:21.953 INFO:tasks.workunit.client.1.smithi163.stdout:9/403: truncate d5/f1a 57689 0 2024-02-16T19:24:21.953 INFO:tasks.workunit.client.1.smithi163.stdout:9/404: chown d5/de/d30/d31/d52/c61 347999 1 2024-02-16T19:24:21.954 INFO:tasks.workunit.client.0.smithi040.stdout:1/756: dread d17/d3c/f80 [0,4194304] 0 2024-02-16T19:24:21.954 INFO:tasks.workunit.client.0.smithi040.stdout:1/757: write d17/db5/fd5 [306414,7196] 0 2024-02-16T19:24:21.954 INFO:tasks.workunit.client.0.smithi040.stdout:1/758: fdatasync d17/d43/d38/f7b 0 2024-02-16T19:24:21.954 INFO:tasks.workunit.client.0.smithi040.stdout:1/759: read d17/f37 [2386003,69661] 0 2024-02-16T19:24:21.956 INFO:tasks.workunit.client.0.smithi040.stdout:0/901: mknod db/d2f/d1d/d21/d2e/d49/d5b/c138 0 2024-02-16T19:24:21.956 INFO:tasks.workunit.client.0.smithi040.stdout:0/902: chown db/d2f/d1d/d21/d2e/d85/d89/d94/f121 327122 1 2024-02-16T19:24:21.957 INFO:tasks.workunit.client.0.smithi040.stdout:4/619: dread - dd/de/d10/fa2 zero size 2024-02-16T19:24:21.960 INFO:tasks.workunit.client.1.smithi163.stdout:1/134: mkdir d7/d20/d2a 0 2024-02-16T19:24:21.960 INFO:tasks.workunit.client.1.smithi163.stdout:1/135: chown d7/d20 621 1 2024-02-16T19:24:21.962 INFO:tasks.workunit.client.0.smithi040.stdout:7/681: rmdir d21/d29/dad/de2 0 2024-02-16T19:24:21.963 INFO:tasks.workunit.client.0.smithi040.stdout:7/682: creat d21/d7e/fe4 x:0 0 0 2024-02-16T19:24:21.963 INFO:tasks.workunit.client.0.smithi040.stdout:0/903: dread db/d2f/d1d/d21/d2e/d49/d5b/f79 [0,4194304] 0 2024-02-16T19:24:21.963 INFO:tasks.workunit.client.0.smithi040.stdout:0/904: write db/d2f/f12b [580959,108499] 0 2024-02-16T19:24:21.963 INFO:tasks.workunit.client.0.smithi040.stdout:0/905: write db/d2f/d23/d6f/d98/ff4 [269859,124865] 0 2024-02-16T19:24:21.963 INFO:tasks.workunit.client.0.smithi040.stdout:0/906: creat db/d2f/d1d/d21/f139 x:0 0 0 2024-02-16T19:24:21.963 INFO:tasks.workunit.client.1.smithi163.stdout:3/231: creat d0/dc/d50/f58 x:0 0 0 2024-02-16T19:24:21.964 INFO:tasks.workunit.client.0.smithi040.stdout:8/861: rmdir d6/d64/d50/dd5/d102 39 2024-02-16T19:24:21.967 INFO:tasks.workunit.client.1.smithi163.stdout:8/153: rmdir d0/d7/d19/d1f/d32 39 2024-02-16T19:24:21.970 INFO:tasks.workunit.client.1.smithi163.stdout:9/405: rename d5/f1e to d5/de/d30/d43/d8f/f90 0 2024-02-16T19:24:21.971 INFO:tasks.workunit.client.0.smithi040.stdout:5/744: mknod d1/d62/d9a/ce3 0 2024-02-16T19:24:21.973 INFO:tasks.workunit.client.1.smithi163.stdout:5/158: getdents d0 0 2024-02-16T19:24:21.975 INFO:tasks.workunit.client.0.smithi040.stdout:9/802: unlink d3/d1c/d73/lbf 0 2024-02-16T19:24:21.980 INFO:tasks.workunit.client.1.smithi163.stdout:1/136: chown c4 1654 1 2024-02-16T19:24:21.980 INFO:tasks.workunit.client.1.smithi163.stdout:1/137: stat d7/d20 0 2024-02-16T19:24:21.980 INFO:tasks.workunit.client.1.smithi163.stdout:8/154: creat d0/d7/d18/f37 x:0 0 0 2024-02-16T19:24:21.980 INFO:tasks.workunit.client.1.smithi163.stdout:5/159: dread d0/d2/fa [0,4194304] 0 2024-02-16T19:24:21.980 INFO:tasks.workunit.client.1.smithi163.stdout:5/160: fdatasync d0/d2/fa 0 2024-02-16T19:24:21.980 INFO:tasks.workunit.client.1.smithi163.stdout:5/161: dread - d0/d22/f33 zero size 2024-02-16T19:24:21.981 INFO:tasks.workunit.client.0.smithi040.stdout:6/925: mkdir d0/d1/d12/d3b/d48/d73/d12f 0 2024-02-16T19:24:21.981 INFO:tasks.workunit.client.0.smithi040.stdout:6/926: stat d0/d37/fb2 0 2024-02-16T19:24:21.984 INFO:tasks.workunit.client.1.smithi163.stdout:3/232: rename d0/f6 to d0/dc/d1d/d49/f59 0 2024-02-16T19:24:21.984 INFO:tasks.workunit.client.0.smithi040.stdout:5/745: write d1/da/d51/f6c [381419,1185] 0 2024-02-16T19:24:21.984 INFO:tasks.workunit.client.0.smithi040.stdout:5/746: write d1/d62/d9a/dc5/fcc [468325,46765] 0 2024-02-16T19:24:21.985 INFO:tasks.workunit.client.0.smithi040.stdout:1/760: mkdir d17/d26/d4c/df9 0 2024-02-16T19:24:21.985 INFO:tasks.workunit.client.0.smithi040.stdout:7/683: dwrite d21/f28 [0,4194304] 0 2024-02-16T19:24:21.986 INFO:tasks.workunit.client.0.smithi040.stdout:1/761: write d17/d3c/f9d [638551,73904] 0 2024-02-16T19:24:21.986 INFO:tasks.workunit.client.1.smithi163.stdout:5/162: unlink d0/f1d 0 2024-02-16T19:24:21.987 INFO:tasks.workunit.client.0.smithi040.stdout:4/620: mknod dd/de/d10/d36/d9e/d32/d7a/cc0 0 2024-02-16T19:24:21.989 INFO:tasks.workunit.client.1.smithi163.stdout:8/155: rename d0/d7/c33 to d0/d7/d17/c38 0 2024-02-16T19:24:21.992 INFO:tasks.workunit.client.1.smithi163.stdout:9/406: getdents d5/de/d16 0 2024-02-16T19:24:21.994 INFO:tasks.workunit.client.1.smithi163.stdout:5/163: mkdir d0/d2/d34 0 2024-02-16T19:24:21.994 INFO:tasks.workunit.client.1.smithi163.stdout:5/164: creat d0/d22/f35 x:0 0 0 2024-02-16T19:24:21.995 INFO:tasks.workunit.client.0.smithi040.stdout:0/907: link db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/d103/l126 db/d2f/d23/d3b/d10c/l13a 0 2024-02-16T19:24:21.997 INFO:tasks.workunit.client.0.smithi040.stdout:8/862: symlink d6/l108 0 2024-02-16T19:24:21.999 INFO:tasks.workunit.client.1.smithi163.stdout:3/233: rename d0/f14 to d0/dc/d1d/f5a 0 2024-02-16T19:24:22.001 INFO:tasks.workunit.client.1.smithi163.stdout:8/156: creat d0/d7/d19/d1f/d32/f39 x:0 0 0 2024-02-16T19:24:22.002 INFO:tasks.workunit.client.0.smithi040.stdout:9/803: unlink d3/d4/d55/d8a/d99/f9d 0 2024-02-16T19:24:22.002 INFO:tasks.workunit.client.0.smithi040.stdout:9/804: fsync d3/d1c/fb9 0 2024-02-16T19:24:22.003 INFO:tasks.workunit.client.0.smithi040.stdout:8/863: write d6/d12/d51/d6e/db7/ddf/df8/d1d/f52 [3445821,98878] 0 2024-02-16T19:24:22.004 INFO:tasks.workunit.client.0.smithi040.stdout:6/927: rmdir d0/d30/d32/dd2/dbc/dfa 39 2024-02-16T19:24:22.006 INFO:tasks.workunit.client.1.smithi163.stdout:5/165: mkdir d0/d22/d28/d36 0 2024-02-16T19:24:22.006 INFO:tasks.workunit.client.1.smithi163.stdout:5/166: truncate d0/d22/f33 170048 0 2024-02-16T19:24:22.006 INFO:tasks.workunit.client.1.smithi163.stdout:5/167: truncate d0/d22/f33 1019635 0 2024-02-16T19:24:22.006 INFO:tasks.workunit.client.0.smithi040.stdout:5/747: truncate d1/f96 3903407 0 2024-02-16T19:24:22.009 INFO:tasks.workunit.client.1.smithi163.stdout:9/407: rename d5/de/d30/d3d/d49/d50/c76 to d5/de/d30/d3d/d49/d6d/c91 0 2024-02-16T19:24:22.010 INFO:tasks.workunit.client.0.smithi040.stdout:9/805: read d3/d4/f96 [2546382,57065] 0 2024-02-16T19:24:22.012 INFO:tasks.workunit.client.1.smithi163.stdout:3/234: symlink d0/dc/d1c/d32/l5b 0 2024-02-16T19:24:22.013 INFO:tasks.workunit.client.0.smithi040.stdout:7/684: mknod d21/d4d/d77/d87/dc9/ddd/ce5 0 2024-02-16T19:24:22.014 INFO:tasks.workunit.client.1.smithi163.stdout:8/157: symlink d0/d7/d19/d1f/d30/l3a 0 2024-02-16T19:24:22.016 INFO:tasks.workunit.client.0.smithi040.stdout:1/762: rename d17/l23 to d17/lfa 0 2024-02-16T19:24:22.018 INFO:tasks.workunit.client.1.smithi163.stdout:5/168: truncate d0/d2/f4 546454 0 2024-02-16T19:24:22.020 INFO:tasks.workunit.client.0.smithi040.stdout:7/685: dread d21/d4d/d77/d87/d9b/fa7 [4194304,4194304] 0 2024-02-16T19:24:22.021 INFO:tasks.workunit.client.0.smithi040.stdout:7/686: readlink d21/d4d/l9a 0 2024-02-16T19:24:22.021 INFO:tasks.workunit.client.0.smithi040.stdout:4/621: mknod dd/de/d10/d36/cc1 0 2024-02-16T19:24:22.023 INFO:tasks.workunit.client.1.smithi163.stdout:9/408: link d5/de/d30/d31/f74 d5/de/d30/d3d/d49/d6e/f92 0 2024-02-16T19:24:22.025 INFO:tasks.workunit.client.1.smithi163.stdout:3/235: mknod d0/c5c 0 2024-02-16T19:24:22.026 INFO:tasks.workunit.client.1.smithi163.stdout:8/158: creat d0/d24/d2c/f3b x:0 0 0 2024-02-16T19:24:22.031 INFO:tasks.workunit.client.1.smithi163.stdout:5/169: symlink d0/d2/l37 0 2024-02-16T19:24:22.034 INFO:tasks.workunit.client.0.smithi040.stdout:7/687: read d21/d4d/f56 [2785419,24615] 0 2024-02-16T19:24:22.037 INFO:tasks.workunit.client.0.smithi040.stdout:0/908: creat db/d25/d34/f13b x:0 0 0 2024-02-16T19:24:22.038 INFO:tasks.workunit.client.0.smithi040.stdout:8/864: rename d6/d12/d51/d6e/fa2 to d6/d12/d51/d6e/db7/dc6/dcc/de8/f109 0 2024-02-16T19:24:22.039 INFO:tasks.workunit.client.0.smithi040.stdout:6/928: symlink d0/d1/d12/d3b/d48/d4f/l130 0 2024-02-16T19:24:22.040 INFO:tasks.workunit.client.1.smithi163.stdout:3/236: write d0/f2 [2312039,5056] 0 2024-02-16T19:24:22.040 INFO:tasks.workunit.client.1.smithi163.stdout:3/237: dread - d0/dc/d1d/d49/f56 zero size 2024-02-16T19:24:22.041 INFO:tasks.workunit.client.0.smithi040.stdout:5/748: link d1/d62/d9a/fa2 d1/da/d1c/d21/daa/fe4 0 2024-02-16T19:24:22.041 INFO:tasks.workunit.client.0.smithi040.stdout:5/749: creat d1/da/d51/fe5 x:0 0 0 2024-02-16T19:24:22.042 INFO:tasks.workunit.client.0.smithi040.stdout:9/806: mkdir d3/d4/d55/d31/d101 0 2024-02-16T19:24:22.042 INFO:tasks.workunit.client.0.smithi040.stdout:9/807: write d3/d4/d55/ff4 [128656,104274] 0 2024-02-16T19:24:22.042 INFO:tasks.workunit.client.0.smithi040.stdout:9/808: fdatasync d3/d1c/f64 0 2024-02-16T19:24:22.043 INFO:tasks.workunit.client.1.smithi163.stdout:3/238: mkdir d0/dc/d1d/d29/d5d 0 2024-02-16T19:24:22.043 INFO:tasks.workunit.client.1.smithi163.stdout:3/239: stat d0/dc/f28 0 2024-02-16T19:24:22.045 INFO:tasks.workunit.client.0.smithi040.stdout:7/688: link d21/d4d/f6f d21/d4d/d77/d7a/dc6/fe6 0 2024-02-16T19:24:22.045 INFO:tasks.workunit.client.0.smithi040.stdout:7/689: dread - d21/d4d/f6e zero size 2024-02-16T19:24:22.045 INFO:tasks.workunit.client.0.smithi040.stdout:8/865: symlink d6/d12/d93/l10a 0 2024-02-16T19:24:22.046 INFO:tasks.workunit.client.0.smithi040.stdout:6/929: mkdir d0/d1/d6f/d90/d131 0 2024-02-16T19:24:22.049 INFO:tasks.workunit.client.0.smithi040.stdout:5/750: rename d1/f9c to d1/d45/fe6 0 2024-02-16T19:24:22.053 INFO:tasks.workunit.client.0.smithi040.stdout:9/809: unlink d3/d4/d11/d29/d38/d46/d6b/d85/fbb 0 2024-02-16T19:24:22.053 INFO:tasks.workunit.client.0.smithi040.stdout:9/810: dread - d3/d4/d4a/f62 zero size 2024-02-16T19:24:22.058 INFO:tasks.workunit.client.0.smithi040.stdout:7/690: mkdir d21/d8b/d9c/da0/db7/de7 0 2024-02-16T19:24:22.058 INFO:tasks.workunit.client.0.smithi040.stdout:5/751: mkdir d1/d45/dab/de7 0 2024-02-16T19:24:22.061 INFO:tasks.workunit.client.0.smithi040.stdout:7/691: unlink d21/cdb 0 2024-02-16T19:24:22.061 INFO:tasks.workunit.client.0.smithi040.stdout:1/763: dwrite d17/d3c/f80 [0,4194304] 0 2024-02-16T19:24:22.062 INFO:tasks.workunit.client.0.smithi040.stdout:7/692: creat d21/d4d/d77/d87/dc9/ddd/dbe/fe8 x:0 0 0 2024-02-16T19:24:22.062 INFO:tasks.workunit.client.0.smithi040.stdout:7/693: truncate d21/d4d/d77/d87/dc9/ddd/dbe/fe8 1041398 0 2024-02-16T19:24:22.066 INFO:tasks.workunit.client.0.smithi040.stdout:7/694: link d21/d7e/faa d21/d7e/fe9 0 2024-02-16T19:24:22.067 INFO:tasks.workunit.client.1.smithi163.stdout:9/409: dwrite d5/f1a [0,4194304] 0 2024-02-16T19:24:22.068 INFO:tasks.workunit.client.0.smithi040.stdout:7/695: symlink d21/d4d/d7b/lea 0 2024-02-16T19:24:22.069 INFO:tasks.workunit.client.0.smithi040.stdout:1/764: rename d17/d43 to d17/d26/d9a/dfb 0 2024-02-16T19:24:22.071 INFO:tasks.workunit.client.0.smithi040.stdout:7/696: unlink d21/d4d/d77/fd3 0 2024-02-16T19:24:22.072 INFO:tasks.workunit.client.0.smithi040.stdout:7/697: read d21/d4d/d77/d87/dc9/ddd/dbe/fe3 [349656,63570] 0 2024-02-16T19:24:22.072 INFO:tasks.workunit.client.0.smithi040.stdout:7/698: chown d21/d4d/f74 22647 1 2024-02-16T19:24:22.072 INFO:tasks.workunit.client.0.smithi040.stdout:7/699: stat d21/d29/d79/dbc 0 2024-02-16T19:24:22.072 INFO:tasks.workunit.client.0.smithi040.stdout:7/700: fdatasync d21/d4d/fbd 0 2024-02-16T19:24:22.073 INFO:tasks.workunit.client.0.smithi040.stdout:1/765: mknod d17/d26/d9a/dfb/d38/cfc 0 2024-02-16T19:24:22.073 INFO:tasks.workunit.client.0.smithi040.stdout:1/766: chown d17/f36 2469408 1 2024-02-16T19:24:22.074 INFO:tasks.workunit.client.1.smithi163.stdout:8/159: fsync d0/d24/d2c/f3b 0 2024-02-16T19:24:22.075 INFO:tasks.workunit.client.0.smithi040.stdout:7/701: rename d21/c42 to d21/d29/ceb 0 2024-02-16T19:24:22.076 INFO:tasks.workunit.client.0.smithi040.stdout:1/767: getdents d17/d3c/d74/d97/da9 0 2024-02-16T19:24:22.076 INFO:tasks.workunit.client.0.smithi040.stdout:1/768: chown d17/d26/d3d/cf1 25515909 1 2024-02-16T19:24:22.079 INFO:tasks.workunit.client.0.smithi040.stdout:7/702: mknod d21/d8b/d9c/cec 0 2024-02-16T19:24:22.079 INFO:tasks.workunit.client.0.smithi040.stdout:7/703: chown d21/d8b/fe0 65 1 2024-02-16T19:24:22.082 INFO:tasks.workunit.client.0.smithi040.stdout:5/752: dread d1/f17 [0,4194304] 0 2024-02-16T19:24:22.082 INFO:tasks.workunit.client.0.smithi040.stdout:5/753: creat d1/da/d1c/dba/db1/dde/ddf/fe8 x:0 0 0 2024-02-16T19:24:22.082 INFO:tasks.workunit.client.0.smithi040.stdout:7/704: rename d21/d4d/d77/d87/dc9/ddd/fb3 to d21/d8b/d9c/da0/db7/de7/fed 0 2024-02-16T19:24:22.082 INFO:tasks.workunit.client.0.smithi040.stdout:7/705: chown d21/d4d/d7d/fce 9545330 1 2024-02-16T19:24:22.083 INFO:tasks.workunit.client.0.smithi040.stdout:7/706: creat d21/d29/d79/fee x:0 0 0 2024-02-16T19:24:22.083 INFO:tasks.workunit.client.0.smithi040.stdout:7/707: chown d21/faf 666816 1 2024-02-16T19:24:22.083 INFO:tasks.workunit.client.0.smithi040.stdout:7/708: truncate d21/d29/fc2 979066 0 2024-02-16T19:24:22.083 INFO:tasks.workunit.client.0.smithi040.stdout:7/709: creat d21/d4d/d77/d87/dc9/fef x:0 0 0 2024-02-16T19:24:22.085 INFO:tasks.workunit.client.1.smithi163.stdout:5/170: dwrite d0/d2/f21 [0,4194304] 0 2024-02-16T19:24:22.085 INFO:tasks.workunit.client.1.smithi163.stdout:5/171: read - d0/d22/f35 zero size 2024-02-16T19:24:22.089 INFO:tasks.workunit.client.1.smithi163.stdout:8/160: write d0/dc/f2d [6980047,47570] 0 2024-02-16T19:24:22.090 INFO:tasks.workunit.client.1.smithi163.stdout:8/161: read - d0/d24/f36 zero size 2024-02-16T19:24:22.091 INFO:tasks.workunit.client.0.smithi040.stdout:1/769: dread d17/f1f [0,4194304] 0 2024-02-16T19:24:22.091 INFO:tasks.workunit.client.0.smithi040.stdout:1/770: stat d17/d26/d3d 0 2024-02-16T19:24:22.093 INFO:tasks.workunit.client.1.smithi163.stdout:8/162: rename d0/d7/d19/d1f/d30/d35 to d0/d24/d3c 0 2024-02-16T19:24:22.094 INFO:tasks.workunit.client.1.smithi163.stdout:3/240: dwrite d0/dc/d1c/d32/f36 [0,4194304] 0 2024-02-16T19:24:22.095 INFO:tasks.workunit.client.0.smithi040.stdout:0/909: dwrite db/d25/d2c/d52/fc1 [4194304,4194304] 0 2024-02-16T19:24:22.095 INFO:tasks.workunit.client.0.smithi040.stdout:0/910: creat db/d25/d34/f13c x:0 0 0 2024-02-16T19:24:22.095 INFO:tasks.workunit.client.1.smithi163.stdout:8/163: rename d0/d7/d17 to d0/d24/d3c/d3d 0 2024-02-16T19:24:22.095 INFO:tasks.workunit.client.1.smithi163.stdout:8/164: truncate d0/d24/f36 181396 0 2024-02-16T19:24:22.095 INFO:tasks.workunit.client.1.smithi163.stdout:8/165: dread d0/d24/f36 [0,4194304] 0 2024-02-16T19:24:22.097 INFO:tasks.workunit.client.0.smithi040.stdout:4/622: dwrite dd/de/d10/fb6 [0,4194304] 0 2024-02-16T19:24:22.097 INFO:tasks.workunit.client.0.smithi040.stdout:6/930: dwrite d0/d1/d12/d3b/d48/d73/d70/d83/f11e [0,4194304] 0 2024-02-16T19:24:22.097 INFO:tasks.workunit.client.0.smithi040.stdout:6/931: fsync d0/d30/d4b/f57 0 2024-02-16T19:24:22.098 INFO:tasks.workunit.client.0.smithi040.stdout:8/866: dwrite d6/d64/d50/dd5/d102/f107 [0,4194304] 0 2024-02-16T19:24:22.098 INFO:tasks.workunit.client.0.smithi040.stdout:8/867: stat d6/d2b/d44/fb4 0 2024-02-16T19:24:22.103 INFO:tasks.workunit.client.1.smithi163.stdout:8/166: symlink d0/d7/d18/l3e 0 2024-02-16T19:24:22.104 INFO:tasks.workunit.client.0.smithi040.stdout:0/911: stat db/d2f/l3d 0 2024-02-16T19:24:22.104 INFO:tasks.workunit.client.0.smithi040.stdout:0/912: read db/d25/d34/d5e/fdb [16774,42612] 0 2024-02-16T19:24:22.104 INFO:tasks.workunit.client.0.smithi040.stdout:0/913: creat db/d2f/d23/d3b/f13d x:0 0 0 2024-02-16T19:24:22.104 INFO:tasks.workunit.client.0.smithi040.stdout:0/914: fsync db/d2f/d1d/d21/d2e/d85/d89/d94/fb2 0 2024-02-16T19:24:22.104 INFO:tasks.workunit.client.0.smithi040.stdout:0/915: chown db/d2f/d23/d56/fe6 0 1 2024-02-16T19:24:22.105 INFO:tasks.workunit.client.1.smithi163.stdout:3/241: getdents d0/dc/d1c/d32 0 2024-02-16T19:24:22.108 INFO:tasks.workunit.client.1.smithi163.stdout:3/242: symlink d0/dc/d1d/d29/d38/l5e 0 2024-02-16T19:24:22.114 INFO:tasks.workunit.client.0.smithi040.stdout:6/932: mknod d0/d1/d12/d3b/d48/dba/dde/c132 0 2024-02-16T19:24:22.116 INFO:tasks.workunit.client.0.smithi040.stdout:3/856: sync 2024-02-16T19:24:22.116 INFO:tasks.workunit.client.0.smithi040.stdout:3/857: creat d1/dc/d35/d53/d64/f114 x:0 0 0 2024-02-16T19:24:22.116 INFO:tasks.workunit.client.0.smithi040.stdout:3/858: dread - d1/d8c/d8e/ff1 zero size 2024-02-16T19:24:22.119 INFO:tasks.workunit.client.0.smithi040.stdout:0/916: creat db/d2f/d23/d3b/d10c/f13e x:0 0 0 2024-02-16T19:24:22.121 INFO:tasks.workunit.client.0.smithi040.stdout:6/933: mkdir d0/d30/d5b/d133 0 2024-02-16T19:24:22.124 INFO:tasks.workunit.client.0.smithi040.stdout:3/859: symlink d1/dc/d42/dee/l115 0 2024-02-16T19:24:22.130 INFO:tasks.workunit.client.0.smithi040.stdout:6/934: dread d0/d1/d12/d3b/f97 [0,4194304] 0 2024-02-16T19:24:22.130 INFO:tasks.workunit.client.0.smithi040.stdout:6/935: write d0/d1/d6f/fdd [525147,58012] 0 2024-02-16T19:24:22.131 INFO:tasks.workunit.client.1.smithi163.stdout:2/292: sync 2024-02-16T19:24:22.131 INFO:tasks.workunit.client.1.smithi163.stdout:7/158: sync 2024-02-16T19:24:22.131 INFO:tasks.workunit.client.1.smithi163.stdout:7/159: stat d11/d24/d25/c31 0 2024-02-16T19:24:22.131 INFO:tasks.workunit.client.1.smithi163.stdout:7/160: write d11/f15 [403461,6946] 0 2024-02-16T19:24:22.131 INFO:tasks.workunit.client.1.smithi163.stdout:7/161: readlink d11/l16 0 2024-02-16T19:24:22.133 INFO:tasks.workunit.client.0.smithi040.stdout:5/754: dwrite d1/da/d1c/da7/fca [0,4194304] 0 2024-02-16T19:24:22.133 INFO:tasks.workunit.client.1.smithi163.stdout:9/410: dwrite d5/f40 [0,4194304] 0 2024-02-16T19:24:22.134 INFO:tasks.workunit.client.0.smithi040.stdout:2/704: sync 2024-02-16T19:24:22.134 INFO:tasks.workunit.client.0.smithi040.stdout:2/705: chown d5/f79 17376316 1 2024-02-16T19:24:22.135 INFO:tasks.workunit.client.1.smithi163.stdout:7/162: truncate f3 1902057 0 2024-02-16T19:24:22.135 INFO:tasks.workunit.client.1.smithi163.stdout:7/163: write f5 [4743661,87479] 0 2024-02-16T19:24:22.137 INFO:tasks.workunit.client.1.smithi163.stdout:2/293: rmdir d2/d26 39 2024-02-16T19:24:22.138 INFO:tasks.workunit.client.0.smithi040.stdout:6/936: rmdir d0/d37 39 2024-02-16T19:24:22.140 INFO:tasks.workunit.client.0.smithi040.stdout:5/755: symlink d1/da/d7e/le9 0 2024-02-16T19:24:22.140 INFO:tasks.workunit.client.0.smithi040.stdout:4/623: dread dd/de/d10/fa9 [0,4194304] 0 2024-02-16T19:24:22.142 INFO:tasks.workunit.client.1.smithi163.stdout:8/167: truncate d0/d24/f36 146807 0 2024-02-16T19:24:22.142 INFO:tasks.workunit.client.1.smithi163.stdout:8/168: truncate d0/d7/d18/f37 267671 0 2024-02-16T19:24:22.143 INFO:tasks.workunit.client.0.smithi040.stdout:2/706: mkdir d5/d7/d12/d1b/de7 0 2024-02-16T19:24:22.146 INFO:tasks.workunit.client.0.smithi040.stdout:5/756: mkdir d1/d3c/dea 0 2024-02-16T19:24:22.147 INFO:tasks.workunit.client.0.smithi040.stdout:3/860: dread d1/d7/f4e [4194304,4194304] 0 2024-02-16T19:24:22.147 INFO:tasks.workunit.client.0.smithi040.stdout:3/861: fdatasync d1/d4/dc4/fe0 0 2024-02-16T19:24:22.150 INFO:tasks.workunit.client.0.smithi040.stdout:2/707: truncate d5/d7/d1e/d4c/fce 1999094 0 2024-02-16T19:24:22.151 INFO:tasks.workunit.client.1.smithi163.stdout:9/411: stat d5/de/d16/c1d 0 2024-02-16T19:24:22.152 INFO:tasks.workunit.client.0.smithi040.stdout:6/937: symlink d0/d30/d32/dd2/d87/d8f/l134 0 2024-02-16T19:24:22.154 INFO:tasks.workunit.client.0.smithi040.stdout:3/862: truncate d1/d7/d36/f97 2795305 0 2024-02-16T19:24:22.155 INFO:tasks.workunit.client.1.smithi163.stdout:7/164: mkdir d11/d26/d36 0 2024-02-16T19:24:22.156 INFO:tasks.workunit.client.0.smithi040.stdout:4/624: rmdir dd/de/daa 39 2024-02-16T19:24:22.157 INFO:tasks.workunit.client.1.smithi163.stdout:2/294: unlink d2/d26/f2b 0 2024-02-16T19:24:22.157 INFO:tasks.workunit.client.1.smithi163.stdout:2/295: chown d2/d2a/d39/f57 328967 1 2024-02-16T19:24:22.158 INFO:tasks.workunit.client.0.smithi040.stdout:2/708: symlink d5/d7/d1e/d31/d4f/le8 0 2024-02-16T19:24:22.158 INFO:tasks.workunit.client.1.smithi163.stdout:8/169: read d0/d7/f15 [4000261,116286] 0 2024-02-16T19:24:22.159 INFO:tasks.workunit.client.1.smithi163.stdout:7/165: unlink d11/f18 0 2024-02-16T19:24:22.159 INFO:tasks.workunit.client.0.smithi040.stdout:6/938: mkdir d0/d30/d4b/dcc/d135 0 2024-02-16T19:24:22.160 INFO:tasks.workunit.client.1.smithi163.stdout:8/170: rename d0/d7/f15 to d0/d7/d18/f3f 0 2024-02-16T19:24:22.162 INFO:tasks.workunit.client.0.smithi040.stdout:4/625: mkdir dd/de/d71/dc2 0 2024-02-16T19:24:22.162 INFO:tasks.workunit.client.0.smithi040.stdout:4/626: chown dd/de/c16 482 1 2024-02-16T19:24:22.163 INFO:tasks.workunit.client.0.smithi040.stdout:2/709: symlink d5/d7/d12/d1b/dae/le9 0 2024-02-16T19:24:22.164 INFO:tasks.workunit.client.1.smithi163.stdout:7/166: getdents d11/d30 0 2024-02-16T19:24:22.164 INFO:tasks.workunit.client.0.smithi040.stdout:3/863: rmdir d1/dc/d42 39 2024-02-16T19:24:22.165 INFO:tasks.workunit.client.0.smithi040.stdout:7/710: dwrite d21/d4d/f66 [0,4194304] 0 2024-02-16T19:24:22.165 INFO:tasks.workunit.client.0.smithi040.stdout:7/711: truncate d21/d8b/fb5 676749 0 2024-02-16T19:24:22.165 INFO:tasks.workunit.client.0.smithi040.stdout:6/939: mkdir d0/d30/d5b/dbe/d136 0 2024-02-16T19:24:22.165 INFO:tasks.workunit.client.0.smithi040.stdout:6/940: getdents d0/d30 0 2024-02-16T19:24:22.167 INFO:tasks.workunit.client.0.smithi040.stdout:4/627: truncate dd/de/d10/d36/d9e/f15 4171917 0 2024-02-16T19:24:22.168 INFO:tasks.workunit.client.1.smithi163.stdout:7/167: mknod d11/d24/d34/c37 0 2024-02-16T19:24:22.171 INFO:tasks.workunit.client.0.smithi040.stdout:6/941: rename d0/d30/d32/dd2/dbc/ff7 to d0/d1/d12/f137 0 2024-02-16T19:24:22.174 INFO:tasks.workunit.client.0.smithi040.stdout:4/628: mkdir dd/de/d10/d36/dc3 0 2024-02-16T19:24:22.175 INFO:tasks.workunit.client.0.smithi040.stdout:6/942: mknod d0/d30/d5b/d133/c138 0 2024-02-16T19:24:22.177 INFO:tasks.workunit.client.0.smithi040.stdout:4/629: write dd/de/d10/f30 [896103,35930] 0 2024-02-16T19:24:22.179 INFO:tasks.workunit.client.1.smithi163.stdout:3/243: dwrite d0/f8 [0,4194304] 0 2024-02-16T19:24:22.179 INFO:tasks.workunit.client.1.smithi163.stdout:3/244: write d0/dc/d1c/d32/f57 [4276049,104123] 0 2024-02-16T19:24:22.179 INFO:tasks.workunit.client.0.smithi040.stdout:2/710: read d5/d7/d10/f86 [359469,123976] 0 2024-02-16T19:24:22.180 INFO:tasks.workunit.client.0.smithi040.stdout:0/917: dwrite db/d2f/f70 [0,4194304] 0 2024-02-16T19:24:22.180 INFO:tasks.workunit.client.0.smithi040.stdout:0/918: creat db/d2f/f13f x:0 0 0 2024-02-16T19:24:22.180 INFO:tasks.workunit.client.0.smithi040.stdout:8/868: dwrite d6/f8 [0,4194304] 0 2024-02-16T19:24:22.180 INFO:tasks.workunit.client.0.smithi040.stdout:6/943: truncate d0/d30/fef 560292 0 2024-02-16T19:24:22.183 INFO:tasks.workunit.client.0.smithi040.stdout:6/944: fdatasync d0/d30/d32/dd2/d87/d8f/fb9 0 2024-02-16T19:24:22.183 INFO:tasks.workunit.client.0.smithi040.stdout:6/945: creat d0/d1/d12/d3b/d48/d73/f139 x:0 0 0 2024-02-16T19:24:22.183 INFO:tasks.workunit.client.0.smithi040.stdout:6/946: chown d0/d37/fb2 9235 1 2024-02-16T19:24:22.183 INFO:tasks.workunit.client.0.smithi040.stdout:7/712: dread d21/d29/d79/f7f [0,4194304] 0 2024-02-16T19:24:22.184 INFO:tasks.workunit.client.0.smithi040.stdout:5/757: dwrite d1/da/fc8 [0,4194304] 0 2024-02-16T19:24:22.184 INFO:tasks.workunit.client.0.smithi040.stdout:5/758: fdatasync d1/da/d1c/da7/fe2 0 2024-02-16T19:24:22.184 INFO:tasks.workunit.client.0.smithi040.stdout:5/759: write d1/f13 [4536852,75809] 0 2024-02-16T19:24:22.184 INFO:tasks.workunit.client.0.smithi040.stdout:5/760: creat d1/da/d1c/dba/db1/dde/ddf/feb x:0 0 0 2024-02-16T19:24:22.184 INFO:tasks.workunit.client.0.smithi040.stdout:4/630: creat dd/de/d45/fc4 x:0 0 0 2024-02-16T19:24:22.185 INFO:tasks.workunit.client.1.smithi163.stdout:6/193: sync 2024-02-16T19:24:22.185 INFO:tasks.workunit.client.1.smithi163.stdout:1/138: sync 2024-02-16T19:24:22.185 INFO:tasks.workunit.client.1.smithi163.stdout:4/159: sync 2024-02-16T19:24:22.185 INFO:tasks.workunit.client.1.smithi163.stdout:0/311: sync 2024-02-16T19:24:22.185 INFO:tasks.workunit.client.1.smithi163.stdout:4/160: getdents d11/d18/d1f/d29 0 2024-02-16T19:24:22.187 INFO:tasks.workunit.client.0.smithi040.stdout:2/711: mkdir d5/d7/d12/dca/dea 0 2024-02-16T19:24:22.189 INFO:tasks.workunit.client.1.smithi163.stdout:3/245: getdents d0/dc 0 2024-02-16T19:24:22.191 INFO:tasks.workunit.client.1.smithi163.stdout:2/296: dwrite d2/d2a/d39/f3b [0,4194304] 0 2024-02-16T19:24:22.192 INFO:tasks.workunit.client.1.smithi163.stdout:1/139: rename d7/dc/c1f to d7/d20/c2b 0 2024-02-16T19:24:22.193 INFO:tasks.workunit.client.0.smithi040.stdout:3/864: dwrite d1/dc/d42/d4a/f50 [0,4194304] 0 2024-02-16T19:24:22.193 INFO:tasks.workunit.client.0.smithi040.stdout:8/869: dread d6/d64/d50/fbe [0,4194304] 0 2024-02-16T19:24:22.194 INFO:tasks.workunit.client.0.smithi040.stdout:0/919: mknod db/c140 0 2024-02-16T19:24:22.196 INFO:tasks.workunit.client.1.smithi163.stdout:0/312: mkdir d11/d21/d50/d58/d6b 0 2024-02-16T19:24:22.198 INFO:tasks.workunit.client.0.smithi040.stdout:7/713: dread f17 [0,4194304] 0 2024-02-16T19:24:22.198 INFO:tasks.workunit.client.0.smithi040.stdout:6/947: dread d0/d1/d12/d3b/d79/f96 [0,4194304] 0 2024-02-16T19:24:22.199 INFO:tasks.workunit.client.1.smithi163.stdout:6/194: creat d0/d6/d9/d2e/d33/f3c x:0 0 0 2024-02-16T19:24:22.199 INFO:tasks.workunit.client.1.smithi163.stdout:6/195: chown d0/d6/le 0 1 2024-02-16T19:24:22.199 INFO:tasks.workunit.client.1.smithi163.stdout:6/196: chown d0/d6/l1b 921325586 1 2024-02-16T19:24:22.201 INFO:tasks.workunit.client.1.smithi163.stdout:4/161: mknod d11/d18/d1f/d29/c2e 0 2024-02-16T19:24:22.207 INFO:tasks.workunit.client.0.smithi040.stdout:3/865: mknod d1/d8c/c116 0 2024-02-16T19:24:22.207 INFO:tasks.workunit.client.0.smithi040.stdout:3/866: chown d1/dc/d35/fbf 1762 1 2024-02-16T19:24:22.208 INFO:tasks.workunit.client.1.smithi163.stdout:8/171: dwrite d0/d7/d18/f37 [0,4194304] 0 2024-02-16T19:24:22.209 INFO:tasks.workunit.client.1.smithi163.stdout:2/297: truncate d2/f1a 446400 0 2024-02-16T19:24:22.209 INFO:tasks.workunit.client.0.smithi040.stdout:8/870: dread d6/d2b/da3/d8b/f9d [0,4194304] 0 2024-02-16T19:24:22.209 INFO:tasks.workunit.client.0.smithi040.stdout:8/871: chown d6/d12/d51/d6e/db7/ddf/df8 7 1 2024-02-16T19:24:22.209 INFO:tasks.workunit.client.0.smithi040.stdout:8/872: chown d6/d12/d51/d6e/db7/ddf/df8/f27 15677097 1 2024-02-16T19:24:22.212 INFO:tasks.workunit.client.0.smithi040.stdout:0/920: unlink db/d2f/d23/d3b/c12e 0 2024-02-16T19:24:22.212 INFO:tasks.workunit.client.0.smithi040.stdout:0/921: chown db/d2f/d23/d6f/f78 21255 1 2024-02-16T19:24:22.213 INFO:tasks.workunit.client.1.smithi163.stdout:7/168: dread f3 [0,4194304] 0 2024-02-16T19:24:22.213 INFO:tasks.workunit.client.0.smithi040.stdout:5/761: dwrite d1/da/d7e/fa0 [0,4194304] 0 2024-02-16T19:24:22.213 INFO:tasks.workunit.client.0.smithi040.stdout:7/714: dread d21/f35 [0,4194304] 0 2024-02-16T19:24:22.216 INFO:tasks.workunit.client.0.smithi040.stdout:6/948: rename d0/d30/d5b/f85 to d0/d30/d32/dd2/d87/d8f/df3/f13a 0 2024-02-16T19:24:22.216 INFO:tasks.workunit.client.0.smithi040.stdout:6/949: creat d0/d1/d6f/d90/db1/f13b x:0 0 0 2024-02-16T19:24:22.219 INFO:tasks.workunit.client.0.smithi040.stdout:8/873: mknod d6/d12/d51/db6/c10b 0 2024-02-16T19:24:22.222 INFO:tasks.workunit.client.1.smithi163.stdout:0/313: rename d11/f35 to d11/d12/f6c 0 2024-02-16T19:24:22.222 INFO:tasks.workunit.client.1.smithi163.stdout:0/314: truncate d11/d22/d61/f69 695473 0 2024-02-16T19:24:22.222 INFO:tasks.workunit.client.1.smithi163.stdout:0/315: write d11/d22/f34 [3945593,87131] 0 2024-02-16T19:24:22.223 INFO:tasks.workunit.client.0.smithi040.stdout:0/922: creat db/d2f/d1d/d21/d2e/d85/d89/f141 x:0 0 0 2024-02-16T19:24:22.223 INFO:tasks.workunit.client.1.smithi163.stdout:6/197: rmdir d0/dd 39 2024-02-16T19:24:22.225 INFO:tasks.workunit.client.1.smithi163.stdout:4/162: creat d11/f2f x:0 0 0 2024-02-16T19:24:22.225 INFO:tasks.workunit.client.1.smithi163.stdout:4/163: write d11/d18/f1d [5970483,96215] 0 2024-02-16T19:24:22.225 INFO:tasks.workunit.client.1.smithi163.stdout:4/164: write d11/d18/f24 [801573,32309] 0 2024-02-16T19:24:22.232 INFO:tasks.workunit.client.0.smithi040.stdout:5/762: link d1/da/d1c/d21/l47 d1/d62/lec 0 2024-02-16T19:24:22.233 INFO:tasks.workunit.client.1.smithi163.stdout:8/172: truncate d0/f20 3508858 0 2024-02-16T19:24:22.233 INFO:tasks.workunit.client.1.smithi163.stdout:8/173: creat d0/d7/d19/d1f/d32/f40 x:0 0 0 2024-02-16T19:24:22.234 INFO:tasks.workunit.client.1.smithi163.stdout:2/298: symlink d2/d2a/l60 0 2024-02-16T19:24:22.236 INFO:tasks.workunit.client.1.smithi163.stdout:7/169: symlink d11/d26/d36/l38 0 2024-02-16T19:24:22.238 INFO:tasks.workunit.client.0.smithi040.stdout:2/712: rmdir d5/d7/d1e/d4c 39 2024-02-16T19:24:22.238 INFO:tasks.workunit.client.0.smithi040.stdout:2/713: fsync d5/d5c/d65/d9d/fc7 0 2024-02-16T19:24:22.239 INFO:tasks.workunit.client.1.smithi163.stdout:6/198: rmdir d0/d6/d9/d15/d1c 39 2024-02-16T19:24:22.239 INFO:tasks.workunit.client.1.smithi163.stdout:8/174: dread d0/d7/d19/f26 [0,4194304] 0 2024-02-16T19:24:22.240 INFO:tasks.workunit.client.1.smithi163.stdout:4/165: dwrite f9 [0,4194304] 0 2024-02-16T19:24:22.245 INFO:tasks.workunit.client.0.smithi040.stdout:7/715: rename d21/d4d/c63 to d21/d29/d79/dbc/cf0 0 2024-02-16T19:24:22.248 INFO:tasks.workunit.client.0.smithi040.stdout:6/950: symlink d0/d30/d32/dd2/dbc/l13c 0 2024-02-16T19:24:22.248 INFO:tasks.workunit.client.0.smithi040.stdout:6/951: chown d0/d1/d12/l5a 312755 1 2024-02-16T19:24:22.250 INFO:tasks.workunit.client.1.smithi163.stdout:8/175: mkdir d0/d24/d3c/d3d/d41 0 2024-02-16T19:24:22.250 INFO:tasks.workunit.client.1.smithi163.stdout:8/176: chown d0/d7/d19/d1f/d32/f40 1 1 2024-02-16T19:24:22.251 INFO:tasks.workunit.client.0.smithi040.stdout:8/874: symlink d6/d12/d51/d6e/db7/dc6/dea/l10c 0 2024-02-16T19:24:22.253 INFO:tasks.workunit.client.0.smithi040.stdout:0/923: getdents db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/d103 0 2024-02-16T19:24:22.255 INFO:tasks.workunit.client.1.smithi163.stdout:0/316: rename d11/d21/d5f to d11/d12/d67/d6a/d6d 0 2024-02-16T19:24:22.255 INFO:tasks.workunit.client.1.smithi163.stdout:0/317: chown d11/d4f/c56 840674 1 2024-02-16T19:24:22.255 INFO:tasks.workunit.client.1.smithi163.stdout:0/318: write d11/d12/d20/f2c [548601,87128] 0 2024-02-16T19:24:22.257 INFO:tasks.workunit.client.0.smithi040.stdout:5/763: creat d1/da/db0/fed x:0 0 0 2024-02-16T19:24:22.257 INFO:tasks.workunit.client.0.smithi040.stdout:5/764: chown d1/da/fa1 24 1 2024-02-16T19:24:22.257 INFO:tasks.workunit.client.0.smithi040.stdout:5/765: creat d1/da/d1c/da7/fee x:0 0 0 2024-02-16T19:24:22.258 INFO:tasks.workunit.client.1.smithi163.stdout:2/299: dwrite d2/f23 [0,4194304] 0 2024-02-16T19:24:22.259 INFO:tasks.workunit.client.0.smithi040.stdout:2/714: rename d5/d7/d1e/d4c/d52/c7c to d5/d7/d1e/d55/d8d/ceb 0 2024-02-16T19:24:22.261 INFO:tasks.workunit.client.1.smithi163.stdout:6/199: rename d0/d6/d11/d17/f2f to d0/d6/d9/d2e/d33/f3d 0 2024-02-16T19:24:22.262 INFO:tasks.workunit.client.0.smithi040.stdout:6/952: mknod d0/d1/d12/d3b/d48/d73/c13d 0 2024-02-16T19:24:22.262 INFO:tasks.workunit.client.0.smithi040.stdout:6/953: chown d0/d1/d12/d3b/d48/d4d/c10d 78736 1 2024-02-16T19:24:22.263 INFO:tasks.workunit.client.1.smithi163.stdout:0/319: symlink d11/d21/d50/d58/d6b/l6e 0 2024-02-16T19:24:22.263 INFO:tasks.workunit.client.1.smithi163.stdout:0/320: read d11/d12/f15 [34415,116836] 0 2024-02-16T19:24:22.263 INFO:tasks.workunit.client.1.smithi163.stdout:0/321: creat d11/d22/d3e/f6f x:0 0 0 2024-02-16T19:24:22.266 INFO:tasks.workunit.client.1.smithi163.stdout:8/177: rename d0/d24/f36 to d0/d24/d2c/f42 0 2024-02-16T19:24:22.266 INFO:tasks.workunit.client.1.smithi163.stdout:8/178: write d0/dc/f16 [2123262,72991] 0 2024-02-16T19:24:22.268 INFO:tasks.workunit.client.0.smithi040.stdout:2/715: rmdir d5/d7/d12/d1b/de7 0 2024-02-16T19:24:22.271 INFO:tasks.workunit.client.0.smithi040.stdout:6/954: mknod d0/d37/de1/dac/c13e 0 2024-02-16T19:24:22.271 INFO:tasks.workunit.client.0.smithi040.stdout:6/955: chown d0/f55 6 1 2024-02-16T19:24:22.271 INFO:tasks.workunit.client.0.smithi040.stdout:8/875: dread d6/f24 [0,4194304] 0 2024-02-16T19:24:22.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/300: dread d2/d2a/f33 [0,4194304] 0 2024-02-16T19:24:22.273 INFO:tasks.workunit.client.0.smithi040.stdout:6/956: mkdir d0/d1/d6f/d90/d131/d13f 0 2024-02-16T19:24:22.273 INFO:tasks.workunit.client.0.smithi040.stdout:6/957: chown d0/d30/d32/ce0 0 1 2024-02-16T19:24:22.274 INFO:tasks.workunit.client.0.smithi040.stdout:8/876: truncate d6/d12/d51/d6e/db7/ddf/df8/d15/f67 2820105 0 2024-02-16T19:24:22.275 INFO:tasks.workunit.client.1.smithi163.stdout:2/301: symlink d2/d26/l61 0 2024-02-16T19:24:22.277 INFO:tasks.workunit.client.1.smithi163.stdout:6/200: write d0/f5 [3370015,53219] 0 2024-02-16T19:24:22.278 INFO:tasks.workunit.client.1.smithi163.stdout:6/201: symlink d0/d6/d9/d15/l3e 0 2024-02-16T19:24:22.279 INFO:tasks.workunit.client.1.smithi163.stdout:7/170: dwrite d11/f19 [0,4194304] 0 2024-02-16T19:24:22.281 INFO:tasks.workunit.client.1.smithi163.stdout:4/166: dwrite d11/d18/d1f/f27 [0,4194304] 0 2024-02-16T19:24:22.281 INFO:tasks.workunit.client.1.smithi163.stdout:7/171: creat d11/d24/d25/f39 x:0 0 0 2024-02-16T19:24:22.282 INFO:tasks.workunit.client.0.smithi040.stdout:0/924: dwrite db/de/d6a/df0/ff8 [0,4194304] 0 2024-02-16T19:24:22.284 INFO:tasks.workunit.client.1.smithi163.stdout:7/172: mkdir d11/d3a 0 2024-02-16T19:24:22.286 INFO:tasks.workunit.client.1.smithi163.stdout:4/167: dread f2 [0,4194304] 0 2024-02-16T19:24:22.286 INFO:tasks.workunit.client.1.smithi163.stdout:4/168: chown d11/d18/d1f/d23/l2a 55935 1 2024-02-16T19:24:22.288 INFO:tasks.workunit.client.0.smithi040.stdout:6/958: read d0/d1/d12/d3b/d79/f96 [4177871,16600] 0 2024-02-16T19:24:22.288 INFO:tasks.workunit.client.0.smithi040.stdout:8/877: dread d6/d2b/f63 [0,4194304] 0 2024-02-16T19:24:22.289 INFO:tasks.workunit.client.0.smithi040.stdout:8/878: creat d6/d64/d50/dd5/f10d x:0 0 0 2024-02-16T19:24:22.289 INFO:tasks.workunit.client.0.smithi040.stdout:8/879: truncate d6/d2b/da3/f3d 579469 0 2024-02-16T19:24:22.289 INFO:tasks.workunit.client.0.smithi040.stdout:8/880: truncate d6/d2b/f63 4644733 0 2024-02-16T19:24:22.293 INFO:tasks.workunit.client.1.smithi163.stdout:0/322: dread d11/d22/f34 [0,4194304] 0 2024-02-16T19:24:22.297 INFO:tasks.workunit.client.0.smithi040.stdout:0/925: write db/d2f/d1d/d21/d2e/d85/f86 [2354941,104155] 0 2024-02-16T19:24:22.299 INFO:tasks.workunit.client.1.smithi163.stdout:2/302: dwrite d2/f9 [0,4194304] 0 2024-02-16T19:24:22.301 INFO:tasks.workunit.client.0.smithi040.stdout:6/959: write d0/d30/d5b/f92 [1065823,46884] 0 2024-02-16T19:24:22.310 INFO:tasks.workunit.client.0.smithi040.stdout:8/881: truncate d6/d12/f1f 994829 0 2024-02-16T19:24:22.312 INFO:tasks.workunit.client.1.smithi163.stdout:0/323: mknod d11/d21/d50/d58/c70 0 2024-02-16T19:24:22.312 INFO:tasks.workunit.client.1.smithi163.stdout:0/324: write d11/d12/d20/f2f [1097003,93149] 0 2024-02-16T19:24:22.312 INFO:tasks.workunit.client.1.smithi163.stdout:0/325: write d11/d22/d3e/f66 [882059,39891] 0 2024-02-16T19:24:22.312 INFO:tasks.workunit.client.1.smithi163.stdout:0/326: stat c9 0 2024-02-16T19:24:22.313 INFO:tasks.workunit.client.1.smithi163.stdout:2/303: rename d2/d2a/l60 to d2/d2a/d2d/l62 0 2024-02-16T19:24:22.318 INFO:tasks.workunit.client.0.smithi040.stdout:6/960: unlink d0/d1/d12/d3b/d48/d4f/f68 0 2024-02-16T19:24:22.319 INFO:tasks.workunit.client.0.smithi040.stdout:6/961: readlink d0/d30/d32/la9 0 2024-02-16T19:24:22.319 INFO:tasks.workunit.client.0.smithi040.stdout:6/962: write d0/d1/d12/d3b/d48/dba/f11d [813636,27802] 0 2024-02-16T19:24:22.319 INFO:tasks.workunit.client.0.smithi040.stdout:6/963: truncate d0/d1/d12/d3b/d48/dba/f11d 1103069 0 2024-02-16T19:24:22.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/882: creat d6/d12/d51/d6e/db7/ddf/df8/d1d/f10e x:0 0 0 2024-02-16T19:24:22.320 INFO:tasks.workunit.client.0.smithi040.stdout:8/883: truncate d6/d2b/da3/f53 2553240 0 2024-02-16T19:24:22.321 INFO:tasks.workunit.client.1.smithi163.stdout:2/304: unlink d2/d2a/d2d/l55 0 2024-02-16T19:24:22.321 INFO:tasks.workunit.client.1.smithi163.stdout:2/305: readlink d2/l20 0 2024-02-16T19:24:22.321 INFO:tasks.workunit.client.1.smithi163.stdout:2/306: readlink d2/d2a/l50 0 2024-02-16T19:24:22.322 INFO:tasks.workunit.client.1.smithi163.stdout:0/327: rmdir d11/d22/d61 39 2024-02-16T19:24:22.322 INFO:tasks.workunit.client.1.smithi163.stdout:0/328: creat d11/d21/d50/d58/d6b/f71 x:0 0 0 2024-02-16T19:24:22.323 INFO:tasks.workunit.client.0.smithi040.stdout:6/964: symlink d0/d37/l140 0 2024-02-16T19:24:22.323 INFO:tasks.workunit.client.0.smithi040.stdout:6/965: creat d0/d37/de1/f141 x:0 0 0 2024-02-16T19:24:22.323 INFO:tasks.workunit.client.0.smithi040.stdout:6/966: truncate d0/d1/d6f/fdd 779140 0 2024-02-16T19:24:22.324 INFO:tasks.workunit.client.1.smithi163.stdout:0/329: mkdir d11/d21/d50/d58/d72 0 2024-02-16T19:24:22.325 INFO:tasks.workunit.client.0.smithi040.stdout:6/967: symlink d0/d30/d5b/dbe/l142 0 2024-02-16T19:24:22.326 INFO:tasks.workunit.client.1.smithi163.stdout:0/330: dread d11/f33 [0,4194304] 0 2024-02-16T19:24:22.326 INFO:tasks.workunit.client.1.smithi163.stdout:0/331: write d11/d22/f24 [619491,63395] 0 2024-02-16T19:24:22.326 INFO:tasks.workunit.client.1.smithi163.stdout:0/332: read d11/d12/f15 [163144,100274] 0 2024-02-16T19:24:22.326 INFO:tasks.workunit.client.0.smithi040.stdout:6/968: symlink d0/d30/d32/dd2/d87/l143 0 2024-02-16T19:24:22.327 INFO:tasks.workunit.client.0.smithi040.stdout:6/969: fsync d0/d1/d12/d3b/d48/dba/dde/fab 0 2024-02-16T19:24:22.327 INFO:tasks.workunit.client.0.smithi040.stdout:6/970: truncate d0/d1/d6f/fe2 953367 0 2024-02-16T19:24:22.327 INFO:tasks.workunit.client.0.smithi040.stdout:6/971: creat d0/d1/d12/d3b/d48/d73/d70/d83/f144 x:0 0 0 2024-02-16T19:24:22.327 INFO:tasks.workunit.client.0.smithi040.stdout:6/972: mknod d0/d30/d32/dd2/d87/d8f/df3/c145 0 2024-02-16T19:24:22.331 INFO:tasks.workunit.client.0.smithi040.stdout:8/884: read d6/d12/d51/d6e/db7/ddf/df8/d15/f85 [650741,100626] 0 2024-02-16T19:24:22.331 INFO:tasks.workunit.client.0.smithi040.stdout:8/885: chown d6/d12/d51/d6e/db9/dcf/fa7 1080 1 2024-02-16T19:24:22.332 INFO:tasks.workunit.client.0.smithi040.stdout:8/886: creat d6/d2b/da3/d8b/f10f x:0 0 0 2024-02-16T19:24:22.333 INFO:tasks.workunit.client.0.smithi040.stdout:8/887: mknod d6/d64/d50/dd5/c110 0 2024-02-16T19:24:22.334 INFO:tasks.workunit.client.0.smithi040.stdout:6/973: dread d0/f1d [0,4194304] 0 2024-02-16T19:24:22.334 INFO:tasks.workunit.client.0.smithi040.stdout:8/888: mknod d6/d12/d51/d6e/db7/dc6/c111 0 2024-02-16T19:24:22.334 INFO:tasks.workunit.client.0.smithi040.stdout:8/889: readlink d6/d2b/d44/l7c 0 2024-02-16T19:24:22.334 INFO:tasks.workunit.client.0.smithi040.stdout:8/890: chown d6/d12/d51/d6e/db7/dc6/fd4 7 1 2024-02-16T19:24:22.335 INFO:tasks.workunit.client.1.smithi163.stdout:0/333: dread d11/d12/d20/f39 [0,4194304] 0 2024-02-16T19:24:22.335 INFO:tasks.workunit.client.0.smithi040.stdout:6/974: symlink d0/df0/l146 0 2024-02-16T19:24:22.335 INFO:tasks.workunit.client.0.smithi040.stdout:6/975: stat d0/d30/d4b/cf5 0 2024-02-16T19:24:22.335 INFO:tasks.workunit.client.0.smithi040.stdout:6/976: chown d0/d1/d12/d3b/d48/dc0 115658 1 2024-02-16T19:24:22.336 INFO:tasks.workunit.client.0.smithi040.stdout:8/891: link d6/d2b/f80 d6/d12/d51/d6e/db7/dc6/dcc/f112 0 2024-02-16T19:24:22.336 INFO:tasks.workunit.client.1.smithi163.stdout:2/307: dwrite d2/fb [0,4194304] 0 2024-02-16T19:24:22.337 INFO:tasks.workunit.client.1.smithi163.stdout:0/334: symlink d11/d21/d50/d58/l73 0 2024-02-16T19:24:22.338 INFO:tasks.workunit.client.1.smithi163.stdout:2/308: mknod d2/d2a/d2d/d3c/c63 0 2024-02-16T19:24:22.339 INFO:tasks.workunit.client.1.smithi163.stdout:2/309: readlink d2/d2a/d2d/d3c/l56 0 2024-02-16T19:24:22.339 INFO:tasks.workunit.client.0.smithi040.stdout:6/977: rmdir d0/d1/d12/d3b 39 2024-02-16T19:24:22.340 INFO:tasks.workunit.client.1.smithi163.stdout:0/335: mkdir d11/d21/d50/d58/d74 0 2024-02-16T19:24:22.340 INFO:tasks.workunit.client.1.smithi163.stdout:0/336: fdatasync d11/d22/f42 0 2024-02-16T19:24:22.340 INFO:tasks.workunit.client.1.smithi163.stdout:0/337: fdatasync f3 0 2024-02-16T19:24:22.340 INFO:tasks.workunit.client.1.smithi163.stdout:2/310: truncate d2/d26/f31 2384971 0 2024-02-16T19:24:22.341 INFO:tasks.workunit.client.1.smithi163.stdout:0/338: creat d11/d12/d67/d6a/f75 x:0 0 0 2024-02-16T19:24:22.341 INFO:tasks.workunit.client.1.smithi163.stdout:0/339: write d11/d22/f34 [5069231,78510] 0 2024-02-16T19:24:22.342 INFO:tasks.workunit.client.1.smithi163.stdout:0/340: readlink d11/d22/d3e/l5d 0 2024-02-16T19:24:22.344 INFO:tasks.workunit.client.1.smithi163.stdout:0/341: unlink d11/d12/d20/f2f 0 2024-02-16T19:24:22.346 INFO:tasks.workunit.client.0.smithi040.stdout:8/892: dread d6/d12/d51/d6e/db7/ddf/df8/d15/f2f [0,4194304] 0 2024-02-16T19:24:22.375 INFO:tasks.workunit.client.0.smithi040.stdout:8/893: read d6/d2b/f48 [4401319,118048] 0 2024-02-16T19:24:22.376 INFO:tasks.workunit.client.0.smithi040.stdout:8/894: symlink d6/d12/d93/dd8/l113 0 2024-02-16T19:24:22.376 INFO:tasks.workunit.client.0.smithi040.stdout:8/895: read d6/d2b/f34 [55874,117048] 0 2024-02-16T19:24:22.376 INFO:tasks.workunit.client.0.smithi040.stdout:8/896: chown d6/d12/d51/d6e/db7/ddf/df8/d15/f67 491759 1 2024-02-16T19:24:22.377 INFO:tasks.workunit.client.0.smithi040.stdout:8/897: symlink d6/d64/l114 0 2024-02-16T19:24:22.378 INFO:tasks.workunit.client.0.smithi040.stdout:8/898: creat d6/d12/d51/d6e/db7/f115 x:0 0 0 2024-02-16T19:24:22.383 INFO:tasks.workunit.client.1.smithi163.stdout:4/169: dwrite d11/d18/d1f/f27 [4194304,4194304] 0 2024-02-16T19:24:22.383 INFO:tasks.workunit.client.1.smithi163.stdout:4/170: stat f9 0 2024-02-16T19:24:22.383 INFO:tasks.workunit.client.1.smithi163.stdout:2/311: dwrite d2/d2a/f33 [0,4194304] 0 2024-02-16T19:24:22.384 INFO:tasks.workunit.client.1.smithi163.stdout:7/173: dwrite d11/d26/f2b [0,4194304] 0 2024-02-16T19:24:22.384 INFO:tasks.workunit.client.1.smithi163.stdout:7/174: chown f3 268554 1 2024-02-16T19:24:22.384 INFO:tasks.workunit.client.0.smithi040.stdout:6/978: dwrite d0/d1/f9c [0,4194304] 0 2024-02-16T19:24:22.384 INFO:tasks.workunit.client.0.smithi040.stdout:6/979: fdatasync d0/d1/d12/f28 0 2024-02-16T19:24:22.385 INFO:tasks.workunit.client.1.smithi163.stdout:0/342: dwrite d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:22.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/179: dwrite d0/d7/d18/f37 [4194304,4194304] 0 2024-02-16T19:24:22.385 INFO:tasks.workunit.client.1.smithi163.stdout:4/171: dread d11/f26 [0,4194304] 0 2024-02-16T19:24:22.386 INFO:tasks.workunit.client.1.smithi163.stdout:4/172: creat d11/f30 x:0 0 0 2024-02-16T19:24:22.391 INFO:tasks.workunit.client.0.smithi040.stdout:9/811: sync 2024-02-16T19:24:22.391 INFO:tasks.workunit.client.0.smithi040.stdout:9/812: chown d3/d4/d11/d71/da8/c93 0 1 2024-02-16T19:24:22.393 INFO:tasks.workunit.client.0.smithi040.stdout:8/899: dwrite d6/f9 [4194304,4194304] 0 2024-02-16T19:24:22.397 INFO:tasks.workunit.client.0.smithi040.stdout:6/980: write d0/d1/d12/f28 [5187530,108898] 0 2024-02-16T19:24:22.397 INFO:tasks.workunit.client.0.smithi040.stdout:6/981: chown d0/d30/d32/dd2/d87/d8f/df3/c145 0 1 2024-02-16T19:24:22.398 INFO:tasks.workunit.client.1.smithi163.stdout:2/312: dread d2/d2a/d39/f3b [0,4194304] 0 2024-02-16T19:24:22.399 INFO:tasks.workunit.client.1.smithi163.stdout:0/343: dread d11/d22/d3e/f66 [0,4194304] 0 2024-02-16T19:24:22.401 INFO:tasks.workunit.client.1.smithi163.stdout:7/175: mknod d11/d30/c3b 0 2024-02-16T19:24:22.403 INFO:tasks.workunit.client.0.smithi040.stdout:9/813: unlink d3/d4/d55/dc3/cdb 0 2024-02-16T19:24:22.403 INFO:tasks.workunit.client.0.smithi040.stdout:9/814: chown d3/d4/d55/ld9 21462 1 2024-02-16T19:24:22.403 INFO:tasks.workunit.client.0.smithi040.stdout:9/815: fsync d3/d4/d11/d29/f8d 0 2024-02-16T19:24:22.403 INFO:tasks.workunit.client.0.smithi040.stdout:9/816: fsync d3/d4/d55/d31/f3b 0 2024-02-16T19:24:22.405 INFO:tasks.workunit.client.0.smithi040.stdout:0/926: write db/d25/d2c/f6d [1867919,66827] 0 2024-02-16T19:24:22.406 INFO:tasks.workunit.client.0.smithi040.stdout:0/927: readlink db/lf1 0 2024-02-16T19:24:22.406 INFO:tasks.workunit.client.0.smithi040.stdout:0/928: write db/d2f/f101 [956362,27468] 0 2024-02-16T19:24:22.406 INFO:tasks.workunit.client.0.smithi040.stdout:0/929: creat db/d2f/f142 x:0 0 0 2024-02-16T19:24:22.406 INFO:tasks.workunit.client.0.smithi040.stdout:0/930: chown db/d2f/d1d/d21/d2e/d83/f133 447 1 2024-02-16T19:24:22.406 INFO:tasks.workunit.client.0.smithi040.stdout:0/931: fsync db/d25/d2c/d52/f55 0 2024-02-16T19:24:22.406 INFO:tasks.workunit.client.0.smithi040.stdout:8/900: mknod d6/d64/d50/dd5/c116 0 2024-02-16T19:24:22.411 INFO:tasks.workunit.client.0.smithi040.stdout:6/982: rename d0/d30/d5b/fce to d0/d30/d4b/dcc/d135/f147 0 2024-02-16T19:24:22.413 INFO:tasks.workunit.client.0.smithi040.stdout:9/817: truncate d3/d4/d11/d71/da8/f58 3996214 0 2024-02-16T19:24:22.413 INFO:tasks.workunit.client.0.smithi040.stdout:9/818: dread - d3/d4/d55/f98 zero size 2024-02-16T19:24:22.416 INFO:tasks.workunit.client.1.smithi163.stdout:8/180: symlink d0/d7/d18/d2e/l43 0 2024-02-16T19:24:22.417 INFO:tasks.workunit.client.0.smithi040.stdout:0/932: creat db/d25/d2c/d52/f143 x:0 0 0 2024-02-16T19:24:22.420 INFO:tasks.workunit.client.0.smithi040.stdout:8/901: creat d6/d12/d51/d6e/db9/dcf/f117 x:0 0 0 2024-02-16T19:24:22.424 INFO:tasks.workunit.client.1.smithi163.stdout:7/176: creat d11/d26/d36/f3c x:0 0 0 2024-02-16T19:24:22.424 INFO:tasks.workunit.client.1.smithi163.stdout:0/344: dwrite d11/d22/d3e/f6f [0,4194304] 0 2024-02-16T19:24:22.424 INFO:tasks.workunit.client.1.smithi163.stdout:4/173: dwrite d11/f21 [4194304,4194304] 0 2024-02-16T19:24:22.426 INFO:tasks.workunit.client.0.smithi040.stdout:6/983: dread d0/fa0 [0,4194304] 0 2024-02-16T19:24:22.427 INFO:tasks.workunit.client.1.smithi163.stdout:2/313: rename d2/d2a/d39/f51 to d2/d26/d54/f64 0 2024-02-16T19:24:22.428 INFO:tasks.workunit.client.0.smithi040.stdout:0/933: mknod db/de/d6a/df0/d106/c144 0 2024-02-16T19:24:22.431 INFO:tasks.workunit.client.0.smithi040.stdout:9/819: write d3/d4/d55/f2e [1474400,125197] 0 2024-02-16T19:24:22.431 INFO:tasks.workunit.client.0.smithi040.stdout:8/902: truncate d6/d12/d51/f89 1191574 0 2024-02-16T19:24:22.437 INFO:tasks.workunit.client.1.smithi163.stdout:0/345: link d11/l37 d11/d21/d50/d58/d6b/l76 0 2024-02-16T19:24:22.438 INFO:tasks.workunit.client.1.smithi163.stdout:8/181: dwrite d0/dc/f2d [0,4194304] 0 2024-02-16T19:24:22.442 INFO:tasks.workunit.client.1.smithi163.stdout:4/174: write d11/f16 [1556278,101682] 0 2024-02-16T19:24:22.442 INFO:tasks.workunit.client.1.smithi163.stdout:7/177: dwrite d11/d26/d36/f3c [0,4194304] 0 2024-02-16T19:24:22.443 INFO:tasks.workunit.client.1.smithi163.stdout:2/314: rename d2/d2a/l50 to d2/d44/l65 0 2024-02-16T19:24:22.443 INFO:tasks.workunit.client.1.smithi163.stdout:2/315: chown d2 775 1 2024-02-16T19:24:22.444 INFO:tasks.workunit.client.1.smithi163.stdout:0/346: mknod d11/d21/d50/d58/c77 0 2024-02-16T19:24:22.445 INFO:tasks.workunit.client.1.smithi163.stdout:0/347: chown d11/d12/d67/d6a/d6d/l5e 159737 1 2024-02-16T19:24:22.445 INFO:tasks.workunit.client.0.smithi040.stdout:6/984: creat d0/d1/d12/d3b/d48/d4d/f148 x:0 0 0 2024-02-16T19:24:22.449 INFO:tasks.workunit.client.0.smithi040.stdout:0/934: dwrite db/d2f/d23/d3b/dd5/f12a [0,4194304] 0 2024-02-16T19:24:22.452 INFO:tasks.workunit.client.1.smithi163.stdout:7/178: creat d11/d24/f3d x:0 0 0 2024-02-16T19:24:22.455 INFO:tasks.workunit.client.1.smithi163.stdout:8/182: rename d0/d24/d2c/f42 to d0/f44 0 2024-02-16T19:24:22.455 INFO:tasks.workunit.client.1.smithi163.stdout:8/183: truncate d0/d7/d19/f2f 195738 0 2024-02-16T19:24:22.456 INFO:tasks.workunit.client.0.smithi040.stdout:8/903: mknod d6/d12/d51/d6e/db7/ddf/df8/d15/de5/c118 0 2024-02-16T19:24:22.456 INFO:tasks.workunit.client.0.smithi040.stdout:8/904: fdatasync d6/d12/f1f 0 2024-02-16T19:24:22.457 INFO:tasks.workunit.client.0.smithi040.stdout:6/985: mkdir d0/d1/d12/d3b/d48/d73/d70/d149 0 2024-02-16T19:24:22.459 INFO:tasks.workunit.client.1.smithi163.stdout:0/348: creat d11/d22/d61/f78 x:0 0 0 2024-02-16T19:24:22.459 INFO:tasks.workunit.client.1.smithi163.stdout:0/349: chown d11/d22 0 1 2024-02-16T19:24:22.460 INFO:tasks.workunit.client.1.smithi163.stdout:7/179: creat d11/d30/f3e x:0 0 0 2024-02-16T19:24:22.462 INFO:tasks.workunit.client.1.smithi163.stdout:8/184: mkdir d0/d24/d27/d45 0 2024-02-16T19:24:22.462 INFO:tasks.workunit.client.1.smithi163.stdout:8/185: dread - d0/d7/d19/d1f/d32/f39 zero size 2024-02-16T19:24:22.464 INFO:tasks.workunit.client.1.smithi163.stdout:2/316: mkdir d2/d2a/d39/d66 0 2024-02-16T19:24:22.467 INFO:tasks.workunit.client.0.smithi040.stdout:8/905: read d6/d12/d22/f4e [1139692,40824] 0 2024-02-16T19:24:22.469 INFO:tasks.workunit.client.1.smithi163.stdout:0/350: creat d11/d21/d50/f79 x:0 0 0 2024-02-16T19:24:22.469 INFO:tasks.workunit.client.1.smithi163.stdout:4/175: dwrite d11/f2f [0,4194304] 0 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/172: sync 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:6/202: sync 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:9/412: sync 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/173: getdents d0/d2/d34 0 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/174: fdatasync d0/d22/f2e 0 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/175: chown d0/ce 1423691478 1 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/176: write d0/d22/f33 [1671381,3121] 0 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:1/140: sync 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:1/141: truncate d7/f26 93185 0 2024-02-16T19:24:22.470 INFO:tasks.workunit.client.1.smithi163.stdout:3/246: sync 2024-02-16T19:24:22.471 INFO:tasks.workunit.client.0.smithi040.stdout:6/986: dread d0/d1/d12/d3b/d48/d73/d70/d83/f11e [0,4194304] 0 2024-02-16T19:24:22.471 INFO:tasks.workunit.client.0.smithi040.stdout:6/987: truncate d0/d1/d6f/d90/f10a 902739 0 2024-02-16T19:24:22.472 INFO:tasks.workunit.client.1.smithi163.stdout:9/413: dread d5/de/d30/d31/f69 [0,4194304] 0 2024-02-16T19:24:22.472 INFO:tasks.workunit.client.1.smithi163.stdout:7/180: creat d11/f3f x:0 0 0 2024-02-16T19:24:22.474 INFO:tasks.workunit.client.0.smithi040.stdout:0/935: unlink db/d2f/d1d/d21/l6c 0 2024-02-16T19:24:22.478 INFO:tasks.workunit.client.1.smithi163.stdout:8/186: mkdir d0/d7/d18/d2e/d46 0 2024-02-16T19:24:22.478 INFO:tasks.workunit.client.1.smithi163.stdout:8/187: truncate d0/d7/d19/d1f/d32/f39 149261 0 2024-02-16T19:24:22.482 INFO:tasks.workunit.client.0.smithi040.stdout:6/988: dread d0/d30/d32/dd2/d87/d8f/f128 [0,4194304] 0 2024-02-16T19:24:22.482 INFO:tasks.workunit.client.0.smithi040.stdout:6/989: write d0/d1/d12/f2c [4343992,98033] 0 2024-02-16T19:24:22.482 INFO:tasks.workunit.client.0.smithi040.stdout:6/990: write d0/d30/d32/f21 [379707,5491] 0 2024-02-16T19:24:22.483 INFO:tasks.workunit.client.0.smithi040.stdout:6/991: write d0/d30/d4b/dcc/f116 [1017568,115294] 0 2024-02-16T19:24:22.484 INFO:tasks.workunit.client.1.smithi163.stdout:4/176: rename d11/f21 to d11/d18/d1f/d29/f31 0 2024-02-16T19:24:22.484 INFO:tasks.workunit.client.0.smithi040.stdout:9/820: rmdir d3/d1c 39 2024-02-16T19:24:22.487 INFO:tasks.workunit.client.1.smithi163.stdout:6/203: symlink d0/l3f 0 2024-02-16T19:24:22.487 INFO:tasks.workunit.client.1.smithi163.stdout:6/204: fsync d0/d6/d11/d1e/f2c 0 2024-02-16T19:24:22.489 INFO:tasks.workunit.client.1.smithi163.stdout:5/177: creat d0/d22/d28/f38 x:0 0 0 2024-02-16T19:24:22.490 INFO:tasks.workunit.client.1.smithi163.stdout:5/178: fdatasync d0/d22/f35 0 2024-02-16T19:24:22.491 INFO:tasks.workunit.client.0.smithi040.stdout:0/936: mknod db/de/d6a/df0/d106/c145 0 2024-02-16T19:24:22.491 INFO:tasks.workunit.client.0.smithi040.stdout:0/937: fdatasync db/d2f/d1d/d21/d2e/d49/d5b/d67/fb7 0 2024-02-16T19:24:22.492 INFO:tasks.workunit.client.1.smithi163.stdout:3/247: truncate d0/dc/d1c/d21/f42 544214 0 2024-02-16T19:24:22.495 INFO:tasks.workunit.client.0.smithi040.stdout:1/771: sync 2024-02-16T19:24:22.495 INFO:tasks.workunit.client.1.smithi163.stdout:9/414: unlink d5/c1f 0 2024-02-16T19:24:22.495 INFO:tasks.workunit.client.0.smithi040.stdout:9/821: rmdir d3/d4/d11/d29/d38/d46/d6b 39 2024-02-16T19:24:22.496 INFO:tasks.workunit.client.1.smithi163.stdout:4/177: dread d11/d18/f1d [0,4194304] 0 2024-02-16T19:24:22.498 INFO:tasks.workunit.client.1.smithi163.stdout:8/188: unlink d0/f9 0 2024-02-16T19:24:22.500 INFO:tasks.workunit.client.1.smithi163.stdout:4/178: write d11/d18/f1d [2747298,58591] 0 2024-02-16T19:24:22.500 INFO:tasks.workunit.client.1.smithi163.stdout:6/205: rename d0/d6/d11 to d0/d6/df/d20/d40 0 2024-02-16T19:24:22.503 INFO:tasks.workunit.client.0.smithi040.stdout:0/938: creat db/d2f/d23/d6f/d98/f146 x:0 0 0 2024-02-16T19:24:22.503 INFO:tasks.workunit.client.0.smithi040.stdout:0/939: stat db/d2f/d1d/d21/f139 0 2024-02-16T19:24:22.503 INFO:tasks.workunit.client.0.smithi040.stdout:0/940: creat db/de/d6a/df0/d106/f147 x:0 0 0 2024-02-16T19:24:22.505 INFO:tasks.workunit.client.1.smithi163.stdout:5/179: link d0/fd d0/f39 0 2024-02-16T19:24:22.505 INFO:tasks.workunit.client.1.smithi163.stdout:5/180: write d0/d22/d28/f38 [436836,1038] 0 2024-02-16T19:24:22.505 INFO:tasks.workunit.client.1.smithi163.stdout:5/181: read d0/d2/f4 [95206,49859] 0 2024-02-16T19:24:22.505 INFO:tasks.workunit.client.1.smithi163.stdout:5/182: creat d0/d22/f3a x:0 0 0 2024-02-16T19:24:22.505 INFO:tasks.workunit.client.1.smithi163.stdout:5/183: readlink d0/d22/l2b 0 2024-02-16T19:24:22.506 INFO:tasks.workunit.client.1.smithi163.stdout:5/184: write d0/d22/f3a [417781,70614] 0 2024-02-16T19:24:22.506 INFO:tasks.workunit.client.1.smithi163.stdout:5/185: chown d0 1834069 1 2024-02-16T19:24:22.507 INFO:tasks.workunit.client.1.smithi163.stdout:2/317: dwrite d2/d2a/f46 [0,4194304] 0 2024-02-16T19:24:22.507 INFO:tasks.workunit.client.1.smithi163.stdout:2/318: write d2/f37 [447074,87494] 0 2024-02-16T19:24:22.507 INFO:tasks.workunit.client.1.smithi163.stdout:1/142: truncate f6 2275138 0 2024-02-16T19:24:22.508 INFO:tasks.workunit.client.0.smithi040.stdout:1/772: getdents d17/d26/d5f/d78/d7e 0 2024-02-16T19:24:22.508 INFO:tasks.workunit.client.1.smithi163.stdout:4/179: write d11/d18/d1f/d29/f31 [665601,53093] 0 2024-02-16T19:24:22.509 INFO:tasks.workunit.client.1.smithi163.stdout:3/248: mknod d0/dc/d1d/d29/d38/c5f 0 2024-02-16T19:24:22.510 INFO:tasks.workunit.client.0.smithi040.stdout:7/716: sync 2024-02-16T19:24:22.511 INFO:tasks.workunit.client.0.smithi040.stdout:4/631: sync 2024-02-16T19:24:22.511 INFO:tasks.workunit.client.0.smithi040.stdout:3/867: sync 2024-02-16T19:24:22.511 INFO:tasks.workunit.client.0.smithi040.stdout:7/717: truncate d21/f62 5518381 0 2024-02-16T19:24:22.511 INFO:tasks.workunit.client.0.smithi040.stdout:4/632: fdatasync dd/d58/d5e/f7f 0 2024-02-16T19:24:22.512 INFO:tasks.workunit.client.0.smithi040.stdout:0/941: creat db/d2f/d23/d3b/dd5/d115/f148 x:0 0 0 2024-02-16T19:24:22.513 INFO:tasks.workunit.client.1.smithi163.stdout:8/189: rename d0/d7/d19/f1b to d0/d7/d19/d1f/d30/f47 0 2024-02-16T19:24:22.514 INFO:tasks.workunit.client.0.smithi040.stdout:6/992: rename d0/d30/d4b/dcc to d0/d1/d12/d3b/d48/d4f/d14a 0 2024-02-16T19:24:22.516 INFO:tasks.workunit.client.0.smithi040.stdout:1/773: getdents d17/d26/d9a/dfb/d92 0 2024-02-16T19:24:22.517 INFO:tasks.workunit.client.0.smithi040.stdout:8/906: dwrite d6/d12/d51/d6e/db7/ddf/df8/d1d/f7a [4194304,4194304] 0 2024-02-16T19:24:22.519 INFO:tasks.workunit.client.1.smithi163.stdout:0/351: dwrite d11/d22/f42 [0,4194304] 0 2024-02-16T19:24:22.519 INFO:tasks.workunit.client.1.smithi163.stdout:7/181: dwrite d11/f15 [0,4194304] 0 2024-02-16T19:24:22.519 INFO:tasks.workunit.client.1.smithi163.stdout:5/186: mkdir d0/d22/d3b 0 2024-02-16T19:24:22.519 INFO:tasks.workunit.client.1.smithi163.stdout:5/187: write d0/d22/f2e [81658,103458] 0 2024-02-16T19:24:22.520 INFO:tasks.workunit.client.0.smithi040.stdout:3/868: mknod d1/d4/d6e/d8a/c117 0 2024-02-16T19:24:22.522 INFO:tasks.workunit.client.0.smithi040.stdout:1/774: dread d17/d26/d5f/f62 [0,4194304] 0 2024-02-16T19:24:22.523 INFO:tasks.workunit.client.0.smithi040.stdout:3/869: dread d1/d7/d30/f81 [0,4194304] 0 2024-02-16T19:24:22.523 INFO:tasks.workunit.client.0.smithi040.stdout:3/870: truncate d1/d8c/fc3 808408 0 2024-02-16T19:24:22.523 INFO:tasks.workunit.client.0.smithi040.stdout:3/871: fsync d1/dc/d42/d58/f72 0 2024-02-16T19:24:22.523 INFO:tasks.workunit.client.0.smithi040.stdout:3/872: write d1/d4/d6e/dc2/dce/fd8 [928595,11165] 0 2024-02-16T19:24:22.523 INFO:tasks.workunit.client.0.smithi040.stdout:3/873: write d1/d4/f41 [1058709,76128] 0 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:5/766: sync 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:2/716: sync 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:2/717: chown d5/d7/d1e/d55/l83 6567651 1 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:2/718: stat d5/d5c/f7f 0 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:5/767: read - d1/da/d1c/dba/db1/dde/ddf/feb zero size 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:8/907: write d6/d2b/d44/f49 [1860001,113147] 0 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:8/908: truncate d6/d2b/f9a 4826869 0 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:8/909: read d6/d2b/f63 [3396598,128821] 0 2024-02-16T19:24:22.528 INFO:tasks.workunit.client.0.smithi040.stdout:7/718: rename d21/d4d/d77/d87/fc7 to d21/d29/ff1 0 2024-02-16T19:24:22.529 INFO:tasks.workunit.client.1.smithi163.stdout:1/143: rmdir d7/dc 39 2024-02-16T19:24:22.530 INFO:tasks.workunit.client.1.smithi163.stdout:1/144: dread d7/f26 [0,4194304] 0 2024-02-16T19:24:22.531 INFO:tasks.workunit.client.0.smithi040.stdout:4/633: write dd/de/d10/f8a [2570772,91508] 0 2024-02-16T19:24:22.532 INFO:tasks.workunit.client.0.smithi040.stdout:2/719: dread d5/d7/d10/d40/f9a [0,4194304] 0 2024-02-16T19:24:22.532 INFO:tasks.workunit.client.0.smithi040.stdout:6/993: mknod d0/d1/d12/d3b/d48/d73/d70/d149/c14b 0 2024-02-16T19:24:22.533 INFO:tasks.workunit.client.1.smithi163.stdout:0/352: write fd [7055079,114992] 0 2024-02-16T19:24:22.533 INFO:tasks.workunit.client.1.smithi163.stdout:0/353: creat d11/d21/d50/d58/d6b/f7a x:0 0 0 2024-02-16T19:24:22.536 INFO:tasks.workunit.client.1.smithi163.stdout:4/180: unlink cb 0 2024-02-16T19:24:22.536 INFO:tasks.workunit.client.1.smithi163.stdout:4/181: chown d11/l1a 30336 1 2024-02-16T19:24:22.536 INFO:tasks.workunit.client.1.smithi163.stdout:8/190: dwrite d0/dc/f2d [0,4194304] 0 2024-02-16T19:24:22.536 INFO:tasks.workunit.client.1.smithi163.stdout:8/191: write d0/d7/d19/d1f/d32/f40 [947697,96400] 0 2024-02-16T19:24:22.536 INFO:tasks.workunit.client.1.smithi163.stdout:8/192: chown d0/d7/c11 0 1 2024-02-16T19:24:22.539 INFO:tasks.workunit.client.1.smithi163.stdout:3/249: creat d0/d3e/f60 x:0 0 0 2024-02-16T19:24:22.542 INFO:tasks.workunit.client.1.smithi163.stdout:9/415: mkdir d5/de/d30/d3d/d93 0 2024-02-16T19:24:22.544 INFO:tasks.workunit.client.0.smithi040.stdout:8/910: dread d6/d64/d50/f74 [4194304,4194304] 0 2024-02-16T19:24:22.544 INFO:tasks.workunit.client.0.smithi040.stdout:8/911: fsync d6/d12/d51/d6e/db7/ddf/df8/f32 0 2024-02-16T19:24:22.544 INFO:tasks.workunit.client.0.smithi040.stdout:8/912: truncate d6/d2b/d44/f8d 1710819 0 2024-02-16T19:24:22.544 INFO:tasks.workunit.client.0.smithi040.stdout:1/775: truncate d17/d26/d5f/d63/f66 2918056 0 2024-02-16T19:24:22.549 INFO:tasks.workunit.client.1.smithi163.stdout:2/319: rename d2/d2a/l30 to d2/d26/d54/l67 0 2024-02-16T19:24:22.549 INFO:tasks.workunit.client.1.smithi163.stdout:7/182: dwrite d11/d26/f2b [4194304,4194304] 0 2024-02-16T19:24:22.558 INFO:tasks.workunit.client.0.smithi040.stdout:7/719: mkdir d21/d4d/d77/d87/d9b/df2 0 2024-02-16T19:24:22.561 INFO:tasks.workunit.client.0.smithi040.stdout:6/994: dwrite d0/d30/d32/f21 [4194304,4194304] 0 2024-02-16T19:24:22.561 INFO:tasks.workunit.client.0.smithi040.stdout:6/995: write d0/d37/d4a/f12d [1008270,13594] 0 2024-02-16T19:24:22.561 INFO:tasks.workunit.client.0.smithi040.stdout:6/996: write d0/d37/f12e [367813,83962] 0 2024-02-16T19:24:22.561 INFO:tasks.workunit.client.0.smithi040.stdout:6/997: dread - d0/d30/d32/dd2/dbc/de7/f102 zero size 2024-02-16T19:24:22.566 INFO:tasks.workunit.client.0.smithi040.stdout:2/720: unlink d5/d7/d1e/d4c/d52/daa/fd5 0 2024-02-16T19:24:22.566 INFO:tasks.workunit.client.1.smithi163.stdout:5/188: mkdir d0/d22/d28/d36/d3c 0 2024-02-16T19:24:22.567 INFO:tasks.workunit.client.0.smithi040.stdout:7/720: dread d21/f48 [0,4194304] 0 2024-02-16T19:24:22.567 INFO:tasks.workunit.client.0.smithi040.stdout:7/721: dread - d21/d4d/d7d/fce zero size 2024-02-16T19:24:22.568 INFO:tasks.workunit.client.0.smithi040.stdout:1/776: link d17/d26/d4c/dab/fb1 d17/d26/d9a/dfb/ffd 0 2024-02-16T19:24:22.569 INFO:tasks.workunit.client.0.smithi040.stdout:1/777: readlink d17/la1 0 2024-02-16T19:24:22.570 INFO:tasks.workunit.client.0.smithi040.stdout:3/874: getdents d1/dc/d42/d4a 0 2024-02-16T19:24:22.572 INFO:tasks.workunit.client.1.smithi163.stdout:8/193: dwrite d0/f20 [0,4194304] 0 2024-02-16T19:24:22.572 INFO:tasks.workunit.client.1.smithi163.stdout:1/145: mknod d7/d20/c2c 0 2024-02-16T19:24:22.574 INFO:tasks.workunit.client.1.smithi163.stdout:7/183: dread d11/f12 [0,4194304] 0 2024-02-16T19:24:22.575 INFO:tasks.workunit.client.1.smithi163.stdout:7/184: write f5 [5497990,53618] 0 2024-02-16T19:24:22.575 INFO:tasks.workunit.client.1.smithi163.stdout:7/185: creat d11/d26/d36/f40 x:0 0 0 2024-02-16T19:24:22.575 INFO:tasks.workunit.client.0.smithi040.stdout:4/634: dread dd/de/d10/f20 [0,4194304] 0 2024-02-16T19:24:22.575 INFO:tasks.workunit.client.0.smithi040.stdout:4/635: chown dd/caf 0 1 2024-02-16T19:24:22.575 INFO:tasks.workunit.client.0.smithi040.stdout:4/636: creat dd/d58/fc5 x:0 0 0 2024-02-16T19:24:22.576 INFO:tasks.workunit.client.0.smithi040.stdout:8/913: dwrite d6/f75 [0,4194304] 0 2024-02-16T19:24:22.577 INFO:tasks.workunit.client.1.smithi163.stdout:0/354: symlink d11/d21/d50/d58/l7b 0 2024-02-16T19:24:22.578 INFO:tasks.workunit.client.1.smithi163.stdout:2/320: dwrite d2/f18 [0,4194304] 0 2024-02-16T19:24:22.580 INFO:tasks.workunit.client.1.smithi163.stdout:4/182: symlink d11/d18/d1f/d23/l32 0 2024-02-16T19:24:22.580 INFO:tasks.workunit.client.1.smithi163.stdout:4/183: getdents d11/d18/d1f 0 2024-02-16T19:24:22.584 INFO:tasks.workunit.client.0.smithi040.stdout:6/998: rename d0/d30/d32/ce0 to d0/d30/d5b/dbe/c14c 0 2024-02-16T19:24:22.589 INFO:tasks.workunit.client.1.smithi163.stdout:3/250: truncate d0/dc/f3c 1853217 0 2024-02-16T19:24:22.591 INFO:tasks.workunit.client.0.smithi040.stdout:2/721: mknod d5/d7/d12/d1b/dae/cec 0 2024-02-16T19:24:22.591 INFO:tasks.workunit.client.0.smithi040.stdout:2/722: chown d5/d8/d42/cb4 9890413 1 2024-02-16T19:24:22.591 INFO:tasks.workunit.client.0.smithi040.stdout:2/723: chown d5/d7/d1e/d55/da3/lb2 135882036 1 2024-02-16T19:24:22.592 INFO:tasks.workunit.client.0.smithi040.stdout:2/724: stat l3 0 2024-02-16T19:24:22.592 INFO:tasks.workunit.client.0.smithi040.stdout:2/725: dread - d5/d8/fc2 zero size 2024-02-16T19:24:22.592 INFO:tasks.workunit.client.0.smithi040.stdout:2/726: creat d5/d7/d1e/d31/d4f/d63/d8b/fed x:0 0 0 2024-02-16T19:24:22.593 INFO:tasks.workunit.client.1.smithi163.stdout:9/416: creat d5/de/d30/d3d/d49/d50/f94 x:0 0 0 2024-02-16T19:24:22.593 INFO:tasks.workunit.client.1.smithi163.stdout:9/417: truncate d5/de/f28 1400020 0 2024-02-16T19:24:22.599 INFO:tasks.workunit.client.0.smithi040.stdout:3/875: unlink d1/dc/d42/d4a/d57/d79/cbe 0 2024-02-16T19:24:22.599 INFO:tasks.workunit.client.0.smithi040.stdout:5/768: write d1/da/d51/f58 [206113,69412] 0 2024-02-16T19:24:22.600 INFO:tasks.workunit.client.0.smithi040.stdout:5/769: write d1/da/d1c/d73/fbe [121575,45307] 0 2024-02-16T19:24:22.601 INFO:tasks.workunit.client.1.smithi163.stdout:8/194: mkdir d0/d7/d19/d1f/d30/d48 0 2024-02-16T19:24:22.603 INFO:tasks.workunit.client.1.smithi163.stdout:1/146: getdents d7/dc 0 2024-02-16T19:24:22.604 INFO:tasks.workunit.client.1.smithi163.stdout:1/147: write d7/dc/f17 [249419,105270] 0 2024-02-16T19:24:22.604 INFO:tasks.workunit.client.1.smithi163.stdout:1/148: dread - d7/d20/f24 zero size 2024-02-16T19:24:22.604 INFO:tasks.workunit.client.1.smithi163.stdout:1/149: dread - d7/d20/f24 zero size 2024-02-16T19:24:22.605 INFO:tasks.workunit.client.0.smithi040.stdout:8/914: dwrite d6/d12/d51/d6e/ffe [0,4194304] 0 2024-02-16T19:24:22.605 INFO:tasks.workunit.client.0.smithi040.stdout:8/915: write d6/d64/faa [267816,94237] 0 2024-02-16T19:24:22.605 INFO:tasks.workunit.client.1.smithi163.stdout:7/186: dwrite d11/f1f [0,4194304] 0 2024-02-16T19:24:22.609 INFO:tasks.workunit.client.1.smithi163.stdout:5/189: dwrite d0/d22/f2e [0,4194304] 0 2024-02-16T19:24:22.616 INFO:tasks.workunit.client.1.smithi163.stdout:0/355: truncate f8 657265 0 2024-02-16T19:24:22.616 INFO:tasks.workunit.client.1.smithi163.stdout:0/356: chown d11/d22/f2d 30571 1 2024-02-16T19:24:22.623 INFO:tasks.workunit.client.1.smithi163.stdout:5/190: write d0/d2/fb [503478,115952] 0 2024-02-16T19:24:22.624 INFO:tasks.workunit.client.1.smithi163.stdout:5/191: fsync d0/d22/f33 0 2024-02-16T19:24:22.624 INFO:tasks.workunit.client.1.smithi163.stdout:5/192: chown d0/d1c 93 1 2024-02-16T19:24:22.624 INFO:tasks.workunit.client.1.smithi163.stdout:8/195: dwrite d0/d7/d19/d1f/d30/f47 [0,4194304] 0 2024-02-16T19:24:22.624 INFO:tasks.workunit.client.1.smithi163.stdout:2/321: dread d2/d2a/d39/f42 [0,4194304] 0 2024-02-16T19:24:22.625 INFO:tasks.workunit.client.0.smithi040.stdout:4/637: mkdir dd/d58/d8c/d99/dc6 0 2024-02-16T19:24:22.628 INFO:tasks.workunit.client.0.smithi040.stdout:6/999: mknod d0/d1/c14d 0 2024-02-16T19:24:22.629 INFO:tasks.workunit.client.1.smithi163.stdout:0/357: dread d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:22.632 INFO:tasks.workunit.client.0.smithi040.stdout:2/727: symlink d5/d7/d10/dbc/lee 0 2024-02-16T19:24:22.635 INFO:tasks.workunit.client.1.smithi163.stdout:3/251: symlink d0/dc/d1c/d32/l61 0 2024-02-16T19:24:22.635 INFO:tasks.workunit.client.1.smithi163.stdout:3/252: readlink d0/l3 0 2024-02-16T19:24:22.636 INFO:tasks.workunit.client.0.smithi040.stdout:7/722: rmdir d21/d29/d79 39 2024-02-16T19:24:22.637 INFO:tasks.workunit.client.1.smithi163.stdout:9/418: creat d5/de/d16/f95 x:0 0 0 2024-02-16T19:24:22.638 INFO:tasks.workunit.client.0.smithi040.stdout:3/876: mknod d1/dc/d35/d53/d64/c118 0 2024-02-16T19:24:22.638 INFO:tasks.workunit.client.0.smithi040.stdout:3/877: dread - d1/d7/d30/f92 zero size 2024-02-16T19:24:22.641 INFO:tasks.workunit.client.0.smithi040.stdout:5/770: symlink d1/da/d1c/d73/lef 0 2024-02-16T19:24:22.641 INFO:tasks.workunit.client.0.smithi040.stdout:5/771: creat d1/da/db0/ff0 x:0 0 0 2024-02-16T19:24:22.644 INFO:tasks.workunit.client.1.smithi163.stdout:1/150: stat f6 0 2024-02-16T19:24:22.644 INFO:tasks.workunit.client.1.smithi163.stdout:1/151: truncate d7/f22 631583 0 2024-02-16T19:24:22.645 INFO:tasks.workunit.client.0.smithi040.stdout:3/878: dread d1/d4/d6e/fc9 [0,4194304] 0 2024-02-16T19:24:22.648 INFO:tasks.workunit.client.0.smithi040.stdout:8/916: creat d6/d12/d51/db6/f119 x:0 0 0 2024-02-16T19:24:22.648 INFO:tasks.workunit.client.0.smithi040.stdout:8/917: stat d6/d64/d50/l8e 0 2024-02-16T19:24:22.649 INFO:tasks.workunit.client.0.smithi040.stdout:4/638: mknod dd/de/d10/d36/d9e/cc7 0 2024-02-16T19:24:22.651 INFO:tasks.workunit.client.1.smithi163.stdout:7/187: rmdir d11/d3a 0 2024-02-16T19:24:22.652 INFO:tasks.workunit.client.1.smithi163.stdout:7/188: dread - d11/d30/f32 zero size 2024-02-16T19:24:22.652 INFO:tasks.workunit.client.0.smithi040.stdout:2/728: symlink d5/d7/d1e/lef 0 2024-02-16T19:24:22.654 INFO:tasks.workunit.client.0.smithi040.stdout:7/723: mknod d21/d29/d79/dbc/cf3 0 2024-02-16T19:24:22.654 INFO:tasks.workunit.client.0.smithi040.stdout:8/918: dread d6/d64/fc4 [0,4194304] 0 2024-02-16T19:24:22.654 INFO:tasks.workunit.client.0.smithi040.stdout:8/919: readlink d6/lf 0 2024-02-16T19:24:22.655 INFO:tasks.workunit.client.1.smithi163.stdout:5/193: link d0/l19 d0/d2/d34/l3d 0 2024-02-16T19:24:22.655 INFO:tasks.workunit.client.1.smithi163.stdout:5/194: fsync d0/d2/f16 0 2024-02-16T19:24:22.665 INFO:tasks.workunit.client.1.smithi163.stdout:2/322: mknod d2/d2a/c68 0 2024-02-16T19:24:22.667 INFO:tasks.workunit.client.1.smithi163.stdout:0/358: mkdir d11/d21/d7c 0 2024-02-16T19:24:22.669 INFO:tasks.workunit.client.1.smithi163.stdout:5/195: dread d0/d2/fb [0,4194304] 0 2024-02-16T19:24:22.670 INFO:tasks.workunit.client.1.smithi163.stdout:5/196: readlink d0/d22/l2b 0 2024-02-16T19:24:22.670 INFO:tasks.workunit.client.1.smithi163.stdout:5/197: creat d0/d22/f3e x:0 0 0 2024-02-16T19:24:22.670 INFO:tasks.workunit.client.1.smithi163.stdout:3/253: mkdir d0/dc/d1c/d32/d62 0 2024-02-16T19:24:22.671 INFO:tasks.workunit.client.0.smithi040.stdout:2/729: link d5/d7/d1e/d55/cd7 d5/d7/d12/dca/cf0 0 2024-02-16T19:24:22.676 INFO:tasks.workunit.client.0.smithi040.stdout:8/920: unlink d6/d64/d50/fbe 0 2024-02-16T19:24:22.677 INFO:tasks.workunit.client.1.smithi163.stdout:1/152: mkdir d7/dc/d2d 0 2024-02-16T19:24:22.679 INFO:tasks.workunit.client.1.smithi163.stdout:1/153: write d7/fb [786587,58835] 0 2024-02-16T19:24:22.679 INFO:tasks.workunit.client.1.smithi163.stdout:1/154: creat d7/f2e x:0 0 0 2024-02-16T19:24:22.679 INFO:tasks.workunit.client.1.smithi163.stdout:1/155: write d7/f1d [5027283,83642] 0 2024-02-16T19:24:22.679 INFO:tasks.workunit.client.1.smithi163.stdout:1/156: chown d7/f22 1734 1 2024-02-16T19:24:22.680 INFO:tasks.workunit.client.0.smithi040.stdout:5/772: rmdir d1/d62/d9a 39 2024-02-16T19:24:22.681 INFO:tasks.workunit.client.1.smithi163.stdout:8/196: symlink d0/d7/d19/l49 0 2024-02-16T19:24:22.682 INFO:tasks.workunit.client.1.smithi163.stdout:8/197: stat d0/d7/d19/d1f/d32 0 2024-02-16T19:24:22.682 INFO:tasks.workunit.client.1.smithi163.stdout:8/198: dread d0/f44 [0,4194304] 0 2024-02-16T19:24:22.682 INFO:tasks.workunit.client.1.smithi163.stdout:8/199: chown d0/d24/d3c 0 1 2024-02-16T19:24:22.682 INFO:tasks.workunit.client.0.smithi040.stdout:5/773: write d1/da/d51/f85 [161659,114378] 0 2024-02-16T19:24:22.683 INFO:tasks.workunit.client.0.smithi040.stdout:4/639: rename dd/de/d45/c92 to dd/d58/d8c/cc8 0 2024-02-16T19:24:22.684 INFO:tasks.workunit.client.1.smithi163.stdout:2/323: creat d2/d26/d54/d58/f69 x:0 0 0 2024-02-16T19:24:22.686 INFO:tasks.workunit.client.0.smithi040.stdout:8/921: mknod d6/d12/d51/c11a 0 2024-02-16T19:24:22.688 INFO:tasks.workunit.client.1.smithi163.stdout:3/254: dwrite d0/dc/d1d/d49/f59 [4194304,4194304] 0 2024-02-16T19:24:22.688 INFO:tasks.workunit.client.1.smithi163.stdout:7/189: dwrite ff [0,4194304] 0 2024-02-16T19:24:22.688 INFO:tasks.workunit.client.1.smithi163.stdout:7/190: dread - d11/d26/d36/f40 zero size 2024-02-16T19:24:22.690 INFO:tasks.workunit.client.1.smithi163.stdout:0/359: creat d11/d12/d67/d6a/d6d/f7d x:0 0 0 2024-02-16T19:24:22.691 INFO:tasks.workunit.client.0.smithi040.stdout:2/730: rename d5/c44 to d5/d7/d1e/d55/da3/cf1 0 2024-02-16T19:24:22.692 INFO:tasks.workunit.client.1.smithi163.stdout:8/200: dread d0/dc/f2d [0,4194304] 0 2024-02-16T19:24:22.694 INFO:tasks.workunit.client.0.smithi040.stdout:4/640: unlink dd/de/d45/fa7 0 2024-02-16T19:24:22.694 INFO:tasks.workunit.client.0.smithi040.stdout:4/641: stat dd/de/d10/d36/d9e/d32/f41 0 2024-02-16T19:24:22.694 INFO:tasks.workunit.client.0.smithi040.stdout:4/642: chown dd/de/d10/d36/d9e/d32/c35 1707 1 2024-02-16T19:24:22.694 INFO:tasks.workunit.client.0.smithi040.stdout:4/643: chown dd/d58/d8c/fbe 6315 1 2024-02-16T19:24:22.694 INFO:tasks.workunit.client.1.smithi163.stdout:5/198: mkdir d0/d22/d3b/d3f 0 2024-02-16T19:24:22.696 INFO:tasks.workunit.client.0.smithi040.stdout:2/731: rename d5/d7/d1e/f70 to d5/d7/d1e/d31/d4f/ff2 0 2024-02-16T19:24:22.696 INFO:tasks.workunit.client.0.smithi040.stdout:2/732: write d5/d7/d10/d40/fa9 [2960216,31194] 0 2024-02-16T19:24:22.697 INFO:tasks.workunit.client.0.smithi040.stdout:4/644: creat dd/de/d71/dc2/fc9 x:0 0 0 2024-02-16T19:24:22.699 INFO:tasks.workunit.client.1.smithi163.stdout:0/360: write d11/d21/d50/f52 [361789,122738] 0 2024-02-16T19:24:22.703 INFO:tasks.workunit.client.1.smithi163.stdout:2/324: creat d2/d2a/d2d/f6a x:0 0 0 2024-02-16T19:24:22.703 INFO:tasks.workunit.client.1.smithi163.stdout:2/325: creat d2/d2a/d2d/f6b x:0 0 0 2024-02-16T19:24:22.703 INFO:tasks.workunit.client.0.smithi040.stdout:5/774: dwrite d1/d45/dab/fc4 [0,4194304] 0 2024-02-16T19:24:22.706 INFO:tasks.workunit.client.1.smithi163.stdout:1/157: dwrite d7/d20/f24 [0,4194304] 0 2024-02-16T19:24:22.707 INFO:tasks.workunit.client.1.smithi163.stdout:2/326: dread d2/d44/f45 [0,4194304] 0 2024-02-16T19:24:22.709 INFO:tasks.workunit.client.1.smithi163.stdout:7/191: symlink d11/d26/d36/l41 0 2024-02-16T19:24:22.709 INFO:tasks.workunit.client.1.smithi163.stdout:7/192: write d11/d24/f3d [821234,110630] 0 2024-02-16T19:24:22.709 INFO:tasks.workunit.client.1.smithi163.stdout:7/193: stat d11/c1a 0 2024-02-16T19:24:22.709 INFO:tasks.workunit.client.1.smithi163.stdout:7/194: stat d11/d24/d25 0 2024-02-16T19:24:22.709 INFO:tasks.workunit.client.1.smithi163.stdout:8/201: creat d0/d24/d27/d45/f4a x:0 0 0 2024-02-16T19:24:22.710 INFO:tasks.workunit.client.1.smithi163.stdout:8/202: dread d0/d7/d19/d1f/d32/f39 [0,4194304] 0 2024-02-16T19:24:22.710 INFO:tasks.workunit.client.1.smithi163.stdout:5/199: rename d0/fd to d0/d1c/f40 0 2024-02-16T19:24:22.710 INFO:tasks.workunit.client.1.smithi163.stdout:5/200: chown d0/d22/d3b 9472878 1 2024-02-16T19:24:22.712 INFO:tasks.workunit.client.0.smithi040.stdout:4/645: dread dd/d58/d5e/f61 [0,4194304] 0 2024-02-16T19:24:22.712 INFO:tasks.workunit.client.1.smithi163.stdout:0/361: creat d11/d21/d50/d58/d74/f7e x:0 0 0 2024-02-16T19:24:22.716 INFO:tasks.workunit.client.0.smithi040.stdout:2/733: dwrite d5/d7/d1e/f25 [0,4194304] 0 2024-02-16T19:24:22.716 INFO:tasks.workunit.client.0.smithi040.stdout:2/734: write d5/d5c/d65/d9d/fc7 [673225,53163] 0 2024-02-16T19:24:22.717 INFO:tasks.workunit.client.1.smithi163.stdout:1/158: getdents d7/d20/d28 0 2024-02-16T19:24:22.717 INFO:tasks.workunit.client.1.smithi163.stdout:1/159: rename d7/d20/d28 to d7/d20/d28/d2f 22 2024-02-16T19:24:22.717 INFO:tasks.workunit.client.1.smithi163.stdout:1/160: write d7/f26 [930670,86302] 0 2024-02-16T19:24:22.717 INFO:tasks.workunit.client.1.smithi163.stdout:1/161: write d7/f2e [589844,107460] 0 2024-02-16T19:24:22.718 INFO:tasks.workunit.client.1.smithi163.stdout:2/327: mkdir d2/d2a/d39/d66/d6c 0 2024-02-16T19:24:22.720 INFO:tasks.workunit.client.1.smithi163.stdout:3/255: getdents d0/dc/d1d/d29 0 2024-02-16T19:24:22.721 INFO:tasks.workunit.client.1.smithi163.stdout:8/203: dread d0/dc/f22 [0,4194304] 0 2024-02-16T19:24:22.721 INFO:tasks.workunit.client.1.smithi163.stdout:8/204: write d0/f44 [902243,12036] 0 2024-02-16T19:24:22.722 INFO:tasks.workunit.client.1.smithi163.stdout:7/195: mknod d11/d24/c42 0 2024-02-16T19:24:22.722 INFO:tasks.workunit.client.1.smithi163.stdout:7/196: chown fa 12656123 1 2024-02-16T19:24:22.722 INFO:tasks.workunit.client.1.smithi163.stdout:7/197: stat d11/d26/f2b 0 2024-02-16T19:24:22.722 INFO:tasks.workunit.client.1.smithi163.stdout:7/198: chown d11/d30/f32 867 1 2024-02-16T19:24:22.722 INFO:tasks.workunit.client.1.smithi163.stdout:7/199: creat d11/f43 x:0 0 0 2024-02-16T19:24:22.723 INFO:tasks.workunit.client.1.smithi163.stdout:5/201: symlink d0/d22/d28/l41 0 2024-02-16T19:24:22.723 INFO:tasks.workunit.client.1.smithi163.stdout:9/419: dwrite d5/de/d16/f2a [0,4194304] 0 2024-02-16T19:24:22.725 INFO:tasks.workunit.client.1.smithi163.stdout:0/362: mkdir d11/d22/d7f 0 2024-02-16T19:24:22.725 INFO:tasks.workunit.client.1.smithi163.stdout:9/420: dread d5/f77 [0,4194304] 0 2024-02-16T19:24:22.728 INFO:tasks.workunit.client.1.smithi163.stdout:1/162: creat d7/dc/f30 x:0 0 0 2024-02-16T19:24:22.728 INFO:tasks.workunit.client.1.smithi163.stdout:1/163: truncate d7/f2e 1056723 0 2024-02-16T19:24:22.730 INFO:tasks.workunit.client.0.smithi040.stdout:2/735: write d5/d5c/d65/f60 [2652740,103151] 0 2024-02-16T19:24:22.730 INFO:tasks.workunit.client.0.smithi040.stdout:2/736: chown d5/d7/d12/d1b 1 1 2024-02-16T19:24:22.731 INFO:tasks.workunit.client.1.smithi163.stdout:3/256: link d0/dc/l35 d0/dc/d1c/d21/l63 0 2024-02-16T19:24:22.731 INFO:tasks.workunit.client.1.smithi163.stdout:3/257: write d0/dc/d1d/d29/f30 [4789658,82358] 0 2024-02-16T19:24:22.731 INFO:tasks.workunit.client.0.smithi040.stdout:2/737: dread d5/d7/d1e/d55/f6c [0,4194304] 0 2024-02-16T19:24:22.733 INFO:tasks.workunit.client.0.smithi040.stdout:2/738: dread d5/d5c/f7f [0,4194304] 0 2024-02-16T19:24:22.733 INFO:tasks.workunit.client.0.smithi040.stdout:2/739: dread - d5/d7/d1e/d31/d4f/d63/d8b/fed zero size 2024-02-16T19:24:22.734 INFO:tasks.workunit.client.1.smithi163.stdout:8/205: rename d0/d7/l8 to d0/d24/d27/d45/l4b 0 2024-02-16T19:24:22.735 INFO:tasks.workunit.client.1.smithi163.stdout:2/328: dwrite d2/d2a/d2d/f5e [0,4194304] 0 2024-02-16T19:24:22.735 INFO:tasks.workunit.client.1.smithi163.stdout:2/329: read - d2/d2a/d2d/f6a zero size 2024-02-16T19:24:22.738 INFO:tasks.workunit.client.0.smithi040.stdout:2/740: unlink d5/d5c/d65/c8a 0 2024-02-16T19:24:22.741 INFO:tasks.workunit.client.1.smithi163.stdout:2/330: dread d2/f37 [0,4194304] 0 2024-02-16T19:24:22.741 INFO:tasks.workunit.client.1.smithi163.stdout:0/363: symlink d11/d12/d67/d6a/d6d/l80 0 2024-02-16T19:24:22.743 INFO:tasks.workunit.client.1.smithi163.stdout:9/421: link d5/de/d16/f6b d5/de/d30/d3d/d49/d6d/f96 0 2024-02-16T19:24:22.743 INFO:tasks.workunit.client.1.smithi163.stdout:9/422: truncate d5/de/d30/d31/f7f 266082 0 2024-02-16T19:24:22.744 INFO:tasks.workunit.client.1.smithi163.stdout:1/164: mknod d7/d20/d28/c31 0 2024-02-16T19:24:22.744 INFO:tasks.workunit.client.1.smithi163.stdout:1/165: chown d7/d20/d2a 0 1 2024-02-16T19:24:22.744 INFO:tasks.workunit.client.0.smithi040.stdout:2/741: dread d5/d7/d1e/d4c/fce [0,4194304] 0 2024-02-16T19:24:22.745 INFO:tasks.workunit.client.1.smithi163.stdout:3/258: mknod d0/dc/d50/c64 0 2024-02-16T19:24:22.747 INFO:tasks.workunit.client.1.smithi163.stdout:8/206: link d0/d7/d18/f3f d0/d24/d3c/f4c 0 2024-02-16T19:24:22.749 INFO:tasks.workunit.client.1.smithi163.stdout:7/200: rmdir d11/d26/d36 39 2024-02-16T19:24:22.749 INFO:tasks.workunit.client.1.smithi163.stdout:7/201: read - d11/f43 zero size 2024-02-16T19:24:22.749 INFO:tasks.workunit.client.1.smithi163.stdout:7/202: chown d11/d24/d25/c2a 0 1 2024-02-16T19:24:22.749 INFO:tasks.workunit.client.1.smithi163.stdout:7/203: chown d11/d24/d25/f2c 5 1 2024-02-16T19:24:22.751 INFO:tasks.workunit.client.1.smithi163.stdout:2/331: creat d2/d2a/d39/d66/d6c/f6d x:0 0 0 2024-02-16T19:24:22.752 INFO:tasks.workunit.client.0.smithi040.stdout:2/742: rename d5/d8 to d5/d7/d10/df3 0 2024-02-16T19:24:22.754 INFO:tasks.workunit.client.1.smithi163.stdout:0/364: truncate d11/d21/f46 2260272 0 2024-02-16T19:24:22.754 INFO:tasks.workunit.client.1.smithi163.stdout:0/365: stat d11/d12/l1e 0 2024-02-16T19:24:22.754 INFO:tasks.workunit.client.1.smithi163.stdout:0/366: truncate d11/d22/f3f 1419462 0 2024-02-16T19:24:22.754 INFO:tasks.workunit.client.1.smithi163.stdout:0/367: truncate d11/f1f 1976942 0 2024-02-16T19:24:22.755 INFO:tasks.workunit.client.1.smithi163.stdout:4/184: sync 2024-02-16T19:24:22.755 INFO:tasks.workunit.client.1.smithi163.stdout:6/206: sync 2024-02-16T19:24:22.755 INFO:tasks.workunit.client.1.smithi163.stdout:6/207: chown d0/d6/d9/cb 0 1 2024-02-16T19:24:22.755 INFO:tasks.workunit.client.1.smithi163.stdout:9/423: mkdir d5/de/d30/d3d/d49/d6d/d97 0 2024-02-16T19:24:22.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/166: mkdir d7/dc/d2d/d32 0 2024-02-16T19:24:22.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/167: fdatasync f2 0 2024-02-16T19:24:22.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/168: chown d7/dc/l11 27024933 1 2024-02-16T19:24:22.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/169: creat d7/dc/f33 x:0 0 0 2024-02-16T19:24:22.757 INFO:tasks.workunit.client.1.smithi163.stdout:0/368: dread d11/d12/d20/f30 [0,4194304] 0 2024-02-16T19:24:22.757 INFO:tasks.workunit.client.1.smithi163.stdout:0/369: chown d11/d21/d50/d58/d72 2824 1 2024-02-16T19:24:22.758 INFO:tasks.workunit.client.1.smithi163.stdout:3/259: symlink d0/dc/d1d/d49/l65 0 2024-02-16T19:24:22.763 INFO:tasks.workunit.client.1.smithi163.stdout:1/170: dread d7/f1d [4194304,4194304] 0 2024-02-16T19:24:22.763 INFO:tasks.workunit.client.1.smithi163.stdout:6/208: rename d0/dd/f12 to d0/d6/df/f41 0 2024-02-16T19:24:22.763 INFO:tasks.workunit.client.1.smithi163.stdout:5/202: dwrite d0/f2c [0,4194304] 0 2024-02-16T19:24:22.763 INFO:tasks.workunit.client.1.smithi163.stdout:5/203: chown d0/c2d 403015 1 2024-02-16T19:24:22.765 INFO:tasks.workunit.client.1.smithi163.stdout:9/424: truncate d5/de/d30/d31/f54 2047805 0 2024-02-16T19:24:22.770 INFO:tasks.workunit.client.1.smithi163.stdout:0/370: creat d11/d21/d7c/f81 x:0 0 0 2024-02-16T19:24:22.771 INFO:tasks.workunit.client.1.smithi163.stdout:4/185: creat d11/d18/d1f/f33 x:0 0 0 2024-02-16T19:24:22.774 INFO:tasks.workunit.client.1.smithi163.stdout:1/171: mknod d7/d20/d28/c34 0 2024-02-16T19:24:22.774 INFO:tasks.workunit.client.1.smithi163.stdout:3/260: read d0/dc/d1d/d29/f45 [2575971,74929] 0 2024-02-16T19:24:22.775 INFO:tasks.workunit.client.1.smithi163.stdout:6/209: symlink d0/l42 0 2024-02-16T19:24:22.775 INFO:tasks.workunit.client.1.smithi163.stdout:6/210: write d0/d6/d9/f2d [4964749,60282] 0 2024-02-16T19:24:22.775 INFO:tasks.workunit.client.1.smithi163.stdout:6/211: fdatasync d0/d6/d9/d2e/d33/f3c 0 2024-02-16T19:24:22.779 INFO:tasks.workunit.client.1.smithi163.stdout:1/172: dread d7/f2e [0,4194304] 0 2024-02-16T19:24:22.779 INFO:tasks.workunit.client.1.smithi163.stdout:1/173: chown d7/dc/c10 1844639 1 2024-02-16T19:24:22.781 INFO:tasks.workunit.client.1.smithi163.stdout:0/371: rename d11/d12/d20/l31 to d11/d21/l82 0 2024-02-16T19:24:22.782 INFO:tasks.workunit.client.1.smithi163.stdout:4/186: mkdir d11/d18/d1f/d34 0 2024-02-16T19:24:22.783 INFO:tasks.workunit.client.1.smithi163.stdout:2/332: dwrite d2/d2a/d2d/f6a [0,4194304] 0 2024-02-16T19:24:22.784 INFO:tasks.workunit.client.1.smithi163.stdout:3/261: truncate d0/dc/d1c/f2c 3792103 0 2024-02-16T19:24:22.784 INFO:tasks.workunit.client.1.smithi163.stdout:6/212: rmdir d0/d6/df 39 2024-02-16T19:24:22.786 INFO:tasks.workunit.client.1.smithi163.stdout:0/372: creat d11/d22/d7f/f83 x:0 0 0 2024-02-16T19:24:22.787 INFO:tasks.workunit.client.1.smithi163.stdout:0/373: read - d11/f62 zero size 2024-02-16T19:24:22.787 INFO:tasks.workunit.client.1.smithi163.stdout:4/187: mknod d11/d2d/c35 0 2024-02-16T19:24:22.788 INFO:tasks.workunit.client.1.smithi163.stdout:6/213: symlink d0/d6/df/d30/l43 0 2024-02-16T19:24:22.790 INFO:tasks.workunit.client.1.smithi163.stdout:4/188: truncate fd 2198654 0 2024-02-16T19:24:22.791 INFO:tasks.workunit.client.1.smithi163.stdout:0/374: unlink d11/d12/d20/f4e 0 2024-02-16T19:24:22.791 INFO:tasks.workunit.client.1.smithi163.stdout:4/189: creat d11/d18/d1f/d29/f36 x:0 0 0 2024-02-16T19:24:22.791 INFO:tasks.workunit.client.1.smithi163.stdout:4/190: chown d11/l22 48 1 2024-02-16T19:24:22.792 INFO:tasks.workunit.client.1.smithi163.stdout:4/191: stat d11/d18/d1f/d29 0 2024-02-16T19:24:22.792 INFO:tasks.workunit.client.0.smithi040.stdout:2/743: dwrite d5/d7/d1e/d31/f7d [0,4194304] 0 2024-02-16T19:24:22.792 INFO:tasks.workunit.client.0.smithi040.stdout:2/744: chown d5/d7/d12/dca/cf0 60407453 1 2024-02-16T19:24:22.792 INFO:tasks.workunit.client.1.smithi163.stdout:0/375: link d11/d22/d3e/l4b d11/d21/d50/d58/l84 0 2024-02-16T19:24:22.793 INFO:tasks.workunit.client.1.smithi163.stdout:4/192: symlink d11/d18/d1f/l37 0 2024-02-16T19:24:22.797 INFO:tasks.workunit.client.0.smithi040.stdout:2/745: rename d5/d7/f24 to d5/d7/d1e/d4c/d52/daa/dcd/ff4 0 2024-02-16T19:24:22.827 INFO:tasks.workunit.client.1.smithi163.stdout:7/204: dwrite ff [0,4194304] 0 2024-02-16T19:24:22.827 INFO:tasks.workunit.client.1.smithi163.stdout:7/205: creat d11/f44 x:0 0 0 2024-02-16T19:24:22.827 INFO:tasks.workunit.client.1.smithi163.stdout:5/204: dwrite d0/d2/fa [0,4194304] 0 2024-02-16T19:24:22.827 INFO:tasks.workunit.client.1.smithi163.stdout:1/174: fsync d7/dc/f33 0 2024-02-16T19:24:22.830 INFO:tasks.workunit.client.1.smithi163.stdout:7/206: symlink d11/d24/d34/l45 0 2024-02-16T19:24:22.830 INFO:tasks.workunit.client.1.smithi163.stdout:7/207: creat d11/d26/f46 x:0 0 0 2024-02-16T19:24:22.832 INFO:tasks.workunit.client.1.smithi163.stdout:7/208: symlink d11/d30/l47 0 2024-02-16T19:24:22.833 INFO:tasks.workunit.client.1.smithi163.stdout:1/175: getdents d7/dc 0 2024-02-16T19:24:22.840 INFO:tasks.workunit.client.1.smithi163.stdout:9/425: dwrite d5/de/d16/f3c [0,4194304] 0 2024-02-16T19:24:22.840 INFO:tasks.workunit.client.1.smithi163.stdout:9/426: dread - d5/de/d16/f95 zero size 2024-02-16T19:24:22.840 INFO:tasks.workunit.client.1.smithi163.stdout:3/262: dwrite d0/dc/d1d/d29/f30 [0,4194304] 0 2024-02-16T19:24:22.841 INFO:tasks.workunit.client.1.smithi163.stdout:2/333: dwrite d2/f37 [0,4194304] 0 2024-02-16T19:24:22.841 INFO:tasks.workunit.client.1.smithi163.stdout:2/334: chown d2/d2a/l5c 722506822 1 2024-02-16T19:24:22.841 INFO:tasks.workunit.client.1.smithi163.stdout:2/335: chown d2/d2a/f5f 1274843 1 2024-02-16T19:24:22.841 INFO:tasks.workunit.client.1.smithi163.stdout:6/214: dwrite d0/d6/df/f32 [0,4194304] 0 2024-02-16T19:24:22.846 INFO:tasks.workunit.client.1.smithi163.stdout:0/376: dwrite d11/d21/d50/d58/d6b/f7a [0,4194304] 0 2024-02-16T19:24:22.847 INFO:tasks.workunit.client.1.smithi163.stdout:0/377: creat d11/f85 x:0 0 0 2024-02-16T19:24:22.847 INFO:tasks.workunit.client.0.smithi040.stdout:2/746: dwrite d5/d7/d10/f86 [4194304,4194304] 0 2024-02-16T19:24:22.848 INFO:tasks.workunit.client.1.smithi163.stdout:4/193: dwrite d11/d18/d1f/d29/f31 [0,4194304] 0 2024-02-16T19:24:22.848 INFO:tasks.workunit.client.1.smithi163.stdout:4/194: creat d11/d18/d1f/f38 x:0 0 0 2024-02-16T19:24:22.848 INFO:tasks.workunit.client.1.smithi163.stdout:2/336: rename d2/d44/f45 to d2/d2a/d2d/f6e 0 2024-02-16T19:24:22.848 INFO:tasks.workunit.client.1.smithi163.stdout:2/337: stat d2/d26/d54 0 2024-02-16T19:24:22.848 INFO:tasks.workunit.client.1.smithi163.stdout:2/338: fdatasync d2/f1f 0 2024-02-16T19:24:22.850 INFO:tasks.workunit.client.0.smithi040.stdout:2/747: dread d5/d7/d10/df3/f69 [0,4194304] 0 2024-02-16T19:24:22.853 INFO:tasks.workunit.client.1.smithi163.stdout:3/263: read d0/f19 [4074684,65410] 0 2024-02-16T19:24:22.853 INFO:tasks.workunit.client.1.smithi163.stdout:3/264: stat d0/dc/d1c/d32/f3f 0 2024-02-16T19:24:22.854 INFO:tasks.workunit.client.1.smithi163.stdout:3/265: fsync d0/dc/d1c/d32/f3f 0 2024-02-16T19:24:22.855 INFO:tasks.workunit.client.1.smithi163.stdout:0/378: symlink d11/d22/l86 0 2024-02-16T19:24:22.857 INFO:tasks.workunit.client.1.smithi163.stdout:5/205: dwrite d0/d22/f33 [0,4194304] 0 2024-02-16T19:24:22.858 INFO:tasks.workunit.client.1.smithi163.stdout:4/195: dread f9 [0,4194304] 0 2024-02-16T19:24:22.860 INFO:tasks.workunit.client.0.smithi040.stdout:2/748: dread d5/d7/d1e/d4c/d52/f9c [0,4194304] 0 2024-02-16T19:24:22.863 INFO:tasks.workunit.client.1.smithi163.stdout:1/176: dwrite d7/dc/f30 [0,4194304] 0 2024-02-16T19:24:22.863 INFO:tasks.workunit.client.1.smithi163.stdout:1/177: chown d7/d20/c2c 36 1 2024-02-16T19:24:22.864 INFO:tasks.workunit.client.1.smithi163.stdout:6/215: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:22.864 INFO:tasks.workunit.client.0.smithi040.stdout:2/749: symlink d5/d7/d1e/d4c/d52/d81/lf5 0 2024-02-16T19:24:22.864 INFO:tasks.workunit.client.0.smithi040.stdout:2/750: read - d5/d7/d10/d40/f91 zero size 2024-02-16T19:24:22.865 INFO:tasks.workunit.client.1.smithi163.stdout:3/266: mkdir d0/dc/d50/d66 0 2024-02-16T19:24:22.865 INFO:tasks.workunit.client.1.smithi163.stdout:0/379: truncate f6 4451216 0 2024-02-16T19:24:22.866 INFO:tasks.workunit.client.0.smithi040.stdout:2/751: link l3 d5/d7/d12/d1b/lf6 0 2024-02-16T19:24:22.867 INFO:tasks.workunit.client.0.smithi040.stdout:2/752: write d5/d7/d10/d40/fa9 [1932869,69544] 0 2024-02-16T19:24:22.867 INFO:tasks.workunit.client.0.smithi040.stdout:2/753: chown d5/d7/d12/fe6 7 1 2024-02-16T19:24:22.867 INFO:tasks.workunit.client.0.smithi040.stdout:2/754: chown d5/d7/d10/d40/f9a 130206 1 2024-02-16T19:24:22.872 INFO:tasks.workunit.client.0.smithi040.stdout:9/822: sync 2024-02-16T19:24:22.872 INFO:tasks.workunit.client.0.smithi040.stdout:2/755: rename d5/d7/d1e/d4c/d52/ca4 to d5/d7/d1e/d4c/d52/dc1/cf7 0 2024-02-16T19:24:22.875 INFO:tasks.workunit.client.1.smithi163.stdout:4/196: creat d11/d18/d1f/d34/f39 x:0 0 0 2024-02-16T19:24:22.878 INFO:tasks.workunit.client.1.smithi163.stdout:2/339: rename d2/d2a/d2d/l62 to d2/d2a/d39/l6f 0 2024-02-16T19:24:22.882 INFO:tasks.workunit.client.1.smithi163.stdout:6/216: mknod d0/d6/df/d20/d40/d1e/c44 0 2024-02-16T19:24:22.882 INFO:tasks.workunit.client.1.smithi163.stdout:6/217: readlink d0/l42 0 2024-02-16T19:24:22.883 INFO:tasks.workunit.client.0.smithi040.stdout:9/823: creat d3/d4/d55/d70/de0/f102 x:0 0 0 2024-02-16T19:24:22.884 INFO:tasks.workunit.client.1.smithi163.stdout:0/380: symlink d11/d21/d50/d58/d6b/l87 0 2024-02-16T19:24:22.885 INFO:tasks.workunit.client.1.smithi163.stdout:5/206: getdents d0/d22 0 2024-02-16T19:24:22.885 INFO:tasks.workunit.client.1.smithi163.stdout:5/207: fsync d0/d2/f7 0 2024-02-16T19:24:22.885 INFO:tasks.workunit.client.1.smithi163.stdout:5/208: write d0/d22/f3e [590791,18813] 0 2024-02-16T19:24:22.886 INFO:tasks.workunit.client.0.smithi040.stdout:9/824: mknod d3/dd3/c103 0 2024-02-16T19:24:22.888 INFO:tasks.workunit.client.0.smithi040.stdout:9/825: creat d3/d4/d11/d71/dc9/dca/f104 x:0 0 0 2024-02-16T19:24:22.888 INFO:tasks.workunit.client.0.smithi040.stdout:9/826: creat d3/d4/d55/d63/f105 x:0 0 0 2024-02-16T19:24:22.889 INFO:tasks.workunit.client.1.smithi163.stdout:3/267: dwrite d0/dc/d1c/d32/f57 [4194304,4194304] 0 2024-02-16T19:24:22.890 INFO:tasks.workunit.client.1.smithi163.stdout:0/381: rename d11/d21/d50/d58/l7b to d11/d22/l88 0 2024-02-16T19:24:22.892 INFO:tasks.workunit.client.1.smithi163.stdout:5/209: creat d0/d22/d3b/f42 x:0 0 0 2024-02-16T19:24:22.895 INFO:tasks.workunit.client.1.smithi163.stdout:4/197: rmdir d11/d18/d1f/d29 39 2024-02-16T19:24:22.899 INFO:tasks.workunit.client.1.smithi163.stdout:5/210: creat d0/d22/d28/d36/d3c/f43 x:0 0 0 2024-02-16T19:24:22.899 INFO:tasks.workunit.client.1.smithi163.stdout:5/211: fsync d0/d2/f30 0 2024-02-16T19:24:22.901 INFO:tasks.workunit.client.1.smithi163.stdout:3/268: dread d0/dc/f24 [0,4194304] 0 2024-02-16T19:24:22.901 INFO:tasks.workunit.client.1.smithi163.stdout:4/198: mkdir d11/d18/d1f/d23/d3a 0 2024-02-16T19:24:22.902 INFO:tasks.workunit.client.1.smithi163.stdout:5/212: mkdir d0/d2/d34/d44 0 2024-02-16T19:24:22.902 INFO:tasks.workunit.client.1.smithi163.stdout:4/199: symlink d11/d2d/l3b 0 2024-02-16T19:24:22.903 INFO:tasks.workunit.client.1.smithi163.stdout:4/200: fdatasync d11/d18/d1f/d34/f39 0 2024-02-16T19:24:22.903 INFO:tasks.workunit.client.1.smithi163.stdout:5/213: symlink d0/d1c/l45 0 2024-02-16T19:24:22.904 INFO:tasks.workunit.client.1.smithi163.stdout:5/214: write d0/d2/fb [922314,64831] 0 2024-02-16T19:24:22.904 INFO:tasks.workunit.client.1.smithi163.stdout:4/201: symlink d11/d18/d1f/d29/l3c 0 2024-02-16T19:24:22.906 INFO:tasks.workunit.client.1.smithi163.stdout:4/202: mkdir d11/d3d 0 2024-02-16T19:24:22.908 INFO:tasks.workunit.client.1.smithi163.stdout:4/203: link fd d11/d18/d1f/d29/f3e 0 2024-02-16T19:24:22.914 INFO:tasks.workunit.client.0.smithi040.stdout:2/756: dwrite d5/d5c/d65/d9d/fd6 [0,4194304] 0 2024-02-16T19:24:22.915 INFO:tasks.workunit.client.0.smithi040.stdout:2/757: getdents d5/d7/d1e/d4c/d52/d81 0 2024-02-16T19:24:22.915 INFO:tasks.workunit.client.0.smithi040.stdout:2/758: write d5/d7/d1e/d4c/d52/f9f [4639765,9600] 0 2024-02-16T19:24:22.917 INFO:tasks.workunit.client.1.smithi163.stdout:1/178: dwrite d7/d20/f24 [0,4194304] 0 2024-02-16T19:24:22.918 INFO:tasks.workunit.client.1.smithi163.stdout:6/218: dwrite d0/d6/f31 [4194304,4194304] 0 2024-02-16T19:24:22.919 INFO:tasks.workunit.client.1.smithi163.stdout:1/179: creat d7/d20/f35 x:0 0 0 2024-02-16T19:24:22.920 INFO:tasks.workunit.client.1.smithi163.stdout:0/382: dwrite d11/d21/d50/f79 [0,4194304] 0 2024-02-16T19:24:22.920 INFO:tasks.workunit.client.1.smithi163.stdout:0/383: fsync d11/d21/d50/d58/d6b/f71 0 2024-02-16T19:24:22.920 INFO:tasks.workunit.client.1.smithi163.stdout:0/384: chown d11/d22/d3e/f57 2729 1 2024-02-16T19:24:22.920 INFO:tasks.workunit.client.1.smithi163.stdout:0/385: write d11/d12/f6c [1732177,58107] 0 2024-02-16T19:24:22.920 INFO:tasks.workunit.client.0.smithi040.stdout:9/827: dwrite d3/d4/d11/d29/d38/d46/d6b/fc8 [0,4194304] 0 2024-02-16T19:24:22.922 INFO:tasks.workunit.client.1.smithi163.stdout:1/180: creat d7/d20/d2a/f36 x:0 0 0 2024-02-16T19:24:22.923 INFO:tasks.workunit.client.1.smithi163.stdout:1/181: write d7/fb [772862,123464] 0 2024-02-16T19:24:22.923 INFO:tasks.workunit.client.0.smithi040.stdout:2/759: dread d5/d7/d12/f15 [0,4194304] 0 2024-02-16T19:24:22.924 INFO:tasks.workunit.client.0.smithi040.stdout:9/828: dread d3/d4/d11/d71/da8/f84 [0,4194304] 0 2024-02-16T19:24:22.924 INFO:tasks.workunit.client.0.smithi040.stdout:2/760: creat d5/d7/d1e/d55/ff8 x:0 0 0 2024-02-16T19:24:22.924 INFO:tasks.workunit.client.0.smithi040.stdout:2/761: fdatasync d5/d5c/d65/d9d/fbb 0 2024-02-16T19:24:22.931 INFO:tasks.workunit.client.1.smithi163.stdout:2/340: write d2/d2a/d39/f42 [489660,31434] 0 2024-02-16T19:24:22.931 INFO:tasks.workunit.client.1.smithi163.stdout:2/341: fsync d2/d26/d54/f64 0 2024-02-16T19:24:22.931 INFO:tasks.workunit.client.1.smithi163.stdout:2/342: creat d2/d2a/d39/d66/d6c/f70 x:0 0 0 2024-02-16T19:24:22.931 INFO:tasks.workunit.client.0.smithi040.stdout:2/762: mknod d5/d7/d1e/d4c/cf9 0 2024-02-16T19:24:22.931 INFO:tasks.workunit.client.0.smithi040.stdout:2/763: creat d5/d7/ffa x:0 0 0 2024-02-16T19:24:22.931 INFO:tasks.workunit.client.0.smithi040.stdout:2/764: fsync d5/d7/f68 0 2024-02-16T19:24:22.932 INFO:tasks.workunit.client.1.smithi163.stdout:0/386: rename d11/d21/d50/d58/c70 to d11/d21/d7c/c89 0 2024-02-16T19:24:22.932 INFO:tasks.workunit.client.1.smithi163.stdout:0/387: fsync d11/d22/d61/f69 0 2024-02-16T19:24:22.933 INFO:tasks.workunit.client.1.smithi163.stdout:6/219: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:22.935 INFO:tasks.workunit.client.1.smithi163.stdout:5/215: rmdir d0/d2 39 2024-02-16T19:24:22.935 INFO:tasks.workunit.client.1.smithi163.stdout:5/216: fdatasync d0/d22/d28/d36/d3c/f43 0 2024-02-16T19:24:22.936 INFO:tasks.workunit.client.1.smithi163.stdout:0/388: dread d11/d12/d20/f39 [0,4194304] 0 2024-02-16T19:24:22.936 INFO:tasks.workunit.client.0.smithi040.stdout:3/879: sync 2024-02-16T19:24:22.936 INFO:tasks.workunit.client.0.smithi040.stdout:1/778: sync 2024-02-16T19:24:22.936 INFO:tasks.workunit.client.0.smithi040.stdout:0/942: sync 2024-02-16T19:24:22.936 INFO:tasks.workunit.client.0.smithi040.stdout:1/779: write d17/d3c/f9d [810754,93103] 0 2024-02-16T19:24:22.936 INFO:tasks.workunit.client.0.smithi040.stdout:0/943: write db/d2f/d1d/f46 [2084230,125972] 0 2024-02-16T19:24:22.936 INFO:tasks.workunit.client.0.smithi040.stdout:0/944: chown db/d25/d34/d5e/c117 96936615 1 2024-02-16T19:24:22.937 INFO:tasks.workunit.client.1.smithi163.stdout:1/182: creat d7/d20/d28/f37 x:0 0 0 2024-02-16T19:24:22.937 INFO:tasks.workunit.client.0.smithi040.stdout:2/765: rmdir d5/d7/d10/dbc 39 2024-02-16T19:24:22.941 INFO:tasks.workunit.client.0.smithi040.stdout:1/780: mknod d17/d26/d5f/da6/cfe 0 2024-02-16T19:24:22.943 INFO:tasks.workunit.client.1.smithi163.stdout:2/343: symlink d2/d2a/l71 0 2024-02-16T19:24:22.944 INFO:tasks.workunit.client.1.smithi163.stdout:2/344: write d2/d26/d54/f64 [729641,11632] 0 2024-02-16T19:24:22.947 INFO:tasks.workunit.client.0.smithi040.stdout:0/945: rename db/d2f/d23/d6f/d98/ff4 to db/de/d6a/df0/d106/f149 0 2024-02-16T19:24:22.948 INFO:tasks.workunit.client.1.smithi163.stdout:0/389: truncate f8 396145 0 2024-02-16T19:24:22.950 INFO:tasks.workunit.client.1.smithi163.stdout:1/183: rename d7/dc/f17 to d7/d20/f38 0 2024-02-16T19:24:22.951 INFO:tasks.workunit.client.0.smithi040.stdout:0/946: mknod db/d2f/d1d/c14a 0 2024-02-16T19:24:22.951 INFO:tasks.workunit.client.0.smithi040.stdout:0/947: dread - db/d25/d34/f13c zero size 2024-02-16T19:24:22.954 INFO:tasks.workunit.client.1.smithi163.stdout:6/220: write d0/d6/d9/f2d [953120,49065] 0 2024-02-16T19:24:22.954 INFO:tasks.workunit.client.1.smithi163.stdout:6/221: dread - d0/d18/f22 zero size 2024-02-16T19:24:22.954 INFO:tasks.workunit.client.1.smithi163.stdout:5/217: dread d0/d1c/f40 [0,4194304] 0 2024-02-16T19:24:22.954 INFO:tasks.workunit.client.1.smithi163.stdout:5/218: truncate d0/f2c 5137288 0 2024-02-16T19:24:22.961 INFO:tasks.workunit.client.1.smithi163.stdout:2/345: symlink d2/d2a/d39/d3a/l72 0 2024-02-16T19:24:22.963 INFO:tasks.workunit.client.1.smithi163.stdout:5/219: symlink d0/d22/d3b/l46 0 2024-02-16T19:24:22.971 INFO:tasks.workunit.client.1.smithi163.stdout:1/184: rmdir d7/d20 39 2024-02-16T19:24:22.973 INFO:tasks.workunit.client.1.smithi163.stdout:2/346: mkdir d2/d2a/d2d/d73 0 2024-02-16T19:24:22.974 INFO:tasks.workunit.client.0.smithi040.stdout:1/781: rmdir d17/d26 39 2024-02-16T19:24:22.975 INFO:tasks.workunit.client.1.smithi163.stdout:0/390: rmdir d11/d12/d67/d6a/d6d/d59 0 2024-02-16T19:24:22.976 INFO:tasks.workunit.client.0.smithi040.stdout:3/880: dwrite d1/d4/d6e/f84 [0,4194304] 0 2024-02-16T19:24:22.977 INFO:tasks.workunit.client.0.smithi040.stdout:1/782: mknod d17/d3c/d6b/cff 0 2024-02-16T19:24:22.977 INFO:tasks.workunit.client.0.smithi040.stdout:1/783: chown d17/d26/d3d/l45 25133635 1 2024-02-16T19:24:22.977 INFO:tasks.workunit.client.0.smithi040.stdout:1/784: write d17/d26/d4c/dab/fb1 [3690675,43932] 0 2024-02-16T19:24:22.977 INFO:tasks.workunit.client.1.smithi163.stdout:1/185: symlink d7/d20/l39 0 2024-02-16T19:24:22.978 INFO:tasks.workunit.client.1.smithi163.stdout:0/391: rename d11/l14 to d11/l8a 0 2024-02-16T19:24:22.979 INFO:tasks.workunit.client.1.smithi163.stdout:0/392: truncate d11/d22/d7f/f83 2747 0 2024-02-16T19:24:22.979 INFO:tasks.workunit.client.0.smithi040.stdout:1/785: symlink d17/d26/d9a/dfb/d92/daa/l100 0 2024-02-16T19:24:22.980 INFO:tasks.workunit.client.1.smithi163.stdout:1/186: symlink d7/d20/d2a/l3a 0 2024-02-16T19:24:22.981 INFO:tasks.workunit.client.1.smithi163.stdout:1/187: mkdir d7/dc/d2d/d32/d3b 0 2024-02-16T19:24:22.981 INFO:tasks.workunit.client.1.smithi163.stdout:0/393: dread d11/f1f [0,4194304] 0 2024-02-16T19:24:22.983 INFO:tasks.workunit.client.1.smithi163.stdout:1/188: rename d7/d20/l39 to d7/dc/l3c 0 2024-02-16T19:24:22.983 INFO:tasks.workunit.client.0.smithi040.stdout:3/881: dread d1/dc/d42/d4a/f7d [0,4194304] 0 2024-02-16T19:24:22.992 INFO:tasks.workunit.client.0.smithi040.stdout:0/948: dwrite db/de/dbe/f10e [0,4194304] 0 2024-02-16T19:24:22.992 INFO:tasks.workunit.client.0.smithi040.stdout:0/949: chown db/d2f/f12b 3 1 2024-02-16T19:24:22.993 INFO:tasks.workunit.client.0.smithi040.stdout:0/950: chown db/d25/d2c/d52/f91 28696747 1 2024-02-16T19:24:22.993 INFO:tasks.workunit.client.0.smithi040.stdout:0/951: chown db/d2f/d1d/d21/d2e/d83/f133 2 1 2024-02-16T19:24:22.993 INFO:tasks.workunit.client.0.smithi040.stdout:0/952: chown db/d25/d2c/d52/ffe 16182 1 2024-02-16T19:24:22.993 INFO:tasks.workunit.client.1.smithi163.stdout:5/220: dwrite d0/d22/d3b/f42 [0,4194304] 0 2024-02-16T19:24:22.995 INFO:tasks.workunit.client.0.smithi040.stdout:0/953: rmdir db/d2f/d23/d56 39 2024-02-16T19:24:22.995 INFO:tasks.workunit.client.0.smithi040.stdout:0/954: chown db/d25/d2c/d52/fc1 806 1 2024-02-16T19:24:22.996 INFO:tasks.workunit.client.0.smithi040.stdout:3/882: dread d1/d4/d6e/fb5 [0,4194304] 0 2024-02-16T19:24:22.998 INFO:tasks.workunit.client.0.smithi040.stdout:0/955: rename db/de/c99 to db/d25/d34/c14b 0 2024-02-16T19:24:22.999 INFO:tasks.workunit.client.1.smithi163.stdout:5/221: symlink d0/l47 0 2024-02-16T19:24:22.999 INFO:tasks.workunit.client.1.smithi163.stdout:5/222: readlink d0/d2/l37 0 2024-02-16T19:24:23.003 INFO:tasks.workunit.client.1.smithi163.stdout:5/223: dread d0/d22/f3a [0,4194304] 0 2024-02-16T19:24:23.003 INFO:tasks.workunit.client.1.smithi163.stdout:5/224: chown d0/d22/d3b/d3f 0 1 2024-02-16T19:24:23.004 INFO:tasks.workunit.client.1.smithi163.stdout:5/225: symlink d0/d2/l48 0 2024-02-16T19:24:23.005 INFO:tasks.workunit.client.1.smithi163.stdout:5/226: dread - d0/d2/f30 zero size 2024-02-16T19:24:23.005 INFO:tasks.workunit.client.1.smithi163.stdout:5/227: write d0/d22/f35 [857534,69233] 0 2024-02-16T19:24:23.012 INFO:tasks.workunit.client.1.smithi163.stdout:2/347: dwrite d2/f1a [0,4194304] 0 2024-02-16T19:24:23.014 INFO:tasks.workunit.client.1.smithi163.stdout:1/189: dwrite d7/d20/f35 [0,4194304] 0 2024-02-16T19:24:23.014 INFO:tasks.workunit.client.1.smithi163.stdout:1/190: dread - d7/f15 zero size 2024-02-16T19:24:23.014 INFO:tasks.workunit.client.1.smithi163.stdout:2/348: symlink d2/l74 0 2024-02-16T19:24:23.015 INFO:tasks.workunit.client.0.smithi040.stdout:1/786: dwrite d17/d29/d8b/fd6 [0,4194304] 0 2024-02-16T19:24:23.015 INFO:tasks.workunit.client.1.smithi163.stdout:0/394: dwrite d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:23.016 INFO:tasks.workunit.client.1.smithi163.stdout:1/191: rename d7/f2e to d7/d20/d28/f3d 0 2024-02-16T19:24:23.019 INFO:tasks.workunit.client.1.smithi163.stdout:0/395: creat d11/d21/d50/d58/f8b x:0 0 0 2024-02-16T19:24:23.019 INFO:tasks.workunit.client.1.smithi163.stdout:0/396: chown d11/d12 5 1 2024-02-16T19:24:23.020 INFO:tasks.workunit.client.0.smithi040.stdout:1/787: dread d17/d29/f2b [0,4194304] 0 2024-02-16T19:24:23.022 INFO:tasks.workunit.client.1.smithi163.stdout:1/192: symlink d7/dc/d2d/l3e 0 2024-02-16T19:24:23.022 INFO:tasks.workunit.client.1.smithi163.stdout:1/193: write d7/f15 [691996,78257] 0 2024-02-16T19:24:23.023 INFO:tasks.workunit.client.1.smithi163.stdout:1/194: chown d7/d20/d28/c31 112467 1 2024-02-16T19:24:23.023 INFO:tasks.workunit.client.1.smithi163.stdout:1/195: write d7/f26 [1992504,59366] 0 2024-02-16T19:24:23.024 INFO:tasks.workunit.client.1.smithi163.stdout:0/397: mknod d11/d22/d3e/c8c 0 2024-02-16T19:24:23.025 INFO:tasks.workunit.client.1.smithi163.stdout:2/349: rename d2/f1e to d2/d2a/d39/f75 0 2024-02-16T19:24:23.025 INFO:tasks.workunit.client.1.smithi163.stdout:2/350: readlink d2/l74 0 2024-02-16T19:24:23.029 INFO:tasks.workunit.client.1.smithi163.stdout:1/196: mkdir d7/dc/d2d/d3f 0 2024-02-16T19:24:23.031 INFO:tasks.workunit.client.1.smithi163.stdout:0/398: rename d11/d12/d20/f2c to d11/d12/d67/f8d 0 2024-02-16T19:24:23.035 INFO:tasks.workunit.client.1.smithi163.stdout:2/351: link d2/f19 d2/d2a/d39/f76 0 2024-02-16T19:24:23.036 INFO:tasks.workunit.client.0.smithi040.stdout:1/788: dwrite f12 [4194304,4194304] 0 2024-02-16T19:24:23.036 INFO:tasks.workunit.client.1.smithi163.stdout:5/228: truncate d0/d22/d3b/f42 748535 0 2024-02-16T19:24:23.037 INFO:tasks.workunit.client.1.smithi163.stdout:2/352: dread d2/d26/f27 [0,4194304] 0 2024-02-16T19:24:23.038 INFO:tasks.workunit.client.1.smithi163.stdout:1/197: dread d7/dc/f30 [0,4194304] 0 2024-02-16T19:24:23.039 INFO:tasks.workunit.client.1.smithi163.stdout:5/229: creat d0/d2/d34/f49 x:0 0 0 2024-02-16T19:24:23.039 INFO:tasks.workunit.client.1.smithi163.stdout:5/230: readlink d0/d1c/l45 0 2024-02-16T19:24:23.040 INFO:tasks.workunit.client.1.smithi163.stdout:2/353: creat d2/d2a/d39/d66/f77 x:0 0 0 2024-02-16T19:24:23.041 INFO:tasks.workunit.client.1.smithi163.stdout:1/198: creat d7/d20/d2a/f40 x:0 0 0 2024-02-16T19:24:23.041 INFO:tasks.workunit.client.1.smithi163.stdout:1/199: write d7/f15 [1130266,64834] 0 2024-02-16T19:24:23.041 INFO:tasks.workunit.client.1.smithi163.stdout:1/200: read d7/fb [306951,61669] 0 2024-02-16T19:24:23.042 INFO:tasks.workunit.client.1.smithi163.stdout:5/231: mknod d0/d22/c4a 0 2024-02-16T19:24:23.044 INFO:tasks.workunit.client.1.smithi163.stdout:7/209: sync 2024-02-16T19:24:23.044 INFO:tasks.workunit.client.1.smithi163.stdout:9/427: sync 2024-02-16T19:24:23.044 INFO:tasks.workunit.client.1.smithi163.stdout:8/207: sync 2024-02-16T19:24:23.044 INFO:tasks.workunit.client.1.smithi163.stdout:8/208: creat d0/d24/d2c/f4d x:0 0 0 2024-02-16T19:24:23.044 INFO:tasks.workunit.client.1.smithi163.stdout:9/428: creat d5/de/d16/f98 x:0 0 0 2024-02-16T19:24:23.045 INFO:tasks.workunit.client.0.smithi040.stdout:7/724: sync 2024-02-16T19:24:23.045 INFO:tasks.workunit.client.0.smithi040.stdout:8/922: sync 2024-02-16T19:24:23.045 INFO:tasks.workunit.client.1.smithi163.stdout:0/399: getdents d11/d21/d50/d58 0 2024-02-16T19:24:23.046 INFO:tasks.workunit.client.1.smithi163.stdout:1/201: dread d7/fb [0,4194304] 0 2024-02-16T19:24:23.046 INFO:tasks.workunit.client.1.smithi163.stdout:1/202: write d7/d20/d28/f3d [1265854,73297] 0 2024-02-16T19:24:23.049 INFO:tasks.workunit.client.0.smithi040.stdout:7/725: mkdir d21/d4d/d77/d7a/df4 0 2024-02-16T19:24:23.051 INFO:tasks.workunit.client.1.smithi163.stdout:9/429: mknod d5/de/d30/d43/d8f/c99 0 2024-02-16T19:24:23.052 INFO:tasks.workunit.client.1.smithi163.stdout:0/400: mkdir d11/d22/d7f/d8e 0 2024-02-16T19:24:23.053 INFO:tasks.workunit.client.0.smithi040.stdout:7/726: creat d21/d29/dad/ff5 x:0 0 0 2024-02-16T19:24:23.053 INFO:tasks.workunit.client.0.smithi040.stdout:8/923: write d6/d12/f4b [2587765,121775] 0 2024-02-16T19:24:23.054 INFO:tasks.workunit.client.1.smithi163.stdout:2/354: dread d2/d26/f31 [0,4194304] 0 2024-02-16T19:24:23.054 INFO:tasks.workunit.client.1.smithi163.stdout:2/355: stat d2/d2a/d39/d3a/f5b 0 2024-02-16T19:24:23.054 INFO:tasks.workunit.client.1.smithi163.stdout:2/356: write d2/fb [9257849,45262] 0 2024-02-16T19:24:23.054 INFO:tasks.workunit.client.1.smithi163.stdout:2/357: readlink d2/l12 0 2024-02-16T19:24:23.054 INFO:tasks.workunit.client.1.smithi163.stdout:8/209: dread d0/d7/d18/f3f [0,4194304] 0 2024-02-16T19:24:23.055 INFO:tasks.workunit.client.1.smithi163.stdout:9/430: mknod d5/de/d30/d3d/d93/c9a 0 2024-02-16T19:24:23.055 INFO:tasks.workunit.client.1.smithi163.stdout:9/431: write d5/de/d30/d43/d8f/f90 [210287,11804] 0 2024-02-16T19:24:23.055 INFO:tasks.workunit.client.1.smithi163.stdout:9/432: write d5/de/d30/d3d/d49/d6e/f8d [97438,53376] 0 2024-02-16T19:24:23.056 INFO:tasks.workunit.client.1.smithi163.stdout:0/401: dread d11/d12/d67/f8d [0,4194304] 0 2024-02-16T19:24:23.057 INFO:tasks.workunit.client.1.smithi163.stdout:8/210: link d0/d7/d19/f26 d0/d7/d19/d1f/f4e 0 2024-02-16T19:24:23.063 INFO:tasks.workunit.client.0.smithi040.stdout:8/924: dread d6/d12/d51/d6e/db7/ddf/df8/d1d/f52 [0,4194304] 0 2024-02-16T19:24:23.064 INFO:tasks.workunit.client.1.smithi163.stdout:0/402: dread fd [4194304,4194304] 0 2024-02-16T19:24:23.064 INFO:tasks.workunit.client.1.smithi163.stdout:0/403: creat d11/d21/d50/d58/d74/f8f x:0 0 0 2024-02-16T19:24:23.069 INFO:tasks.workunit.client.1.smithi163.stdout:0/404: rmdir d11/d21/d50/d58/d6b 39 2024-02-16T19:24:23.071 INFO:tasks.workunit.client.0.smithi040.stdout:7/727: dread d21/d4d/d7b/f8f [4194304,4194304] 0 2024-02-16T19:24:23.071 INFO:tasks.workunit.client.0.smithi040.stdout:7/728: chown d21/d29/l78 14807 1 2024-02-16T19:24:23.072 INFO:tasks.workunit.client.0.smithi040.stdout:7/729: symlink d21/d29/lf6 0 2024-02-16T19:24:23.072 INFO:tasks.workunit.client.0.smithi040.stdout:7/730: write d21/d4d/d77/d7a/fbf [901566,108067] 0 2024-02-16T19:24:23.072 INFO:tasks.workunit.client.0.smithi040.stdout:7/731: readlink d21/d29/l58 0 2024-02-16T19:24:23.073 INFO:tasks.workunit.client.1.smithi163.stdout:0/405: symlink d11/l90 0 2024-02-16T19:24:23.073 INFO:tasks.workunit.client.1.smithi163.stdout:0/406: write f10 [3796369,43917] 0 2024-02-16T19:24:23.073 INFO:tasks.workunit.client.1.smithi163.stdout:0/407: dread - d11/f85 zero size 2024-02-16T19:24:23.073 INFO:tasks.workunit.client.0.smithi040.stdout:7/732: creat d21/d8b/d9c/ff7 x:0 0 0 2024-02-16T19:24:23.080 INFO:tasks.workunit.client.1.smithi163.stdout:9/433: dread d5/de/d30/f6c [0,4194304] 0 2024-02-16T19:24:23.082 INFO:tasks.workunit.client.1.smithi163.stdout:9/434: mkdir d5/de/d16/d46/d9b 0 2024-02-16T19:24:23.085 INFO:tasks.workunit.client.1.smithi163.stdout:9/435: rename d5/de/d30/d3d/d49/f58 to d5/de/d16/d46/f9c 0 2024-02-16T19:24:23.086 INFO:tasks.workunit.client.1.smithi163.stdout:9/436: rmdir d5/de/d30/d43 39 2024-02-16T19:24:23.089 INFO:tasks.workunit.client.1.smithi163.stdout:9/437: dread d5/de/d30/d31/f69 [0,4194304] 0 2024-02-16T19:24:23.091 INFO:tasks.workunit.client.1.smithi163.stdout:5/232: dread d0/d22/d3b/f42 [0,4194304] 0 2024-02-16T19:24:23.091 INFO:tasks.workunit.client.1.smithi163.stdout:5/233: creat d0/d2/d34/f4b x:0 0 0 2024-02-16T19:24:23.091 INFO:tasks.workunit.client.1.smithi163.stdout:9/438: creat d5/de/d16/d46/d9b/f9d x:0 0 0 2024-02-16T19:24:23.092 INFO:tasks.workunit.client.1.smithi163.stdout:9/439: getdents d5/de/d84 0 2024-02-16T19:24:23.095 INFO:tasks.workunit.client.1.smithi163.stdout:7/210: dwrite d11/f1b [4194304,4194304] 0 2024-02-16T19:24:23.095 INFO:tasks.workunit.client.1.smithi163.stdout:9/440: unlink d5/de/d16/d46/l4d 0 2024-02-16T19:24:23.095 INFO:tasks.workunit.client.1.smithi163.stdout:9/441: write d5/de/d30/d31/d52/f81 [1030273,22576] 0 2024-02-16T19:24:23.095 INFO:tasks.workunit.client.0.smithi040.stdout:7/733: dread d21/d4d/f4b [0,4194304] 0 2024-02-16T19:24:23.097 INFO:tasks.workunit.client.1.smithi163.stdout:7/211: rmdir d11/d26/d36 39 2024-02-16T19:24:23.097 INFO:tasks.workunit.client.1.smithi163.stdout:7/212: creat d11/d24/d25/f48 x:0 0 0 2024-02-16T19:24:23.097 INFO:tasks.workunit.client.1.smithi163.stdout:7/213: fsync d11/f3f 0 2024-02-16T19:24:23.097 INFO:tasks.workunit.client.1.smithi163.stdout:7/214: chown d11/c22 16 1 2024-02-16T19:24:23.098 INFO:tasks.workunit.client.1.smithi163.stdout:9/442: mknod d5/de/d84/c9e 0 2024-02-16T19:24:23.099 INFO:tasks.workunit.client.0.smithi040.stdout:7/734: unlink d21/d4d/d77/ca1 0 2024-02-16T19:24:23.099 INFO:tasks.workunit.client.0.smithi040.stdout:7/735: stat d21/d4d/c71 0 2024-02-16T19:24:23.100 INFO:tasks.workunit.client.1.smithi163.stdout:9/443: write d5/f6 [219707,10966] 0 2024-02-16T19:24:23.101 INFO:tasks.workunit.client.1.smithi163.stdout:7/215: mkdir d11/d24/d49 0 2024-02-16T19:24:23.101 INFO:tasks.workunit.client.1.smithi163.stdout:7/216: creat d11/d24/d25/f4a x:0 0 0 2024-02-16T19:24:23.101 INFO:tasks.workunit.client.1.smithi163.stdout:7/217: write d11/d24/f3d [1398608,125693] 0 2024-02-16T19:24:23.103 INFO:tasks.workunit.client.1.smithi163.stdout:1/203: dwrite d7/d20/d28/f37 [0,4194304] 0 2024-02-16T19:24:23.103 INFO:tasks.workunit.client.1.smithi163.stdout:1/204: readlink d7/dc/l11 0 2024-02-16T19:24:23.109 INFO:tasks.workunit.client.1.smithi163.stdout:5/234: dread d0/d2/f21 [0,4194304] 0 2024-02-16T19:24:23.109 INFO:tasks.workunit.client.1.smithi163.stdout:5/235: creat d0/d22/d3b/f4c x:0 0 0 2024-02-16T19:24:23.110 INFO:tasks.workunit.client.1.smithi163.stdout:8/211: dwrite d0/d7/d19/d1f/f4e [4194304,4194304] 0 2024-02-16T19:24:23.111 INFO:tasks.workunit.client.0.smithi040.stdout:1/789: dwrite d17/d26/d5f/d63/f66 [0,4194304] 0 2024-02-16T19:24:23.111 INFO:tasks.workunit.client.0.smithi040.stdout:1/790: fsync d17/f61 0 2024-02-16T19:24:23.112 INFO:tasks.workunit.client.1.smithi163.stdout:2/358: dwrite d2/d26/f31 [0,4194304] 0 2024-02-16T19:24:23.113 INFO:tasks.workunit.client.0.smithi040.stdout:8/925: dwrite d6/d12/d51/d6e/db7/ddf/df8/d15/f85 [4194304,4194304] 0 2024-02-16T19:24:23.113 INFO:tasks.workunit.client.1.smithi163.stdout:0/408: dwrite d11/d12/d67/f8d [0,4194304] 0 2024-02-16T19:24:23.122 INFO:tasks.workunit.client.0.smithi040.stdout:1/791: dread d17/f1c [0,4194304] 0 2024-02-16T19:24:23.122 INFO:tasks.workunit.client.0.smithi040.stdout:1/792: fdatasync d17/d26/d4c/f5b 0 2024-02-16T19:24:23.123 INFO:tasks.workunit.client.1.smithi163.stdout:8/212: dread d0/d7/d18/f37 [4194304,4194304] 0 2024-02-16T19:24:23.132 INFO:tasks.workunit.client.1.smithi163.stdout:5/236: dwrite d0/d22/f2e [0,4194304] 0 2024-02-16T19:24:23.141 INFO:tasks.workunit.client.1.smithi163.stdout:9/444: symlink d5/de/d16/d47/l9f 0 2024-02-16T19:24:23.145 INFO:tasks.workunit.client.1.smithi163.stdout:7/218: mkdir d11/d24/d25/d4b 0 2024-02-16T19:24:23.145 INFO:tasks.workunit.client.1.smithi163.stdout:7/219: creat d11/d26/f4c x:0 0 0 2024-02-16T19:24:23.145 INFO:tasks.workunit.client.1.smithi163.stdout:7/220: stat d11/d30/f3e 0 2024-02-16T19:24:23.152 INFO:tasks.workunit.client.1.smithi163.stdout:1/205: link d7/dc/f30 d7/d20/d28/f41 0 2024-02-16T19:24:23.153 INFO:tasks.workunit.client.1.smithi163.stdout:1/206: readlink d7/dc/l3c 0 2024-02-16T19:24:23.164 INFO:tasks.workunit.client.1.smithi163.stdout:2/359: mknod d2/d44/c78 0 2024-02-16T19:24:23.164 INFO:tasks.workunit.client.1.smithi163.stdout:2/360: creat d2/d2a/d39/d66/d6c/f79 x:0 0 0 2024-02-16T19:24:23.172 INFO:tasks.workunit.client.1.smithi163.stdout:1/207: dwrite d7/dc/f33 [0,4194304] 0 2024-02-16T19:24:23.200 INFO:tasks.workunit.client.1.smithi163.stdout:0/409: creat d11/d12/d67/f91 x:0 0 0 2024-02-16T19:24:23.201 INFO:tasks.workunit.client.1.smithi163.stdout:0/410: write d11/d21/d50/d58/d74/f8f [212994,101275] 0 2024-02-16T19:24:23.201 INFO:tasks.workunit.client.1.smithi163.stdout:0/411: truncate d11/d12/d67/f91 1038699 0 2024-02-16T19:24:23.205 INFO:tasks.workunit.client.1.smithi163.stdout:5/237: unlink d0/d2/fa 0 2024-02-16T19:24:23.207 INFO:tasks.workunit.client.1.smithi163.stdout:5/238: dread d0/d22/d28/f38 [0,4194304] 0 2024-02-16T19:24:23.211 INFO:tasks.workunit.client.1.smithi163.stdout:0/412: write d11/d12/d67/f8d [2793430,99091] 0 2024-02-16T19:24:23.211 INFO:tasks.workunit.client.1.smithi163.stdout:0/413: chown f8 16692 1 2024-02-16T19:24:23.211 INFO:tasks.workunit.client.1.smithi163.stdout:9/445: creat d5/fa0 x:0 0 0 2024-02-16T19:24:23.222 INFO:tasks.workunit.client.0.smithi040.stdout:8/926: mknod d6/d12/c11b 0 2024-02-16T19:24:23.226 INFO:tasks.workunit.client.1.smithi163.stdout:1/208: mknod d7/dc/d2d/c42 0 2024-02-16T19:24:23.227 INFO:tasks.workunit.client.1.smithi163.stdout:9/446: dwrite d5/f77 [0,4194304] 0 2024-02-16T19:24:23.230 INFO:tasks.workunit.client.1.smithi163.stdout:8/213: getdents d0/d7/d18 0 2024-02-16T19:24:23.239 INFO:tasks.workunit.client.1.smithi163.stdout:0/414: creat d11/d21/d50/d58/f92 x:0 0 0 2024-02-16T19:24:23.239 INFO:tasks.workunit.client.1.smithi163.stdout:0/415: creat d11/d12/d67/d6a/d6d/f93 x:0 0 0 2024-02-16T19:24:23.239 INFO:tasks.workunit.client.1.smithi163.stdout:0/416: readlink d11/l8a 0 2024-02-16T19:24:23.239 INFO:tasks.workunit.client.1.smithi163.stdout:0/417: creat d11/d22/d61/f94 x:0 0 0 2024-02-16T19:24:23.239 INFO:tasks.workunit.client.1.smithi163.stdout:0/418: chown d11/d12/d67/f8d 68368030 1 2024-02-16T19:24:23.239 INFO:tasks.workunit.client.1.smithi163.stdout:0/419: truncate d11/d12/d20/f3a 4460586 0 2024-02-16T19:24:23.239 INFO:tasks.workunit.client.1.smithi163.stdout:0/420: write f6 [4973873,130914] 0 2024-02-16T19:24:23.241 INFO:tasks.workunit.client.1.smithi163.stdout:0/421: dread d11/d22/d3e/f57 [0,4194304] 0 2024-02-16T19:24:23.241 INFO:tasks.workunit.client.1.smithi163.stdout:0/422: creat d11/d22/d7f/f95 x:0 0 0 2024-02-16T19:24:23.241 INFO:tasks.workunit.client.1.smithi163.stdout:7/221: rename l6 to d11/l4d 0 2024-02-16T19:24:23.241 INFO:tasks.workunit.client.1.smithi163.stdout:7/222: chown d11/d24/c35 344149903 1 2024-02-16T19:24:23.248 INFO:tasks.workunit.client.1.smithi163.stdout:1/209: dwrite d7/f22 [0,4194304] 0 2024-02-16T19:24:23.249 INFO:tasks.workunit.client.1.smithi163.stdout:8/214: rmdir d0/d7/d18 39 2024-02-16T19:24:23.255 INFO:tasks.workunit.client.1.smithi163.stdout:1/210: dread d7/d20/d28/f41 [0,4194304] 0 2024-02-16T19:24:23.266 INFO:tasks.workunit.client.1.smithi163.stdout:0/423: link d11/d21/d50/d58/f8b d11/d22/d7f/f96 0 2024-02-16T19:24:23.269 INFO:tasks.workunit.client.1.smithi163.stdout:7/223: creat d11/d26/d36/f4e x:0 0 0 2024-02-16T19:24:23.269 INFO:tasks.workunit.client.1.smithi163.stdout:7/224: creat d11/d24/d25/f4f x:0 0 0 2024-02-16T19:24:23.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/361: rename d2/d2a/d39/f5a to d2/d2a/d39/f7a 0 2024-02-16T19:24:23.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/362: write d2/d26/d54/f64 [1039724,65262] 0 2024-02-16T19:24:23.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/363: write d2/d2a/f5f [284433,118360] 0 2024-02-16T19:24:23.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/364: write d2/d26/f3d [4780622,118401] 0 2024-02-16T19:24:23.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/365: creat d2/d2a/d39/d3a/f7b x:0 0 0 2024-02-16T19:24:23.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/366: chown d2/d26/l32 826930 1 2024-02-16T19:24:23.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/367: readlink d2/d26/l32 0 2024-02-16T19:24:23.277 INFO:tasks.workunit.client.1.smithi163.stdout:7/225: write ff [1588158,10467] 0 2024-02-16T19:24:23.278 INFO:tasks.workunit.client.1.smithi163.stdout:7/226: chown d11/l14 2 1 2024-02-16T19:24:23.278 INFO:tasks.workunit.client.1.smithi163.stdout:8/215: link d0/dc/l29 d0/dc/l4f 0 2024-02-16T19:24:23.278 INFO:tasks.workunit.client.1.smithi163.stdout:8/216: chown d0/d24/d2c 44788424 1 2024-02-16T19:24:23.278 INFO:tasks.workunit.client.1.smithi163.stdout:8/217: chown d0/d7/d18/f3f 636151459 1 2024-02-16T19:24:23.281 INFO:tasks.workunit.client.1.smithi163.stdout:1/211: unlink d7/l19 0 2024-02-16T19:24:23.282 INFO:tasks.workunit.client.0.smithi040.stdout:8/927: mknod d6/d12/d93/c11c 0 2024-02-16T19:24:23.283 INFO:tasks.workunit.client.1.smithi163.stdout:8/218: dread d0/d7/d18/f3f [0,4194304] 0 2024-02-16T19:24:23.284 INFO:tasks.workunit.client.1.smithi163.stdout:0/424: rename d11/c5a to d11/c97 0 2024-02-16T19:24:23.287 INFO:tasks.workunit.client.1.smithi163.stdout:8/219: dread d0/dc/f22 [0,4194304] 0 2024-02-16T19:24:23.290 INFO:tasks.workunit.client.1.smithi163.stdout:2/368: mkdir d2/d2a/d2d/d3c/d7c 0 2024-02-16T19:24:23.295 INFO:tasks.workunit.client.1.smithi163.stdout:7/227: mkdir d11/d50 0 2024-02-16T19:24:23.295 INFO:tasks.workunit.client.1.smithi163.stdout:7/228: fsync f3 0 2024-02-16T19:24:23.297 INFO:tasks.workunit.client.1.smithi163.stdout:1/212: creat d7/dc/d2d/d32/f43 x:0 0 0 2024-02-16T19:24:23.300 INFO:tasks.workunit.client.1.smithi163.stdout:8/220: mknod d0/d24/d2c/c50 0 2024-02-16T19:24:23.301 INFO:tasks.workunit.client.1.smithi163.stdout:2/369: mkdir d2/d26/d54/d58/d7d 0 2024-02-16T19:24:23.303 INFO:tasks.workunit.client.1.smithi163.stdout:7/229: creat d11/d24/d49/f51 x:0 0 0 2024-02-16T19:24:23.307 INFO:tasks.workunit.client.1.smithi163.stdout:1/213: rename d7/dc/d2d/c42 to d7/dc/d2d/d32/c44 0 2024-02-16T19:24:23.307 INFO:tasks.workunit.client.1.smithi163.stdout:1/214: chown d7 3740781 1 2024-02-16T19:24:23.309 INFO:tasks.workunit.client.1.smithi163.stdout:0/425: getdents d11/d21/d50 0 2024-02-16T19:24:23.311 INFO:tasks.workunit.client.1.smithi163.stdout:8/221: getdents d0/d7 0 2024-02-16T19:24:23.311 INFO:tasks.workunit.client.1.smithi163.stdout:8/222: write d0/d24/d2c/f3b [100576,63083] 0 2024-02-16T19:24:23.313 INFO:tasks.workunit.client.1.smithi163.stdout:1/215: creat d7/d20/d21/f45 x:0 0 0 2024-02-16T19:24:23.314 INFO:tasks.workunit.client.1.smithi163.stdout:0/426: rename d11/d22/d61/f69 to d11/f98 0 2024-02-16T19:24:23.316 INFO:tasks.workunit.client.1.smithi163.stdout:2/370: getdents d2 0 2024-02-16T19:24:23.316 INFO:tasks.workunit.client.1.smithi163.stdout:2/371: chown d2/d2a/d2d/d3c/d7c 7983329 1 2024-02-16T19:24:23.318 INFO:tasks.workunit.client.1.smithi163.stdout:1/216: rename d7/d20/d2a/f36 to d7/dc/d2d/d32/f46 0 2024-02-16T19:24:23.318 INFO:tasks.workunit.client.1.smithi163.stdout:1/217: chown d7/dc/f33 14242299 1 2024-02-16T19:24:23.320 INFO:tasks.workunit.client.1.smithi163.stdout:0/427: rename d11/d12/d67/d6a/d6d/l80 to d11/d21/d7c/l99 0 2024-02-16T19:24:23.321 INFO:tasks.workunit.client.1.smithi163.stdout:0/428: mknod d11/d21/d50/c9a 0 2024-02-16T19:24:23.326 INFO:tasks.workunit.client.1.smithi163.stdout:2/372: write d2/f23 [346736,74415] 0 2024-02-16T19:24:23.329 INFO:tasks.workunit.client.1.smithi163.stdout:8/223: dwrite d0/d7/d19/d1f/d32/f40 [0,4194304] 0 2024-02-16T19:24:23.331 INFO:tasks.workunit.client.1.smithi163.stdout:5/239: truncate d0/d22/f2e 732277 0 2024-02-16T19:24:23.336 INFO:tasks.workunit.client.1.smithi163.stdout:0/429: dread d11/d21/d50/d58/d6b/f7a [0,4194304] 0 2024-02-16T19:24:23.336 INFO:tasks.workunit.client.1.smithi163.stdout:0/430: dread - d11/d22/d61/f78 zero size 2024-02-16T19:24:23.337 INFO:tasks.workunit.client.1.smithi163.stdout:0/431: getdents d11/d21/d50/d58/d6b 0 2024-02-16T19:24:23.337 INFO:tasks.workunit.client.1.smithi163.stdout:0/432: write d11/d21/d50/d58/d6b/f71 [112070,83982] 0 2024-02-16T19:24:23.339 INFO:tasks.workunit.client.1.smithi163.stdout:0/433: link d11/d21/d50/f79 d11/d22/d61/f9b 0 2024-02-16T19:24:23.339 INFO:tasks.workunit.client.1.smithi163.stdout:0/434: read d11/d12/d67/f8d [3488027,41922] 0 2024-02-16T19:24:23.339 INFO:tasks.workunit.client.1.smithi163.stdout:0/435: write d11/d22/f34 [5834480,105873] 0 2024-02-16T19:24:23.339 INFO:tasks.workunit.client.1.smithi163.stdout:0/436: readlink d11/d12/d67/d6a/d6d/l5e 0 2024-02-16T19:24:23.339 INFO:tasks.workunit.client.1.smithi163.stdout:0/437: write d11/d21/d50/d58/d74/f8f [536610,78041] 0 2024-02-16T19:24:23.340 INFO:tasks.workunit.client.1.smithi163.stdout:0/438: stat d11/l37 0 2024-02-16T19:24:23.343 INFO:tasks.workunit.client.1.smithi163.stdout:8/224: write d0/d7/d19/d1f/d32/f34 [3174376,130858] 0 2024-02-16T19:24:23.343 INFO:tasks.workunit.client.1.smithi163.stdout:1/218: dwrite d7/dc/d2d/d32/f43 [0,4194304] 0 2024-02-16T19:24:23.343 INFO:tasks.workunit.client.1.smithi163.stdout:2/373: dwrite d2/f1a [0,4194304] 0 2024-02-16T19:24:23.343 INFO:tasks.workunit.client.1.smithi163.stdout:2/374: fdatasync d2/d26/f31 0 2024-02-16T19:24:23.345 INFO:tasks.workunit.client.1.smithi163.stdout:7/230: getdents d11/d26/d36 0 2024-02-16T19:24:23.345 INFO:tasks.workunit.client.1.smithi163.stdout:7/231: write fa [4302893,15898] 0 2024-02-16T19:24:23.345 INFO:tasks.workunit.client.1.smithi163.stdout:7/232: stat d11/f43 0 2024-02-16T19:24:23.346 INFO:tasks.workunit.client.1.smithi163.stdout:5/240: dread d0/d2/fb [0,4194304] 0 2024-02-16T19:24:23.346 INFO:tasks.workunit.client.1.smithi163.stdout:2/375: dread d2/f19 [0,4194304] 0 2024-02-16T19:24:23.351 INFO:tasks.workunit.client.1.smithi163.stdout:8/225: rename d0/c21 to d0/d7/d18/c51 0 2024-02-16T19:24:23.351 INFO:tasks.workunit.client.1.smithi163.stdout:8/226: fsync d0/d24/d27/d45/f4a 0 2024-02-16T19:24:23.351 INFO:tasks.workunit.client.1.smithi163.stdout:8/227: readlink d0/dc/l4f 0 2024-02-16T19:24:23.352 INFO:tasks.workunit.client.1.smithi163.stdout:7/233: rmdir d11/d24/d34 39 2024-02-16T19:24:23.353 INFO:tasks.workunit.client.1.smithi163.stdout:2/376: symlink d2/d2a/d39/d3a/l7e 0 2024-02-16T19:24:23.353 INFO:tasks.workunit.client.1.smithi163.stdout:2/377: fdatasync d2/d2a/d2d/f2e 0 2024-02-16T19:24:23.356 INFO:tasks.workunit.client.1.smithi163.stdout:1/219: rename d7/l1b to d7/d20/l47 0 2024-02-16T19:24:23.356 INFO:tasks.workunit.client.1.smithi163.stdout:5/241: write d0/d22/f33 [3107397,20626] 0 2024-02-16T19:24:23.356 INFO:tasks.workunit.client.1.smithi163.stdout:1/220: fsync f6 0 2024-02-16T19:24:23.358 INFO:tasks.workunit.client.1.smithi163.stdout:8/228: creat d0/d7/d19/d1f/d30/f52 x:0 0 0 2024-02-16T19:24:23.359 INFO:tasks.workunit.client.1.smithi163.stdout:7/234: symlink d11/l52 0 2024-02-16T19:24:23.364 INFO:tasks.workunit.client.1.smithi163.stdout:0/439: dwrite d11/d12/d67/d6a/d6d/f93 [0,4194304] 0 2024-02-16T19:24:23.364 INFO:tasks.workunit.client.1.smithi163.stdout:0/440: creat d11/d22/f9c x:0 0 0 2024-02-16T19:24:23.365 INFO:tasks.workunit.client.1.smithi163.stdout:1/221: mknod d7/dc/d2d/d32/d3b/c48 0 2024-02-16T19:24:23.366 INFO:tasks.workunit.client.1.smithi163.stdout:8/229: mkdir d0/d7/d18/d53 0 2024-02-16T19:24:23.366 INFO:tasks.workunit.client.1.smithi163.stdout:7/235: mknod d11/d50/c53 0 2024-02-16T19:24:23.367 INFO:tasks.workunit.client.1.smithi163.stdout:5/242: dread d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:23.367 INFO:tasks.workunit.client.1.smithi163.stdout:2/378: getdents d2/d2a 0 2024-02-16T19:24:23.367 INFO:tasks.workunit.client.1.smithi163.stdout:2/379: chown d2/d2a/d2d/d3c/c63 2223479 1 2024-02-16T19:24:23.373 INFO:tasks.workunit.client.1.smithi163.stdout:0/441: rename d11/d21/d50/d58/d6b/l76 to d11/d21/d50/d58/d6b/l9d 0 2024-02-16T19:24:23.374 INFO:tasks.workunit.client.1.smithi163.stdout:2/380: read d2/d2a/d2d/f6a [2081764,64668] 0 2024-02-16T19:24:23.374 INFO:tasks.workunit.client.1.smithi163.stdout:2/381: getdents d2/d2a/d2d/d73 0 2024-02-16T19:24:23.380 INFO:tasks.workunit.client.0.smithi040.stdout:7/736: rename d21/d8b/d9c/da0 to d21/d7e/df8 0 2024-02-16T19:24:23.381 INFO:tasks.workunit.client.1.smithi163.stdout:1/222: stat d7/dc/l13 0 2024-02-16T19:24:23.381 INFO:tasks.workunit.client.1.smithi163.stdout:1/223: creat d7/f49 x:0 0 0 2024-02-16T19:24:23.381 INFO:tasks.workunit.client.1.smithi163.stdout:1/224: write d7/fb [91578,110449] 0 2024-02-16T19:24:23.382 INFO:tasks.workunit.client.0.smithi040.stdout:8/928: rename d6/d2b/f9c to d6/d64/d50/d69/df2/f11d 0 2024-02-16T19:24:23.383 INFO:tasks.workunit.client.1.smithi163.stdout:1/225: write d7/f14 [605539,115428] 0 2024-02-16T19:24:23.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/230: mkdir d0/d24/d3c/d3d/d41/d54 0 2024-02-16T19:24:23.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/231: write d0/d7/d19/d1f/d32/f39 [169318,124741] 0 2024-02-16T19:24:23.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/232: readlink d0/d24/d3c/d3d/l1d 0 2024-02-16T19:24:23.386 INFO:tasks.workunit.client.1.smithi163.stdout:7/236: mknod d11/d24/d25/d4b/c54 0 2024-02-16T19:24:23.386 INFO:tasks.workunit.client.1.smithi163.stdout:7/237: write d11/d26/d36/f40 [738989,100288] 0 2024-02-16T19:24:23.387 INFO:tasks.workunit.client.0.smithi040.stdout:8/929: dread d6/d2b/d44/f49 [0,4194304] 0 2024-02-16T19:24:23.388 INFO:tasks.workunit.client.1.smithi163.stdout:5/243: unlink d0/l8 0 2024-02-16T19:24:23.391 INFO:tasks.workunit.client.1.smithi163.stdout:0/442: unlink d11/d12/l26 0 2024-02-16T19:24:23.392 INFO:tasks.workunit.client.1.smithi163.stdout:0/443: dread d11/f98 [0,4194304] 0 2024-02-16T19:24:23.392 INFO:tasks.workunit.client.1.smithi163.stdout:0/444: dread - d11/d12/d67/d6a/d6d/f7d zero size 2024-02-16T19:24:23.392 INFO:tasks.workunit.client.1.smithi163.stdout:0/445: dread - d11/d21/d50/d58/f92 zero size 2024-02-16T19:24:23.392 INFO:tasks.workunit.client.1.smithi163.stdout:0/446: chown d11/d21/d50 260689 1 2024-02-16T19:24:23.393 INFO:tasks.workunit.client.0.smithi040.stdout:8/930: link d6/d2b/ld2 d6/d12/d51/d6e/db7/dc6/dce/l11e 0 2024-02-16T19:24:23.393 INFO:tasks.workunit.client.0.smithi040.stdout:8/931: creat d6/d12/d51/db6/f11f x:0 0 0 2024-02-16T19:24:23.393 INFO:tasks.workunit.client.0.smithi040.stdout:8/932: fsync d6/d12/d51/d6e/db7/ddf/df8/d1d/f7a 0 2024-02-16T19:24:23.401 INFO:tasks.workunit.client.1.smithi163.stdout:8/233: mkdir d0/d7/d55 0 2024-02-16T19:24:23.404 INFO:tasks.workunit.client.1.smithi163.stdout:7/238: link d11/d24/d34/l45 d11/d50/l55 0 2024-02-16T19:24:23.405 INFO:tasks.workunit.client.1.smithi163.stdout:7/239: dread - d11/d24/d25/f4a zero size 2024-02-16T19:24:23.405 INFO:tasks.workunit.client.1.smithi163.stdout:7/240: creat d11/d30/f56 x:0 0 0 2024-02-16T19:24:23.406 INFO:tasks.workunit.client.1.smithi163.stdout:5/244: rmdir d0/d22/d3b/d3f 0 2024-02-16T19:24:23.408 INFO:tasks.workunit.client.1.smithi163.stdout:0/447: mkdir d11/d21/d50/d58/d72/d9e 0 2024-02-16T19:24:23.408 INFO:tasks.workunit.client.1.smithi163.stdout:0/448: chown d11/d12/d20 0 1 2024-02-16T19:24:23.408 INFO:tasks.workunit.client.1.smithi163.stdout:0/449: chown d11/d12/d20/l2e 1505674 1 2024-02-16T19:24:23.408 INFO:tasks.workunit.client.1.smithi163.stdout:5/245: dread d0/d22/d3b/f42 [0,4194304] 0 2024-02-16T19:24:23.408 INFO:tasks.workunit.client.1.smithi163.stdout:1/226: dwrite d7/dc/f30 [4194304,4194304] 0 2024-02-16T19:24:23.409 INFO:tasks.workunit.client.1.smithi163.stdout:2/382: getdents d2/d2a/d39/d66 0 2024-02-16T19:24:23.410 INFO:tasks.workunit.client.1.smithi163.stdout:8/234: dread d0/d7/d19/d1f/f4e [4194304,4194304] 0 2024-02-16T19:24:23.412 INFO:tasks.workunit.client.1.smithi163.stdout:8/235: write d0/f44 [786670,51924] 0 2024-02-16T19:24:23.412 INFO:tasks.workunit.client.1.smithi163.stdout:8/236: stat d0/d24/d3c/d3d/l1d 0 2024-02-16T19:24:23.413 INFO:tasks.workunit.client.1.smithi163.stdout:7/241: symlink d11/d50/l57 0 2024-02-16T19:24:23.413 INFO:tasks.workunit.client.1.smithi163.stdout:7/242: stat d11/d26/f4c 0 2024-02-16T19:24:23.413 INFO:tasks.workunit.client.1.smithi163.stdout:7/243: chown d11/d30/f56 17447464 1 2024-02-16T19:24:23.416 INFO:tasks.workunit.client.0.smithi040.stdout:8/933: dread f3 [4194304,4194304] 0 2024-02-16T19:24:23.416 INFO:tasks.workunit.client.1.smithi163.stdout:1/227: unlink d7/d20/d28/f37 0 2024-02-16T19:24:23.418 INFO:tasks.workunit.client.1.smithi163.stdout:2/383: rmdir d2 39 2024-02-16T19:24:23.419 INFO:tasks.workunit.client.1.smithi163.stdout:8/237: dread d0/f44 [0,4194304] 0 2024-02-16T19:24:23.419 INFO:tasks.workunit.client.1.smithi163.stdout:7/244: creat d11/d24/d34/f58 x:0 0 0 2024-02-16T19:24:23.421 INFO:tasks.workunit.client.1.smithi163.stdout:5/246: rename d0/d2/d34/f49 to d0/f4d 0 2024-02-16T19:24:23.421 INFO:tasks.workunit.client.1.smithi163.stdout:5/247: dread - d0/d2/d34/f4b zero size 2024-02-16T19:24:23.422 INFO:tasks.workunit.client.1.smithi163.stdout:8/238: dread d0/f44 [0,4194304] 0 2024-02-16T19:24:23.422 INFO:tasks.workunit.client.1.smithi163.stdout:8/239: fdatasync d0/dc/f2d 0 2024-02-16T19:24:23.422 INFO:tasks.workunit.client.1.smithi163.stdout:2/384: rmdir d2 39 2024-02-16T19:24:23.422 INFO:tasks.workunit.client.1.smithi163.stdout:8/240: chown d0/d7/d19/d1f/d30/d48 17 1 2024-02-16T19:24:23.422 INFO:tasks.workunit.client.1.smithi163.stdout:2/385: dread - d2/d2a/d39/f75 zero size 2024-02-16T19:24:23.423 INFO:tasks.workunit.client.1.smithi163.stdout:8/241: chown d0/d7/d19/d1f/d32/f39 4486638 1 2024-02-16T19:24:23.423 INFO:tasks.workunit.client.1.smithi163.stdout:4/204: sync 2024-02-16T19:24:23.423 INFO:tasks.workunit.client.1.smithi163.stdout:3/269: sync 2024-02-16T19:24:23.423 INFO:tasks.workunit.client.1.smithi163.stdout:6/222: sync 2024-02-16T19:24:23.423 INFO:tasks.workunit.client.1.smithi163.stdout:6/223: fsync d0/d6/d9/f2d 0 2024-02-16T19:24:23.423 INFO:tasks.workunit.client.1.smithi163.stdout:3/270: dread - d0/dc/d1c/f23 zero size 2024-02-16T19:24:23.424 INFO:tasks.workunit.client.1.smithi163.stdout:8/242: creat d0/dc/f56 x:0 0 0 2024-02-16T19:24:23.425 INFO:tasks.workunit.client.1.smithi163.stdout:2/386: mkdir d2/d2a/d39/d66/d6c/d7f 0 2024-02-16T19:24:23.426 INFO:tasks.workunit.client.1.smithi163.stdout:4/205: creat d11/d18/d1f/d23/d3a/f3f x:0 0 0 2024-02-16T19:24:23.426 INFO:tasks.workunit.client.1.smithi163.stdout:4/206: chown d11/d18/d1f/d23/l2c 119296 1 2024-02-16T19:24:23.427 INFO:tasks.workunit.client.0.smithi040.stdout:8/934: read d6/d12/d51/d6e/db7/ddf/df8/f9b [660353,107955] 0 2024-02-16T19:24:23.427 INFO:tasks.workunit.client.0.smithi040.stdout:8/935: readlink d6/l108 0 2024-02-16T19:24:23.427 INFO:tasks.workunit.client.0.smithi040.stdout:8/936: dread - d6/d12/d51/db6/f119 zero size 2024-02-16T19:24:23.429 INFO:tasks.workunit.client.1.smithi163.stdout:4/207: rename c7 to d11/d18/d1f/c40 0 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:5/775: sync 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:9/829: sync 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:3/883: sync 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:9/830: stat d3/d4/d11/d29/d38/d46/d6b/lf2 0 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:4/646: sync 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:3/884: fdatasync d1/d7/d36/ff9 0 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:5/776: truncate d1/da/f35 1709357 0 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:2/766: sync 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:1/793: sync 2024-02-16T19:24:23.435 INFO:tasks.workunit.client.0.smithi040.stdout:5/777: chown d1/d45/l48 0 1 2024-02-16T19:24:23.436 INFO:tasks.workunit.client.0.smithi040.stdout:2/767: chown d5/d7/d10/d40/f91 31 1 2024-02-16T19:24:23.436 INFO:tasks.workunit.client.0.smithi040.stdout:2/768: fdatasync d5/d7/d10/df3/d42/fc8 0 2024-02-16T19:24:23.436 INFO:tasks.workunit.client.0.smithi040.stdout:2/769: truncate d5/d7/d1e/f80 2809857 0 2024-02-16T19:24:23.436 INFO:tasks.workunit.client.0.smithi040.stdout:0/956: sync 2024-02-16T19:24:23.436 INFO:tasks.workunit.client.0.smithi040.stdout:9/831: symlink d3/d4/d55/dc3/l106 0 2024-02-16T19:24:23.439 INFO:tasks.workunit.client.1.smithi163.stdout:5/248: dread d0/d2/f21 [0,4194304] 0 2024-02-16T19:24:23.446 INFO:tasks.workunit.client.1.smithi163.stdout:6/224: write d0/d6/f8 [230095,18175] 0 2024-02-16T19:24:23.446 INFO:tasks.workunit.client.1.smithi163.stdout:6/225: truncate d0/d6/f1a 9038217 0 2024-02-16T19:24:23.447 INFO:tasks.workunit.client.0.smithi040.stdout:4/647: creat dd/d58/d8c/fca x:0 0 0 2024-02-16T19:24:23.447 INFO:tasks.workunit.client.0.smithi040.stdout:5/778: truncate d1/da/d1c/dba/fc2 157968 0 2024-02-16T19:24:23.448 INFO:tasks.workunit.client.0.smithi040.stdout:1/794: mkdir d17/d26/d101 0 2024-02-16T19:24:23.448 INFO:tasks.workunit.client.0.smithi040.stdout:1/795: chown d17/d29/d8b/d93/dcf/fd8 14 1 2024-02-16T19:24:23.448 INFO:tasks.workunit.client.0.smithi040.stdout:8/937: dread d6/d12/d51/d6e/fc3 [0,4194304] 0 2024-02-16T19:24:23.450 INFO:tasks.workunit.client.0.smithi040.stdout:9/832: link d3/d4/d55/d31/fad d3/d4/dbe/de9/f107 0 2024-02-16T19:24:23.453 INFO:tasks.workunit.client.0.smithi040.stdout:0/957: write db/d2f/d1d/d21/d2e/d49/d5b/fb3 [2430651,13327] 0 2024-02-16T19:24:23.453 INFO:tasks.workunit.client.0.smithi040.stdout:4/648: creat dd/d58/d68/fcb x:0 0 0 2024-02-16T19:24:23.453 INFO:tasks.workunit.client.0.smithi040.stdout:0/958: chown db/d2f/d23/d6f/d98/f146 753 1 2024-02-16T19:24:23.456 INFO:tasks.workunit.client.1.smithi163.stdout:0/450: dwrite d11/d22/d3e/f57 [0,4194304] 0 2024-02-16T19:24:23.457 INFO:tasks.workunit.client.0.smithi040.stdout:5/779: creat d1/d62/d9a/dc5/ff1 x:0 0 0 2024-02-16T19:24:23.459 INFO:tasks.workunit.client.1.smithi163.stdout:6/226: rmdir d0/d6/df/d30 39 2024-02-16T19:24:23.459 INFO:tasks.workunit.client.1.smithi163.stdout:6/227: readlink d0/d6/le 0 2024-02-16T19:24:23.462 INFO:tasks.workunit.client.1.smithi163.stdout:3/271: dwrite d0/f1b [4194304,4194304] 0 2024-02-16T19:24:23.465 INFO:tasks.workunit.client.1.smithi163.stdout:8/243: dwrite d0/d24/d27/d45/f4a [0,4194304] 0 2024-02-16T19:24:23.466 INFO:tasks.workunit.client.0.smithi040.stdout:8/938: creat d6/d12/d51/d6e/db7/dc6/dea/f120 x:0 0 0 2024-02-16T19:24:23.468 INFO:tasks.workunit.client.0.smithi040.stdout:3/885: rename d1/dc/ddf to d1/dc/d42/d119 0 2024-02-16T19:24:23.470 INFO:tasks.workunit.client.1.smithi163.stdout:5/249: dread d0/d22/f33 [0,4194304] 0 2024-02-16T19:24:23.470 INFO:tasks.workunit.client.1.smithi163.stdout:0/451: mknod d11/d21/d50/d58/d72/d9e/c9f 0 2024-02-16T19:24:23.476 INFO:tasks.workunit.client.0.smithi040.stdout:0/959: write db/de/dbe/fb1 [3077189,35296] 0 2024-02-16T19:24:23.476 INFO:tasks.workunit.client.0.smithi040.stdout:0/960: dread - db/d25/d2c/d52/f143 zero size 2024-02-16T19:24:23.476 INFO:tasks.workunit.client.0.smithi040.stdout:0/961: creat db/de/d6a/df0/d106/f14c x:0 0 0 2024-02-16T19:24:23.482 INFO:tasks.workunit.client.0.smithi040.stdout:5/780: mknod d1/da/d1c/dba/cf2 0 2024-02-16T19:24:23.482 INFO:tasks.workunit.client.0.smithi040.stdout:5/781: readlink d1/da/d1c/d73/lda 0 2024-02-16T19:24:23.488 INFO:tasks.workunit.client.0.smithi040.stdout:8/939: creat d6/d2b/d44/f121 x:0 0 0 2024-02-16T19:24:23.491 INFO:tasks.workunit.client.0.smithi040.stdout:3/886: read d1/dc/fd [1782568,120456] 0 2024-02-16T19:24:23.491 INFO:tasks.workunit.client.0.smithi040.stdout:3/887: readlink d1/d4/ld7 0 2024-02-16T19:24:23.491 INFO:tasks.workunit.client.1.smithi163.stdout:7/245: dwrite d11/d26/d36/f4e [0,4194304] 0 2024-02-16T19:24:23.491 INFO:tasks.workunit.client.1.smithi163.stdout:7/246: chown d11/d24/d34/l45 86721499 1 2024-02-16T19:24:23.491 INFO:tasks.workunit.client.1.smithi163.stdout:7/247: write d11/d30/f56 [262318,122908] 0 2024-02-16T19:24:23.493 INFO:tasks.workunit.client.1.smithi163.stdout:0/452: mknod d11/d22/d61/ca0 0 2024-02-16T19:24:23.494 INFO:tasks.workunit.client.0.smithi040.stdout:5/782: link d1/d62/fdd d1/da/d1c/dba/db1/dde/ddf/ff3 0 2024-02-16T19:24:23.495 INFO:tasks.workunit.client.1.smithi163.stdout:8/244: symlink d0/d7/d19/l57 0 2024-02-16T19:24:23.502 INFO:tasks.workunit.client.0.smithi040.stdout:3/888: read d1/dc/d35/d48/d85/f9e [316415,127775] 0 2024-02-16T19:24:23.503 INFO:tasks.workunit.client.0.smithi040.stdout:5/783: creat d1/d45/ff4 x:0 0 0 2024-02-16T19:24:23.504 INFO:tasks.workunit.client.1.smithi163.stdout:0/453: mknod d11/d22/ca1 0 2024-02-16T19:24:23.506 INFO:tasks.workunit.client.0.smithi040.stdout:8/940: getdents d6/d64 0 2024-02-16T19:24:23.512 INFO:tasks.workunit.client.1.smithi163.stdout:1/228: dwrite f2 [0,4194304] 0 2024-02-16T19:24:23.512 INFO:tasks.workunit.client.0.smithi040.stdout:2/770: dwrite d5/f3a [0,4194304] 0 2024-02-16T19:24:23.514 INFO:tasks.workunit.client.0.smithi040.stdout:9/833: dwrite d3/d4/d4a/f8c [4194304,4194304] 0 2024-02-16T19:24:23.514 INFO:tasks.workunit.client.0.smithi040.stdout:9/834: chown d3/d4/d55/d63/f105 7124917 1 2024-02-16T19:24:23.516 INFO:tasks.workunit.client.0.smithi040.stdout:3/889: mkdir d1/d11a 0 2024-02-16T19:24:23.518 INFO:tasks.workunit.client.1.smithi163.stdout:6/228: dwrite d0/d6/df/f41 [0,4194304] 0 2024-02-16T19:24:23.518 INFO:tasks.workunit.client.1.smithi163.stdout:6/229: chown d0/l3f 428340042 1 2024-02-16T19:24:23.518 INFO:tasks.workunit.client.1.smithi163.stdout:6/230: truncate d0/d18/f22 942902 0 2024-02-16T19:24:23.519 INFO:tasks.workunit.client.1.smithi163.stdout:3/272: dwrite d0/dc/d1c/f2c [0,4194304] 0 2024-02-16T19:24:23.519 INFO:tasks.workunit.client.1.smithi163.stdout:2/387: dwrite d2/f1f [0,4194304] 0 2024-02-16T19:24:23.521 INFO:tasks.workunit.client.1.smithi163.stdout:8/245: mknod d0/d7/d19/c58 0 2024-02-16T19:24:23.525 INFO:tasks.workunit.client.0.smithi040.stdout:4/649: dwrite dd/de/d45/f73 [0,4194304] 0 2024-02-16T19:24:23.525 INFO:tasks.workunit.client.0.smithi040.stdout:4/650: write dd/de/d10/d36/dad/fb4 [380209,50290] 0 2024-02-16T19:24:23.525 INFO:tasks.workunit.client.0.smithi040.stdout:4/651: readlink dd/de/d10/d36/d9e/d32/d7a/l83 0 2024-02-16T19:24:23.526 INFO:tasks.workunit.client.0.smithi040.stdout:0/962: dwrite db/d2f/d23/f33 [0,4194304] 0 2024-02-16T19:24:23.528 INFO:tasks.workunit.client.1.smithi163.stdout:1/229: dread d7/dc/d2d/d32/f43 [0,4194304] 0 2024-02-16T19:24:23.529 INFO:tasks.workunit.client.1.smithi163.stdout:8/246: dread d0/f44 [0,4194304] 0 2024-02-16T19:24:23.529 INFO:tasks.workunit.client.1.smithi163.stdout:8/247: creat d0/d7/d19/d1f/d32/f59 x:0 0 0 2024-02-16T19:24:23.529 INFO:tasks.workunit.client.1.smithi163.stdout:8/248: write d0/dc/f56 [950253,18682] 0 2024-02-16T19:24:23.529 INFO:tasks.workunit.client.1.smithi163.stdout:8/249: chown d0/d24/d27/d45/l4b 33234558 1 2024-02-16T19:24:23.532 INFO:tasks.workunit.client.0.smithi040.stdout:9/835: getdents d3/d4/d11/d29/d38/d46/d6b 0 2024-02-16T19:24:23.532 INFO:tasks.workunit.client.0.smithi040.stdout:9/836: chown d3/d4/d11/d29/d36/l87 16484830 1 2024-02-16T19:24:23.536 INFO:tasks.workunit.client.0.smithi040.stdout:3/890: getdents d1/d7/d33 0 2024-02-16T19:24:23.536 INFO:tasks.workunit.client.0.smithi040.stdout:3/891: chown d1/dc/d42/d4a/d57/l5d 163646898 1 2024-02-16T19:24:23.536 INFO:tasks.workunit.client.0.smithi040.stdout:3/892: creat d1/d8c/d8e/dde/f11b x:0 0 0 2024-02-16T19:24:23.536 INFO:tasks.workunit.client.0.smithi040.stdout:3/893: write d1/dc/d42/d58/f72 [3062184,73326] 0 2024-02-16T19:24:23.536 INFO:tasks.workunit.client.0.smithi040.stdout:3/894: write d1/dc/d42/d5c/f6d [362553,98035] 0 2024-02-16T19:24:23.537 INFO:tasks.workunit.client.1.smithi163.stdout:3/273: link d0/l12 d0/dc/d1d/d49/l67 0 2024-02-16T19:24:23.540 INFO:tasks.workunit.client.0.smithi040.stdout:4/652: creat dd/de/d10/fcc x:0 0 0 2024-02-16T19:24:23.540 INFO:tasks.workunit.client.1.smithi163.stdout:7/248: dwrite d11/f15 [0,4194304] 0 2024-02-16T19:24:23.541 INFO:tasks.workunit.client.1.smithi163.stdout:2/388: symlink d2/d2a/d39/d66/d6c/l80 0 2024-02-16T19:24:23.542 INFO:tasks.workunit.client.1.smithi163.stdout:2/389: write d2/d26/f27 [1509075,78217] 0 2024-02-16T19:24:23.542 INFO:tasks.workunit.client.1.smithi163.stdout:2/390: chown d2/l14 1061351393 1 2024-02-16T19:24:23.542 INFO:tasks.workunit.client.1.smithi163.stdout:2/391: truncate d2/d26/f35 1964141 0 2024-02-16T19:24:23.542 INFO:tasks.workunit.client.1.smithi163.stdout:1/230: mknod d7/c4a 0 2024-02-16T19:24:23.545 INFO:tasks.workunit.client.0.smithi040.stdout:4/653: mknod dd/d58/d8c/d99/d9d/ccd 0 2024-02-16T19:24:23.547 INFO:tasks.workunit.client.1.smithi163.stdout:3/274: unlink d0/l9 0 2024-02-16T19:24:23.547 INFO:tasks.workunit.client.1.smithi163.stdout:3/275: fdatasync d0/dc/d1c/d32/f57 0 2024-02-16T19:24:23.547 INFO:tasks.workunit.client.0.smithi040.stdout:9/837: rmdir d3/d4/d11/d29/d36 39 2024-02-16T19:24:23.547 INFO:tasks.workunit.client.0.smithi040.stdout:9/838: write d3/d4/d11/fdd [3883929,34423] 0 2024-02-16T19:24:23.548 INFO:tasks.workunit.client.0.smithi040.stdout:4/654: truncate dd/de/d10/d36/d9e/d77/f3b 3048186 0 2024-02-16T19:24:23.549 INFO:tasks.workunit.client.1.smithi163.stdout:7/249: mknod d11/d50/c59 0 2024-02-16T19:24:23.551 INFO:tasks.workunit.client.1.smithi163.stdout:1/231: unlink f2 0 2024-02-16T19:24:23.551 INFO:tasks.workunit.client.0.smithi040.stdout:9/839: mknod d3/d4/dbe/de9/c108 0 2024-02-16T19:24:23.552 INFO:tasks.workunit.client.1.smithi163.stdout:7/250: rename d11/d24/f3d to d11/d24/d25/f5a 0 2024-02-16T19:24:23.555 INFO:tasks.workunit.client.0.smithi040.stdout:4/655: creat dd/d58/d8c/d99/d9d/fce x:0 0 0 2024-02-16T19:24:23.555 INFO:tasks.workunit.client.0.smithi040.stdout:4/656: chown dd/de/d10/f2e 0 1 2024-02-16T19:24:23.556 INFO:tasks.workunit.client.1.smithi163.stdout:1/232: symlink d7/dc/l4b 0 2024-02-16T19:24:23.557 INFO:tasks.workunit.client.1.smithi163.stdout:7/251: truncate d11/f19 2436580 0 2024-02-16T19:24:23.559 INFO:tasks.workunit.client.1.smithi163.stdout:8/250: write d0/d7/d19/d1f/d30/f47 [2465545,119435] 0 2024-02-16T19:24:23.559 INFO:tasks.workunit.client.1.smithi163.stdout:8/251: write d0/d24/d27/d45/f4a [4685859,71287] 0 2024-02-16T19:24:23.559 INFO:tasks.workunit.client.0.smithi040.stdout:4/657: dread dd/de/f80 [0,4194304] 0 2024-02-16T19:24:23.564 INFO:tasks.workunit.client.0.smithi040.stdout:3/895: dread d1/dc/d35/d53/d64/d65/fb9 [0,4194304] 0 2024-02-16T19:24:23.564 INFO:tasks.workunit.client.0.smithi040.stdout:5/784: dwrite d1/da/d1c/da7/fe2 [0,4194304] 0 2024-02-16T19:24:23.564 INFO:tasks.workunit.client.0.smithi040.stdout:5/785: write d1/da/d1c/d73/fbe [1425301,117670] 0 2024-02-16T19:24:23.564 INFO:tasks.workunit.client.0.smithi040.stdout:5/786: write d1/d3c/f72 [1009684,118973] 0 2024-02-16T19:24:23.564 INFO:tasks.workunit.client.0.smithi040.stdout:5/787: stat d1/da/d1c/dba/db1/dde/ddf/feb 0 2024-02-16T19:24:23.565 INFO:tasks.workunit.client.1.smithi163.stdout:3/276: dread d0/dc/d1d/f5a [0,4194304] 0 2024-02-16T19:24:23.565 INFO:tasks.workunit.client.0.smithi040.stdout:5/788: truncate d1/d45/fe6 3691576 0 2024-02-16T19:24:23.567 INFO:tasks.workunit.client.0.smithi040.stdout:5/789: rename d1/d62/fb6 to d1/d62/ff5 0 2024-02-16T19:24:23.567 INFO:tasks.workunit.client.0.smithi040.stdout:5/790: chown d1/da/d1c/dba/cf2 54868810 1 2024-02-16T19:24:23.569 INFO:tasks.workunit.client.0.smithi040.stdout:5/791: creat d1/da/d1c/d21/daa/ff6 x:0 0 0 2024-02-16T19:24:23.570 INFO:tasks.workunit.client.1.smithi163.stdout:1/233: write d7/dc/f33 [808333,94567] 0 2024-02-16T19:24:23.575 INFO:tasks.workunit.client.1.smithi163.stdout:1/234: symlink d7/l4c 0 2024-02-16T19:24:23.576 INFO:tasks.workunit.client.0.smithi040.stdout:3/896: dread d1/dc/d35/d53/d64/d65/f98 [0,4194304] 0 2024-02-16T19:24:23.578 INFO:tasks.workunit.client.1.smithi163.stdout:1/235: creat d7/dc/d2d/d32/d3b/f4d x:0 0 0 2024-02-16T19:24:23.579 INFO:tasks.workunit.client.1.smithi163.stdout:7/252: dread d11/f15 [0,4194304] 0 2024-02-16T19:24:23.579 INFO:tasks.workunit.client.1.smithi163.stdout:7/253: creat d11/d30/f5b x:0 0 0 2024-02-16T19:24:23.581 INFO:tasks.workunit.client.0.smithi040.stdout:2/771: dwrite d5/d7/d12/d1b/dae/fc4 [0,4194304] 0 2024-02-16T19:24:23.581 INFO:tasks.workunit.client.0.smithi040.stdout:8/941: dwrite d6/d12/d22/f4e [0,4194304] 0 2024-02-16T19:24:23.581 INFO:tasks.workunit.client.0.smithi040.stdout:2/772: write d5/d7/d1e/d4c/d52/f9c [3002635,114934] 0 2024-02-16T19:24:23.581 INFO:tasks.workunit.client.1.smithi163.stdout:6/231: dwrite d0/d6/df/f41 [0,4194304] 0 2024-02-16T19:24:23.582 INFO:tasks.workunit.client.0.smithi040.stdout:1/796: dwrite d17/d26/d3d/f83 [0,4194304] 0 2024-02-16T19:24:23.585 INFO:tasks.workunit.client.1.smithi163.stdout:0/454: dwrite d11/d22/d7f/f96 [0,4194304] 0 2024-02-16T19:24:23.585 INFO:tasks.workunit.client.1.smithi163.stdout:2/392: dwrite d2/f1c [0,4194304] 0 2024-02-16T19:24:23.586 INFO:tasks.workunit.client.0.smithi040.stdout:9/840: dwrite d3/d4/d55/f98 [0,4194304] 0 2024-02-16T19:24:23.586 INFO:tasks.workunit.client.0.smithi040.stdout:9/841: creat d3/d4/d4a/f109 x:0 0 0 2024-02-16T19:24:23.588 INFO:tasks.workunit.client.0.smithi040.stdout:0/963: dwrite db/d2f/d23/f137 [0,4194304] 0 2024-02-16T19:24:23.590 INFO:tasks.workunit.client.1.smithi163.stdout:3/277: dwrite d0/dc/d1c/d32/f55 [0,4194304] 0 2024-02-16T19:24:23.590 INFO:tasks.workunit.client.1.smithi163.stdout:7/254: rename d11/d24/d34/c37 to d11/d24/d34/c5c 0 2024-02-16T19:24:23.590 INFO:tasks.workunit.client.1.smithi163.stdout:7/255: chown d11/c22 5582 1 2024-02-16T19:24:23.596 INFO:tasks.workunit.client.1.smithi163.stdout:6/232: creat d0/d6/f45 x:0 0 0 2024-02-16T19:24:23.596 INFO:tasks.workunit.client.1.smithi163.stdout:6/233: write d0/f5 [8991566,2945] 0 2024-02-16T19:24:23.599 INFO:tasks.workunit.client.0.smithi040.stdout:3/897: symlink d1/dc/d42/d4a/d57/l11c 0 2024-02-16T19:24:23.601 INFO:tasks.workunit.client.0.smithi040.stdout:3/898: dread d1/dc/d42/d4a/f7d [0,4194304] 0 2024-02-16T19:24:23.601 INFO:tasks.workunit.client.0.smithi040.stdout:3/899: readlink d1/l1b 0 2024-02-16T19:24:23.602 INFO:tasks.workunit.client.1.smithi163.stdout:0/455: dread d11/f1f [0,4194304] 0 2024-02-16T19:24:23.610 INFO:tasks.workunit.client.0.smithi040.stdout:2/773: mknod d5/d7/d1e/d31/d4f/d63/d8b/cfb 0 2024-02-16T19:24:23.610 INFO:tasks.workunit.client.1.smithi163.stdout:1/236: getdents d7/dc 0 2024-02-16T19:24:23.611 INFO:tasks.workunit.client.0.smithi040.stdout:3/900: write d1/d7/d30/fa8 [172362,56655] 0 2024-02-16T19:24:23.612 INFO:tasks.workunit.client.0.smithi040.stdout:5/792: dwrite d1/d45/f52 [0,4194304] 0 2024-02-16T19:24:23.612 INFO:tasks.workunit.client.0.smithi040.stdout:5/793: dread - d1/da/d1c/f94 zero size 2024-02-16T19:24:23.612 INFO:tasks.workunit.client.0.smithi040.stdout:5/794: chown d1/fc3 0 1 2024-02-16T19:24:23.613 INFO:tasks.workunit.client.0.smithi040.stdout:1/797: dwrite d17/d26/d9a/dfb/d92/dca/fda [0,4194304] 0 2024-02-16T19:24:23.613 INFO:tasks.workunit.client.1.smithi163.stdout:2/393: dwrite d2/ff [0,4194304] 0 2024-02-16T19:24:23.614 INFO:tasks.workunit.client.0.smithi040.stdout:3/901: dread d1/ff0 [0,4194304] 0 2024-02-16T19:24:23.614 INFO:tasks.workunit.client.0.smithi040.stdout:5/795: dread d1/da/d51/f58 [0,4194304] 0 2024-02-16T19:24:23.625 INFO:tasks.workunit.client.0.smithi040.stdout:9/842: mknod d3/d4/d11/d29/d38/d46/d6b/c10a 0 2024-02-16T19:24:23.626 INFO:tasks.workunit.client.0.smithi040.stdout:5/796: dread d1/fd [0,4194304] 0 2024-02-16T19:24:23.629 INFO:tasks.workunit.client.0.smithi040.stdout:0/964: rmdir db/d25/d34 39 2024-02-16T19:24:23.631 INFO:tasks.workunit.client.1.smithi163.stdout:2/394: dwrite d2/d2a/d39/d66/d6c/f6d [0,4194304] 0 2024-02-16T19:24:23.632 INFO:tasks.workunit.client.1.smithi163.stdout:7/256: mkdir d11/d50/d5d 0 2024-02-16T19:24:23.632 INFO:tasks.workunit.client.1.smithi163.stdout:7/257: write d11/d30/f32 [403815,114030] 0 2024-02-16T19:24:23.632 INFO:tasks.workunit.client.1.smithi163.stdout:7/258: write d11/d26/d36/f40 [1868513,42599] 0 2024-02-16T19:24:23.633 INFO:tasks.workunit.client.0.smithi040.stdout:8/942: rmdir d6/d12/d51/d6e/db7/dc6/dcc/de8 39 2024-02-16T19:24:23.633 INFO:tasks.workunit.client.0.smithi040.stdout:8/943: creat d6/d12/d51/d6e/db9/dcf/f122 x:0 0 0 2024-02-16T19:24:23.633 INFO:tasks.workunit.client.0.smithi040.stdout:8/944: write d6/d12/d51/d6e/db7/dc6/dcc/f112 [158377,82429] 0 2024-02-16T19:24:23.634 INFO:tasks.workunit.client.1.smithi163.stdout:6/234: mknod d0/d6/d9/d15/c46 0 2024-02-16T19:24:23.634 INFO:tasks.workunit.client.1.smithi163.stdout:6/235: dread - d0/d6/f45 zero size 2024-02-16T19:24:23.639 INFO:tasks.workunit.client.0.smithi040.stdout:1/798: mkdir d17/d26/d4c/d102 0 2024-02-16T19:24:23.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/799: read - d17/d26/d9a/dfb/fef zero size 2024-02-16T19:24:23.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/800: write d17/db5/fd5 [343943,82529] 0 2024-02-16T19:24:23.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/801: fdatasync d17/d3c/d74/d97/fc5 0 2024-02-16T19:24:23.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/802: readlink d17/d3c/d74/d97/da9/ld0 0 2024-02-16T19:24:23.640 INFO:tasks.workunit.client.0.smithi040.stdout:1/803: chown d17/d29 27 1 2024-02-16T19:24:23.643 INFO:tasks.workunit.client.0.smithi040.stdout:3/902: mkdir d1/d4/d6e/d8a/da5/d11d 0 2024-02-16T19:24:23.643 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:23 smithi040 ceph-mon[29201]: pgmap v250: 65 pgs: 65 active+clean; 1.9 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 141 MiB/s rd, 176 MiB/s wr, 240 op/s 2024-02-16T19:24:23.647 INFO:tasks.workunit.client.0.smithi040.stdout:9/843: mknod d3/d4/d11/d29/d38/d46/c10b 0 2024-02-16T19:24:23.650 INFO:tasks.workunit.client.0.smithi040.stdout:0/965: rename db/d25/d2c/c7e to db/d2f/d23/d6f/c14d 0 2024-02-16T19:24:23.651 INFO:tasks.workunit.client.0.smithi040.stdout:5/797: dwrite d1/da/d1c/f26 [0,4194304] 0 2024-02-16T19:24:23.651 INFO:tasks.workunit.client.1.smithi163.stdout:6/236: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:23.654 INFO:tasks.workunit.client.1.smithi163.stdout:2/395: rename d2/d2a/d39/d66/d6c/l80 to d2/d26/l81 0 2024-02-16T19:24:23.655 INFO:tasks.workunit.client.1.smithi163.stdout:7/259: link d11/d26/d36/l41 d11/d26/l5e 0 2024-02-16T19:24:23.656 INFO:tasks.workunit.client.1.smithi163.stdout:7/260: write fa [4789848,78554] 0 2024-02-16T19:24:23.658 INFO:tasks.workunit.client.1.smithi163.stdout:0/456: mknod d11/d12/d20/ca2 0 2024-02-16T19:24:23.659 INFO:tasks.workunit.client.0.smithi040.stdout:1/804: creat d17/d26/d5f/d78/d7e/ddb/f103 x:0 0 0 2024-02-16T19:24:23.659 INFO:tasks.workunit.client.1.smithi163.stdout:0/457: dread d11/f33 [0,4194304] 0 2024-02-16T19:24:23.659 INFO:tasks.workunit.client.1.smithi163.stdout:0/458: write d11/d22/d7f/f83 [840283,128758] 0 2024-02-16T19:24:23.659 INFO:tasks.workunit.client.1.smithi163.stdout:0/459: creat d11/d22/fa3 x:0 0 0 2024-02-16T19:24:23.663 INFO:tasks.workunit.client.1.smithi163.stdout:1/237: mkdir d7/d20/d21/d4e 0 2024-02-16T19:24:23.663 INFO:tasks.workunit.client.1.smithi163.stdout:9/447: sync 2024-02-16T19:24:23.665 INFO:tasks.workunit.client.0.smithi040.stdout:3/903: mkdir d1/dc/d42/d119/d11e 0 2024-02-16T19:24:23.665 INFO:tasks.workunit.client.0.smithi040.stdout:3/904: chown d1/c4c 1877 1 2024-02-16T19:24:23.669 INFO:tasks.workunit.client.1.smithi163.stdout:6/237: mkdir d0/d6/df/d20/d40/d17/d47 0 2024-02-16T19:24:23.672 INFO:tasks.workunit.client.0.smithi040.stdout:5/798: unlink d1/da/d7e/le9 0 2024-02-16T19:24:23.672 INFO:tasks.workunit.client.0.smithi040.stdout:5/799: dread - d1/da/d1c/d21/f9d zero size 2024-02-16T19:24:23.672 INFO:tasks.workunit.client.0.smithi040.stdout:5/800: getdents d1/da/dd5 0 2024-02-16T19:24:23.674 INFO:tasks.workunit.client.0.smithi040.stdout:1/805: link d17/d26/d5f/d63/cae d17/d26/d4c/dab/c104 0 2024-02-16T19:24:23.676 INFO:tasks.workunit.client.1.smithi163.stdout:0/460: creat d11/d22/d7f/d8e/fa4 x:0 0 0 2024-02-16T19:24:23.677 INFO:tasks.workunit.client.0.smithi040.stdout:0/966: getdents db/de/d12d 0 2024-02-16T19:24:23.679 INFO:tasks.workunit.client.0.smithi040.stdout:0/967: symlink db/d25/d34/l14e 0 2024-02-16T19:24:23.679 INFO:tasks.workunit.client.0.smithi040.stdout:0/968: write db/d25/d34/d5e/fdb [766233,114303] 0 2024-02-16T19:24:23.679 INFO:tasks.workunit.client.0.smithi040.stdout:0/969: readlink db/d25/lea 0 2024-02-16T19:24:23.680 INFO:tasks.workunit.client.1.smithi163.stdout:1/238: creat d7/f4f x:0 0 0 2024-02-16T19:24:23.681 INFO:tasks.workunit.client.0.smithi040.stdout:1/806: dread d17/d26/d4c/dab/fbd [0,4194304] 0 2024-02-16T19:24:23.682 INFO:tasks.workunit.client.0.smithi040.stdout:1/807: write d17/d26/d5f/fd4 [238519,68723] 0 2024-02-16T19:24:23.682 INFO:tasks.workunit.client.1.smithi163.stdout:9/448: rename d5/de/d16/d46/d8b to d5/de/d30/d3d/d49/d6e/da1 0 2024-02-16T19:24:23.684 INFO:tasks.workunit.client.1.smithi163.stdout:6/238: write d0/d6/df/f41 [318534,104392] 0 2024-02-16T19:24:23.684 INFO:tasks.workunit.client.1.smithi163.stdout:6/239: read - d0/d6/d9/d2e/d33/f3c zero size 2024-02-16T19:24:23.685 INFO:tasks.workunit.client.0.smithi040.stdout:0/970: mkdir db/d2f/d23/d3b/dd5/d14f 0 2024-02-16T19:24:23.687 INFO:tasks.workunit.client.0.smithi040.stdout:0/971: mknod db/d2f/d1d/d21/d2e/d85/c150 0 2024-02-16T19:24:23.688 INFO:tasks.workunit.client.0.smithi040.stdout:5/801: dread d1/d3c/f77 [0,4194304] 0 2024-02-16T19:24:23.688 INFO:tasks.workunit.client.1.smithi163.stdout:0/461: getdents d11/d22/d7f 0 2024-02-16T19:24:23.691 INFO:tasks.workunit.client.0.smithi040.stdout:5/802: mkdir d1/da/d1c/dba/dd6/df7 0 2024-02-16T19:24:23.691 INFO:tasks.workunit.client.0.smithi040.stdout:5/803: truncate d1/da/d1c/dba/db1/fdb 16463 0 2024-02-16T19:24:23.693 INFO:tasks.workunit.client.1.smithi163.stdout:0/462: unlink d11/d22/d3e/f57 0 2024-02-16T19:24:23.693 INFO:tasks.workunit.client.0.smithi040.stdout:5/804: dread d1/da/da4/fc1 [0,4194304] 0 2024-02-16T19:24:23.694 INFO:tasks.workunit.client.0.smithi040.stdout:1/808: dread d17/d26/f7a [0,4194304] 0 2024-02-16T19:24:23.695 INFO:tasks.workunit.client.0.smithi040.stdout:5/805: truncate d1/da/d1c/f37 7971675 0 2024-02-16T19:24:23.695 INFO:tasks.workunit.client.0.smithi040.stdout:5/806: fsync d1/da/da4/fc1 0 2024-02-16T19:24:23.696 INFO:tasks.workunit.client.0.smithi040.stdout:1/809: symlink d17/d26/d5f/l105 0 2024-02-16T19:24:23.697 INFO:tasks.workunit.client.0.smithi040.stdout:9/844: dwrite d3/d4/d11/d29/d36/f74 [0,4194304] 0 2024-02-16T19:24:23.698 INFO:tasks.workunit.client.0.smithi040.stdout:1/810: unlink d17/d3c/d74/d97/lb3 0 2024-02-16T19:24:23.702 INFO:tasks.workunit.client.0.smithi040.stdout:9/845: creat d3/d4/d11/d29/d38/d46/d6b/d85/dfc/f10c x:0 0 0 2024-02-16T19:24:23.703 INFO:tasks.workunit.client.1.smithi163.stdout:2/396: dwrite d2/d26/f35 [0,4194304] 0 2024-02-16T19:24:23.703 INFO:tasks.workunit.client.0.smithi040.stdout:8/945: dwrite d6/d64/ffa [0,4194304] 0 2024-02-16T19:24:23.703 INFO:tasks.workunit.client.0.smithi040.stdout:8/946: stat d6/d12/d22/f42 0 2024-02-16T19:24:23.703 INFO:tasks.workunit.client.1.smithi163.stdout:6/240: fdatasync d0/d6/df/f41 0 2024-02-16T19:24:23.705 INFO:tasks.workunit.client.0.smithi040.stdout:3/905: dwrite d1/d4/d6e/d8a/da5/fd0 [0,4194304] 0 2024-02-16T19:24:23.709 INFO:tasks.workunit.client.1.smithi163.stdout:7/261: dwrite d11/d26/f33 [0,4194304] 0 2024-02-16T19:24:23.709 INFO:tasks.workunit.client.0.smithi040.stdout:1/811: mknod d17/d26/d3d/c106 0 2024-02-16T19:24:23.714 INFO:tasks.workunit.client.0.smithi040.stdout:0/972: dwrite db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/fc6 [0,4194304] 0 2024-02-16T19:24:23.714 INFO:tasks.workunit.client.0.smithi040.stdout:0/973: readlink db/l88 0 2024-02-16T19:24:23.714 INFO:tasks.workunit.client.0.smithi040.stdout:0/974: dread - db/d25/d34/f13b zero size 2024-02-16T19:24:23.715 INFO:tasks.workunit.client.1.smithi163.stdout:6/241: creat d0/d18/f48 x:0 0 0 2024-02-16T19:24:23.720 INFO:tasks.workunit.client.1.smithi163.stdout:2/397: write d2/d26/f35 [136855,67322] 0 2024-02-16T19:24:23.723 INFO:tasks.workunit.client.0.smithi040.stdout:0/975: dread db/de/d6a/df0/d106/f149 [0,4194304] 0 2024-02-16T19:24:23.725 INFO:tasks.workunit.client.1.smithi163.stdout:1/239: fsync d7/dc/d2d/d32/f43 0 2024-02-16T19:24:23.731 INFO:tasks.workunit.client.1.smithi163.stdout:7/262: mknod d11/d26/c5f 0 2024-02-16T19:24:23.732 INFO:tasks.workunit.client.0.smithi040.stdout:1/812: truncate d17/f44 475645 0 2024-02-16T19:24:23.733 INFO:tasks.workunit.client.1.smithi163.stdout:6/242: getdents d0/d6/d9/d15/d1c 0 2024-02-16T19:24:23.735 INFO:tasks.workunit.client.1.smithi163.stdout:2/398: symlink d2/d2a/d2d/d73/l82 0 2024-02-16T19:24:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:23 smithi163 ceph-mon[36452]: pgmap v250: 65 pgs: 65 active+clean; 1.9 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 141 MiB/s rd, 176 MiB/s wr, 240 op/s 2024-02-16T19:24:23.737 INFO:tasks.workunit.client.0.smithi040.stdout:0/976: mkdir db/d2f/d1d/d151 0 2024-02-16T19:24:23.739 INFO:tasks.workunit.client.1.smithi163.stdout:2/399: creat d2/d26/d54/d58/d7d/f83 x:0 0 0 2024-02-16T19:24:23.739 INFO:tasks.workunit.client.1.smithi163.stdout:2/400: dread - d2/d2a/d39/d66/d6c/f70 zero size 2024-02-16T19:24:23.739 INFO:tasks.workunit.client.1.smithi163.stdout:2/401: write d2/d26/f28 [2751963,107745] 0 2024-02-16T19:24:23.739 INFO:tasks.workunit.client.0.smithi040.stdout:0/977: mknod db/d2f/d1d/c152 0 2024-02-16T19:24:23.741 INFO:tasks.workunit.client.1.smithi163.stdout:1/240: rename d7/f1c to d7/dc/d2d/d32/d3b/f50 0 2024-02-16T19:24:23.742 INFO:tasks.workunit.client.0.smithi040.stdout:0/978: link db/d25/d2c/d52/cc9 db/de/d12d/c153 0 2024-02-16T19:24:23.742 INFO:tasks.workunit.client.0.smithi040.stdout:0/979: fdatasync db/d25/d34/d5e/f6b 0 2024-02-16T19:24:23.742 INFO:tasks.workunit.client.0.smithi040.stdout:0/980: fdatasync db/d2f/d1d/d21/d2e/d83/fa8 0 2024-02-16T19:24:23.742 INFO:tasks.workunit.client.0.smithi040.stdout:0/981: chown db/d25/d2c/d52/cfd 614 1 2024-02-16T19:24:23.743 INFO:tasks.workunit.client.1.smithi163.stdout:2/402: mkdir d2/d2a/d2d/d73/d84 0 2024-02-16T19:24:23.743 INFO:tasks.workunit.client.1.smithi163.stdout:2/403: write d2/d2a/d39/f76 [1035814,70156] 0 2024-02-16T19:24:23.743 INFO:tasks.workunit.client.1.smithi163.stdout:2/404: chown d2/ff 1666615 1 2024-02-16T19:24:23.744 INFO:tasks.workunit.client.0.smithi040.stdout:1/813: rmdir d17/d26/d5f/d63 39 2024-02-16T19:24:23.744 INFO:tasks.workunit.client.0.smithi040.stdout:3/906: dwrite d1/dc/d42/d4a/f110 [0,4194304] 0 2024-02-16T19:24:23.744 INFO:tasks.workunit.client.1.smithi163.stdout:0/463: dwrite d11/d21/d50/d58/f92 [0,4194304] 0 2024-02-16T19:24:23.746 INFO:tasks.workunit.client.0.smithi040.stdout:9/846: dwrite d3/d4/d55/d31/f9b [0,4194304] 0 2024-02-16T19:24:23.746 INFO:tasks.workunit.client.0.smithi040.stdout:9/847: chown d3/d4/d55/d8a/d99/fb2 6085 1 2024-02-16T19:24:23.746 INFO:tasks.workunit.client.0.smithi040.stdout:5/807: dwrite d1/fc3 [0,4194304] 0 2024-02-16T19:24:23.746 INFO:tasks.workunit.client.0.smithi040.stdout:0/982: link db/de/c73 db/d2f/d23/d56/c154 0 2024-02-16T19:24:23.749 INFO:tasks.workunit.client.0.smithi040.stdout:8/947: rename d6/d2b/d44 to d6/d12/d51/d6e/db9/d123 0 2024-02-16T19:24:23.751 INFO:tasks.workunit.client.1.smithi163.stdout:6/243: fsync d0/d18/f48 0 2024-02-16T19:24:23.751 INFO:tasks.workunit.client.1.smithi163.stdout:6/244: fsync d0/d6/d9/f2d 0 2024-02-16T19:24:23.751 INFO:tasks.workunit.client.1.smithi163.stdout:6/245: fsync d0/dd/f13 0 2024-02-16T19:24:23.751 INFO:tasks.workunit.client.1.smithi163.stdout:6/246: write d0/d6/df/f2a [299992,88870] 0 2024-02-16T19:24:23.752 INFO:tasks.workunit.client.0.smithi040.stdout:3/907: mkdir d1/d4/d6e/dc2/dce/d11f 0 2024-02-16T19:24:23.757 INFO:tasks.workunit.client.0.smithi040.stdout:9/848: mkdir d3/dd3/d10d 0 2024-02-16T19:24:23.759 INFO:tasks.workunit.client.1.smithi163.stdout:1/241: dread d7/f22 [0,4194304] 0 2024-02-16T19:24:23.759 INFO:tasks.workunit.client.1.smithi163.stdout:1/242: write d7/d20/f35 [5218487,77839] 0 2024-02-16T19:24:23.759 INFO:tasks.workunit.client.1.smithi163.stdout:1/243: chown d7/dc/l13 62462691 1 2024-02-16T19:24:23.760 INFO:tasks.workunit.client.1.smithi163.stdout:0/464: dread d11/d12/d67/f8d [0,4194304] 0 2024-02-16T19:24:23.760 INFO:tasks.workunit.client.1.smithi163.stdout:0/465: creat d11/d22/d3e/fa5 x:0 0 0 2024-02-16T19:24:23.761 INFO:tasks.workunit.client.1.smithi163.stdout:2/405: symlink d2/d2a/d39/l85 0 2024-02-16T19:24:23.764 INFO:tasks.workunit.client.1.smithi163.stdout:6/247: rename d0/l3 to d0/d6/d9/d2e/l49 0 2024-02-16T19:24:23.770 INFO:tasks.workunit.client.1.smithi163.stdout:0/466: unlink d11/d22/ca1 0 2024-02-16T19:24:23.770 INFO:tasks.workunit.client.1.smithi163.stdout:0/467: creat d11/d12/d67/fa6 x:0 0 0 2024-02-16T19:24:23.772 INFO:tasks.workunit.client.0.smithi040.stdout:1/814: dwrite d17/d26/d5f/d78/dac/fea [0,4194304] 0 2024-02-16T19:24:23.772 INFO:tasks.workunit.client.0.smithi040.stdout:1/815: write d17/d3c/f9d [1402038,3778] 0 2024-02-16T19:24:23.772 INFO:tasks.workunit.client.0.smithi040.stdout:0/983: dwrite db/d2f/d1d/d21/d2e/d85/fe8 [0,4194304] 0 2024-02-16T19:24:23.774 INFO:tasks.workunit.client.1.smithi163.stdout:6/248: getdents d0/d6/df 0 2024-02-16T19:24:23.774 INFO:tasks.workunit.client.1.smithi163.stdout:6/249: fdatasync d0/d6/f8 0 2024-02-16T19:24:23.776 INFO:tasks.workunit.client.0.smithi040.stdout:3/908: mknod d1/dc/d42/c120 0 2024-02-16T19:24:23.777 INFO:tasks.workunit.client.0.smithi040.stdout:3/909: write d1/d8c/d8e/ff1 [545411,116319] 0 2024-02-16T19:24:23.777 INFO:tasks.workunit.client.0.smithi040.stdout:5/808: dread d1/d45/fe6 [0,4194304] 0 2024-02-16T19:24:23.779 INFO:tasks.workunit.client.0.smithi040.stdout:9/849: mknod d3/dd3/d10d/c10e 0 2024-02-16T19:24:23.780 INFO:tasks.workunit.client.1.smithi163.stdout:1/244: rmdir d7/dc/d2d/d32 39 2024-02-16T19:24:23.784 INFO:tasks.workunit.client.1.smithi163.stdout:0/468: rmdir d11/d21/d50 39 2024-02-16T19:24:23.784 INFO:tasks.workunit.client.1.smithi163.stdout:0/469: creat d11/d22/fa7 x:0 0 0 2024-02-16T19:24:23.785 INFO:tasks.workunit.client.0.smithi040.stdout:5/809: write d1/da/d1c/da7/fe2 [513142,53315] 0 2024-02-16T19:24:23.785 INFO:tasks.workunit.client.0.smithi040.stdout:5/810: readlink d1/da/d1c/la8 0 2024-02-16T19:24:23.785 INFO:tasks.workunit.client.0.smithi040.stdout:5/811: fdatasync d1/da/d1c/dba/db1/dde/ddf/ff3 0 2024-02-16T19:24:23.785 INFO:tasks.workunit.client.0.smithi040.stdout:5/812: creat d1/d62/d9a/dc5/ff8 x:0 0 0 2024-02-16T19:24:23.785 INFO:tasks.workunit.client.0.smithi040.stdout:5/813: creat d1/da/d1c/d21/daa/ff9 x:0 0 0 2024-02-16T19:24:23.785 INFO:tasks.workunit.client.0.smithi040.stdout:1/816: creat d17/d26/d5f/d63/f107 x:0 0 0 2024-02-16T19:24:23.787 INFO:tasks.workunit.client.0.smithi040.stdout:2/774: sync 2024-02-16T19:24:23.787 INFO:tasks.workunit.client.0.smithi040.stdout:7/737: sync 2024-02-16T19:24:23.787 INFO:tasks.workunit.client.0.smithi040.stdout:4/658: sync 2024-02-16T19:24:23.787 INFO:tasks.workunit.client.0.smithi040.stdout:7/738: write d21/d4d/f4b [1734667,128251] 0 2024-02-16T19:24:23.787 INFO:tasks.workunit.client.0.smithi040.stdout:7/739: write d21/d4d/d77/fa3 [1042660,54714] 0 2024-02-16T19:24:23.788 INFO:tasks.workunit.client.1.smithi163.stdout:4/208: sync 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:8/252: sync 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:5/250: sync 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:8/253: creat d0/d24/d27/d45/f5a x:0 0 0 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:4/209: write d11/d18/d1f/f38 [626585,79220] 0 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:2/406: rename d2/d26/f27 to d2/d2a/d39/f86 0 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:2/407: chown d2/d26/f31 0 1 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:2/408: chown d2/d44/c78 35073 1 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:2/409: creat d2/d26/d54/d58/f87 x:0 0 0 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:3/278: sync 2024-02-16T19:24:23.789 INFO:tasks.workunit.client.1.smithi163.stdout:3/279: truncate d0/dc/d1d/f1f 9053603 0 2024-02-16T19:24:23.792 INFO:tasks.workunit.client.1.smithi163.stdout:6/250: getdents d0/d6/df/d30 0 2024-02-16T19:24:23.797 INFO:tasks.workunit.client.1.smithi163.stdout:1/245: link d7/dc/d2d/d32/d3b/f50 d7/dc/d2d/d32/f51 0 2024-02-16T19:24:23.797 INFO:tasks.workunit.client.1.smithi163.stdout:1/246: chown d7/d20/f38 909 1 2024-02-16T19:24:23.798 INFO:tasks.workunit.client.1.smithi163.stdout:5/251: dread d0/d1c/f40 [4194304,4194304] 0 2024-02-16T19:24:23.802 INFO:tasks.workunit.client.1.smithi163.stdout:0/470: dwrite d11/d22/d3e/fa5 [0,4194304] 0 2024-02-16T19:24:23.802 INFO:tasks.workunit.client.1.smithi163.stdout:0/471: fdatasync d11/d12/d20/f30 0 2024-02-16T19:24:23.803 INFO:tasks.workunit.client.0.smithi040.stdout:8/948: truncate d6/d12/d93/fa9 173843 0 2024-02-16T19:24:23.806 INFO:tasks.workunit.client.1.smithi163.stdout:8/254: creat d0/d24/d27/f5b x:0 0 0 2024-02-16T19:24:23.808 INFO:tasks.workunit.client.0.smithi040.stdout:9/850: link d3/d4/dbe/cd6 d3/d4/d11/d71/da8/dde/c10f 0 2024-02-16T19:24:23.810 INFO:tasks.workunit.client.1.smithi163.stdout:2/410: mknod d2/d2a/d2d/d73/c88 0 2024-02-16T19:24:23.812 INFO:tasks.workunit.client.1.smithi163.stdout:3/280: mknod d0/dc/d1d/d29/d38/d46/c68 0 2024-02-16T19:24:23.815 INFO:tasks.workunit.client.0.smithi040.stdout:5/814: creat d1/da/d1c/d21/ffa x:0 0 0 2024-02-16T19:24:23.819 INFO:tasks.workunit.client.0.smithi040.stdout:1/817: link d17/d26/d9a/dfb/d92/fc2 d17/d26/d9a/dfb/d92/dca/f108 0 2024-02-16T19:24:23.819 INFO:tasks.workunit.client.0.smithi040.stdout:1/818: dread - d17/d3c/d74/d97/fe5 zero size 2024-02-16T19:24:23.819 INFO:tasks.workunit.client.1.smithi163.stdout:1/247: chown d7/l16 1285 1 2024-02-16T19:24:23.820 INFO:tasks.workunit.client.0.smithi040.stdout:9/851: dread d3/d4/d11/d29/d38/d46/f5b [0,4194304] 0 2024-02-16T19:24:23.820 INFO:tasks.workunit.client.0.smithi040.stdout:9/852: truncate d3/d4/d55/d63/fd4 568507 0 2024-02-16T19:24:23.820 INFO:tasks.workunit.client.0.smithi040.stdout:4/659: unlink dd/d58/fa1 0 2024-02-16T19:24:23.823 INFO:tasks.workunit.client.0.smithi040.stdout:2/775: creat d5/d7/d1e/d4c/d52/daa/dcd/ffc x:0 0 0 2024-02-16T19:24:23.824 INFO:tasks.workunit.client.0.smithi040.stdout:7/740: mkdir d21/df9 0 2024-02-16T19:24:23.824 INFO:tasks.workunit.client.0.smithi040.stdout:7/741: truncate d21/d8b/fb5 1278374 0 2024-02-16T19:24:23.824 INFO:tasks.workunit.client.0.smithi040.stdout:7/742: readlink d21/d29/d79/lc5 0 2024-02-16T19:24:23.825 INFO:tasks.workunit.client.0.smithi040.stdout:8/949: getdents d6/d12/d51 0 2024-02-16T19:24:23.825 INFO:tasks.workunit.client.1.smithi163.stdout:0/472: creat d11/d21/d50/d58/d72/fa8 x:0 0 0 2024-02-16T19:24:23.826 INFO:tasks.workunit.client.0.smithi040.stdout:3/910: getdents d1/d4/d6e/dc2/dce 0 2024-02-16T19:24:23.826 INFO:tasks.workunit.client.0.smithi040.stdout:3/911: write d1/d4/dc4/fe0 [1131882,8561] 0 2024-02-16T19:24:23.829 INFO:tasks.workunit.client.0.smithi040.stdout:0/984: dwrite db/d2f/d1d/d21/d2e/d85/d89/d94/fb2 [0,4194304] 0 2024-02-16T19:24:23.829 INFO:tasks.workunit.client.1.smithi163.stdout:8/255: link d0/d7/d19/f26 d0/d7/d18/d2e/f5c 0 2024-02-16T19:24:23.829 INFO:tasks.workunit.client.1.smithi163.stdout:8/256: chown d0/d24/d3c/d3d 731 1 2024-02-16T19:24:23.831 INFO:tasks.workunit.client.1.smithi163.stdout:6/251: dwrite d0/d18/f48 [0,4194304] 0 2024-02-16T19:24:23.831 INFO:tasks.workunit.client.0.smithi040.stdout:5/815: rmdir d1/d45/dab/de7 0 2024-02-16T19:24:23.831 INFO:tasks.workunit.client.1.smithi163.stdout:1/248: dread d7/d20/f24 [0,4194304] 0 2024-02-16T19:24:23.831 INFO:tasks.workunit.client.1.smithi163.stdout:1/249: chown d7/dc/d2d/l3e 151870 1 2024-02-16T19:24:23.831 INFO:tasks.workunit.client.1.smithi163.stdout:1/250: write d7/dc/f30 [8750171,32532] 0 2024-02-16T19:24:23.834 INFO:tasks.workunit.client.1.smithi163.stdout:2/411: rename d2/d26/f3d to d2/d2a/d2d/d73/d84/f89 0 2024-02-16T19:24:23.834 INFO:tasks.workunit.client.1.smithi163.stdout:2/412: truncate d2/d2a/d39/d66/f77 474743 0 2024-02-16T19:24:23.834 INFO:tasks.workunit.client.1.smithi163.stdout:2/413: truncate d2/d26/d54/d58/d7d/f83 765493 0 2024-02-16T19:24:23.834 INFO:tasks.workunit.client.1.smithi163.stdout:3/281: rename d0 to d0/d3e/d69 22 2024-02-16T19:24:23.834 INFO:tasks.workunit.client.1.smithi163.stdout:3/282: fdatasync d0/dc/f2e 0 2024-02-16T19:24:23.834 INFO:tasks.workunit.client.1.smithi163.stdout:3/283: chown d0/dc/d1c/d21/l2f 1 1 2024-02-16T19:24:23.837 INFO:tasks.workunit.client.0.smithi040.stdout:1/819: mknod d17/d26/d9a/dfb/d92/daa/c109 0 2024-02-16T19:24:23.837 INFO:tasks.workunit.client.0.smithi040.stdout:1/820: write d17/d29/d8b/f8d [4256600,13283] 0 2024-02-16T19:24:23.837 INFO:tasks.workunit.client.0.smithi040.stdout:1/821: fdatasync d17/d26/d5f/d78/fa8 0 2024-02-16T19:24:23.838 INFO:tasks.workunit.client.0.smithi040.stdout:9/853: mkdir d3/d4/d55/d31/d110 0 2024-02-16T19:24:23.841 INFO:tasks.workunit.client.0.smithi040.stdout:4/660: getdents dd/de/d10/d36/d9e 0 2024-02-16T19:24:23.843 INFO:tasks.workunit.client.0.smithi040.stdout:2/776: creat d5/d5c/d65/d9d/ffd x:0 0 0 2024-02-16T19:24:23.845 INFO:tasks.workunit.client.0.smithi040.stdout:8/950: symlink d6/d2b/da3/l124 0 2024-02-16T19:24:23.845 INFO:tasks.workunit.client.0.smithi040.stdout:8/951: write d6/d2b/da3/d8b/f90 [4247624,61089] 0 2024-02-16T19:24:23.846 INFO:tasks.workunit.client.1.smithi163.stdout:2/414: rename d2/d2a/l71 to d2/l8a 0 2024-02-16T19:24:23.848 INFO:tasks.workunit.client.0.smithi040.stdout:3/912: write d1/ff [3146517,24819] 0 2024-02-16T19:24:23.848 INFO:tasks.workunit.client.0.smithi040.stdout:3/913: chown d1/d7/d36/f97 755 1 2024-02-16T19:24:23.848 INFO:tasks.workunit.client.0.smithi040.stdout:2/777: dread d5/d7/d1e/d55/f6f [0,4194304] 0 2024-02-16T19:24:23.848 INFO:tasks.workunit.client.0.smithi040.stdout:0/985: write db/de/dbe/f10e [2797387,25261] 0 2024-02-16T19:24:23.849 INFO:tasks.workunit.client.1.smithi163.stdout:0/473: symlink d11/d12/la9 0 2024-02-16T19:24:23.851 INFO:tasks.workunit.client.1.smithi163.stdout:1/251: mknod d7/c52 0 2024-02-16T19:24:23.851 INFO:tasks.workunit.client.1.smithi163.stdout:1/252: fdatasync d7/d20/f38 0 2024-02-16T19:24:23.854 INFO:tasks.workunit.client.0.smithi040.stdout:1/822: creat d17/d26/d3d/f10a x:0 0 0 2024-02-16T19:24:23.856 INFO:tasks.workunit.client.1.smithi163.stdout:5/252: dwrite d0/d2/f30 [0,4194304] 0 2024-02-16T19:24:23.857 INFO:tasks.workunit.client.0.smithi040.stdout:7/743: dwrite d21/d4d/d7b/f8f [8388608,4194304] 0 2024-02-16T19:24:23.857 INFO:tasks.workunit.client.0.smithi040.stdout:5/816: dwrite d1/da/d1c/f91 [0,4194304] 0 2024-02-16T19:24:23.858 INFO:tasks.workunit.client.1.smithi163.stdout:8/257: rename d0/dc/f56 to d0/d7/d18/d2e/f5d 0 2024-02-16T19:24:23.862 INFO:tasks.workunit.client.1.smithi163.stdout:3/284: getdents d0/dc/d1d/d29/d38 0 2024-02-16T19:24:23.862 INFO:tasks.workunit.client.1.smithi163.stdout:3/285: chown d0/dc/d1c/d21/c4f 51 1 2024-02-16T19:24:23.864 INFO:tasks.workunit.client.0.smithi040.stdout:4/661: truncate dd/de/d10/f30 180740 0 2024-02-16T19:24:23.869 INFO:tasks.workunit.client.0.smithi040.stdout:3/914: dwrite d1/dc/d42/d5c/df3/f102 [0,4194304] 0 2024-02-16T19:24:23.870 INFO:tasks.workunit.client.0.smithi040.stdout:3/915: chown d1/d7/d30/c68 7609 1 2024-02-16T19:24:23.870 INFO:tasks.workunit.client.0.smithi040.stdout:3/916: readlink d1/d7/d33/lfd 0 2024-02-16T19:24:23.870 INFO:tasks.workunit.client.0.smithi040.stdout:2/778: dread d5/d5c/fdf [0,4194304] 0 2024-02-16T19:24:23.870 INFO:tasks.workunit.client.0.smithi040.stdout:8/952: mknod d6/d12/d51/d6e/db7/c125 0 2024-02-16T19:24:23.871 INFO:tasks.workunit.client.0.smithi040.stdout:8/953: write d6/d12/d93/fda [184644,59040] 0 2024-02-16T19:24:23.877 INFO:tasks.workunit.client.0.smithi040.stdout:8/954: dread d6/d12/d51/d6e/db7/ddf/df8/d1d/f29 [4194304,4194304] 0 2024-02-16T19:24:23.879 INFO:tasks.workunit.client.0.smithi040.stdout:0/986: mkdir db/d2f/d1d/d21/d2e/d49/d5b/d155 0 2024-02-16T19:24:23.880 INFO:tasks.workunit.client.0.smithi040.stdout:3/917: write d1/d7/d30/f7f [3566528,130570] 0 2024-02-16T19:24:23.884 INFO:tasks.workunit.client.0.smithi040.stdout:9/854: rmdir d3/d4/d55/d8a/d99 39 2024-02-16T19:24:23.884 INFO:tasks.workunit.client.0.smithi040.stdout:5/817: dwrite d1/da/f99 [0,4194304] 0 2024-02-16T19:24:23.885 INFO:tasks.workunit.client.1.smithi163.stdout:5/253: fdatasync d0/d2/f29 0 2024-02-16T19:24:23.885 INFO:tasks.workunit.client.1.smithi163.stdout:5/254: truncate d0/d2/f30 5025084 0 2024-02-16T19:24:23.887 INFO:tasks.workunit.client.0.smithi040.stdout:5/818: read d1/d45/fce [48769,86103] 0 2024-02-16T19:24:23.888 INFO:tasks.workunit.client.0.smithi040.stdout:5/819: write d1/da/d1c/da7/fee [268489,79736] 0 2024-02-16T19:24:23.888 INFO:tasks.workunit.client.0.smithi040.stdout:5/820: chown d1/da/d51/f58 4360 1 2024-02-16T19:24:23.888 INFO:tasks.workunit.client.1.smithi163.stdout:2/415: rename d2/l3e to d2/d2a/d2d/d73/d84/l8b 0 2024-02-16T19:24:23.888 INFO:tasks.workunit.client.0.smithi040.stdout:4/662: dwrite dd/de/d71/f9a [0,4194304] 0 2024-02-16T19:24:23.889 INFO:tasks.workunit.client.0.smithi040.stdout:2/779: dwrite d5/d7/d1e/d4c/d52/daa/dcd/ffc [0,4194304] 0 2024-02-16T19:24:23.889 INFO:tasks.workunit.client.0.smithi040.stdout:2/780: write d5/d7/d10/fdd [325945,105441] 0 2024-02-16T19:24:23.890 INFO:tasks.workunit.client.1.smithi163.stdout:3/286: symlink d0/dc/d1d/d49/l6a 0 2024-02-16T19:24:23.896 INFO:tasks.workunit.client.1.smithi163.stdout:2/416: mkdir d2/d26/d54/d58/d8c 0 2024-02-16T19:24:23.899 INFO:tasks.workunit.client.1.smithi163.stdout:0/474: rename d11/d21/d50/d58/d6b/l87 to d11/d12/d67/d6a/laa 0 2024-02-16T19:24:23.899 INFO:tasks.workunit.client.1.smithi163.stdout:0/475: chown d11/d21/d50/d58/c77 3529200 1 2024-02-16T19:24:23.900 INFO:tasks.workunit.client.1.smithi163.stdout:9/449: sync 2024-02-16T19:24:23.900 INFO:tasks.workunit.client.1.smithi163.stdout:7/263: sync 2024-02-16T19:24:23.900 INFO:tasks.workunit.client.1.smithi163.stdout:7/264: creat d11/d24/d49/f60 x:0 0 0 2024-02-16T19:24:23.901 INFO:tasks.workunit.client.1.smithi163.stdout:3/287: truncate d0/dc/d1c/d32/f57 1851657 0 2024-02-16T19:24:23.902 INFO:tasks.workunit.client.1.smithi163.stdout:2/417: mknod d2/d2a/d39/d3a/c8d 0 2024-02-16T19:24:23.903 INFO:tasks.workunit.client.0.smithi040.stdout:0/987: dwrite db/d2f/d23/d6f/f72 [0,4194304] 0 2024-02-16T19:24:23.903 INFO:tasks.workunit.client.0.smithi040.stdout:0/988: write db/d25/fec [408080,36050] 0 2024-02-16T19:24:23.903 INFO:tasks.workunit.client.0.smithi040.stdout:8/955: mkdir d6/d12/d51/d6e/db9/dc9/d126 0 2024-02-16T19:24:23.903 INFO:tasks.workunit.client.0.smithi040.stdout:8/956: chown d6/d12/d51/d6e/db7/dc6/ce3 13507802 1 2024-02-16T19:24:23.906 INFO:tasks.workunit.client.1.smithi163.stdout:0/476: creat d11/d21/d50/d58/d72/d9e/fab x:0 0 0 2024-02-16T19:24:23.906 INFO:tasks.workunit.client.1.smithi163.stdout:0/477: chown d11/d21/f47 333 1 2024-02-16T19:24:23.907 INFO:tasks.workunit.client.0.smithi040.stdout:3/918: unlink d1/d7/c21 0 2024-02-16T19:24:23.908 INFO:tasks.workunit.client.1.smithi163.stdout:4/210: sync 2024-02-16T19:24:23.908 INFO:tasks.workunit.client.1.smithi163.stdout:6/252: sync 2024-02-16T19:24:23.908 INFO:tasks.workunit.client.1.smithi163.stdout:4/211: chown d11/c13 1726863 1 2024-02-16T19:24:23.909 INFO:tasks.workunit.client.0.smithi040.stdout:7/744: rmdir d21/d8b/d9c 39 2024-02-16T19:24:23.909 INFO:tasks.workunit.client.0.smithi040.stdout:7/745: chown d21/d4d/d7b/ccc 3 1 2024-02-16T19:24:23.910 INFO:tasks.workunit.client.1.smithi163.stdout:7/265: write d11/d26/f2b [4914010,42595] 0 2024-02-16T19:24:23.911 INFO:tasks.workunit.client.1.smithi163.stdout:3/288: symlink d0/dc/d1d/d29/d38/d46/l6b 0 2024-02-16T19:24:23.912 INFO:tasks.workunit.client.1.smithi163.stdout:3/289: read d0/dc/d1d/d29/f30 [4296452,5004] 0 2024-02-16T19:24:23.913 INFO:tasks.workunit.client.1.smithi163.stdout:2/418: creat d2/d2a/d2d/d73/d84/f8e x:0 0 0 2024-02-16T19:24:23.915 INFO:tasks.workunit.client.1.smithi163.stdout:5/255: dwrite d0/d2/d34/f4b [0,4194304] 0 2024-02-16T19:24:23.918 INFO:tasks.workunit.client.1.smithi163.stdout:1/253: rename d7/c1e to d7/dc/d2d/d32/d3b/c53 0 2024-02-16T19:24:23.919 INFO:tasks.workunit.client.0.smithi040.stdout:5/821: mkdir d1/da/dd5/dfb 0 2024-02-16T19:24:23.921 INFO:tasks.workunit.client.0.smithi040.stdout:0/989: dread db/d2f/f66 [0,4194304] 0 2024-02-16T19:24:23.924 INFO:tasks.workunit.client.1.smithi163.stdout:6/253: truncate d0/dd/f13 4105514 0 2024-02-16T19:24:23.924 INFO:tasks.workunit.client.1.smithi163.stdout:6/254: dread - d0/d6/df/d20/d40/d1e/f2c zero size 2024-02-16T19:24:23.924 INFO:tasks.workunit.client.1.smithi163.stdout:6/255: stat d0/d6/d9/d15/c19 0 2024-02-16T19:24:23.927 INFO:tasks.workunit.client.1.smithi163.stdout:3/290: dread d0/dc/d1c/d32/f3f [0,4194304] 0 2024-02-16T19:24:23.928 INFO:tasks.workunit.client.1.smithi163.stdout:3/291: fsync d0/dc/d1c/d21/f42 0 2024-02-16T19:24:23.931 INFO:tasks.workunit.client.1.smithi163.stdout:5/256: mknod d0/c4e 0 2024-02-16T19:24:23.931 INFO:tasks.workunit.client.1.smithi163.stdout:5/257: chown d0/l12 9998 1 2024-02-16T19:24:23.933 INFO:tasks.workunit.client.1.smithi163.stdout:1/254: rename d7/dc/d2d/d32/d3b/c48 to d7/dc/d2d/d32/c54 0 2024-02-16T19:24:23.933 INFO:tasks.workunit.client.1.smithi163.stdout:8/258: rename d0 to d0/d7/d18/d2e/d5e 22 2024-02-16T19:24:23.933 INFO:tasks.workunit.client.0.smithi040.stdout:2/781: mknod d5/d7/d1e/d4c/d52/daa/cfe 0 2024-02-16T19:24:23.934 INFO:tasks.workunit.client.0.smithi040.stdout:2/782: read - d5/d7/d12/dca/fd1 zero size 2024-02-16T19:24:23.936 INFO:tasks.workunit.client.1.smithi163.stdout:0/478: mknod d11/d12/cac 0 2024-02-16T19:24:23.936 INFO:tasks.workunit.client.1.smithi163.stdout:0/479: read - d11/d22/d61/f78 zero size 2024-02-16T19:24:23.936 INFO:tasks.workunit.client.0.smithi040.stdout:8/957: chown d6/d64/d50/f8c 491 1 2024-02-16T19:24:23.941 INFO:tasks.workunit.client.0.smithi040.stdout:0/990: write db/d2f/d1d/d21/d2e/d83/fe4 [3612065,85911] 0 2024-02-16T19:24:23.943 INFO:tasks.workunit.client.0.smithi040.stdout:7/746: unlink d21/d29/d79/dbc/fd7 0 2024-02-16T19:24:23.943 INFO:tasks.workunit.client.0.smithi040.stdout:7/747: readlink d21/d4d/d77/d87/lb9 0 2024-02-16T19:24:23.943 INFO:tasks.workunit.client.0.smithi040.stdout:7/748: write d21/f23 [1622093,45715] 0 2024-02-16T19:24:23.943 INFO:tasks.workunit.client.0.smithi040.stdout:7/749: dread - d21/d29/d79/fc4 zero size 2024-02-16T19:24:23.943 INFO:tasks.workunit.client.0.smithi040.stdout:7/750: stat d21/d29/d79/dbc/fc0 0 2024-02-16T19:24:23.943 INFO:tasks.workunit.client.1.smithi163.stdout:8/259: creat d0/d7/d18/d2e/f5f x:0 0 0 2024-02-16T19:24:23.947 INFO:tasks.workunit.client.0.smithi040.stdout:5/822: symlink d1/da/db0/lfc 0 2024-02-16T19:24:23.947 INFO:tasks.workunit.client.0.smithi040.stdout:5/823: creat d1/da/d51/ffd x:0 0 0 2024-02-16T19:24:23.947 INFO:tasks.workunit.client.0.smithi040.stdout:5/824: getdents d1/da/d1c/dba/dd6/df7 0 2024-02-16T19:24:23.948 INFO:tasks.workunit.client.1.smithi163.stdout:0/480: creat d11/d21/fad x:0 0 0 2024-02-16T19:24:23.948 INFO:tasks.workunit.client.1.smithi163.stdout:0/481: fsync d11/d22/d7f/f83 0 2024-02-16T19:24:23.948 INFO:tasks.workunit.client.1.smithi163.stdout:0/482: dread - d11/f62 zero size 2024-02-16T19:24:23.952 INFO:tasks.workunit.client.0.smithi040.stdout:3/919: dwrite d1/dc/d42/d5c/df3/f102 [0,4194304] 0 2024-02-16T19:24:23.952 INFO:tasks.workunit.client.1.smithi163.stdout:2/419: dwrite d2/d2a/d39/f86 [0,4194304] 0 2024-02-16T19:24:23.953 INFO:tasks.workunit.client.0.smithi040.stdout:5/825: dread d1/da/d1c/d73/fbe [0,4194304] 0 2024-02-16T19:24:23.953 INFO:tasks.workunit.client.0.smithi040.stdout:9/855: rmdir d3 39 2024-02-16T19:24:23.954 INFO:tasks.workunit.client.0.smithi040.stdout:8/958: read d6/d12/d51/d6e/db7/ddf/df8/f7b [925684,45876] 0 2024-02-16T19:24:23.956 INFO:tasks.workunit.client.0.smithi040.stdout:5/826: read d1/da/d1c/d73/fbd [393915,89410] 0 2024-02-16T19:24:23.957 INFO:tasks.workunit.client.1.smithi163.stdout:5/258: dread d0/d2/fc [0,4194304] 0 2024-02-16T19:24:23.958 INFO:tasks.workunit.client.0.smithi040.stdout:4/663: getdents dd/de/d10 0 2024-02-16T19:24:23.958 INFO:tasks.workunit.client.0.smithi040.stdout:4/664: write dd/de/daa/fab [230777,80438] 0 2024-02-16T19:24:23.959 INFO:tasks.workunit.client.0.smithi040.stdout:2/783: dwrite d5/d7/d1e/d31/d4f/fd3 [0,4194304] 0 2024-02-16T19:24:23.962 INFO:tasks.workunit.client.1.smithi163.stdout:2/420: mknod d2/d2a/d2d/d73/c8f 0 2024-02-16T19:24:23.964 INFO:tasks.workunit.client.1.smithi163.stdout:0/483: write d11/d12/d67/d6a/d6d/f93 [2992133,84107] 0 2024-02-16T19:24:23.964 INFO:tasks.workunit.client.1.smithi163.stdout:0/484: dread - d11/d12/d67/fa6 zero size 2024-02-16T19:24:23.966 INFO:tasks.workunit.client.1.smithi163.stdout:5/259: mkdir d0/d22/d28/d36/d3c/d4f 0 2024-02-16T19:24:23.966 INFO:tasks.workunit.client.1.smithi163.stdout:5/260: write d0/d22/f23 [185466,73605] 0 2024-02-16T19:24:23.968 INFO:tasks.workunit.client.0.smithi040.stdout:9/856: unlink d3/d4/d11/d29/d38/d46/d6b/c92 0 2024-02-16T19:24:23.968 INFO:tasks.workunit.client.0.smithi040.stdout:9/857: read - d3/d4/d11/d29/d38/d86/faa zero size 2024-02-16T19:24:23.969 INFO:tasks.workunit.client.0.smithi040.stdout:8/959: write d6/d2b/da3/d8b/f90 [2600424,82460] 0 2024-02-16T19:24:23.970 INFO:tasks.workunit.client.0.smithi040.stdout:8/960: fdatasync d6/d2b/da3/f53 0 2024-02-16T19:24:23.970 INFO:tasks.workunit.client.0.smithi040.stdout:5/827: rename d1/da/f2a to d1/da/dd5/ffe 0 2024-02-16T19:24:23.973 INFO:tasks.workunit.client.1.smithi163.stdout:2/421: mknod d2/d2a/d39/d66/d6c/d7f/c90 0 2024-02-16T19:24:23.973 INFO:tasks.workunit.client.1.smithi163.stdout:2/422: write d2/d2a/d39/d3a/f7b [190933,61350] 0 2024-02-16T19:24:23.973 INFO:tasks.workunit.client.1.smithi163.stdout:7/266: dwrite f5 [4194304,4194304] 0 2024-02-16T19:24:23.979 INFO:tasks.workunit.client.1.smithi163.stdout:2/423: unlink d2/d2a/d2d/d3c/l56 0 2024-02-16T19:24:23.979 INFO:tasks.workunit.client.1.smithi163.stdout:5/261: creat d0/d22/d28/d36/d3c/d4f/f50 x:0 0 0 2024-02-16T19:24:23.979 INFO:tasks.workunit.client.1.smithi163.stdout:5/262: fdatasync d0/d22/f2e 0 2024-02-16T19:24:23.979 INFO:tasks.workunit.client.1.smithi163.stdout:2/424: chown d2/d2a/d2d/f6b 192 1 2024-02-16T19:24:23.979 INFO:tasks.workunit.client.1.smithi163.stdout:2/425: chown d2/ce 0 1 2024-02-16T19:24:23.981 INFO:tasks.workunit.client.1.smithi163.stdout:3/292: dwrite d0/dc/d1d/d29/f30 [4194304,4194304] 0 2024-02-16T19:24:23.982 INFO:tasks.workunit.client.0.smithi040.stdout:8/961: readlink d6/d64/l114 0 2024-02-16T19:24:23.985 INFO:tasks.workunit.client.0.smithi040.stdout:4/665: write dd/f8e [3850661,6975] 0 2024-02-16T19:24:23.985 INFO:tasks.workunit.client.0.smithi040.stdout:4/666: read dd/de/f5d [816524,116409] 0 2024-02-16T19:24:23.985 INFO:tasks.workunit.client.0.smithi040.stdout:9/858: dread d3/d4/d11/d29/d38/d46/f6a [0,4194304] 0 2024-02-16T19:24:23.986 INFO:tasks.workunit.client.0.smithi040.stdout:4/667: fsync dd/f66 0 2024-02-16T19:24:23.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/263: creat d0/d2/d34/d44/f51 x:0 0 0 2024-02-16T19:24:23.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/264: chown d0/d22/d3b 421429384 1 2024-02-16T19:24:23.987 INFO:tasks.workunit.client.1.smithi163.stdout:5/265: read - d0/d2/f29 zero size 2024-02-16T19:24:23.988 INFO:tasks.workunit.client.0.smithi040.stdout:0/991: dwrite db/d2f/d1d/d21/d2e/d49/d5b/d67/d84/fc6 [4194304,4194304] 0 2024-02-16T19:24:23.989 INFO:tasks.workunit.client.1.smithi163.stdout:2/426: unlink d2/c36 0 2024-02-16T19:24:23.989 INFO:tasks.workunit.client.1.smithi163.stdout:2/427: fsync d2/d2a/d39/d3a/f5b 0 2024-02-16T19:24:23.990 INFO:tasks.workunit.client.1.smithi163.stdout:7/267: write d11/f20 [788638,103021] 0 2024-02-16T19:24:23.993 INFO:tasks.workunit.client.1.smithi163.stdout:3/293: symlink d0/dc/d1c/d32/d62/l6c 0 2024-02-16T19:24:23.993 INFO:tasks.workunit.client.1.smithi163.stdout:5/266: truncate d0/d2/f21 748055 0 2024-02-16T19:24:23.994 INFO:tasks.workunit.client.1.smithi163.stdout:5/267: fsync d0/d2/f29 0 2024-02-16T19:24:23.994 INFO:tasks.workunit.client.1.smithi163.stdout:5/268: chown d0/l19 1836982 1 2024-02-16T19:24:23.994 INFO:tasks.workunit.client.0.smithi040.stdout:4/668: rmdir dd/d58/d5e 39 2024-02-16T19:24:23.995 INFO:tasks.workunit.client.0.smithi040.stdout:7/751: dwrite d21/d29/f4c [0,4194304] 0 2024-02-16T19:24:23.995 INFO:tasks.workunit.client.0.smithi040.stdout:7/752: creat d21/d29/dad/ffa x:0 0 0 2024-02-16T19:24:23.995 INFO:tasks.workunit.client.0.smithi040.stdout:7/753: chown f1f 154 1 2024-02-16T19:24:23.996 INFO:tasks.workunit.client.1.smithi163.stdout:3/294: rmdir d0/dc/d1c/d32/d62 39 2024-02-16T19:24:23.996 INFO:tasks.workunit.client.1.smithi163.stdout:3/295: write d0/f1b [2378934,94825] 0 2024-02-16T19:24:23.996 INFO:tasks.workunit.client.1.smithi163.stdout:3/296: creat d0/f6d x:0 0 0 2024-02-16T19:24:23.996 INFO:tasks.workunit.client.1.smithi163.stdout:3/297: chown d0/dc/f2e 96 1 2024-02-16T19:24:24.001 INFO:tasks.workunit.client.1.smithi163.stdout:8/260: dwrite d0/d24/d3c/f4c [0,4194304] 0 2024-02-16T19:24:24.001 INFO:tasks.workunit.client.1.smithi163.stdout:8/261: fdatasync d0/d7/d19/d1f/d30/f47 0 2024-02-16T19:24:24.001 INFO:tasks.workunit.client.1.smithi163.stdout:8/262: readlink d0/d7/d18/l3e 0 2024-02-16T19:24:24.002 INFO:tasks.workunit.client.0.smithi040.stdout:0/992: unlink db/de/d6a/f110 0 2024-02-16T19:24:24.002 INFO:tasks.workunit.client.0.smithi040.stdout:0/993: chown db/de/c73 32336 1 2024-02-16T19:24:24.002 INFO:tasks.workunit.client.1.smithi163.stdout:5/269: read d0/d2/fc [1762401,112121] 0 2024-02-16T19:24:24.003 INFO:tasks.workunit.client.0.smithi040.stdout:4/669: truncate dd/de/d10/d36/d9e/d32/f41 672346 0 2024-02-16T19:24:24.005 INFO:tasks.workunit.client.1.smithi163.stdout:7/268: dread d11/f1c [0,4194304] 0 2024-02-16T19:24:24.007 INFO:tasks.workunit.client.0.smithi040.stdout:0/994: dread db/d2f/d1d/d21/d2e/d49/d5b/d67/fb7 [0,4194304] 0 2024-02-16T19:24:24.007 INFO:tasks.workunit.client.1.smithi163.stdout:8/263: rename d0/d7/d19/d1f/d32/f40 to d0/d24/f60 0 2024-02-16T19:24:24.010 INFO:tasks.workunit.client.1.smithi163.stdout:5/270: getdents d0/d2 0 2024-02-16T19:24:24.011 INFO:tasks.workunit.client.0.smithi040.stdout:3/920: dwrite d1/dc/d42/d4a/f110 [0,4194304] 0 2024-02-16T19:24:24.011 INFO:tasks.workunit.client.0.smithi040.stdout:3/921: read d1/dc/d42/d5c/f6d [419714,70338] 0 2024-02-16T19:24:24.011 INFO:tasks.workunit.client.1.smithi163.stdout:0/485: getdents d11/d12 0 2024-02-16T19:24:24.015 INFO:tasks.workunit.client.1.smithi163.stdout:8/264: link d0/d24/d2c/c50 d0/d7/d18/d2e/c61 0 2024-02-16T19:24:24.016 INFO:tasks.workunit.client.0.smithi040.stdout:7/754: truncate f1f 1990778 0 2024-02-16T19:24:24.017 INFO:tasks.workunit.client.1.smithi163.stdout:5/271: symlink d0/d22/d28/d36/l52 0 2024-02-16T19:24:24.018 INFO:tasks.workunit.client.1.smithi163.stdout:5/272: chown d0/f2c 3 1 2024-02-16T19:24:24.018 INFO:tasks.workunit.client.1.smithi163.stdout:5/273: fsync d0/d2/f16 0 2024-02-16T19:24:24.018 INFO:tasks.workunit.client.1.smithi163.stdout:0/486: rmdir d11/d12 39 2024-02-16T19:24:24.019 INFO:tasks.workunit.client.1.smithi163.stdout:0/487: readlink d11/d21/l82 0 2024-02-16T19:24:24.020 INFO:tasks.workunit.client.0.smithi040.stdout:4/670: link dd/de/d10/d36/d9e/d32/f64 dd/de/fcf 0 2024-02-16T19:24:24.021 INFO:tasks.workunit.client.1.smithi163.stdout:8/265: symlink d0/d24/d2c/l62 0 2024-02-16T19:24:24.022 INFO:tasks.workunit.client.0.smithi040.stdout:0/995: mkdir db/d2f/d1d/d21/d2e/d156 0 2024-02-16T19:24:24.025 INFO:tasks.workunit.client.0.smithi040.stdout:3/922: creat d1/d7/f121 x:0 0 0 2024-02-16T19:24:24.025 INFO:tasks.workunit.client.1.smithi163.stdout:1/255: dwrite d7/d20/d28/f41 [4194304,4194304] 0 2024-02-16T19:24:24.027 INFO:tasks.workunit.client.1.smithi163.stdout:6/256: write d0/d6/f1a [3532256,45668] 0 2024-02-16T19:24:24.027 INFO:tasks.workunit.client.1.smithi163.stdout:6/257: write d0/f5 [3942158,123658] 0 2024-02-16T19:24:24.027 INFO:tasks.workunit.client.1.smithi163.stdout:8/266: mknod d0/d7/d55/c63 0 2024-02-16T19:24:24.030 INFO:tasks.workunit.client.1.smithi163.stdout:1/256: write d7/dc/d2d/d32/f43 [2865286,10672] 0 2024-02-16T19:24:24.031 INFO:tasks.workunit.client.1.smithi163.stdout:1/257: creat d7/d20/d21/f55 x:0 0 0 2024-02-16T19:24:24.031 INFO:tasks.workunit.client.1.smithi163.stdout:1/258: stat d7/d20/f35 0 2024-02-16T19:24:24.031 INFO:tasks.workunit.client.1.smithi163.stdout:1/259: chown d7/d20/c2c 433790 1 2024-02-16T19:24:24.031 INFO:tasks.workunit.client.1.smithi163.stdout:1/260: write d7/d20/d2a/f40 [45135,97745] 0 2024-02-16T19:24:24.031 INFO:tasks.workunit.client.0.smithi040.stdout:7/755: symlink d21/d7e/df8/db7/lfb 0 2024-02-16T19:24:24.032 INFO:tasks.workunit.client.0.smithi040.stdout:2/784: dwrite d5/d5c/d65/d9d/fc7 [0,4194304] 0 2024-02-16T19:24:24.032 INFO:tasks.workunit.client.0.smithi040.stdout:2/785: chown d5/d7/d10/df3/f69 40433 1 2024-02-16T19:24:24.033 INFO:tasks.workunit.client.1.smithi163.stdout:0/488: rmdir d11/d12/d20 39 2024-02-16T19:24:24.034 INFO:tasks.workunit.client.0.smithi040.stdout:4/671: symlink dd/de/d71/ld0 0 2024-02-16T19:24:24.034 INFO:tasks.workunit.client.0.smithi040.stdout:4/672: write dd/de/d10/d36/dad/fb4 [1455851,34746] 0 2024-02-16T19:24:24.034 INFO:tasks.workunit.client.0.smithi040.stdout:4/673: dread - dd/de/d45/f5c zero size 2024-02-16T19:24:24.034 INFO:tasks.workunit.client.0.smithi040.stdout:4/674: chown fb 0 1 2024-02-16T19:24:24.034 INFO:tasks.workunit.client.0.smithi040.stdout:4/675: read - dd/d58/fbb zero size 2024-02-16T19:24:24.036 INFO:tasks.workunit.client.0.smithi040.stdout:7/756: rename d21/d7e/df8/db7/de7/fed to d21/d4d/d7b/ffc 0 2024-02-16T19:24:24.036 INFO:tasks.workunit.client.0.smithi040.stdout:7/757: chown d21/d8b/fb5 87495 1 2024-02-16T19:24:24.042 INFO:tasks.workunit.client.0.smithi040.stdout:2/786: write d5/d7/d1e/f6d [1034280,50788] 0 2024-02-16T19:24:24.043 INFO:tasks.workunit.client.0.smithi040.stdout:8/962: dwrite d6/d12/d51/d6e/db9/dcf/fa7 [0,4194304] 0 2024-02-16T19:24:24.043 INFO:tasks.workunit.client.0.smithi040.stdout:8/963: write d6/d12/d51/d6e/db7/ddf/df8/f2e [874199,29332] 0 2024-02-16T19:24:24.043 INFO:tasks.workunit.client.0.smithi040.stdout:4/676: unlink dd/d58/d5e/f69 0 2024-02-16T19:24:24.043 INFO:tasks.workunit.client.0.smithi040.stdout:5/828: dwrite d1/da/d1c/f26 [0,4194304] 0 2024-02-16T19:24:24.046 INFO:tasks.workunit.client.0.smithi040.stdout:2/787: mkdir d5/d7/d1e/d31/d4f/d63/d8b/dff 0 2024-02-16T19:24:24.046 INFO:tasks.workunit.client.0.smithi040.stdout:2/788: write d5/d7/d1e/d31/f34 [1540309,21139] 0 2024-02-16T19:24:24.047 INFO:tasks.workunit.client.1.smithi163.stdout:1/261: symlink d7/dc/l56 0 2024-02-16T19:24:24.051 INFO:tasks.workunit.client.0.smithi040.stdout:8/964: rename d6/d12/d51/d6e/db7/ddf/df8/d15/f61 to d6/d12/d51/d6e/db7/dc6/dce/f127 0 2024-02-16T19:24:24.051 INFO:tasks.workunit.client.0.smithi040.stdout:8/965: fdatasync d6/d2b/f9a 0 2024-02-16T19:24:24.054 INFO:tasks.workunit.client.0.smithi040.stdout:2/789: mknod d5/d7/d12/da6/c100 0 2024-02-16T19:24:24.056 INFO:tasks.workunit.client.0.smithi040.stdout:8/966: mkdir d6/d12/d51/d6e/db9/dc9/d126/d128 0 2024-02-16T19:24:24.056 INFO:tasks.workunit.client.0.smithi040.stdout:8/967: chown d6/d2b/da3/dcb 81383 1 2024-02-16T19:24:24.057 INFO:tasks.workunit.client.1.smithi163.stdout:1/262: getdents d7/dc/d2d/d32 0 2024-02-16T19:24:24.059 INFO:tasks.workunit.client.1.smithi163.stdout:0/489: getdents d11/d21/d7c 0 2024-02-16T19:24:24.061 INFO:tasks.workunit.client.1.smithi163.stdout:1/263: creat d7/f57 x:0 0 0 2024-02-16T19:24:24.062 INFO:tasks.workunit.client.1.smithi163.stdout:2/428: dwrite d2/d26/f2f [0,4194304] 0 2024-02-16T19:24:24.062 INFO:tasks.workunit.client.1.smithi163.stdout:7/269: dwrite d11/f1b [4194304,4194304] 0 2024-02-16T19:24:24.062 INFO:tasks.workunit.client.1.smithi163.stdout:7/270: write d11/d24/d25/f5a [2200975,119997] 0 2024-02-16T19:24:24.062 INFO:tasks.workunit.client.1.smithi163.stdout:7/271: chown d11/d26/f4c 55 1 2024-02-16T19:24:24.062 INFO:tasks.workunit.client.0.smithi040.stdout:4/677: dread dd/d58/d5e/f7f [0,4194304] 0 2024-02-16T19:24:24.064 INFO:tasks.workunit.client.1.smithi163.stdout:1/264: creat d7/dc/d2d/d32/d3b/f58 x:0 0 0 2024-02-16T19:24:24.068 INFO:tasks.workunit.client.0.smithi040.stdout:2/790: write d5/d7/d1e/d4c/d52/daa/dcd/ffc [3221937,58023] 0 2024-02-16T19:24:24.068 INFO:tasks.workunit.client.0.smithi040.stdout:0/996: dwrite db/d2f/d23/d3b/f47 [4194304,4194304] 0 2024-02-16T19:24:24.068 INFO:tasks.workunit.client.0.smithi040.stdout:3/923: dwrite d1/dc/f3c [0,4194304] 0 2024-02-16T19:24:24.071 INFO:tasks.workunit.client.1.smithi163.stdout:2/429: truncate d2/d2a/d39/f3b 170635 0 2024-02-16T19:24:24.071 INFO:tasks.workunit.client.1.smithi163.stdout:5/274: dwrite d0/d22/d28/f38 [0,4194304] 0 2024-02-16T19:24:24.072 INFO:tasks.workunit.client.0.smithi040.stdout:4/678: rename dd/d58/d8c/d99 to dd/d58/d5e/dd1 0 2024-02-16T19:24:24.077 INFO:tasks.workunit.client.1.smithi163.stdout:5/275: dread d0/d22/f23 [0,4194304] 0 2024-02-16T19:24:24.077 INFO:tasks.workunit.client.0.smithi040.stdout:3/924: unlink d1/dc/d35/d55/c6b 0 2024-02-16T19:24:24.079 INFO:tasks.workunit.client.1.smithi163.stdout:2/430: mknod d2/c91 0 2024-02-16T19:24:24.081 INFO:tasks.workunit.client.0.smithi040.stdout:5/829: dwrite d1/da/d1c/d73/fbe [0,4194304] 0 2024-02-16T19:24:24.086 INFO:tasks.workunit.client.0.smithi040.stdout:8/968: dwrite d6/d12/d51/fa1 [4194304,4194304] 0 2024-02-16T19:24:24.086 INFO:tasks.workunit.client.0.smithi040.stdout:8/969: fdatasync d6/d64/d50/d69/fa4 0 2024-02-16T19:24:24.086 INFO:tasks.workunit.client.0.smithi040.stdout:8/970: read - d6/d12/d51/d6e/db7/ddf/df8/d1d/f77 zero size 2024-02-16T19:24:24.087 INFO:tasks.workunit.client.1.smithi163.stdout:0/490: dwrite d11/d21/f47 [0,4194304] 0 2024-02-16T19:24:24.087 INFO:tasks.workunit.client.1.smithi163.stdout:0/491: stat d11/f85 0 2024-02-16T19:24:24.087 INFO:tasks.workunit.client.0.smithi040.stdout:2/791: rename d5/d5c/d65/d9d/fd6 to d5/d7/d1e/d4c/d52/f101 0 2024-02-16T19:24:24.087 INFO:tasks.workunit.client.0.smithi040.stdout:2/792: write d5/d7/d1e/d31/d4f/d63/f2c [3743184,22226] 0 2024-02-16T19:24:24.087 INFO:tasks.workunit.client.0.smithi040.stdout:2/793: write d5/d7/d1e/d4c/d52/daa/dcd/ffc [4738990,69319] 0 2024-02-16T19:24:24.087 INFO:tasks.workunit.client.0.smithi040.stdout:7/758: dwrite d21/d4d/d7b/fd6 [0,4194304] 0 2024-02-16T19:24:24.088 INFO:tasks.workunit.client.0.smithi040.stdout:4/679: dread dd/de/d71/f9a [0,4194304] 0 2024-02-16T19:24:24.089 INFO:tasks.workunit.client.0.smithi040.stdout:3/925: creat d1/d11a/f122 x:0 0 0 2024-02-16T19:24:24.090 INFO:tasks.workunit.client.1.smithi163.stdout:1/265: unlink d7/le 0 2024-02-16T19:24:24.092 INFO:tasks.workunit.client.1.smithi163.stdout:5/276: mkdir d0/d2/d34/d44/d53 0 2024-02-16T19:24:24.093 INFO:tasks.workunit.client.1.smithi163.stdout:5/277: creat d0/d22/d28/d36/d3c/f54 x:0 0 0 2024-02-16T19:24:24.093 INFO:tasks.workunit.client.1.smithi163.stdout:5/278: write d0/d22/d28/d36/d3c/f43 [634187,61788] 0 2024-02-16T19:24:24.094 INFO:tasks.workunit.client.1.smithi163.stdout:7/272: dread d11/d24/d25/f5a [0,4194304] 0 2024-02-16T19:24:24.094 INFO:tasks.workunit.client.1.smithi163.stdout:7/273: chown d11/d50 113754032 1 2024-02-16T19:24:24.098 INFO:tasks.workunit.client.0.smithi040.stdout:7/759: write d21/d29/f73 [5561260,14540] 0 2024-02-16T19:24:24.099 INFO:tasks.workunit.client.1.smithi163.stdout:2/431: dwrite d2/d2a/d2d/f6b [0,4194304] 0 2024-02-16T19:24:24.099 INFO:tasks.workunit.client.1.smithi163.stdout:2/432: creat d2/d2a/d2d/f92 x:0 0 0 2024-02-16T19:24:24.099 INFO:tasks.workunit.client.1.smithi163.stdout:2/433: dread - d2/d26/f5d zero size 2024-02-16T19:24:24.100 INFO:tasks.workunit.client.0.smithi040.stdout:8/971: dread f5 [0,4194304] 0 2024-02-16T19:24:24.102 INFO:tasks.workunit.client.0.smithi040.stdout:8/972: write d6/d12/d51/db6/fe9 [191652,109563] 0 2024-02-16T19:24:24.102 INFO:tasks.workunit.client.0.smithi040.stdout:8/973: chown d6/d64/d50/d69/cc7 7845 1 2024-02-16T19:24:24.102 INFO:tasks.workunit.client.0.smithi040.stdout:8/974: fdatasync d6/d64/d50/d69/fa4 0 2024-02-16T19:24:24.103 INFO:tasks.workunit.client.1.smithi163.stdout:1/266: mknod d7/dc/d2d/d32/c59 0 2024-02-16T19:24:24.109 INFO:tasks.workunit.client.1.smithi163.stdout:7/274: creat d11/d50/d5d/f61 x:0 0 0 2024-02-16T19:24:24.109 INFO:tasks.workunit.client.1.smithi163.stdout:7/275: creat d11/d24/d25/f62 x:0 0 0 2024-02-16T19:24:24.109 INFO:tasks.workunit.client.1.smithi163.stdout:7/276: fsync d11/d24/d25/f4f 0 2024-02-16T19:24:24.111 INFO:tasks.workunit.client.0.smithi040.stdout:5/830: mknod d1/da/dd5/dfb/cff 0 2024-02-16T19:24:24.116 INFO:tasks.workunit.client.0.smithi040.stdout:2/794: truncate d5/d7/d1e/d31/d4f/fd3 1448487 0 2024-02-16T19:24:24.119 INFO:tasks.workunit.client.1.smithi163.stdout:2/434: symlink d2/d2a/d2d/d73/d84/l93 0 2024-02-16T19:24:24.121 INFO:tasks.workunit.client.1.smithi163.stdout:0/492: getdents d11/d12/d67/d6a/d6d 0 2024-02-16T19:24:24.121 INFO:tasks.workunit.client.1.smithi163.stdout:5/279: dwrite d0/d22/d3b/f4c [0,4194304] 0 2024-02-16T19:24:24.122 INFO:tasks.workunit.client.0.smithi040.stdout:4/680: dwrite dd/de/fcf [0,4194304] 0 2024-02-16T19:24:24.124 INFO:tasks.workunit.client.0.smithi040.stdout:4/681: dread dd/d58/d8c/fbe [0,4194304] 0 2024-02-16T19:24:24.124 INFO:tasks.workunit.client.0.smithi040.stdout:3/926: symlink d1/dc/d42/d4a/d57/d79/de3/l123 0 2024-02-16T19:24:24.124 INFO:tasks.workunit.client.0.smithi040.stdout:3/927: readlink d1/dc/d42/d5c/l9d 0 2024-02-16T19:24:24.124 INFO:tasks.workunit.client.0.smithi040.stdout:3/928: stat d1/d4/d6e 0 2024-02-16T19:24:24.124 INFO:tasks.workunit.client.0.smithi040.stdout:3/929: truncate d1/f37 1150220 0 2024-02-16T19:24:24.125 INFO:tasks.workunit.client.1.smithi163.stdout:1/267: creat d7/dc/f5a x:0 0 0 2024-02-16T19:24:24.125 INFO:tasks.workunit.client.1.smithi163.stdout:1/268: dread - d7/dc/f5a zero size 2024-02-16T19:24:24.129 INFO:tasks.workunit.client.1.smithi163.stdout:7/277: mknod d11/d26/d36/c63 0 2024-02-16T19:24:24.130 INFO:tasks.workunit.client.0.smithi040.stdout:7/760: link d21/d29/l5a d21/d4d/d7d/lfd 0 2024-02-16T19:24:24.131 INFO:tasks.workunit.client.1.smithi163.stdout:2/435: mkdir d2/d2a/d2d/d73/d94 0 2024-02-16T19:24:24.131 INFO:tasks.workunit.client.1.smithi163.stdout:2/436: chown d2/d2a/d39/d66/d6c/f79 59927 1 2024-02-16T19:24:24.132 INFO:tasks.workunit.client.0.smithi040.stdout:5/831: dwrite d1/d62/ff5 [0,4194304] 0 2024-02-16T19:24:24.133 INFO:tasks.workunit.client.0.smithi040.stdout:8/975: link d6/d2b/da3/d8b/cdb d6/d64/d50/d69/c129 0 2024-02-16T19:24:24.134 INFO:tasks.workunit.client.1.smithi163.stdout:0/493: mkdir d11/d12/d67/d6a/dae 0 2024-02-16T19:24:24.134 INFO:tasks.workunit.client.1.smithi163.stdout:0/494: truncate d11/d21/f46 2500223 0 2024-02-16T19:24:24.134 INFO:tasks.workunit.client.1.smithi163.stdout:0/495: write f10 [2436289,67021] 0 2024-02-16T19:24:24.136 INFO:tasks.workunit.client.0.smithi040.stdout:2/795: mkdir d5/d7/d1e/d55/d102 0 2024-02-16T19:24:24.136 INFO:tasks.workunit.client.0.smithi040.stdout:2/796: chown d5/d7/d12/d1b/l2a 7 1 2024-02-16T19:24:24.138 INFO:tasks.workunit.client.0.smithi040.stdout:4/682: mkdir dd/dd2 0 2024-02-16T19:24:24.140 INFO:tasks.workunit.client.1.smithi163.stdout:9/450: sync 2024-02-16T19:24:24.140 INFO:tasks.workunit.client.0.smithi040.stdout:3/930: symlink d1/d8c/d8e/l124 0 2024-02-16T19:24:24.142 INFO:tasks.workunit.client.0.smithi040.stdout:7/761: mkdir d21/df9/dfe 0 2024-02-16T19:24:24.146 INFO:tasks.workunit.client.1.smithi163.stdout:2/437: mknod d2/d2a/d2d/d73/c95 0 2024-02-16T19:24:24.146 INFO:tasks.workunit.client.1.smithi163.stdout:2/438: chown d2/d2a/d2d/d73 333956 1 2024-02-16T19:24:24.147 INFO:tasks.workunit.client.1.smithi163.stdout:5/280: dwrite d0/d22/d28/f38 [0,4194304] 0 2024-02-16T19:24:24.148 INFO:tasks.workunit.client.1.smithi163.stdout:2/439: write d2/d26/f28 [1250540,90920] 0 2024-02-16T19:24:24.148 INFO:tasks.workunit.client.1.smithi163.stdout:2/440: fdatasync d2/d2a/f46 0 2024-02-16T19:24:24.149 INFO:tasks.workunit.client.1.smithi163.stdout:0/496: symlink d11/d12/d20/laf 0 2024-02-16T19:24:24.152 INFO:tasks.workunit.client.1.smithi163.stdout:9/451: rename d5/de/l11 to d5/de/d16/d46/la2 0 2024-02-16T19:24:24.156 INFO:tasks.workunit.client.1.smithi163.stdout:1/269: mkdir d7/d5b 0 2024-02-16T19:24:24.158 INFO:tasks.workunit.client.0.smithi040.stdout:5/832: stat d1/d62/lec 0 2024-02-16T19:24:24.159 INFO:tasks.workunit.client.1.smithi163.stdout:2/441: symlink d2/d2a/d39/d3a/l96 0 2024-02-16T19:24:24.159 INFO:tasks.workunit.client.1.smithi163.stdout:2/442: write d2/d26/f35 [5130561,78277] 0 2024-02-16T19:24:24.161 INFO:tasks.workunit.client.0.smithi040.stdout:8/976: symlink d6/d12/d51/d6e/db7/dc6/dcc/de8/l12a 0 2024-02-16T19:24:24.161 INFO:tasks.workunit.client.0.smithi040.stdout:8/977: chown d6/f3c 2 1 2024-02-16T19:24:24.161 INFO:tasks.workunit.client.0.smithi040.stdout:8/978: readlink d6/d64/d50/l7e 0 2024-02-16T19:24:24.161 INFO:tasks.workunit.client.0.smithi040.stdout:8/979: truncate d6/d12/d22/f86 5346313 0 2024-02-16T19:24:24.161 INFO:tasks.workunit.client.0.smithi040.stdout:2/797: dwrite d5/d5c/d65/f75 [0,4194304] 0 2024-02-16T19:24:24.162 INFO:tasks.workunit.client.1.smithi163.stdout:5/281: dwrite d0/d22/f3e [0,4194304] 0 2024-02-16T19:24:24.163 INFO:tasks.workunit.client.0.smithi040.stdout:8/980: read d6/d12/d51/d6e/db7/ddf/df8/f70 [32517,24398] 0 2024-02-16T19:24:24.163 INFO:tasks.workunit.client.0.smithi040.stdout:8/981: write d6/fa5 [2336830,109383] 0 2024-02-16T19:24:24.163 INFO:tasks.workunit.client.0.smithi040.stdout:8/982: fdatasync d6/d12/f1f 0 2024-02-16T19:24:24.163 INFO:tasks.workunit.client.0.smithi040.stdout:8/983: read d6/f9 [1306456,52160] 0 2024-02-16T19:24:24.164 INFO:tasks.workunit.client.1.smithi163.stdout:0/497: link f8 d11/d4f/fb0 0 2024-02-16T19:24:24.165 INFO:tasks.workunit.client.0.smithi040.stdout:4/683: symlink dd/d58/d5e/ld3 0 2024-02-16T19:24:24.165 INFO:tasks.workunit.client.0.smithi040.stdout:4/684: truncate dd/d58/fb7 207689 0 2024-02-16T19:24:24.167 INFO:tasks.workunit.client.1.smithi163.stdout:9/452: mknod d5/de/d30/d3d/d49/d6d/d97/ca3 0 2024-02-16T19:24:24.167 INFO:tasks.workunit.client.1.smithi163.stdout:9/453: read d5/f9 [815214,41945] 0 2024-02-16T19:24:24.168 INFO:tasks.workunit.client.0.smithi040.stdout:3/931: mknod d1/c125 0 2024-02-16T19:24:24.170 INFO:tasks.workunit.client.1.smithi163.stdout:1/270: truncate d7/d20/d28/f41 3955996 0 2024-02-16T19:24:24.170 INFO:tasks.workunit.client.1.smithi163.stdout:1/271: creat d7/d20/d2a/f5c x:0 0 0 2024-02-16T19:24:24.170 INFO:tasks.workunit.client.1.smithi163.stdout:1/272: readlink d7/dc/l56 0 2024-02-16T19:24:24.171 INFO:tasks.workunit.client.0.smithi040.stdout:7/762: creat d21/d8b/d9c/fff x:0 0 0 2024-02-16T19:24:24.173 INFO:tasks.workunit.client.1.smithi163.stdout:2/443: mkdir d2/d26/d97 0 2024-02-16T19:24:24.174 INFO:tasks.workunit.client.0.smithi040.stdout:5/833: stat d1/da/d1c/d73/c7f 0 2024-02-16T19:24:24.178 INFO:tasks.workunit.client.1.smithi163.stdout:7/278: getdents d11/d50/d5d 0 2024-02-16T19:24:24.178 INFO:tasks.workunit.client.1.smithi163.stdout:7/279: fsync f3 0 2024-02-16T19:24:24.178 INFO:tasks.workunit.client.1.smithi163.stdout:7/280: write d11/d26/d36/f3c [4257555,41838] 0 2024-02-16T19:24:24.179 INFO:tasks.workunit.client.1.smithi163.stdout:7/281: write d11/d24/d25/f62 [149659,69934] 0 2024-02-16T19:24:24.179 INFO:tasks.workunit.client.1.smithi163.stdout:7/282: chown d11/d50/c53 12046 1 2024-02-16T19:24:24.179 INFO:tasks.workunit.client.1.smithi163.stdout:7/283: write d11/d24/d49/f51 [9879,124661] 0 2024-02-16T19:24:24.179 INFO:tasks.workunit.client.1.smithi163.stdout:4/212: sync 2024-02-16T19:24:24.179 INFO:tasks.workunit.client.1.smithi163.stdout:3/298: sync 2024-02-16T19:24:24.179 INFO:tasks.workunit.client.1.smithi163.stdout:4/213: chown d11/d18/d1f/f33 6665876 1 2024-02-16T19:24:24.180 INFO:tasks.workunit.client.0.smithi040.stdout:2/798: creat d5/d7/d10/d40/f103 x:0 0 0 2024-02-16T19:24:24.180 INFO:tasks.workunit.client.1.smithi163.stdout:7/284: dread fc [0,4194304] 0 2024-02-16T19:24:24.181 INFO:tasks.workunit.client.1.smithi163.stdout:0/498: symlink d11/d12/d67/lb1 0 2024-02-16T19:24:24.181 INFO:tasks.workunit.client.1.smithi163.stdout:0/499: dread - d11/d22/d7f/f95 zero size 2024-02-16T19:24:24.184 INFO:tasks.workunit.client.0.smithi040.stdout:8/984: unlink d6/d12/d51/d6e/db9/d123/fb4 0 2024-02-16T19:24:24.185 INFO:tasks.workunit.client.1.smithi163.stdout:1/273: rename d7/d20/c2c to d7/d20/d2a/c5d 0 2024-02-16T19:24:24.186 INFO:tasks.workunit.client.0.smithi040.stdout:4/685: creat dd/de/d45/fd4 x:0 0 0 2024-02-16T19:24:24.187 INFO:tasks.workunit.client.1.smithi163.stdout:2/444: mkdir d2/d26/d54/d58/d98 0 2024-02-16T19:24:24.187 INFO:tasks.workunit.client.1.smithi163.stdout:2/445: readlink d2/l14 0 2024-02-16T19:24:24.189 INFO:tasks.workunit.client.0.smithi040.stdout:3/932: mkdir d1/dc/d35/d48/d85/d126 0 2024-02-16T19:24:24.192 INFO:tasks.workunit.client.0.smithi040.stdout:7/763: mkdir d21/d4d/d77/d87/d9b/df2/d100 0 2024-02-16T19:24:24.192 INFO:tasks.workunit.client.1.smithi163.stdout:9/454: dwrite d5/f2f [0,4194304] 0 2024-02-16T19:24:24.194 INFO:tasks.workunit.client.1.smithi163.stdout:5/282: getdents d0/d2/d34 0 2024-02-16T19:24:24.196 INFO:tasks.workunit.client.0.smithi040.stdout:5/834: mkdir d1/da/d1c/da7/d100 0 2024-02-16T19:24:24.196 INFO:tasks.workunit.client.0.smithi040.stdout:5/835: chown d1 1156731 1 2024-02-16T19:24:24.197 INFO:tasks.workunit.client.1.smithi163.stdout:2/446: dread d2/d2a/d39/f86 [0,4194304] 0 2024-02-16T19:24:24.198 INFO:tasks.workunit.client.1.smithi163.stdout:4/214: unlink d11/d18/d1f/c40 0 2024-02-16T19:24:24.198 INFO:tasks.workunit.client.1.smithi163.stdout:4/215: readlink d11/d18/d1f/d29/l3c 0 2024-02-16T19:24:24.198 INFO:tasks.workunit.client.1.smithi163.stdout:4/216: creat d11/f41 x:0 0 0 2024-02-16T19:24:24.200 INFO:tasks.workunit.client.0.smithi040.stdout:2/799: link d5/d7/d1e/d55/f6f d5/d7/d12/dca/f104 0 2024-02-16T19:24:24.201 INFO:tasks.workunit.client.1.smithi163.stdout:3/299: creat d0/dc/f6e x:0 0 0 2024-02-16T19:24:24.201 INFO:tasks.workunit.client.1.smithi163.stdout:9/455: dread d5/f40 [0,4194304] 0 2024-02-16T19:24:24.204 INFO:tasks.workunit.client.1.smithi163.stdout:7/285: mknod d11/d26/c64 0 2024-02-16T19:24:24.207 INFO:tasks.workunit.client.1.smithi163.stdout:0/500: symlink d11/d12/d20/lb2 0 2024-02-16T19:24:24.208 INFO:tasks.workunit.client.1.smithi163.stdout:0/501: write f10 [4199787,22474] 0 2024-02-16T19:24:24.210 INFO:tasks.workunit.client.1.smithi163.stdout:1/274: symlink d7/d5b/l5e 0 2024-02-16T19:24:24.211 INFO:tasks.workunit.client.1.smithi163.stdout:1/275: write d7/d20/d2a/f40 [498465,63442] 0 2024-02-16T19:24:24.211 INFO:tasks.workunit.client.1.smithi163.stdout:1/276: stat d7/f57 0 2024-02-16T19:24:24.211 INFO:tasks.workunit.client.1.smithi163.stdout:8/267: sync 2024-02-16T19:24:24.211 INFO:tasks.workunit.client.1.smithi163.stdout:6/258: sync 2024-02-16T19:24:24.211 INFO:tasks.workunit.client.1.smithi163.stdout:6/259: chown d0/d6/df/d20 132061488 1 2024-02-16T19:24:24.211 INFO:tasks.workunit.client.1.smithi163.stdout:5/283: dread d0/d22/d28/f38 [0,4194304] 0 2024-02-16T19:24:24.211 INFO:tasks.workunit.client.1.smithi163.stdout:5/284: truncate d0/f4d 394430 0 2024-02-16T19:24:24.213 INFO:tasks.workunit.client.1.smithi163.stdout:2/447: rename d2/d2a/d39/f76 to d2/d2a/d2d/d3c/d7c/f99 0 2024-02-16T19:24:24.213 INFO:tasks.workunit.client.0.smithi040.stdout:4/686: dwrite dd/de/d10/d36/d9e/d32/f95 [0,4194304] 0 2024-02-16T19:24:24.213 INFO:tasks.workunit.client.0.smithi040.stdout:4/687: fsync dd/de/d10/f2e 0 2024-02-16T19:24:24.214 INFO:tasks.workunit.client.0.smithi040.stdout:7/764: dwrite d21/d4d/d77/d87/dc9/ddd/dbe/fe8 [0,4194304] 0 2024-02-16T19:24:24.214 INFO:tasks.workunit.client.0.smithi040.stdout:7/765: write d21/d4d/d77/d87/dc9/ddd/dbe/fe3 [424833,90210] 0 2024-02-16T19:24:24.214 INFO:tasks.workunit.client.0.smithi040.stdout:7/766: chown d21/d4d/l9a 357 1 2024-02-16T19:24:24.215 INFO:tasks.workunit.client.0.smithi040.stdout:8/985: link d6/d64/d50/d69/cac d6/d12/d51/d6e/db9/d123/c12b 0 2024-02-16T19:24:24.215 INFO:tasks.workunit.client.1.smithi163.stdout:8/268: dread d0/dc/f22 [0,4194304] 0 2024-02-16T19:24:24.215 INFO:tasks.workunit.client.1.smithi163.stdout:8/269: creat d0/d24/d27/f64 x:0 0 0 2024-02-16T19:24:24.215 INFO:tasks.workunit.client.1.smithi163.stdout:8/270: write d0/d24/f60 [5156530,63779] 0 2024-02-16T19:24:24.218 INFO:tasks.workunit.client.0.smithi040.stdout:3/933: symlink d1/dc/d35/d10a/l127 0 2024-02-16T19:24:24.219 INFO:tasks.workunit.client.1.smithi163.stdout:4/217: symlink d11/d18/d1f/l42 0 2024-02-16T19:24:24.220 INFO:tasks.workunit.client.0.smithi040.stdout:5/836: symlink d1/da/d1c/d73/l101 0 2024-02-16T19:24:24.221 INFO:tasks.workunit.client.1.smithi163.stdout:3/300: link d0/dc/d1d/c27 d0/dc/d1c/d32/c6f 0 2024-02-16T19:24:24.223 INFO:tasks.workunit.client.0.smithi040.stdout:2/800: link d5/d7/d1e/d31/c3e d5/d7/d1e/d4c/d52/d81/c105 0 2024-02-16T19:24:24.224 INFO:tasks.workunit.client.1.smithi163.stdout:7/286: symlink d11/d24/l65 0 2024-02-16T19:24:24.224 INFO:tasks.workunit.client.1.smithi163.stdout:7/287: write d11/d24/d25/f39 [304622,4748] 0 2024-02-16T19:24:24.227 INFO:tasks.workunit.client.1.smithi163.stdout:0/502: mknod d11/d4f/cb3 0 2024-02-16T19:24:24.227 INFO:tasks.workunit.client.1.smithi163.stdout:0/503: truncate d11/d21/d50/d58/d6b/f71 615582 0 2024-02-16T19:24:24.227 INFO:tasks.workunit.client.1.smithi163.stdout:0/504: readlink d11/d4f/l55 0 2024-02-16T19:24:24.227 INFO:tasks.workunit.client.1.smithi163.stdout:0/505: write d11/d21/d7c/f81 [255999,121255] 0 2024-02-16T19:24:24.227 INFO:tasks.workunit.client.1.smithi163.stdout:5/285: dwrite d0/f32 [0,4194304] 0 2024-02-16T19:24:24.229 INFO:tasks.workunit.client.0.smithi040.stdout:4/688: mknod dd/de/d71/dc2/cd5 0 2024-02-16T19:24:24.231 INFO:tasks.workunit.client.1.smithi163.stdout:1/277: symlink d7/d20/l5f 0 2024-02-16T19:24:24.231 INFO:tasks.workunit.client.1.smithi163.stdout:3/301: write d0/dc/d1d/f5a [4132307,49580] 0 2024-02-16T19:24:24.238 INFO:tasks.workunit.client.0.smithi040.stdout:7/767: creat d21/d4d/d77/d7a/dc6/f101 x:0 0 0 2024-02-16T19:24:24.239 INFO:tasks.workunit.client.0.smithi040.stdout:7/768: fdatasync d21/d8b/d9c/ff7 0 2024-02-16T19:24:24.239 INFO:tasks.workunit.client.0.smithi040.stdout:7/769: chown d21/d7e/lca 125789975 1 2024-02-16T19:24:24.239 INFO:tasks.workunit.client.0.smithi040.stdout:7/770: creat d21/d8b/d9c/f102 x:0 0 0 2024-02-16T19:24:24.239 INFO:tasks.workunit.client.1.smithi163.stdout:6/260: mknod d0/d6/df/c4a 0 2024-02-16T19:24:24.241 INFO:tasks.workunit.client.1.smithi163.stdout:2/448: creat d2/d2a/d39/d66/f9a x:0 0 0 2024-02-16T19:24:24.241 INFO:tasks.workunit.client.1.smithi163.stdout:2/449: dread - d2/d2a/d39/d66/d6c/f79 zero size 2024-02-16T19:24:24.241 INFO:tasks.workunit.client.1.smithi163.stdout:2/450: readlink d2/d2a/d2d/d73/d84/l8b 0 2024-02-16T19:24:24.241 INFO:tasks.workunit.client.1.smithi163.stdout:2/451: truncate d2/f19 1927781 0 2024-02-16T19:24:24.243 INFO:tasks.workunit.client.1.smithi163.stdout:3/302: write d0/dc/f24 [3278830,126931] 0 2024-02-16T19:24:24.243 INFO:tasks.workunit.client.0.smithi040.stdout:8/986: link d6/d12/d51/d6e/db9/d123/c8a d6/d12/d93/dd8/c12c 0 2024-02-16T19:24:24.244 INFO:tasks.workunit.client.0.smithi040.stdout:8/987: creat d6/d12/d51/d6e/db7/f12d x:0 0 0 2024-02-16T19:24:24.248 INFO:tasks.workunit.client.0.smithi040.stdout:3/934: creat d1/d4/d6e/f128 x:0 0 0 2024-02-16T19:24:24.248 INFO:tasks.workunit.client.0.smithi040.stdout:3/935: truncate d1/d7/d33/fdb 450135 0 2024-02-16T19:24:24.251 INFO:tasks.workunit.client.0.smithi040.stdout:5/837: symlink d1/da/d7e/l102 0 2024-02-16T19:24:24.258 INFO:tasks.workunit.client.0.smithi040.stdout:3/936: dread d1/dc/d35/f109 [0,4194304] 0 2024-02-16T19:24:24.259 INFO:tasks.workunit.client.0.smithi040.stdout:4/689: mkdir dd/d58/d5e/dd1/d9d/dd6 0 2024-02-16T19:24:24.259 INFO:tasks.workunit.client.0.smithi040.stdout:4/690: chown dd/de/f5d 6 1 2024-02-16T19:24:24.261 INFO:tasks.workunit.client.1.smithi163.stdout:8/271: creat d0/d7/d19/d1f/d30/d48/f65 x:0 0 0 2024-02-16T19:24:24.266 INFO:tasks.workunit.client.0.smithi040.stdout:8/988: creat d6/d64/d50/dd5/d102/f12e x:0 0 0 2024-02-16T19:24:24.266 INFO:tasks.workunit.client.0.smithi040.stdout:8/989: dread - d6/d12/d51/d6e/db7/dc6/fde zero size 2024-02-16T19:24:24.267 INFO:tasks.workunit.client.0.smithi040.stdout:7/771: dwrite d21/d7e/fe9 [0,4194304] 0 2024-02-16T19:24:24.267 INFO:tasks.workunit.client.0.smithi040.stdout:7/772: write d21/d4d/d77/d7a/fb1 [312259,68799] 0 2024-02-16T19:24:24.267 INFO:tasks.workunit.client.0.smithi040.stdout:7/773: stat d21/d4d/d77 0 2024-02-16T19:24:24.269 INFO:tasks.workunit.client.0.smithi040.stdout:3/937: symlink d1/dc/l129 0 2024-02-16T19:24:24.274 INFO:tasks.workunit.client.1.smithi163.stdout:8/272: read d0/d24/f60 [2761882,16229] 0 2024-02-16T19:24:24.274 INFO:tasks.workunit.client.1.smithi163.stdout:8/273: truncate d0/d24/d2c/f3b 1095504 0 2024-02-16T19:24:24.274 INFO:tasks.workunit.client.1.smithi163.stdout:6/261: dwrite d0/d6/f31 [0,4194304] 0 2024-02-16T19:24:24.274 INFO:tasks.workunit.client.1.smithi163.stdout:6/262: write d0/d6/d9/d15/d1c/f2b [5181356,14341] 0 2024-02-16T19:24:24.274 INFO:tasks.workunit.client.1.smithi163.stdout:6/263: readlink d0/l14 0 2024-02-16T19:24:24.275 INFO:tasks.workunit.client.0.smithi040.stdout:3/938: mknod d1/dc/d42/d4a/d57/c12a 0 2024-02-16T19:24:24.275 INFO:tasks.workunit.client.0.smithi040.stdout:4/691: dread dd/de/d10/d36/d9e/d32/f95 [0,4194304] 0 2024-02-16T19:24:24.278 INFO:tasks.workunit.client.1.smithi163.stdout:4/218: creat d11/d18/f43 x:0 0 0 2024-02-16T19:24:24.278 INFO:tasks.workunit.client.0.smithi040.stdout:2/801: fsync d5/d7/d1e/d55/f6f 0 2024-02-16T19:24:24.279 INFO:tasks.workunit.client.0.smithi040.stdout:3/939: truncate d1/d4/d6e/d8a/da5/fd0 2102220 0 2024-02-16T19:24:24.281 INFO:tasks.workunit.client.1.smithi163.stdout:7/288: mkdir d11/d24/d25/d4b/d66 0 2024-02-16T19:24:24.282 INFO:tasks.workunit.client.0.smithi040.stdout:4/692: creat dd/de/d10/d36/fd7 x:0 0 0 2024-02-16T19:24:24.285 INFO:tasks.workunit.client.0.smithi040.stdout:9/859: sync 2024-02-16T19:24:24.285 INFO:tasks.workunit.client.0.smithi040.stdout:1/823: sync 2024-02-16T19:24:24.286 INFO:tasks.workunit.client.0.smithi040.stdout:1/824: write d17/d26/d5f/d78/d7e/ddb/ff5 [887797,6789] 0 2024-02-16T19:24:24.286 INFO:tasks.workunit.client.0.smithi040.stdout:2/802: link d5/d7/d1e/d31/f34 d5/d7/d1e/d4c/d52/d81/f106 0 2024-02-16T19:24:24.288 INFO:tasks.workunit.client.0.smithi040.stdout:3/940: dread d1/f3 [0,4194304] 0 2024-02-16T19:24:24.288 INFO:tasks.workunit.client.1.smithi163.stdout:3/303: dwrite d0/dc/d50/f58 [0,4194304] 0 2024-02-16T19:24:24.288 INFO:tasks.workunit.client.0.smithi040.stdout:4/693: rmdir dd/de/d10/d36/d9e/d32/d7a 39 2024-02-16T19:24:24.292 INFO:tasks.workunit.client.1.smithi163.stdout:5/286: mkdir d0/d2/d55 0 2024-02-16T19:24:24.295 INFO:tasks.workunit.client.0.smithi040.stdout:1/825: mkdir d17/d3c/d74/d97/da9/d10b 0 2024-02-16T19:24:24.295 INFO:tasks.workunit.client.0.smithi040.stdout:1/826: creat d17/d29/d8b/f10c x:0 0 0 2024-02-16T19:24:24.295 INFO:tasks.workunit.client.0.smithi040.stdout:5/838: dwrite d1/da/d1c/da7/fee [0,4194304] 0 2024-02-16T19:24:24.298 INFO:tasks.workunit.client.1.smithi163.stdout:3/304: dread d0/dc/f24 [0,4194304] 0 2024-02-16T19:24:24.298 INFO:tasks.workunit.client.1.smithi163.stdout:3/305: chown d0/dc/d1d/d49/f56 470 1 2024-02-16T19:24:24.298 INFO:tasks.workunit.client.1.smithi163.stdout:3/306: fdatasync d0/f6d 0 2024-02-16T19:24:24.299 INFO:tasks.workunit.client.0.smithi040.stdout:2/803: rename d5/l6e to d5/d5c/d65/d9d/l107 0 2024-02-16T19:24:24.299 INFO:tasks.workunit.client.0.smithi040.stdout:7/774: dwrite d21/d4d/d77/d87/d9b/f98 [0,4194304] 0 2024-02-16T19:24:24.300 INFO:tasks.workunit.client.0.smithi040.stdout:2/804: dread d5/d7/d1e/d4c/d52/f96 [0,4194304] 0 2024-02-16T19:24:24.300 INFO:tasks.workunit.client.0.smithi040.stdout:2/805: truncate d5/d7/d10/dbc/fe3 4551593 0 2024-02-16T19:24:24.300 INFO:tasks.workunit.client.0.smithi040.stdout:2/806: creat d5/d7/d1e/d31/d4f/d63/d8b/f108 x:0 0 0 2024-02-16T19:24:24.301 INFO:tasks.workunit.client.0.smithi040.stdout:9/860: dread d3/d4/d55/d70/dcc/ffa [0,4194304] 0 2024-02-16T19:24:24.301 INFO:tasks.workunit.client.0.smithi040.stdout:9/861: chown d3/d4/d11/d71/da8/fd2 2980647 1 2024-02-16T19:24:24.301 INFO:tasks.workunit.client.1.smithi163.stdout:1/278: rename d7/c52 to d7/d20/d21/c60 0 2024-02-16T19:24:24.305 INFO:tasks.workunit.client.0.smithi040.stdout:1/827: rmdir d17/d3c/d6b 39 2024-02-16T19:24:24.305 INFO:tasks.workunit.client.0.smithi040.stdout:8/990: dwrite d6/d12/d51/d6e/db7/ddf/df8/d15/f67 [0,4194304] 0 2024-02-16T19:24:24.305 INFO:tasks.workunit.client.0.smithi040.stdout:8/991: chown d6/d2b/da3/dcb 114 1 2024-02-16T19:24:24.305 INFO:tasks.workunit.client.0.smithi040.stdout:3/941: dread d1/d4/f25 [0,4194304] 0 2024-02-16T19:24:24.306 INFO:tasks.workunit.client.1.smithi163.stdout:2/452: chown d2/d2a/d39/f57 52742999 1 2024-02-16T19:24:24.311 INFO:tasks.workunit.client.0.smithi040.stdout:0/997: sync 2024-02-16T19:24:24.316 INFO:tasks.workunit.client.1.smithi163.stdout:8/274: unlink d0/f20 0 2024-02-16T19:24:24.316 INFO:tasks.workunit.client.1.smithi163.stdout:8/275: truncate d0/d24/d27/f64 996546 0 2024-02-16T19:24:24.318 INFO:tasks.workunit.client.0.smithi040.stdout:7/775: creat d21/d4d/d77/d87/dc9/dcb/f103 x:0 0 0 2024-02-16T19:24:24.324 INFO:tasks.workunit.client.1.smithi163.stdout:6/264: creat d0/d6/df/d20/d40/d17/f4b x:0 0 0 2024-02-16T19:24:24.325 INFO:tasks.workunit.client.0.smithi040.stdout:9/862: mknod d3/c111 0 2024-02-16T19:24:24.325 INFO:tasks.workunit.client.0.smithi040.stdout:9/863: chown d3/d1c/d73/f83 870 1 2024-02-16T19:24:24.326 INFO:tasks.workunit.client.1.smithi163.stdout:4/219: symlink d11/d3d/l44 0 2024-02-16T19:24:24.328 INFO:tasks.workunit.client.1.smithi163.stdout:7/289: unlink d11/f43 0 2024-02-16T19:24:24.332 INFO:tasks.workunit.client.0.smithi040.stdout:8/992: truncate d6/d12/d51/d6e/db7/dc6/fdd 2291502 0 2024-02-16T19:24:24.333 INFO:tasks.workunit.client.0.smithi040.stdout:4/694: dwrite dd/f6e [0,4194304] 0 2024-02-16T19:24:24.333 INFO:tasks.workunit.client.0.smithi040.stdout:3/942: dwrite d1/dc/fd [0,4194304] 0 2024-02-16T19:24:24.334 INFO:tasks.workunit.client.1.smithi163.stdout:3/307: dwrite d0/dc/f28 [0,4194304] 0 2024-02-16T19:24:24.334 INFO:tasks.workunit.client.1.smithi163.stdout:3/308: stat d0/dc/d1c/d32/f57 0 2024-02-16T19:24:24.335 INFO:tasks.workunit.client.1.smithi163.stdout:3/309: fsync d0/dc/f28 0 2024-02-16T19:24:24.335 INFO:tasks.workunit.client.0.smithi040.stdout:8/993: dread d6/d12/d51/d6e/db7/ddf/df8/d15/f1c [4194304,4194304] 0 2024-02-16T19:24:24.335 INFO:tasks.workunit.client.0.smithi040.stdout:5/839: dwrite d1/da/dd5/ffe [0,4194304] 0 2024-02-16T19:24:24.335 INFO:tasks.workunit.client.0.smithi040.stdout:0/998: creat db/f157 x:0 0 0 2024-02-16T19:24:24.338 INFO:tasks.workunit.client.0.smithi040.stdout:7/776: rename d21/d4d/ca2 to d21/d4d/d77/d87/dc9/ddd/dbe/c104 0 2024-02-16T19:24:24.339 INFO:tasks.workunit.client.0.smithi040.stdout:7/777: creat d21/d7e/f105 x:0 0 0 2024-02-16T19:24:24.340 INFO:tasks.workunit.client.0.smithi040.stdout:2/807: getdents d5/d7/d1e/d4c/d52/daa 0 2024-02-16T19:24:24.343 INFO:tasks.workunit.client.1.smithi163.stdout:1/279: truncate d7/d20/f24 1780659 0 2024-02-16T19:24:24.345 INFO:tasks.workunit.client.0.smithi040.stdout:4/695: getdents dd/de/d10/d36/d9e/d32 0 2024-02-16T19:24:24.345 INFO:tasks.workunit.client.0.smithi040.stdout:4/696: readlink dd/l78 0 2024-02-16T19:24:24.345 INFO:tasks.workunit.client.0.smithi040.stdout:4/697: write dd/de/f5d [1585571,77564] 0 2024-02-16T19:24:24.345 INFO:tasks.workunit.client.0.smithi040.stdout:4/698: dread - dd/de/d45/fc4 zero size 2024-02-16T19:24:24.346 INFO:tasks.workunit.client.0.smithi040.stdout:4/699: truncate dd/de/d45/f5c 466583 0 2024-02-16T19:24:24.347 INFO:tasks.workunit.client.1.smithi163.stdout:2/453: creat d2/d26/d54/f9b x:0 0 0 2024-02-16T19:24:24.347 INFO:tasks.workunit.client.1.smithi163.stdout:2/454: fdatasync d2/d26/d54/d58/f69 0 2024-02-16T19:24:24.347 INFO:tasks.workunit.client.0.smithi040.stdout:5/840: write d1/da/d1c/dba/db1/f7b [2411415,57777] 0 2024-02-16T19:24:24.351 INFO:tasks.workunit.client.1.smithi163.stdout:8/276: mknod d0/d7/d19/d1f/d32/c66 0 2024-02-16T19:24:24.351 INFO:tasks.workunit.client.1.smithi163.stdout:8/277: dread - d0/d24/d27/d45/f5a zero size 2024-02-16T19:24:24.351 INFO:tasks.workunit.client.1.smithi163.stdout:8/278: readlink d0/d24/d3c/d3d/l1d 0 2024-02-16T19:24:24.352 INFO:tasks.workunit.client.1.smithi163.stdout:6/265: creat d0/d6/d9/f4c x:0 0 0 2024-02-16T19:24:24.354 INFO:tasks.workunit.client.0.smithi040.stdout:1/828: dwrite d17/d2d/f2f [8388608,4194304] 0 2024-02-16T19:24:24.354 INFO:tasks.workunit.client.0.smithi040.stdout:1/829: stat d17/d26/d5f/fd4 0 2024-02-16T19:24:24.354 INFO:tasks.workunit.client.0.smithi040.stdout:1/830: write d17/d26/d9a/dfb/ffd [1231773,127114] 0 2024-02-16T19:24:24.354 INFO:tasks.workunit.client.0.smithi040.stdout:3/943: dwrite d1/d8c/d8e/dde/f111 [0,4194304] 0 2024-02-16T19:24:24.355 INFO:tasks.workunit.client.0.smithi040.stdout:8/994: creat d6/d64/f12f x:0 0 0 2024-02-16T19:24:24.358 INFO:tasks.workunit.client.1.smithi163.stdout:4/220: rename d11/l1a to d11/d18/d1f/d29/l45 0 2024-02-16T19:24:24.358 INFO:tasks.workunit.client.1.smithi163.stdout:4/221: creat d11/f46 x:0 0 0 2024-02-16T19:24:24.358 INFO:tasks.workunit.client.1.smithi163.stdout:4/222: readlink d11/d18/d1f/d29/l3c 0 2024-02-16T19:24:24.360 INFO:tasks.workunit.client.0.smithi040.stdout:0/999: truncate db/d2f/d1d/d21/d2e/d83/fe4 1576364 0 2024-02-16T19:24:24.363 INFO:tasks.workunit.client.1.smithi163.stdout:7/290: mkdir d11/d26/d36/d67 0 2024-02-16T19:24:24.366 INFO:tasks.workunit.client.0.smithi040.stdout:9/864: rename d3/d4/d55 to d3/d4/d11/d29/d38/d46/d6b/d85/d112 0 2024-02-16T19:24:24.366 INFO:tasks.workunit.client.0.smithi040.stdout:9/865: fdatasync d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/f3b 0 2024-02-16T19:24:24.370 INFO:tasks.workunit.client.0.smithi040.stdout:7/778: link d21/d29/d79/f7f d21/d4d/d77/d87/dc9/dcb/f106 0 2024-02-16T19:24:24.370 INFO:tasks.workunit.client.0.smithi040.stdout:7/779: dread - d21/f9f zero size 2024-02-16T19:24:24.370 INFO:tasks.workunit.client.0.smithi040.stdout:7/780: truncate f16 4940712 0 2024-02-16T19:24:24.370 INFO:tasks.workunit.client.1.smithi163.stdout:5/287: getdents d0/d22/d3b 0 2024-02-16T19:24:24.372 INFO:tasks.workunit.client.0.smithi040.stdout:5/841: dwrite d1/d45/ff4 [0,4194304] 0 2024-02-16T19:24:24.372 INFO:tasks.workunit.client.0.smithi040.stdout:5/842: chown d1/d45/dab 11882 1 2024-02-16T19:24:24.372 INFO:tasks.workunit.client.0.smithi040.stdout:5/843: stat d1/da/d1c/dba/dd6/df7 0 2024-02-16T19:24:24.372 INFO:tasks.workunit.client.0.smithi040.stdout:2/808: creat d5/d7/d1e/d55/da3/f109 x:0 0 0 2024-02-16T19:24:24.372 INFO:tasks.workunit.client.1.smithi163.stdout:1/280: mknod d7/dc/d2d/c61 0 2024-02-16T19:24:24.373 INFO:tasks.workunit.client.0.smithi040.stdout:1/831: symlink d17/db5/l10d 0 2024-02-16T19:24:24.374 INFO:tasks.workunit.client.1.smithi163.stdout:2/455: rmdir d2/d2a/d39/d66/d6c/d7f 39 2024-02-16T19:24:24.376 INFO:tasks.workunit.client.1.smithi163.stdout:6/266: truncate d0/d6/f8 3159581 0 2024-02-16T19:24:24.378 INFO:tasks.workunit.client.1.smithi163.stdout:3/310: rename d0/dc/d1c/d21/c4f to d0/dc/d1d/d49/c70 0 2024-02-16T19:24:24.380 INFO:tasks.workunit.client.0.smithi040.stdout:3/944: symlink d1/dc/d35/d53/d64/l12b 0 2024-02-16T19:24:24.380 INFO:tasks.workunit.client.1.smithi163.stdout:8/279: dwrite d0/d7/d19/d1f/d30/f47 [0,4194304] 0 2024-02-16T19:24:24.381 INFO:tasks.workunit.client.0.smithi040.stdout:8/995: dwrite d6/d2b/ffb [0,4194304] 0 2024-02-16T19:24:24.383 INFO:tasks.workunit.client.1.smithi163.stdout:4/223: fdatasync d11/d18/f24 0 2024-02-16T19:24:24.383 INFO:tasks.workunit.client.1.smithi163.stdout:4/224: chown d11/d18/d1f/l37 1013690 1 2024-02-16T19:24:24.384 INFO:tasks.workunit.client.0.smithi040.stdout:5/844: write d1/d45/dab/fc4 [88109,76703] 0 2024-02-16T19:24:24.384 INFO:tasks.workunit.client.0.smithi040.stdout:5/845: chown d1/da/d1c/d73/c9e 450427212 1 2024-02-16T19:24:24.385 INFO:tasks.workunit.client.0.smithi040.stdout:3/945: dread d1/d4/dc4/fe0 [0,4194304] 0 2024-02-16T19:24:24.385 INFO:tasks.workunit.client.0.smithi040.stdout:3/946: stat d1/dc/d42/d5c/l9d 0 2024-02-16T19:24:24.385 INFO:tasks.workunit.client.1.smithi163.stdout:7/291: symlink d11/d24/d49/l68 0 2024-02-16T19:24:24.385 INFO:tasks.workunit.client.1.smithi163.stdout:7/292: creat d11/d24/d49/f69 x:0 0 0 2024-02-16T19:24:24.385 INFO:tasks.workunit.client.1.smithi163.stdout:5/288: dwrite d0/d2/d34/d44/f51 [0,4194304] 0 2024-02-16T19:24:24.386 INFO:tasks.workunit.client.0.smithi040.stdout:7/781: truncate d21/d29/f37 4602134 0 2024-02-16T19:24:24.387 INFO:tasks.workunit.client.1.smithi163.stdout:4/225: dread d11/d18/d1f/f38 [0,4194304] 0 2024-02-16T19:24:24.387 INFO:tasks.workunit.client.1.smithi163.stdout:2/456: truncate d2/d2a/d39/f42 978787 0 2024-02-16T19:24:24.391 INFO:tasks.workunit.client.1.smithi163.stdout:6/267: creat d0/d6/d9/d15/f4d x:0 0 0 2024-02-16T19:24:24.393 INFO:tasks.workunit.client.1.smithi163.stdout:1/281: rename d7/d20/d21/c25 to d7/dc/d2d/c62 0 2024-02-16T19:24:24.394 INFO:tasks.workunit.client.1.smithi163.stdout:9/456: sync 2024-02-16T19:24:24.395 INFO:tasks.workunit.client.0.smithi040.stdout:2/809: mknod d5/d7/d1e/c10a 0 2024-02-16T19:24:24.395 INFO:tasks.workunit.client.0.smithi040.stdout:2/810: fdatasync d5/d7/d1e/d31/f43 0 2024-02-16T19:24:24.396 INFO:tasks.workunit.client.1.smithi163.stdout:7/293: write d11/f1b [1974334,54688] 0 2024-02-16T19:24:24.396 INFO:tasks.workunit.client.1.smithi163.stdout:7/294: dread - d11/d30/f5b zero size 2024-02-16T19:24:24.396 INFO:tasks.workunit.client.1.smithi163.stdout:6/268: dread d0/d6/df/f2a [0,4194304] 0 2024-02-16T19:24:24.396 INFO:tasks.workunit.client.1.smithi163.stdout:3/311: mkdir d0/dc/d1d/d29/d5d/d71 0 2024-02-16T19:24:24.397 INFO:tasks.workunit.client.0.smithi040.stdout:3/947: dwrite d1/d8c/d8e/dde/f112 [0,4194304] 0 2024-02-16T19:24:24.398 INFO:tasks.workunit.client.0.smithi040.stdout:1/832: rmdir d17/d26/d9a 39 2024-02-16T19:24:24.400 INFO:tasks.workunit.client.1.smithi163.stdout:8/280: unlink d0/d24/d27/f5b 0 2024-02-16T19:24:24.401 INFO:tasks.workunit.client.1.smithi163.stdout:8/281: fdatasync d0/d24/d27/d45/f4a 0 2024-02-16T19:24:24.401 INFO:tasks.workunit.client.1.smithi163.stdout:0/506: sync 2024-02-16T19:24:24.409 INFO:tasks.workunit.client.1.smithi163.stdout:5/289: creat d0/d22/d3b/f56 x:0 0 0 2024-02-16T19:24:24.411 INFO:tasks.workunit.client.0.smithi040.stdout:3/948: dread d1/dc/d42/d4a/f110 [0,4194304] 0 2024-02-16T19:24:24.411 INFO:tasks.workunit.client.0.smithi040.stdout:5/846: unlink d1/d62/d9a/f9b 0 2024-02-16T19:24:24.411 INFO:tasks.workunit.client.0.smithi040.stdout:5/847: creat d1/da/d51/f103 x:0 0 0 2024-02-16T19:24:24.411 INFO:tasks.workunit.client.0.smithi040.stdout:5/848: readlink d1/da/d1c/l9f 0 2024-02-16T19:24:24.412 INFO:tasks.workunit.client.1.smithi163.stdout:0/507: write d11/d12/d20/f5c [2228920,33448] 0 2024-02-16T19:24:24.412 INFO:tasks.workunit.client.1.smithi163.stdout:0/508: creat d11/fb4 x:0 0 0 2024-02-16T19:24:24.412 INFO:tasks.workunit.client.0.smithi040.stdout:3/949: dread d1/dc/d35/d53/fa9 [0,4194304] 0 2024-02-16T19:24:24.413 INFO:tasks.workunit.client.1.smithi163.stdout:4/226: mknod d11/c47 0 2024-02-16T19:24:24.413 INFO:tasks.workunit.client.1.smithi163.stdout:4/227: dread - d11/f14 zero size 2024-02-16T19:24:24.414 INFO:tasks.workunit.client.1.smithi163.stdout:2/457: mkdir d2/d2a/d2d/d3c/d7c/d9c 0 2024-02-16T19:24:24.416 INFO:tasks.workunit.client.0.smithi040.stdout:5/849: dread d1/da/d51/f85 [0,4194304] 0 2024-02-16T19:24:24.417 INFO:tasks.workunit.client.0.smithi040.stdout:2/811: unlink d5/d7/d10/df3/d42/f50 0 2024-02-16T19:24:24.417 INFO:tasks.workunit.client.0.smithi040.stdout:2/812: getdents d5/d7/d1e/d55/d102 0 2024-02-16T19:24:24.418 INFO:tasks.workunit.client.1.smithi163.stdout:1/282: mkdir d7/d5b/d63 0 2024-02-16T19:24:24.420 INFO:tasks.workunit.client.1.smithi163.stdout:9/457: symlink d5/de/d30/d3d/d93/la4 0 2024-02-16T19:24:24.422 INFO:tasks.workunit.client.1.smithi163.stdout:7/295: rename d11/f12 to d11/d30/f6a 0 2024-02-16T19:24:24.422 INFO:tasks.workunit.client.1.smithi163.stdout:7/296: dread - d11/d30/f5b zero size 2024-02-16T19:24:24.424 INFO:tasks.workunit.client.0.smithi040.stdout:1/833: mkdir d17/d29/d8b/d10e 0 2024-02-16T19:24:24.424 INFO:tasks.workunit.client.0.smithi040.stdout:1/834: fdatasync d17/d26/d5f/f84 0 2024-02-16T19:24:24.424 INFO:tasks.workunit.client.0.smithi040.stdout:5/850: dread d1/da/d1c/d21/f64 [0,4194304] 0 2024-02-16T19:24:24.425 INFO:tasks.workunit.client.1.smithi163.stdout:4/228: dread d11/d18/d1f/f27 [4194304,4194304] 0 2024-02-16T19:24:24.425 INFO:tasks.workunit.client.1.smithi163.stdout:4/229: truncate d11/d18/d1f/d34/f39 79943 0 2024-02-16T19:24:24.425 INFO:tasks.workunit.client.1.smithi163.stdout:6/269: mknod d0/d6/c4e 0 2024-02-16T19:24:24.426 INFO:tasks.workunit.client.0.smithi040.stdout:8/996: truncate d6/d12/d93/fa9 241498 0 2024-02-16T19:24:24.427 INFO:tasks.workunit.client.0.smithi040.stdout:8/997: readlink d6/d2b/ld2 0 2024-02-16T19:24:24.427 INFO:tasks.workunit.client.1.smithi163.stdout:5/290: dwrite d0/d22/d3b/f4c [0,4194304] 0 2024-02-16T19:24:24.427 INFO:tasks.workunit.client.1.smithi163.stdout:8/282: dwrite d0/d7/d19/d1f/d30/d48/f65 [0,4194304] 0 2024-02-16T19:24:24.429 INFO:tasks.workunit.client.1.smithi163.stdout:3/312: creat d0/dc/d1d/d29/f72 x:0 0 0 2024-02-16T19:24:24.429 INFO:tasks.workunit.client.1.smithi163.stdout:3/313: stat d0/dc/d1c/d32/f36 0 2024-02-16T19:24:24.429 INFO:tasks.workunit.client.1.smithi163.stdout:3/314: write d0/f39 [1076806,75827] 0 2024-02-16T19:24:24.431 INFO:tasks.workunit.client.0.smithi040.stdout:5/851: dread d1/da/fa1 [0,4194304] 0 2024-02-16T19:24:24.431 INFO:tasks.workunit.client.0.smithi040.stdout:4/700: rename dd/de/d71/dc2 to dd/de/daa/dd8 0 2024-02-16T19:24:24.434 INFO:tasks.workunit.client.1.smithi163.stdout:4/230: dread d11/d18/d1f/d29/f31 [0,4194304] 0 2024-02-16T19:24:24.436 INFO:tasks.workunit.client.1.smithi163.stdout:2/458: creat d2/f9d x:0 0 0 2024-02-16T19:24:24.437 INFO:tasks.workunit.client.0.smithi040.stdout:3/950: creat d1/dc/d35/d53/d64/f12c x:0 0 0 2024-02-16T19:24:24.437 INFO:tasks.workunit.client.0.smithi040.stdout:3/951: stat d1/d4/d6e/d8a/da5 0 2024-02-16T19:24:24.438 INFO:tasks.workunit.client.1.smithi163.stdout:1/283: creat d7/dc/d2d/d32/f64 x:0 0 0 2024-02-16T19:24:24.439 INFO:tasks.workunit.client.0.smithi040.stdout:2/813: rmdir d5/d7/d1e/d31/d4f/d63 39 2024-02-16T19:24:24.440 INFO:tasks.workunit.client.0.smithi040.stdout:2/814: creat d5/d7/f10b x:0 0 0 2024-02-16T19:24:24.441 INFO:tasks.workunit.client.1.smithi163.stdout:9/458: symlink d5/de/d30/d3d/d49/la5 0 2024-02-16T19:24:24.443 INFO:tasks.workunit.client.0.smithi040.stdout:1/835: creat d17/d3c/d6b/de8/f10f x:0 0 0 2024-02-16T19:24:24.443 INFO:tasks.workunit.client.0.smithi040.stdout:1/836: creat d17/d3c/d6b/de8/f110 x:0 0 0 2024-02-16T19:24:24.445 INFO:tasks.workunit.client.1.smithi163.stdout:0/509: rename d11/d21/d50/d58/d6b/f71 to d11/d12/d20/fb5 0 2024-02-16T19:24:24.447 INFO:tasks.workunit.client.0.smithi040.stdout:8/998: unlink d6/d64/d50/dd5/ff3 0 2024-02-16T19:24:24.448 INFO:tasks.workunit.client.1.smithi163.stdout:7/297: mknod d11/d24/d25/d4b/d66/c6b 0 2024-02-16T19:24:24.450 INFO:tasks.workunit.client.0.smithi040.stdout:5/852: link d1/da/d1c/f94 d1/da/d1c/d73/f104 0 2024-02-16T19:24:24.454 INFO:tasks.workunit.client.0.smithi040.stdout:9/866: rename d3/d4/d11/d71/fed to d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/f113 0 2024-02-16T19:24:24.455 INFO:tasks.workunit.client.0.smithi040.stdout:8/999: dread d6/d12/f5a [0,4194304] 0 2024-02-16T19:24:24.456 INFO:tasks.workunit.client.1.smithi163.stdout:5/291: link d0/d2/f30 d0/d2/f57 0 2024-02-16T19:24:24.457 INFO:tasks.workunit.client.0.smithi040.stdout:4/701: mknod dd/de/cd9 0 2024-02-16T19:24:24.457 INFO:tasks.workunit.client.0.smithi040.stdout:9/867: dread d3/d4/dbe/de9/f107 [0,4194304] 0 2024-02-16T19:24:24.459 INFO:tasks.workunit.client.0.smithi040.stdout:3/952: mknod d1/dc/d35/d48/c12d 0 2024-02-16T19:24:24.462 INFO:tasks.workunit.client.1.smithi163.stdout:8/283: truncate d0/d7/d19/d1f/d32/f34 1985028 0 2024-02-16T19:24:24.462 INFO:tasks.workunit.client.1.smithi163.stdout:8/284: chown d0/l1c 4015601 1 2024-02-16T19:24:24.464 INFO:tasks.workunit.client.0.smithi040.stdout:2/815: link d5/d7/c48 d5/d7/d1e/d55/dd4/c10c 0 2024-02-16T19:24:24.465 INFO:tasks.workunit.client.0.smithi040.stdout:2/816: readlink d5/d7/d89/le5 0 2024-02-16T19:24:24.465 INFO:tasks.workunit.client.0.smithi040.stdout:2/817: write d5/d7/d1e/d55/da3/f109 [892164,97929] 0 2024-02-16T19:24:24.465 INFO:tasks.workunit.client.0.smithi040.stdout:2/818: chown d5 49 1 2024-02-16T19:24:24.467 INFO:tasks.workunit.client.1.smithi163.stdout:8/285: dread d0/dc/f2d [4194304,4194304] 0 2024-02-16T19:24:24.468 INFO:tasks.workunit.client.0.smithi040.stdout:1/837: rmdir d17/d26/d9a/dfb/d92/dca 39 2024-02-16T19:24:24.468 INFO:tasks.workunit.client.0.smithi040.stdout:1/838: chown d17/d26/d5f/d78/dac 4432 1 2024-02-16T19:24:24.468 INFO:tasks.workunit.client.0.smithi040.stdout:1/839: write d17/d26/d5f/fe1 [778387,59000] 0 2024-02-16T19:24:24.471 INFO:tasks.workunit.client.0.smithi040.stdout:5/853: creat d1/da/d1c/f105 x:0 0 0 2024-02-16T19:24:24.471 INFO:tasks.workunit.client.0.smithi040.stdout:5/854: chown d1/da/fc8 1001 1 2024-02-16T19:24:24.471 INFO:tasks.workunit.client.0.smithi040.stdout:5/855: fdatasync d1/da/d51/f103 0 2024-02-16T19:24:24.473 INFO:tasks.workunit.client.0.smithi040.stdout:7/782: rename d21/d4d/d77/d87/dc9/ddd/lb2 to d21/d29/d79/dbc/l107 0 2024-02-16T19:24:24.475 INFO:tasks.workunit.client.1.smithi163.stdout:1/284: write d7/f49 [826209,76914] 0 2024-02-16T19:24:24.476 INFO:tasks.workunit.client.0.smithi040.stdout:9/868: rmdir d3/d4/d11/d29/d38/d46/d6b/d85/d112/d8a/d99 39 2024-02-16T19:24:24.478 INFO:tasks.workunit.client.1.smithi163.stdout:9/459: link d5/de/d30/d3d/l41 d5/de/d30/d3d/d93/la6 0 2024-02-16T19:24:24.478 INFO:tasks.workunit.client.0.smithi040.stdout:4/702: rmdir dd/de/daa/dd8 39 2024-02-16T19:24:24.478 INFO:tasks.workunit.client.0.smithi040.stdout:4/703: creat dd/de/d10/d36/fda x:0 0 0 2024-02-16T19:24:24.479 INFO:tasks.workunit.client.0.smithi040.stdout:3/953: creat d1/d4/d6e/dc2/f12e x:0 0 0 2024-02-16T19:24:24.482 INFO:tasks.workunit.client.1.smithi163.stdout:3/315: rename d0/dc/d1c/d21/c34 to d0/dc/d1d/d29/d5d/d71/c73 0 2024-02-16T19:24:24.485 INFO:tasks.workunit.client.0.smithi040.stdout:2/819: link d5/d7/d1e/d31/fb8 d5/d7/d1e/d31/dc3/f10d 0 2024-02-16T19:24:24.486 INFO:tasks.workunit.client.1.smithi163.stdout:0/510: creat d11/d22/d7f/fb6 x:0 0 0 2024-02-16T19:24:24.486 INFO:tasks.workunit.client.1.smithi163.stdout:0/511: dread - d11/d22/fa7 zero size 2024-02-16T19:24:24.487 INFO:tasks.workunit.client.0.smithi040.stdout:9/869: write d3/d4/d11/d29/d38/d46/f5b [673777,114674] 0 2024-02-16T19:24:24.487 INFO:tasks.workunit.client.0.smithi040.stdout:1/840: chown d17/d2d/fad 165013 1 2024-02-16T19:24:24.487 INFO:tasks.workunit.client.0.smithi040.stdout:1/841: fsync d17/d26/f53 0 2024-02-16T19:24:24.488 INFO:tasks.workunit.client.1.smithi163.stdout:7/298: mknod d11/d26/d36/c6c 0 2024-02-16T19:24:24.489 INFO:tasks.workunit.client.0.smithi040.stdout:4/704: write dd/de/d10/f4a [3967224,113840] 0 2024-02-16T19:24:24.490 INFO:tasks.workunit.client.0.smithi040.stdout:5/856: link d1/da/d1c/d21/c3b d1/da/db0/c106 0 2024-02-16T19:24:24.490 INFO:tasks.workunit.client.0.smithi040.stdout:5/857: creat d1/da/d51/f107 x:0 0 0 2024-02-16T19:24:24.492 INFO:tasks.workunit.client.1.smithi163.stdout:4/231: symlink d11/d18/l48 0 2024-02-16T19:24:24.492 INFO:tasks.workunit.client.1.smithi163.stdout:4/232: chown d11/f17 15620553 1 2024-02-16T19:24:24.492 INFO:tasks.workunit.client.1.smithi163.stdout:4/233: chown d11 1 1 2024-02-16T19:24:24.493 INFO:tasks.workunit.client.1.smithi163.stdout:4/234: chown d11/d18/d1f/d29/l3c 405303 1 2024-02-16T19:24:24.493 INFO:tasks.workunit.client.1.smithi163.stdout:4/235: truncate d11/f14 849142 0 2024-02-16T19:24:24.494 INFO:tasks.workunit.client.0.smithi040.stdout:7/783: symlink d21/d4d/d77/d7a/dc6/l108 0 2024-02-16T19:24:24.495 INFO:tasks.workunit.client.0.smithi040.stdout:2/820: dread d5/d7/d10/f86 [0,4194304] 0 2024-02-16T19:24:24.495 INFO:tasks.workunit.client.0.smithi040.stdout:2/821: truncate d5/d7/d1e/d31/d4f/fd3 2238348 0 2024-02-16T19:24:24.498 INFO:tasks.workunit.client.1.smithi163.stdout:8/286: unlink d0/d24/d27/d45/l4b 0 2024-02-16T19:24:24.499 INFO:tasks.workunit.client.0.smithi040.stdout:9/870: dread d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/f9b [0,4194304] 0 2024-02-16T19:24:24.500 INFO:tasks.workunit.client.1.smithi163.stdout:2/459: mknod d2/d2a/d39/c9e 0 2024-02-16T19:24:24.504 INFO:tasks.workunit.client.1.smithi163.stdout:9/460: creat d5/de/d72/fa7 x:0 0 0 2024-02-16T19:24:24.504 INFO:tasks.workunit.client.1.smithi163.stdout:9/461: dread - d5/de/d30/f89 zero size 2024-02-16T19:24:24.508 INFO:tasks.workunit.client.1.smithi163.stdout:3/316: mkdir d0/dc/d1d/d29/d38/d46/d74 0 2024-02-16T19:24:24.508 INFO:tasks.workunit.client.1.smithi163.stdout:3/317: readlink d0/dc/d1c/d21/l2a 0 2024-02-16T19:24:24.509 INFO:tasks.workunit.client.0.smithi040.stdout:1/842: mkdir d17/d26/d4c/dab/dd3/d111 0 2024-02-16T19:24:24.509 INFO:tasks.workunit.client.0.smithi040.stdout:4/705: dwrite dd/de/d10/d36/d9e/d77/f96 [0,4194304] 0 2024-02-16T19:24:24.511 INFO:tasks.workunit.client.1.smithi163.stdout:0/512: creat d11/d21/d7c/fb7 x:0 0 0 2024-02-16T19:24:24.511 INFO:tasks.workunit.client.1.smithi163.stdout:0/513: fdatasync fd 0 2024-02-16T19:24:24.512 INFO:tasks.workunit.client.1.smithi163.stdout:9/462: dread d5/de/d30/d3d/d49/d6e/f8d [0,4194304] 0 2024-02-16T19:24:24.514 INFO:tasks.workunit.client.1.smithi163.stdout:7/299: rename d11/d26/d36/f40 to d11/d26/f6d 0 2024-02-16T19:24:24.516 INFO:tasks.workunit.client.1.smithi163.stdout:4/236: creat d11/d18/d1f/d29/f49 x:0 0 0 2024-02-16T19:24:24.516 INFO:tasks.workunit.client.1.smithi163.stdout:4/237: truncate d11/d18/f43 405440 0 2024-02-16T19:24:24.520 INFO:tasks.workunit.client.1.smithi163.stdout:8/287: link d0/d7/d19/l57 d0/d7/d18/d2e/d46/l67 0 2024-02-16T19:24:24.520 INFO:tasks.workunit.client.1.smithi163.stdout:8/288: write d0/dc/f2d [5557280,124026] 0 2024-02-16T19:24:24.522 INFO:tasks.workunit.client.1.smithi163.stdout:2/460: mkdir d2/d2a/d9f 0 2024-02-16T19:24:24.526 INFO:tasks.workunit.client.1.smithi163.stdout:0/514: symlink d11/d22/lb8 0 2024-02-16T19:24:24.529 INFO:tasks.workunit.client.1.smithi163.stdout:9/463: mkdir d5/de/d30/d8a/da8 0 2024-02-16T19:24:24.531 INFO:tasks.workunit.client.1.smithi163.stdout:7/300: rename d11/l27 to d11/d24/d25/d4b/l6e 0 2024-02-16T19:24:24.532 INFO:tasks.workunit.client.0.smithi040.stdout:5/858: dwrite d1/da/d1c/da7/fca [4194304,4194304] 0 2024-02-16T19:24:24.534 INFO:tasks.workunit.client.1.smithi163.stdout:8/289: dread d0/dc/f16 [0,4194304] 0 2024-02-16T19:24:24.534 INFO:tasks.workunit.client.0.smithi040.stdout:7/784: creat d21/d4d/d77/d87/d9b/df2/f109 x:0 0 0 2024-02-16T19:24:24.536 INFO:tasks.workunit.client.0.smithi040.stdout:2/822: truncate d5/f3c 768572 0 2024-02-16T19:24:24.536 INFO:tasks.workunit.client.1.smithi163.stdout:7/301: dread d11/d24/d25/f5a [0,4194304] 0 2024-02-16T19:24:24.536 INFO:tasks.workunit.client.1.smithi163.stdout:0/515: creat d11/d12/d20/fb9 x:0 0 0 2024-02-16T19:24:24.537 INFO:tasks.workunit.client.1.smithi163.stdout:7/302: dread fc [0,4194304] 0 2024-02-16T19:24:24.537 INFO:tasks.workunit.client.0.smithi040.stdout:9/871: unlink d3/d4/d11/d29/d38/d46/d6b/d85/d112/d70/de0/ce7 0 2024-02-16T19:24:24.537 INFO:tasks.workunit.client.0.smithi040.stdout:9/872: write d3/d4/d11/d29/d38/d46/d6b/dac/fb5 [628761,65243] 0 2024-02-16T19:24:24.538 INFO:tasks.workunit.client.1.smithi163.stdout:9/464: mkdir d5/de/d72/da9 0 2024-02-16T19:24:24.539 INFO:tasks.workunit.client.1.smithi163.stdout:9/465: fsync d5/de/d30/d3d/d49/d50/f94 0 2024-02-16T19:24:24.543 INFO:tasks.workunit.client.1.smithi163.stdout:2/461: rename d2/d2a/d39/f7a to d2/d2a/d39/d66/d6c/fa0 0 2024-02-16T19:24:24.544 INFO:tasks.workunit.client.1.smithi163.stdout:2/462: write d2/d26/f2f [4459760,118338] 0 2024-02-16T19:24:24.544 INFO:tasks.workunit.client.1.smithi163.stdout:7/303: dread f3 [0,4194304] 0 2024-02-16T19:24:24.544 INFO:tasks.workunit.client.1.smithi163.stdout:8/290: creat d0/d7/f68 x:0 0 0 2024-02-16T19:24:24.544 INFO:tasks.workunit.client.0.smithi040.stdout:5/859: dread d1/da/d51/f8b [0,4194304] 0 2024-02-16T19:24:24.545 INFO:tasks.workunit.client.1.smithi163.stdout:1/285: dwrite d7/f15 [0,4194304] 0 2024-02-16T19:24:24.545 INFO:tasks.workunit.client.1.smithi163.stdout:1/286: truncate d7/d20/d21/f45 322102 0 2024-02-16T19:24:24.545 INFO:tasks.workunit.client.0.smithi040.stdout:5/860: write d1/da/d1c/d21/daa/fe4 [1026182,115197] 0 2024-02-16T19:24:24.546 INFO:tasks.workunit.client.1.smithi163.stdout:3/318: dwrite d0/dc/d1d/d29/f33 [4194304,4194304] 0 2024-02-16T19:24:24.546 INFO:tasks.workunit.client.1.smithi163.stdout:3/319: write d0/f6d [863928,126201] 0 2024-02-16T19:24:24.546 INFO:tasks.workunit.client.1.smithi163.stdout:0/516: mknod d11/d4f/cba 0 2024-02-16T19:24:24.546 INFO:tasks.workunit.client.0.smithi040.stdout:4/706: dwrite dd/d58/d5e/f7f [0,4194304] 0 2024-02-16T19:24:24.546 INFO:tasks.workunit.client.0.smithi040.stdout:4/707: read dd/f43 [3769253,2907] 0 2024-02-16T19:24:24.547 INFO:tasks.workunit.client.0.smithi040.stdout:4/708: truncate dd/de/d10/d36/fd7 870776 0 2024-02-16T19:24:24.547 INFO:tasks.workunit.client.0.smithi040.stdout:7/785: rename d21/d29/f37 to d21/df9/f10a 0 2024-02-16T19:24:24.550 INFO:tasks.workunit.client.1.smithi163.stdout:9/466: mknod d5/de/d16/d47/caa 0 2024-02-16T19:24:24.555 INFO:tasks.workunit.client.0.smithi040.stdout:2/823: mkdir d5/d7/d10/d10e 0 2024-02-16T19:24:24.558 INFO:tasks.workunit.client.1.smithi163.stdout:3/320: dread d0/dc/d1c/f2c [0,4194304] 0 2024-02-16T19:24:24.558 INFO:tasks.workunit.client.0.smithi040.stdout:1/843: rmdir d17/d26/d5f/d78/d7e/ddb 39 2024-02-16T19:24:24.559 INFO:tasks.workunit.client.0.smithi040.stdout:1/844: read - d17/d26/d9a/dfb/fef zero size 2024-02-16T19:24:24.559 INFO:tasks.workunit.client.0.smithi040.stdout:1/845: dread - d17/d26/d9a/dfb/d38/feb zero size 2024-02-16T19:24:24.563 INFO:tasks.workunit.client.1.smithi163.stdout:4/238: dwrite d11/d18/d1f/d34/f39 [0,4194304] 0 2024-02-16T19:24:24.563 INFO:tasks.workunit.client.1.smithi163.stdout:2/463: dread d2/fc [0,4194304] 0 2024-02-16T19:24:24.570 INFO:tasks.workunit.client.0.smithi040.stdout:5/861: rename d1/d45/dab/cc0 to d1/da/c108 0 2024-02-16T19:24:24.572 INFO:tasks.workunit.client.0.smithi040.stdout:7/786: mknod d21/d4d/d77/d87/dc9/ddd/c10b 0 2024-02-16T19:24:24.573 INFO:tasks.workunit.client.1.smithi163.stdout:0/517: symlink d11/d22/d61/lbb 0 2024-02-16T19:24:24.573 INFO:tasks.workunit.client.1.smithi163.stdout:7/304: dwrite d11/f15 [0,4194304] 0 2024-02-16T19:24:24.576 INFO:tasks.workunit.client.1.smithi163.stdout:3/321: mknod d0/dc/d1d/d29/d38/d46/d74/c75 0 2024-02-16T19:24:24.576 INFO:tasks.workunit.client.1.smithi163.stdout:3/322: fsync d0/d3e/f60 0 2024-02-16T19:24:24.576 INFO:tasks.workunit.client.1.smithi163.stdout:3/323: dread - d0/dc/f6e zero size 2024-02-16T19:24:24.576 INFO:tasks.workunit.client.0.smithi040.stdout:2/824: mkdir d5/d7/d1e/d55/d102/d10f 0 2024-02-16T19:24:24.580 INFO:tasks.workunit.client.0.smithi040.stdout:4/709: rename dd/de/d10/d36/l7c to dd/de/db0/ldb 0 2024-02-16T19:24:24.583 INFO:tasks.workunit.client.0.smithi040.stdout:5/862: mknod d1/da/d7e/c109 0 2024-02-16T19:24:24.583 INFO:tasks.workunit.client.0.smithi040.stdout:5/863: readlink d1/da/d51/l7c 0 2024-02-16T19:24:24.583 INFO:tasks.workunit.client.0.smithi040.stdout:5/864: chown d1/da/d1c/d21/daa/ff9 401538803 1 2024-02-16T19:24:24.584 INFO:tasks.workunit.client.0.smithi040.stdout:1/846: dwrite d17/d26/d3d/f83 [4194304,4194304] 0 2024-02-16T19:24:24.585 INFO:tasks.workunit.client.1.smithi163.stdout:9/467: dwrite d5/de/d30/f5e [0,4194304] 0 2024-02-16T19:24:24.585 INFO:tasks.workunit.client.1.smithi163.stdout:0/518: dread d11/d22/d7f/f96 [0,4194304] 0 2024-02-16T19:24:24.586 INFO:tasks.workunit.client.1.smithi163.stdout:4/239: mkdir d11/d18/d1f/d23/d3a/d4a 0 2024-02-16T19:24:24.586 INFO:tasks.workunit.client.1.smithi163.stdout:4/240: creat d11/f4b x:0 0 0 2024-02-16T19:24:24.587 INFO:tasks.workunit.client.0.smithi040.stdout:7/787: truncate f9 4644735 0 2024-02-16T19:24:24.587 INFO:tasks.workunit.client.0.smithi040.stdout:7/788: fdatasync d21/d4d/d77/d87/dc9/fef 0 2024-02-16T19:24:24.590 INFO:tasks.workunit.client.0.smithi040.stdout:2/825: creat d5/d7/d12/da6/f110 x:0 0 0 2024-02-16T19:24:24.592 INFO:tasks.workunit.client.0.smithi040.stdout:5/865: mknod d1/da/d7e/db2/c10a 0 2024-02-16T19:24:24.593 INFO:tasks.workunit.client.0.smithi040.stdout:5/866: write d1/da/d1c/dba/fc2 [687409,83163] 0 2024-02-16T19:24:24.596 INFO:tasks.workunit.client.1.smithi163.stdout:1/287: rename d7/dc/d2d/d32/c54 to d7/c65 0 2024-02-16T19:24:24.597 INFO:tasks.workunit.client.0.smithi040.stdout:1/847: mkdir d17/d29/d8b/d10e/d112 0 2024-02-16T19:24:24.599 INFO:tasks.workunit.client.0.smithi040.stdout:7/789: mkdir d21/d29/d79/d10c 0 2024-02-16T19:24:24.601 INFO:tasks.workunit.client.1.smithi163.stdout:3/324: link d0/dc/d1d/d29/d38/l3a d0/dc/d1c/d32/l76 0 2024-02-16T19:24:24.604 INFO:tasks.workunit.client.0.smithi040.stdout:2/826: rename d5/d5c/fdf to d5/d7/d1e/d55/da3/f111 0 2024-02-16T19:24:24.606 INFO:tasks.workunit.client.0.smithi040.stdout:5/867: creat d1/d3c/f10b x:0 0 0 2024-02-16T19:24:24.606 INFO:tasks.workunit.client.0.smithi040.stdout:5/868: chown d1/da/d1c/l53 619 1 2024-02-16T19:24:24.606 INFO:tasks.workunit.client.0.smithi040.stdout:5/869: chown d1/da/d51/f6c 221 1 2024-02-16T19:24:24.607 INFO:tasks.workunit.client.1.smithi163.stdout:4/241: dwrite d11/d18/d1f/f33 [0,4194304] 0 2024-02-16T19:24:24.607 INFO:tasks.workunit.client.1.smithi163.stdout:4/242: chown d11/f30 532264 1 2024-02-16T19:24:24.609 INFO:tasks.workunit.client.1.smithi163.stdout:7/305: dwrite d11/f44 [0,4194304] 0 2024-02-16T19:24:24.609 INFO:tasks.workunit.client.0.smithi040.stdout:4/710: dwrite dd/d58/fb7 [0,4194304] 0 2024-02-16T19:24:24.610 INFO:tasks.workunit.client.1.smithi163.stdout:0/519: symlink d11/d21/d50/d58/lbc 0 2024-02-16T19:24:24.610 INFO:tasks.workunit.client.1.smithi163.stdout:0/520: read - d11/d22/d61/f94 zero size 2024-02-16T19:24:24.611 INFO:tasks.workunit.client.0.smithi040.stdout:1/848: creat d17/d29/d8b/dd2/f113 x:0 0 0 2024-02-16T19:24:24.614 INFO:tasks.workunit.client.0.smithi040.stdout:2/827: creat d5/d7/d1e/d4c/d52/dc1/f112 x:0 0 0 2024-02-16T19:24:24.617 INFO:tasks.workunit.client.1.smithi163.stdout:9/468: rename d5/de/d30/f5e to d5/de/d30/d3d/d93/fab 0 2024-02-16T19:24:24.618 INFO:tasks.workunit.client.0.smithi040.stdout:5/870: mkdir d1/d3c/dea/d10c 0 2024-02-16T19:24:24.620 INFO:tasks.workunit.client.1.smithi163.stdout:1/288: link d7/dc/f30 d7/d20/f66 0 2024-02-16T19:24:24.621 INFO:tasks.workunit.client.1.smithi163.stdout:3/325: truncate d0/f39 559196 0 2024-02-16T19:24:24.626 INFO:tasks.workunit.client.1.smithi163.stdout:4/243: symlink d11/d18/d1f/d29/l4c 0 2024-02-16T19:24:24.627 INFO:tasks.workunit.client.0.smithi040.stdout:1/849: truncate d17/d26/d4c/f8a 196989 0 2024-02-16T19:24:24.629 INFO:tasks.workunit.client.0.smithi040.stdout:2/828: rename d5/d7/d1e/d31/d4f/d63/l1c to d5/d7/d10/df3/l113 0 2024-02-16T19:24:24.630 INFO:tasks.workunit.client.1.smithi163.stdout:7/306: dwrite d11/f44 [0,4194304] 0 2024-02-16T19:24:24.630 INFO:tasks.workunit.client.1.smithi163.stdout:7/307: truncate d11/d30/f3e 309366 0 2024-02-16T19:24:24.631 INFO:tasks.workunit.client.0.smithi040.stdout:5/871: rmdir d1/da/d1c/da7/d100 0 2024-02-16T19:24:24.632 INFO:tasks.workunit.client.1.smithi163.stdout:0/521: mknod d11/d12/d20/cbd 0 2024-02-16T19:24:24.632 INFO:tasks.workunit.client.1.smithi163.stdout:0/522: chown ca 105 1 2024-02-16T19:24:24.632 INFO:tasks.workunit.client.1.smithi163.stdout:0/523: write d11/d12/d67/d6a/d6d/f7d [692941,999] 0 2024-02-16T19:24:24.633 INFO:tasks.workunit.client.1.smithi163.stdout:3/326: read d0/dc/d1c/d32/f55 [367850,60609] 0 2024-02-16T19:24:24.633 INFO:tasks.workunit.client.1.smithi163.stdout:3/327: fdatasync d0/f39 0 2024-02-16T19:24:24.634 INFO:tasks.workunit.client.1.smithi163.stdout:9/469: creat d5/de/d30/d3d/d49/d6d/fac x:0 0 0 2024-02-16T19:24:24.634 INFO:tasks.workunit.client.1.smithi163.stdout:9/470: fsync d5/f40 0 2024-02-16T19:24:24.634 INFO:tasks.workunit.client.1.smithi163.stdout:9/471: readlink d5/de/d30/d3d/d93/la4 0 2024-02-16T19:24:24.635 INFO:tasks.workunit.client.0.smithi040.stdout:1/850: mknod d17/d26/d54/c114 0 2024-02-16T19:24:24.635 INFO:tasks.workunit.client.0.smithi040.stdout:1/851: truncate d17/d26/d9a/dfb/fc9 659141 0 2024-02-16T19:24:24.635 INFO:tasks.workunit.client.1.smithi163.stdout:4/244: readlink d11/d18/l25 0 2024-02-16T19:24:24.635 INFO:tasks.workunit.client.1.smithi163.stdout:4/245: dread - d11/f30 zero size 2024-02-16T19:24:24.636 INFO:tasks.workunit.client.0.smithi040.stdout:2/829: mkdir d5/d7/d12/da6/d114 0 2024-02-16T19:24:24.636 INFO:tasks.workunit.client.0.smithi040.stdout:2/830: write d5/d7/d1e/d31/d4f/d63/f2c [296144,95311] 0 2024-02-16T19:24:24.638 INFO:tasks.workunit.client.0.smithi040.stdout:5/872: symlink d1/d62/l10d 0 2024-02-16T19:24:24.640 INFO:tasks.workunit.client.0.smithi040.stdout:5/873: dread d1/da/d7e/f86 [0,4194304] 0 2024-02-16T19:24:24.640 INFO:tasks.workunit.client.0.smithi040.stdout:5/874: truncate d1/da/d1c/dba/db1/dde/ddf/fe8 1046935 0 2024-02-16T19:24:24.640 INFO:tasks.workunit.client.1.smithi163.stdout:7/308: creat d11/d24/d25/d4b/d66/f6f x:0 0 0 2024-02-16T19:24:24.641 INFO:tasks.workunit.client.0.smithi040.stdout:2/831: dread d5/d7/d1e/d55/da3/f111 [0,4194304] 0 2024-02-16T19:24:24.641 INFO:tasks.workunit.client.0.smithi040.stdout:2/832: creat d5/d7/d12/da6/f115 x:0 0 0 2024-02-16T19:24:24.641 INFO:tasks.workunit.client.0.smithi040.stdout:1/852: unlink d17/d26/d4c/f98 0 2024-02-16T19:24:24.641 INFO:tasks.workunit.client.0.smithi040.stdout:1/853: chown d17/d26/d5f/d63/f75 195 1 2024-02-16T19:24:24.651 INFO:tasks.workunit.client.0.smithi040.stdout:2/833: dread d5/d7/d1e/d31/d4f/d63/f8e [4194304,4194304] 0 2024-02-16T19:24:24.661 INFO:tasks.workunit.client.1.smithi163.stdout:0/524: rename d11/d21/d7c/c89 to d11/d21/d7c/cbe 0 2024-02-16T19:24:24.663 INFO:tasks.workunit.client.1.smithi163.stdout:3/328: mknod d0/dc/d1d/d29/d38/d46/d74/c77 0 2024-02-16T19:24:24.663 INFO:tasks.workunit.client.0.smithi040.stdout:5/875: getdents d1/da/d1c/dba/db1/dde 0 2024-02-16T19:24:24.664 INFO:tasks.workunit.client.0.smithi040.stdout:1/854: dwrite d17/d26/f7d [8388608,4194304] 0 2024-02-16T19:24:24.665 INFO:tasks.workunit.client.1.smithi163.stdout:9/472: creat d5/de/d30/d8a/da8/fad x:0 0 0 2024-02-16T19:24:24.666 INFO:tasks.workunit.client.1.smithi163.stdout:4/246: mknod d11/d18/d1f/d34/c4d 0 2024-02-16T19:24:24.668 INFO:tasks.workunit.client.1.smithi163.stdout:7/309: truncate d11/d30/f6a 923656 0 2024-02-16T19:24:24.669 INFO:tasks.workunit.client.0.smithi040.stdout:2/834: mknod d5/d7/d1e/c116 0 2024-02-16T19:24:24.669 INFO:tasks.workunit.client.0.smithi040.stdout:2/835: dread - d5/d7/d1e/d31/d4f/d63/d8b/fed zero size 2024-02-16T19:24:24.673 INFO:tasks.workunit.client.0.smithi040.stdout:5/876: symlink d1/da/d1c/dba/l10e 0 2024-02-16T19:24:24.678 INFO:tasks.workunit.client.1.smithi163.stdout:0/525: mkdir d11/d21/d50/d58/d72/d9e/dbf 0 2024-02-16T19:24:24.678 INFO:tasks.workunit.client.1.smithi163.stdout:0/526: fdatasync d11/f85 0 2024-02-16T19:24:24.682 INFO:tasks.workunit.client.1.smithi163.stdout:3/329: rename d0/dc/d1c/d32/f55 to d0/d3e/f78 0 2024-02-16T19:24:24.684 INFO:tasks.workunit.client.1.smithi163.stdout:9/473: mknod d5/de/d30/d43/cae 0 2024-02-16T19:24:24.685 INFO:tasks.workunit.client.1.smithi163.stdout:4/247: creat d11/d18/d1f/d23/d3a/d4a/f4e x:0 0 0 2024-02-16T19:24:24.687 INFO:tasks.workunit.client.1.smithi163.stdout:0/527: unlink d11/d12/d20/fb5 0 2024-02-16T19:24:24.689 INFO:tasks.workunit.client.1.smithi163.stdout:4/248: mkdir d11/d2d/d4f 0 2024-02-16T19:24:24.689 INFO:tasks.workunit.client.1.smithi163.stdout:4/249: read - d11/d18/d1f/d23/d3a/d4a/f4e zero size 2024-02-16T19:24:24.690 INFO:tasks.workunit.client.1.smithi163.stdout:7/310: dwrite d11/f1f [0,4194304] 0 2024-02-16T19:24:24.690 INFO:tasks.workunit.client.1.smithi163.stdout:7/311: creat d11/d24/d25/d4b/d66/f70 x:0 0 0 2024-02-16T19:24:24.692 INFO:tasks.workunit.client.0.smithi040.stdout:1/855: dwrite d17/d26/d5f/d63/f66 [0,4194304] 0 2024-02-16T19:24:24.695 INFO:tasks.workunit.client.0.smithi040.stdout:1/856: write d17/d26/d9a/dfb/f6a [1960899,96296] 0 2024-02-16T19:24:24.695 INFO:tasks.workunit.client.0.smithi040.stdout:1/857: chown d17/d26/d54/fe6 478324602 1 2024-02-16T19:24:24.695 INFO:tasks.workunit.client.0.smithi040.stdout:1/858: creat d17/d29/d8b/f115 x:0 0 0 2024-02-16T19:24:24.695 INFO:tasks.workunit.client.0.smithi040.stdout:1/859: chown d17/f28 59 1 2024-02-16T19:24:24.695 INFO:tasks.workunit.client.0.smithi040.stdout:1/860: fsync d17/d3c/f9d 0 2024-02-16T19:24:24.695 INFO:tasks.workunit.client.1.smithi163.stdout:3/330: dread d0/dc/d1d/d29/f33 [0,4194304] 0 2024-02-16T19:24:24.696 INFO:tasks.workunit.client.1.smithi163.stdout:3/331: readlink d0/d3e/l54 0 2024-02-16T19:24:24.701 INFO:tasks.workunit.client.1.smithi163.stdout:7/312: mkdir d11/d26/d36/d67/d71 0 2024-02-16T19:24:24.701 INFO:tasks.workunit.client.1.smithi163.stdout:7/313: fsync d11/d26/d36/f4e 0 2024-02-16T19:24:24.701 INFO:tasks.workunit.client.1.smithi163.stdout:0/528: write d11/d22/d3e/f6f [2152825,13980] 0 2024-02-16T19:24:24.703 INFO:tasks.workunit.client.0.smithi040.stdout:2/836: mkdir d5/d7/d10/d40/d117 0 2024-02-16T19:24:24.703 INFO:tasks.workunit.client.1.smithi163.stdout:3/332: dread d0/dc/d1d/d29/d38/f40 [0,4194304] 0 2024-02-16T19:24:24.705 INFO:tasks.workunit.client.1.smithi163.stdout:9/474: dwrite d5/de/f5f [0,4194304] 0 2024-02-16T19:24:24.705 INFO:tasks.workunit.client.1.smithi163.stdout:9/475: readlink d5/de/l26 0 2024-02-16T19:24:24.706 INFO:tasks.workunit.client.0.smithi040.stdout:5/877: unlink d1/d62/d9a/dc5/ff1 0 2024-02-16T19:24:24.708 INFO:tasks.workunit.client.1.smithi163.stdout:3/333: rename d0/dc/f2e to d0/f79 0 2024-02-16T19:24:24.709 INFO:tasks.workunit.client.1.smithi163.stdout:4/250: dwrite f10 [0,4194304] 0 2024-02-16T19:24:24.709 INFO:tasks.workunit.client.1.smithi163.stdout:4/251: truncate d11/d18/d1f/f33 5053138 0 2024-02-16T19:24:24.711 INFO:tasks.workunit.client.1.smithi163.stdout:6/270: sync 2024-02-16T19:24:24.711 INFO:tasks.workunit.client.1.smithi163.stdout:9/476: dread d5/de/d30/d43/d8f/f90 [0,4194304] 0 2024-02-16T19:24:24.712 INFO:tasks.workunit.client.1.smithi163.stdout:9/477: dread d5/de/d30/d3d/d49/d6e/f8d [0,4194304] 0 2024-02-16T19:24:24.715 INFO:tasks.workunit.client.1.smithi163.stdout:3/334: link d0/dc/d1c/f3b d0/dc/d1c/d21/f7a 0 2024-02-16T19:24:24.719 INFO:tasks.workunit.client.1.smithi163.stdout:6/271: truncate d0/d6/df/f32 840102 0 2024-02-16T19:24:24.720 INFO:tasks.workunit.client.1.smithi163.stdout:4/252: write f10 [1477466,97427] 0 2024-02-16T19:24:24.721 INFO:tasks.workunit.client.0.smithi040.stdout:2/837: creat d5/d7/d12/da6/d114/f118 x:0 0 0 2024-02-16T19:24:24.722 INFO:tasks.workunit.client.1.smithi163.stdout:9/478: symlink d5/de/d30/d3d/d49/d6e/laf 0 2024-02-16T19:24:24.724 INFO:tasks.workunit.client.1.smithi163.stdout:5/292: sync 2024-02-16T19:24:24.724 INFO:tasks.workunit.client.1.smithi163.stdout:5/293: chown d0/d22/d28/d36/d3c/d4f 803554 1 2024-02-16T19:24:24.724 INFO:tasks.workunit.client.1.smithi163.stdout:4/253: write d11/d18/d1f/f27 [8337181,68566] 0 2024-02-16T19:24:24.724 INFO:tasks.workunit.client.1.smithi163.stdout:7/314: dwrite d11/d50/d5d/f61 [0,4194304] 0 2024-02-16T19:24:24.725 INFO:tasks.workunit.client.1.smithi163.stdout:9/479: dread d5/de/d30/d31/f69 [0,4194304] 0 2024-02-16T19:24:24.725 INFO:tasks.workunit.client.1.smithi163.stdout:0/529: dwrite d11/f1f [4194304,4194304] 0 2024-02-16T19:24:24.725 INFO:tasks.workunit.client.1.smithi163.stdout:0/530: stat d11/f1f 0 2024-02-16T19:24:24.725 INFO:tasks.workunit.client.1.smithi163.stdout:9/480: creat d5/de/d72/fb0 x:0 0 0 2024-02-16T19:24:24.725 INFO:tasks.workunit.client.1.smithi163.stdout:0/531: write d11/d22/d7f/fb6 [719960,61134] 0 2024-02-16T19:24:24.726 INFO:tasks.workunit.client.1.smithi163.stdout:3/335: mkdir d0/dc/d1d/d7b 0 2024-02-16T19:24:24.727 INFO:tasks.workunit.client.0.smithi040.stdout:1/861: rmdir d17/d2d 39 2024-02-16T19:24:24.729 INFO:tasks.workunit.client.1.smithi163.stdout:6/272: creat d0/d6/d9/d2e/f4f x:0 0 0 2024-02-16T19:24:24.729 INFO:tasks.workunit.client.1.smithi163.stdout:6/273: chown d0/d6/df/l25 16012216 1 2024-02-16T19:24:24.730 INFO:tasks.workunit.client.1.smithi163.stdout:5/294: symlink d0/d22/d28/d36/d3c/l58 0 2024-02-16T19:24:24.731 INFO:tasks.workunit.client.0.smithi040.stdout:1/862: creat d17/d26/d5f/da6/f116 x:0 0 0 2024-02-16T19:24:24.734 INFO:tasks.workunit.client.1.smithi163.stdout:4/254: unlink cf 0 2024-02-16T19:24:24.742 INFO:tasks.workunit.client.1.smithi163.stdout:3/336: dread d0/dc/f31 [0,4194304] 0 2024-02-16T19:24:24.743 INFO:tasks.workunit.client.1.smithi163.stdout:9/481: symlink d5/de/d16/lb1 0 2024-02-16T19:24:24.748 INFO:tasks.workunit.client.1.smithi163.stdout:4/255: dread d11/d18/d1f/d34/f39 [0,4194304] 0 2024-02-16T19:24:24.748 INFO:tasks.workunit.client.1.smithi163.stdout:3/337: getdents d0/dc/d1d/d29 0 2024-02-16T19:24:24.749 INFO:tasks.workunit.client.1.smithi163.stdout:4/256: symlink d11/d18/d1f/d34/l50 0 2024-02-16T19:24:24.749 INFO:tasks.workunit.client.1.smithi163.stdout:4/257: chown d11/d2d/l3b 9936801 1 2024-02-16T19:24:24.750 INFO:tasks.workunit.client.1.smithi163.stdout:4/258: fsync d11/d18/f43 0 2024-02-16T19:24:24.750 INFO:tasks.workunit.client.1.smithi163.stdout:4/259: write d11/f14 [1227161,4202] 0 2024-02-16T19:24:24.750 INFO:tasks.workunit.client.1.smithi163.stdout:4/260: chown d11/c1c 71883261 1 2024-02-16T19:24:24.757 INFO:tasks.workunit.client.1.smithi163.stdout:5/295: read d0/d2/d34/d44/f51 [1472495,126750] 0 2024-02-16T19:24:24.757 INFO:tasks.workunit.client.1.smithi163.stdout:5/296: write d0/d22/f33 [4454549,129467] 0 2024-02-16T19:24:24.757 INFO:tasks.workunit.client.1.smithi163.stdout:5/297: fdatasync d0/d22/d28/d36/d3c/f43 0 2024-02-16T19:24:24.759 INFO:tasks.workunit.client.1.smithi163.stdout:5/298: rename d0/d22/c4a to d0/c59 0 2024-02-16T19:24:24.760 INFO:tasks.workunit.client.1.smithi163.stdout:5/299: mkdir d0/d1c/d5a 0 2024-02-16T19:24:24.767 INFO:tasks.workunit.client.1.smithi163.stdout:3/338: read d0/dc/d1c/f20 [2626543,15273] 0 2024-02-16T19:24:24.768 INFO:tasks.workunit.client.1.smithi163.stdout:0/532: dwrite d11/d21/d50/d58/d72/fa8 [0,4194304] 0 2024-02-16T19:24:24.769 INFO:tasks.workunit.client.0.smithi040.stdout:1/863: dwrite d17/d26/d9a/dfb/f5d [0,4194304] 0 2024-02-16T19:24:24.771 INFO:tasks.workunit.client.0.smithi040.stdout:2/838: dwrite d5/f79 [0,4194304] 0 2024-02-16T19:24:24.771 INFO:tasks.workunit.client.0.smithi040.stdout:2/839: creat d5/d7/d12/da6/d114/f119 x:0 0 0 2024-02-16T19:24:24.772 INFO:tasks.workunit.client.0.smithi040.stdout:1/864: creat d17/d26/d4c/dab/f117 x:0 0 0 2024-02-16T19:24:24.772 INFO:tasks.workunit.client.0.smithi040.stdout:1/865: fdatasync d17/d26/d9a/dfb/f5d 0 2024-02-16T19:24:24.773 INFO:tasks.workunit.client.1.smithi163.stdout:9/482: dwrite d5/de/d30/d3d/f88 [4194304,4194304] 0 2024-02-16T19:24:24.773 INFO:tasks.workunit.client.1.smithi163.stdout:6/274: dwrite d0/d6/f1a [0,4194304] 0 2024-02-16T19:24:24.773 INFO:tasks.workunit.client.1.smithi163.stdout:0/533: getdents d11/d22/d3e 0 2024-02-16T19:24:24.782 INFO:tasks.workunit.client.1.smithi163.stdout:6/275: creat d0/dd/f50 x:0 0 0 2024-02-16T19:24:24.784 INFO:tasks.workunit.client.0.smithi040.stdout:2/840: link d5/d7/d1e/d31/d4f/fd3 d5/d7/d1e/f11a 0 2024-02-16T19:24:24.785 INFO:tasks.workunit.client.1.smithi163.stdout:0/534: getdents d11/d21/d50 0 2024-02-16T19:24:24.785 INFO:tasks.workunit.client.1.smithi163.stdout:0/535: truncate d11/d12/d67/fa6 877712 0 2024-02-16T19:24:24.786 INFO:tasks.workunit.client.1.smithi163.stdout:5/300: dwrite d0/d22/d28/f38 [0,4194304] 0 2024-02-16T19:24:24.786 INFO:tasks.workunit.client.1.smithi163.stdout:5/301: dread - d0/d2/f29 zero size 2024-02-16T19:24:24.790 INFO:tasks.workunit.client.1.smithi163.stdout:3/339: dread d0/dc/f24 [0,4194304] 0 2024-02-16T19:24:24.790 INFO:tasks.workunit.client.1.smithi163.stdout:3/340: readlink d0/d3e/l54 0 2024-02-16T19:24:24.790 INFO:tasks.workunit.client.1.smithi163.stdout:3/341: write d0/dc/d1c/f3b [1056530,1057] 0 2024-02-16T19:24:24.790 INFO:tasks.workunit.client.1.smithi163.stdout:0/536: rename d11/d21/d50/d58/d6b/l6e to d11/d22/d7f/d8e/lc0 0 2024-02-16T19:24:24.792 INFO:tasks.workunit.client.0.smithi040.stdout:2/841: symlink d5/d7/d12/dca/dea/l11b 0 2024-02-16T19:24:24.792 INFO:tasks.workunit.client.0.smithi040.stdout:2/842: readlink d5/d7/d12/lb3 0 2024-02-16T19:24:24.792 INFO:tasks.workunit.client.1.smithi163.stdout:4/261: rmdir d11/d18/d1f/d34 39 2024-02-16T19:24:24.794 INFO:tasks.workunit.client.1.smithi163.stdout:0/537: dread d11/f33 [0,4194304] 0 2024-02-16T19:24:24.796 INFO:tasks.workunit.client.0.smithi040.stdout:1/866: link d17/d2d/f81 d17/d3c/d74/f118 0 2024-02-16T19:24:24.799 INFO:tasks.workunit.client.0.smithi040.stdout:2/843: creat d5/d7/d1e/d31/dc3/f11c x:0 0 0 2024-02-16T19:24:24.800 INFO:tasks.workunit.client.1.smithi163.stdout:9/483: dwrite d5/de/d72/fa7 [0,4194304] 0 2024-02-16T19:24:24.808 INFO:tasks.workunit.client.0.smithi040.stdout:3/954: sync 2024-02-16T19:24:24.808 INFO:tasks.workunit.client.0.smithi040.stdout:9/873: sync 2024-02-16T19:24:24.808 INFO:tasks.workunit.client.0.smithi040.stdout:3/955: dread - d1/dc/d42/d5c/f101 zero size 2024-02-16T19:24:24.808 INFO:tasks.workunit.client.0.smithi040.stdout:3/956: chown d1/dc/d42/d119/f10c 658457 1 2024-02-16T19:24:24.811 INFO:tasks.workunit.client.0.smithi040.stdout:3/957: creat d1/dc/d35/d48/d85/d126/f12f x:0 0 0 2024-02-16T19:24:24.813 INFO:tasks.workunit.client.1.smithi163.stdout:3/342: rmdir d0/dc/d1d/d7b 0 2024-02-16T19:24:24.814 INFO:tasks.workunit.client.0.smithi040.stdout:1/867: dwrite d17/d26/d4c/dab/fbd [0,4194304] 0 2024-02-16T19:24:24.814 INFO:tasks.workunit.client.0.smithi040.stdout:1/868: stat d17/d29/d8b/dd2/f113 0 2024-02-16T19:24:24.821 INFO:tasks.workunit.client.1.smithi163.stdout:9/484: creat d5/de/d16/fb2 x:0 0 0 2024-02-16T19:24:24.822 INFO:tasks.workunit.client.1.smithi163.stdout:3/343: rename d0/l37 to d0/dc/d1d/d49/l7c 0 2024-02-16T19:24:24.825 INFO:tasks.workunit.client.1.smithi163.stdout:4/262: link f9 d11/d18/d1f/d34/f51 0 2024-02-16T19:24:24.830 INFO:tasks.workunit.client.0.smithi040.stdout:4/711: sync 2024-02-16T19:24:24.830 INFO:tasks.workunit.client.0.smithi040.stdout:5/878: sync 2024-02-16T19:24:24.830 INFO:tasks.workunit.client.0.smithi040.stdout:7/790: sync 2024-02-16T19:24:24.830 INFO:tasks.workunit.client.0.smithi040.stdout:4/712: fdatasync dd/f8e 0 2024-02-16T19:24:24.830 INFO:tasks.workunit.client.0.smithi040.stdout:5/879: dread - d1/da/db0/ff0 zero size 2024-02-16T19:24:24.830 INFO:tasks.workunit.client.0.smithi040.stdout:1/869: symlink d17/d26/d101/l119 0 2024-02-16T19:24:24.831 INFO:tasks.workunit.client.1.smithi163.stdout:3/344: dread d0/dc/d1d/f5a [0,4194304] 0 2024-02-16T19:24:24.832 INFO:tasks.workunit.client.1.smithi163.stdout:4/263: mkdir d11/d18/d1f/d29/d52 0 2024-02-16T19:24:24.834 INFO:tasks.workunit.client.1.smithi163.stdout:2/464: sync 2024-02-16T19:24:24.834 INFO:tasks.workunit.client.1.smithi163.stdout:1/289: sync 2024-02-16T19:24:24.834 INFO:tasks.workunit.client.1.smithi163.stdout:1/290: read - d7/dc/d2d/d32/f46 zero size 2024-02-16T19:24:24.834 INFO:tasks.workunit.client.1.smithi163.stdout:1/291: write f6 [2760622,29013] 0 2024-02-16T19:24:24.834 INFO:tasks.workunit.client.1.smithi163.stdout:8/291: sync 2024-02-16T19:24:24.835 INFO:tasks.workunit.client.0.smithi040.stdout:7/791: rename d21/d4d/d77/d87/dc9/ddd/dbe/c104 to d21/d8b/d9c/c10d 0 2024-02-16T19:24:24.835 INFO:tasks.workunit.client.0.smithi040.stdout:4/713: creat dd/d58/fdc x:0 0 0 2024-02-16T19:24:24.838 INFO:tasks.workunit.client.0.smithi040.stdout:5/880: rename d1/da/lc7 to d1/da/db0/l10f 0 2024-02-16T19:24:24.840 INFO:tasks.workunit.client.0.smithi040.stdout:2/844: dwrite d5/d7/d10/df3/d42/fc8 [0,4194304] 0 2024-02-16T19:24:24.840 INFO:tasks.workunit.client.1.smithi163.stdout:8/292: symlink d0/d7/d19/d1f/d32/l69 0 2024-02-16T19:24:24.840 INFO:tasks.workunit.client.1.smithi163.stdout:3/345: write d0/f1b [1814878,102800] 0 2024-02-16T19:24:24.841 INFO:tasks.workunit.client.1.smithi163.stdout:5/302: dwrite d0/f2c [0,4194304] 0 2024-02-16T19:24:24.841 INFO:tasks.workunit.client.0.smithi040.stdout:3/958: dwrite d1/dc/d42/dee/fd9 [0,4194304] 0 2024-02-16T19:24:24.841 INFO:tasks.workunit.client.0.smithi040.stdout:3/959: read d1/dc/d35/fbf [3965596,68844] 0 2024-02-16T19:24:24.841 INFO:tasks.workunit.client.0.smithi040.stdout:3/960: creat d1/d4/d6e/dc2/f130 x:0 0 0 2024-02-16T19:24:24.842 INFO:tasks.workunit.client.1.smithi163.stdout:4/264: rmdir d11/d18/d1f/d23/d3a/d4a 39 2024-02-16T19:24:24.842 INFO:tasks.workunit.client.1.smithi163.stdout:4/265: chown d11/d18/d1f/d23/d3a 1079419 1 2024-02-16T19:24:24.843 INFO:tasks.workunit.client.0.smithi040.stdout:5/881: mknod d1/da/d1c/da9/c110 0 2024-02-16T19:24:24.844 INFO:tasks.workunit.client.1.smithi163.stdout:8/293: mkdir d0/d7/d19/d1f/d30/d6a 0 2024-02-16T19:24:24.844 INFO:tasks.workunit.client.1.smithi163.stdout:8/294: fdatasync d0/d7/d19/d1f/f4e 0 2024-02-16T19:24:24.846 INFO:tasks.workunit.client.1.smithi163.stdout:3/346: rmdir d0/dc/d1d 39 2024-02-16T19:24:24.849 INFO:tasks.workunit.client.1.smithi163.stdout:5/303: symlink d0/l5b 0 2024-02-16T19:24:24.852 INFO:tasks.workunit.client.0.smithi040.stdout:2/845: rename d5/d7/d12/dca/dea/l11b to d5/d7/d1e/d31/d4f/l11d 0 2024-02-16T19:24:24.852 INFO:tasks.workunit.client.0.smithi040.stdout:5/882: dread d1/d3c/f72 [0,4194304] 0 2024-02-16T19:24:24.852 INFO:tasks.workunit.client.0.smithi040.stdout:5/883: creat d1/da/d1c/f111 x:0 0 0 2024-02-16T19:24:24.857 INFO:tasks.workunit.client.1.smithi163.stdout:8/295: link d0/dc/f2d d0/d7/f6b 0 2024-02-16T19:24:24.857 INFO:tasks.workunit.client.1.smithi163.stdout:8/296: chown d0/d7/f68 122957047 1 2024-02-16T19:24:24.859 INFO:tasks.workunit.client.1.smithi163.stdout:5/304: truncate d0/d22/f2e 352645 0 2024-02-16T19:24:24.863 INFO:tasks.workunit.client.0.smithi040.stdout:3/961: unlink d1/d8c/d8e/dde/f111 0 2024-02-16T19:24:24.865 INFO:tasks.workunit.client.1.smithi163.stdout:4/266: rename fd to d11/d2d/d4f/f53 0 2024-02-16T19:24:24.865 INFO:tasks.workunit.client.1.smithi163.stdout:4/267: stat d11/d18/l25 0 2024-02-16T19:24:24.866 INFO:tasks.workunit.client.0.smithi040.stdout:4/714: write dd/de/d10/d36/d9e/d77/f7b [2708509,22679] 0 2024-02-16T19:24:24.866 INFO:tasks.workunit.client.1.smithi163.stdout:4/268: write d11/d18/f24 [654310,11626] 0 2024-02-16T19:24:24.867 INFO:tasks.workunit.client.0.smithi040.stdout:2/846: truncate d5/d7/d1e/d31/d4f/d63/f2c 1405626 0 2024-02-16T19:24:24.869 INFO:tasks.workunit.client.0.smithi040.stdout:5/884: link d1/da/d7e/f92 d1/da/d1c/da7/f112 0 2024-02-16T19:24:24.872 INFO:tasks.workunit.client.0.smithi040.stdout:3/962: symlink d1/dc/d35/d48/d85/l131 0 2024-02-16T19:24:24.874 INFO:tasks.workunit.client.1.smithi163.stdout:4/269: mknod d11/d18/c54 0 2024-02-16T19:24:24.878 INFO:tasks.workunit.client.0.smithi040.stdout:1/870: dwrite d17/d26/d9a/dfb/f6a [0,4194304] 0 2024-02-16T19:24:24.880 INFO:tasks.workunit.client.0.smithi040.stdout:1/871: symlink d17/d26/d5f/da6/l11a 0 2024-02-16T19:24:24.882 INFO:tasks.workunit.client.1.smithi163.stdout:5/305: write d0/f39 [2755921,11521] 0 2024-02-16T19:24:24.882 INFO:tasks.workunit.client.0.smithi040.stdout:1/872: creat d17/d26/d5f/d78/dac/f11b x:0 0 0 2024-02-16T19:24:24.882 INFO:tasks.workunit.client.0.smithi040.stdout:1/873: fsync d17/d29/d8b/d93/fa3 0 2024-02-16T19:24:24.883 INFO:tasks.workunit.client.0.smithi040.stdout:1/874: getdents d17/d26/d5f/d78/d7e 0 2024-02-16T19:24:24.883 INFO:tasks.workunit.client.0.smithi040.stdout:1/875: truncate d17/d3c/d74/d97/fc5 715556 0 2024-02-16T19:24:24.883 INFO:tasks.workunit.client.0.smithi040.stdout:1/876: write d17/d26/d4c/dab/fb1 [424559,54091] 0 2024-02-16T19:24:24.885 INFO:tasks.workunit.client.1.smithi163.stdout:5/306: mkdir d0/d2/d5c 0 2024-02-16T19:24:24.885 INFO:tasks.workunit.client.1.smithi163.stdout:5/307: chown d0/d2/l37 1146 1 2024-02-16T19:24:24.887 INFO:tasks.workunit.client.0.smithi040.stdout:1/877: dread d17/d26/d4c/f5b [0,4194304] 0 2024-02-16T19:24:24.887 INFO:tasks.workunit.client.0.smithi040.stdout:1/878: write d17/d26/d4c/f5b [558001,121809] 0 2024-02-16T19:24:24.888 INFO:tasks.workunit.client.1.smithi163.stdout:3/347: dwrite d0/dc/d1d/d49/f56 [0,4194304] 0 2024-02-16T19:24:24.888 INFO:tasks.workunit.client.1.smithi163.stdout:3/348: creat d0/dc/d1d/d49/f7d x:0 0 0 2024-02-16T19:24:24.888 INFO:tasks.workunit.client.0.smithi040.stdout:1/879: symlink d17/d29/d8b/l11c 0 2024-02-16T19:24:24.889 INFO:tasks.workunit.client.0.smithi040.stdout:5/885: dread d1/f24 [0,4194304] 0 2024-02-16T19:24:24.891 INFO:tasks.workunit.client.0.smithi040.stdout:1/880: mkdir d17/d26/d9a/dfb/d92/dca/d11d 0 2024-02-16T19:24:24.892 INFO:tasks.workunit.client.1.smithi163.stdout:1/292: dwrite d7/dc/d2d/d32/f64 [0,4194304] 0 2024-02-16T19:24:24.893 INFO:tasks.workunit.client.1.smithi163.stdout:1/293: read d7/f49 [250897,40518] 0 2024-02-16T19:24:24.894 INFO:tasks.workunit.client.0.smithi040.stdout:5/886: link d1/d3c/c4d d1/da/d1c/da7/c113 0 2024-02-16T19:24:24.894 INFO:tasks.workunit.client.0.smithi040.stdout:5/887: read - d1/da/d1c/da7/fd8 zero size 2024-02-16T19:24:24.894 INFO:tasks.workunit.client.0.smithi040.stdout:5/888: creat d1/d45/f114 x:0 0 0 2024-02-16T19:24:24.896 INFO:tasks.workunit.client.0.smithi040.stdout:1/881: mkdir d17/d26/d4c/dab/dd3/d11e 0 2024-02-16T19:24:24.898 INFO:tasks.workunit.client.1.smithi163.stdout:3/349: unlink d0/f19 0 2024-02-16T19:24:24.898 INFO:tasks.workunit.client.1.smithi163.stdout:3/350: fsync d0/dc/f28 0 2024-02-16T19:24:24.900 INFO:tasks.workunit.client.0.smithi040.stdout:5/889: symlink d1/da/d1c/dba/db1/dde/ddf/l115 0 2024-02-16T19:24:24.901 INFO:tasks.workunit.client.0.smithi040.stdout:1/882: rmdir d17/d26/d9a 39 2024-02-16T19:24:24.903 INFO:tasks.workunit.client.1.smithi163.stdout:1/294: creat d7/dc/d2d/d3f/f67 x:0 0 0 2024-02-16T19:24:24.904 INFO:tasks.workunit.client.0.smithi040.stdout:5/890: symlink d1/l116 0 2024-02-16T19:24:24.905 INFO:tasks.workunit.client.1.smithi163.stdout:3/351: symlink d0/dc/l7e 0 2024-02-16T19:24:24.905 INFO:tasks.workunit.client.1.smithi163.stdout:5/308: dread d0/d22/d28/f38 [0,4194304] 0 2024-02-16T19:24:24.907 INFO:tasks.workunit.client.1.smithi163.stdout:1/295: creat d7/d20/d21/d4e/f68 x:0 0 0 2024-02-16T19:24:24.908 INFO:tasks.workunit.client.1.smithi163.stdout:9/485: dwrite d5/de/d30/d3d/f3b [0,4194304] 0 2024-02-16T19:24:24.908 INFO:tasks.workunit.client.0.smithi040.stdout:4/715: dwrite dd/de/d10/f8a [0,4194304] 0 2024-02-16T19:24:24.909 INFO:tasks.workunit.client.0.smithi040.stdout:4/716: creat dd/d58/fdd x:0 0 0 2024-02-16T19:24:24.909 INFO:tasks.workunit.client.1.smithi163.stdout:3/352: mkdir d0/dc/d1c/d32/d7f 0 2024-02-16T19:24:24.909 INFO:tasks.workunit.client.1.smithi163.stdout:5/309: creat d0/d2/d5c/f5d x:0 0 0 2024-02-16T19:24:24.909 INFO:tasks.workunit.client.1.smithi163.stdout:5/310: fsync d0/d2/f4 0 2024-02-16T19:24:24.909 INFO:tasks.workunit.client.1.smithi163.stdout:5/311: write d0/d22/d28/d36/d3c/d4f/f50 [929724,124269] 0 2024-02-16T19:24:24.909 INFO:tasks.workunit.client.1.smithi163.stdout:5/312: chown d0/d22/f23 8390788 1 2024-02-16T19:24:24.911 INFO:tasks.workunit.client.0.smithi040.stdout:3/963: dwrite d1/dc/d35/d53/d64/d65/f98 [0,4194304] 0 2024-02-16T19:24:24.911 INFO:tasks.workunit.client.1.smithi163.stdout:4/270: dwrite d11/d18/d1f/d29/f49 [0,4194304] 0 2024-02-16T19:24:24.913 INFO:tasks.workunit.client.0.smithi040.stdout:2/847: dwrite d5/d5c/d65/f75 [0,4194304] 0 2024-02-16T19:24:24.913 INFO:tasks.workunit.client.0.smithi040.stdout:2/848: write d5/d7/f10b [355442,56350] 0 2024-02-16T19:24:24.913 INFO:tasks.workunit.client.0.smithi040.stdout:2/849: readlink d5/d7/d1e/d55/d8d/la8 0 2024-02-16T19:24:24.914 INFO:tasks.workunit.client.1.smithi163.stdout:3/353: dread d0/dc/d1c/f20 [0,4194304] 0 2024-02-16T19:24:24.914 INFO:tasks.workunit.client.0.smithi040.stdout:5/891: write d1/da/d1c/f37 [5987703,15322] 0 2024-02-16T19:24:24.916 INFO:tasks.workunit.client.1.smithi163.stdout:8/297: dwrite d0/d7/d18/d2e/f5d [0,4194304] 0 2024-02-16T19:24:24.916 INFO:tasks.workunit.client.1.smithi163.stdout:8/298: creat d0/dc/f6c x:0 0 0 2024-02-16T19:24:24.922 INFO:tasks.workunit.client.0.smithi040.stdout:2/850: mkdir d5/d7/d10/d10e/d11e 0 2024-02-16T19:24:24.924 INFO:tasks.workunit.client.0.smithi040.stdout:1/883: dwrite d17/db5/fd5 [0,4194304] 0 2024-02-16T19:24:24.927 INFO:tasks.workunit.client.0.smithi040.stdout:5/892: symlink d1/da/d1c/dba/dd6/l117 0 2024-02-16T19:24:24.927 INFO:tasks.workunit.client.0.smithi040.stdout:5/893: readlink d1/da/db0/lfc 0 2024-02-16T19:24:24.927 INFO:tasks.workunit.client.0.smithi040.stdout:5/894: readlink d1/da/l33 0 2024-02-16T19:24:24.927 INFO:tasks.workunit.client.0.smithi040.stdout:5/895: chown d1/da/d1c/dba/db1/dde/ddf/l115 6655 1 2024-02-16T19:24:24.930 INFO:tasks.workunit.client.1.smithi163.stdout:1/296: symlink d7/l69 0 2024-02-16T19:24:24.931 INFO:tasks.workunit.client.1.smithi163.stdout:1/297: read d7/d20/d2a/f40 [262027,41374] 0 2024-02-16T19:24:24.934 INFO:tasks.workunit.client.1.smithi163.stdout:1/298: write d7/f22 [494509,25917] 0 2024-02-16T19:24:24.938 INFO:tasks.workunit.client.0.smithi040.stdout:1/884: link d17/d26/d3d/f56 d17/d3c/d71/f11f 0 2024-02-16T19:24:24.939 INFO:tasks.workunit.client.1.smithi163.stdout:9/486: getdents d5/de/d72 0 2024-02-16T19:24:24.942 INFO:tasks.workunit.client.0.smithi040.stdout:5/896: rename d1/d45/f114 to d1/da/db0/f118 0 2024-02-16T19:24:24.942 INFO:tasks.workunit.client.0.smithi040.stdout:5/897: truncate d1/da/d1c/dba/db1/fdb 673169 0 2024-02-16T19:24:24.943 INFO:tasks.workunit.client.1.smithi163.stdout:3/354: stat d0/dc/l51 0 2024-02-16T19:24:24.948 INFO:tasks.workunit.client.1.smithi163.stdout:8/299: dwrite d0/d7/f6b [4194304,4194304] 0 2024-02-16T19:24:24.949 INFO:tasks.workunit.client.1.smithi163.stdout:8/300: truncate d0/dc/f6c 963353 0 2024-02-16T19:24:24.949 INFO:tasks.workunit.client.1.smithi163.stdout:8/301: fsync d0/d24/d27/f64 0 2024-02-16T19:24:24.949 INFO:tasks.workunit.client.1.smithi163.stdout:8/302: chown d0/dc 1564822499 1 2024-02-16T19:24:24.949 INFO:tasks.workunit.client.0.smithi040.stdout:2/851: dwrite d5/d7/d10/d40/f9a [0,4194304] 0 2024-02-16T19:24:24.950 INFO:tasks.workunit.client.1.smithi163.stdout:4/271: dwrite d11/d18/d1f/d29/f49 [0,4194304] 0 2024-02-16T19:24:24.950 INFO:tasks.workunit.client.1.smithi163.stdout:4/272: dread - d11/d18/d1f/d23/d3a/d4a/f4e zero size 2024-02-16T19:24:24.950 INFO:tasks.workunit.client.1.smithi163.stdout:5/313: truncate d0/d22/f2e 623343 0 2024-02-16T19:24:24.953 INFO:tasks.workunit.client.1.smithi163.stdout:1/299: creat d7/d5b/f6a x:0 0 0 2024-02-16T19:24:24.956 INFO:tasks.workunit.client.1.smithi163.stdout:9/487: symlink d5/de/d30/d3d/d49/d50/lb3 0 2024-02-16T19:24:24.956 INFO:tasks.workunit.client.1.smithi163.stdout:9/488: creat d5/de/d16/d47/fb4 x:0 0 0 2024-02-16T19:24:24.956 INFO:tasks.workunit.client.0.smithi040.stdout:1/885: symlink d17/d26/d5f/d63/l120 0 2024-02-16T19:24:24.957 INFO:tasks.workunit.client.1.smithi163.stdout:3/355: mkdir d0/dc/d50/d66/d80 0 2024-02-16T19:24:24.957 INFO:tasks.workunit.client.1.smithi163.stdout:3/356: read - d0/dc/d1d/d29/f72 zero size 2024-02-16T19:24:24.958 INFO:tasks.workunit.client.0.smithi040.stdout:5/898: truncate d1/d45/fe6 1665337 0 2024-02-16T19:24:24.966 INFO:tasks.workunit.client.0.smithi040.stdout:1/886: dread d17/d26/d9a/dfb/f5d [0,4194304] 0 2024-02-16T19:24:24.967 INFO:tasks.workunit.client.0.smithi040.stdout:2/852: rename d5/d7/f10b to d5/d7/d1e/d55/d102/f11f 0 2024-02-16T19:24:24.967 INFO:tasks.workunit.client.0.smithi040.stdout:2/853: creat d5/d7/d10/d40/f120 x:0 0 0 2024-02-16T19:24:24.967 INFO:tasks.workunit.client.0.smithi040.stdout:2/854: dread - d5/d7/ffa zero size 2024-02-16T19:24:24.971 INFO:tasks.workunit.client.0.smithi040.stdout:5/899: rename d1/da/d1c/d73/lef to d1/da/d1c/da7/l119 0 2024-02-16T19:24:24.971 INFO:tasks.workunit.client.0.smithi040.stdout:5/900: creat d1/da/d1c/f11a x:0 0 0 2024-02-16T19:24:24.975 INFO:tasks.workunit.client.0.smithi040.stdout:1/887: write d17/d26/d4c/f8a [3433327,23724] 0 2024-02-16T19:24:24.976 INFO:tasks.workunit.client.1.smithi163.stdout:3/357: dwrite d0/dc/d50/f58 [0,4194304] 0 2024-02-16T19:24:24.976 INFO:tasks.workunit.client.1.smithi163.stdout:3/358: chown d0/d3e/c47 0 1 2024-02-16T19:24:24.977 INFO:tasks.workunit.client.0.smithi040.stdout:2/855: creat d5/d7/d12/d1b/f121 x:0 0 0 2024-02-16T19:24:24.977 INFO:tasks.workunit.client.0.smithi040.stdout:2/856: chown d5/d5c/d65/d9d 489081452 1 2024-02-16T19:24:24.978 INFO:tasks.workunit.client.1.smithi163.stdout:5/314: creat d0/d22/d28/d36/f5e x:0 0 0 2024-02-16T19:24:24.980 INFO:tasks.workunit.client.1.smithi163.stdout:4/273: dread d11/f14 [0,4194304] 0 2024-02-16T19:24:24.982 INFO:tasks.workunit.client.0.smithi040.stdout:5/901: rename d1/da/d1c/d21/daa/fe4 to d1/da/d1c/dba/dd6/f11b 0 2024-02-16T19:24:24.987 INFO:tasks.workunit.client.1.smithi163.stdout:9/489: symlink d5/de/d30/d8a/da8/lb5 0 2024-02-16T19:24:24.987 INFO:tasks.workunit.client.0.smithi040.stdout:1/888: mkdir d17/d29/d8b/d10e/d121 0 2024-02-16T19:24:24.988 INFO:tasks.workunit.client.1.smithi163.stdout:8/303: getdents d0/d7 0 2024-02-16T19:24:24.989 INFO:tasks.workunit.client.0.smithi040.stdout:5/902: mknod d1/da/d1c/c11c 0 2024-02-16T19:24:24.990 INFO:tasks.workunit.client.0.smithi040.stdout:5/903: creat d1/f11d x:0 0 0 2024-02-16T19:24:24.990 INFO:tasks.workunit.client.1.smithi163.stdout:3/359: truncate d0/dc/d1d/d29/f30 7352405 0 2024-02-16T19:24:24.991 INFO:tasks.workunit.client.1.smithi163.stdout:5/315: symlink d0/d2/d55/l5f 0 2024-02-16T19:24:24.994 INFO:tasks.workunit.client.0.smithi040.stdout:5/904: dread d1/d62/d9a/fa2 [0,4194304] 0 2024-02-16T19:24:24.995 INFO:tasks.workunit.client.1.smithi163.stdout:1/300: rename d7/f15 to d7/f6b 0 2024-02-16T19:24:24.996 INFO:tasks.workunit.client.0.smithi040.stdout:2/857: dwrite d5/d7/d1e/d4c/d52/dc1/f112 [0,4194304] 0 2024-02-16T19:24:24.996 INFO:tasks.workunit.client.1.smithi163.stdout:9/490: rename d5/de/d30/d3d/d49/d6d/d97/ca3 to d5/de/d84/cb6 0 2024-02-16T19:24:24.999 INFO:tasks.workunit.client.0.smithi040.stdout:5/905: unlink d1/d3c/f68 0 2024-02-16T19:24:24.999 INFO:tasks.workunit.client.0.smithi040.stdout:5/906: truncate d1/da/d1c/dba/db1/dde/ddf/fe8 1804760 0 2024-02-16T19:24:25.000 INFO:tasks.workunit.client.1.smithi163.stdout:4/274: dread d11/f26 [0,4194304] 0 2024-02-16T19:24:25.000 INFO:tasks.workunit.client.1.smithi163.stdout:4/275: write d11/d18/d1f/f20 [4579660,34883] 0 2024-02-16T19:24:25.000 INFO:tasks.workunit.client.1.smithi163.stdout:8/304: link d0/dc/f6c d0/dc/f6d 0 2024-02-16T19:24:25.001 INFO:tasks.workunit.client.0.smithi040.stdout:1/889: read d17/d3c/f80 [2260934,51854] 0 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:0/538: sync 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:6/276: sync 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:2/465: sync 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:7/315: sync 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:0/539: readlink d11/l37 0 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:6/277: stat d0/d6/d9/d15/c1f 0 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:2/466: fsync d2/d2a/d39/d66/d6c/f70 0 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:0/540: truncate d11/d22/d7f/fb6 1167033 0 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:0/541: fsync d11/d22/f34 0 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:7/316: write d11/d24/d49/f51 [1132621,34954] 0 2024-02-16T19:24:25.004 INFO:tasks.workunit.client.1.smithi163.stdout:2/467: write d2/f23 [1310236,24025] 0 2024-02-16T19:24:25.005 INFO:tasks.workunit.client.1.smithi163.stdout:2/468: chown d2/d2a/d39/d66/d6c 9006 1 2024-02-16T19:24:25.005 INFO:tasks.workunit.client.1.smithi163.stdout:2/469: chown d2/d26/f5d 5301654 1 2024-02-16T19:24:25.005 INFO:tasks.workunit.client.1.smithi163.stdout:1/301: rmdir d7/dc/d2d 39 2024-02-16T19:24:25.005 INFO:tasks.workunit.client.1.smithi163.stdout:1/302: creat d7/d20/d21/f6c x:0 0 0 2024-02-16T19:24:25.005 INFO:tasks.workunit.client.0.smithi040.stdout:9/874: sync 2024-02-16T19:24:25.005 INFO:tasks.workunit.client.1.smithi163.stdout:7/317: dread d11/f1b [0,4194304] 0 2024-02-16T19:24:25.006 INFO:tasks.workunit.client.1.smithi163.stdout:9/491: truncate d5/de/d30/d3d/f88 3806554 0 2024-02-16T19:24:25.008 INFO:tasks.workunit.client.0.smithi040.stdout:5/907: truncate d1/da/d1c/dba/db1/f7b 2395996 0 2024-02-16T19:24:25.009 INFO:tasks.workunit.client.1.smithi163.stdout:4/276: getdents d11/d18/d1f/d23/d3a/d4a 0 2024-02-16T19:24:25.018 INFO:tasks.workunit.client.1.smithi163.stdout:6/278: mkdir d0/d18/d51 0 2024-02-16T19:24:25.018 INFO:tasks.workunit.client.1.smithi163.stdout:6/279: read - d0/d6/d9/d15/f4d zero size 2024-02-16T19:24:25.023 INFO:tasks.workunit.client.1.smithi163.stdout:1/303: creat d7/d20/f6d x:0 0 0 2024-02-16T19:24:25.024 INFO:tasks.workunit.client.1.smithi163.stdout:4/277: creat d11/d2d/d4f/f55 x:0 0 0 2024-02-16T19:24:25.026 INFO:tasks.workunit.client.1.smithi163.stdout:8/305: link d0/d7/d19/d1f/d30/l3a d0/d7/d19/l6e 0 2024-02-16T19:24:25.027 INFO:tasks.workunit.client.1.smithi163.stdout:8/306: stat d0/d24/f60 0 2024-02-16T19:24:25.027 INFO:tasks.workunit.client.1.smithi163.stdout:8/307: write d0/d7/d18/d2e/f5f [290276,78759] 0 2024-02-16T19:24:25.028 INFO:tasks.workunit.client.1.smithi163.stdout:6/280: rename d0/d6/df/f41 to d0/d6/df/d20/d40/d1e/f52 0 2024-02-16T19:24:25.030 INFO:tasks.workunit.client.1.smithi163.stdout:1/304: link d7/dc/d2d/c61 d7/d20/c6e 0 2024-02-16T19:24:25.031 INFO:tasks.workunit.client.1.smithi163.stdout:4/278: write d11/f41 [201607,35881] 0 2024-02-16T19:24:25.032 INFO:tasks.workunit.client.0.smithi040.stdout:1/890: rename d17/d26/d4c to d17/d29/d8b/d10e/d121/d122 0 2024-02-16T19:24:25.033 INFO:tasks.workunit.client.1.smithi163.stdout:6/281: mkdir d0/d6/df/d20/d40/d53 0 2024-02-16T19:24:25.033 INFO:tasks.workunit.client.1.smithi163.stdout:6/282: fdatasync d0/d6/df/f2a 0 2024-02-16T19:24:25.033 INFO:tasks.workunit.client.1.smithi163.stdout:6/283: chown d0/dd/f50 55031 1 2024-02-16T19:24:25.033 INFO:tasks.workunit.client.1.smithi163.stdout:6/284: fsync d0/d6/d9/d2e/d33/f3c 0 2024-02-16T19:24:25.033 INFO:tasks.workunit.client.0.smithi040.stdout:1/891: rmdir d17/d3c/dde 39 2024-02-16T19:24:25.037 INFO:tasks.workunit.client.0.smithi040.stdout:1/892: mknod d17/d26/d9a/dfb/d92/dca/d11d/c123 0 2024-02-16T19:24:25.038 INFO:tasks.workunit.client.1.smithi163.stdout:6/285: unlink d0/d18/f22 0 2024-02-16T19:24:25.038 INFO:tasks.workunit.client.1.smithi163.stdout:6/286: readlink d0/d6/l1b 0 2024-02-16T19:24:25.038 INFO:tasks.workunit.client.1.smithi163.stdout:6/287: chown d0/d6/df/d20/l3b 450691 1 2024-02-16T19:24:25.038 INFO:tasks.workunit.client.0.smithi040.stdout:1/893: symlink d17/d26/d9a/dfb/l124 0 2024-02-16T19:24:25.040 INFO:tasks.workunit.client.0.smithi040.stdout:1/894: rename d17/f1f to d17/d3c/d74/d97/da9/d10b/f125 0 2024-02-16T19:24:25.041 INFO:tasks.workunit.client.0.smithi040.stdout:1/895: mkdir d17/d3c/d74/d126 0 2024-02-16T19:24:25.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/858: dread d5/d7/d1e/f80 [0,4194304] 0 2024-02-16T19:24:25.043 INFO:tasks.workunit.client.0.smithi040.stdout:2/859: dread - d5/d7/d12/da6/d114/f118 zero size 2024-02-16T19:24:25.045 INFO:tasks.workunit.client.0.smithi040.stdout:1/896: unlink d17/d26/d9a/le0 0 2024-02-16T19:24:25.047 INFO:tasks.workunit.client.0.smithi040.stdout:2/860: mknod d5/d7/d1e/d31/d4f/d63/c122 0 2024-02-16T19:24:25.047 INFO:tasks.workunit.client.0.smithi040.stdout:2/861: chown d5/cd0 5 1 2024-02-16T19:24:25.048 INFO:tasks.workunit.client.0.smithi040.stdout:1/897: creat d17/d3c/d74/d97/da9/d10b/f127 x:0 0 0 2024-02-16T19:24:25.048 INFO:tasks.workunit.client.0.smithi040.stdout:1/898: fsync d17/d26/d9a/dfb/fef 0 2024-02-16T19:24:25.048 INFO:tasks.workunit.client.0.smithi040.stdout:2/862: mkdir d5/d7/d12/dca/dea/d123 0 2024-02-16T19:24:25.049 INFO:tasks.workunit.client.0.smithi040.stdout:1/899: rename d17/d3c/d74/d97/da9/d10b/f127 to d17/d3c/f128 0 2024-02-16T19:24:25.050 INFO:tasks.workunit.client.0.smithi040.stdout:2/863: rmdir d5/d7/d1e/d55/dd4 39 2024-02-16T19:24:25.051 INFO:tasks.workunit.client.0.smithi040.stdout:1/900: unlink d17/d29/d8b/d93/fdd 0 2024-02-16T19:24:25.053 INFO:tasks.workunit.client.1.smithi163.stdout:6/288: write d0/d6/f1a [3472856,113768] 0 2024-02-16T19:24:25.053 INFO:tasks.workunit.client.1.smithi163.stdout:6/289: write d0/d6/f8 [4076512,125212] 0 2024-02-16T19:24:25.056 INFO:tasks.workunit.client.1.smithi163.stdout:6/290: symlink d0/d6/df/d30/l54 0 2024-02-16T19:24:25.060 INFO:tasks.workunit.client.1.smithi163.stdout:6/291: write d0/d6/d9/f2d [4253410,37299] 0 2024-02-16T19:24:25.061 INFO:tasks.workunit.client.1.smithi163.stdout:0/542: dwrite d11/d22/d7f/d8e/fa4 [0,4194304] 0 2024-02-16T19:24:25.064 INFO:tasks.workunit.client.1.smithi163.stdout:0/543: mknod d11/d12/d67/d6a/cc1 0 2024-02-16T19:24:25.064 INFO:tasks.workunit.client.1.smithi163.stdout:0/544: readlink d11/d12/d20/l2e 0 2024-02-16T19:24:25.077 INFO:tasks.workunit.client.1.smithi163.stdout:1/305: dwrite d7/dc/d2d/d32/f51 [0,4194304] 0 2024-02-16T19:24:25.085 INFO:tasks.workunit.client.1.smithi163.stdout:2/470: dwrite d2/f1c [0,4194304] 0 2024-02-16T19:24:25.086 INFO:tasks.workunit.client.1.smithi163.stdout:9/492: dwrite d5/de/d30/d3d/d49/f62 [0,4194304] 0 2024-02-16T19:24:25.086 INFO:tasks.workunit.client.1.smithi163.stdout:9/493: stat d5/de/d16/d47/f4c 0 2024-02-16T19:24:25.086 INFO:tasks.workunit.client.0.smithi040.stdout:9/875: dwrite d3/d4/d11/d29/d36/f3a [0,4194304] 0 2024-02-16T19:24:25.086 INFO:tasks.workunit.client.0.smithi040.stdout:9/876: write d3/d4/f2c [4518639,103366] 0 2024-02-16T19:24:25.086 INFO:tasks.workunit.client.0.smithi040.stdout:9/877: fdatasync d3/d4/d11/d29/d38/d46/d6b/d85/d112/d8a/f9f 0 2024-02-16T19:24:25.088 INFO:tasks.workunit.client.1.smithi163.stdout:2/471: symlink d2/d44/la1 0 2024-02-16T19:24:25.089 INFO:tasks.workunit.client.1.smithi163.stdout:1/306: dread d7/dc/d2d/d32/f51 [0,4194304] 0 2024-02-16T19:24:25.090 INFO:tasks.workunit.client.0.smithi040.stdout:2/864: dwrite d5/d7/d1e/f6d [0,4194304] 0 2024-02-16T19:24:25.090 INFO:tasks.workunit.client.0.smithi040.stdout:1/901: dwrite d17/d29/d8b/d10e/d121/d122/dab/dd3/ff3 [0,4194304] 0 2024-02-16T19:24:25.090 INFO:tasks.workunit.client.0.smithi040.stdout:5/908: dwrite d1/d3c/f77 [0,4194304] 0 2024-02-16T19:24:25.091 INFO:tasks.workunit.client.1.smithi163.stdout:4/279: dwrite d11/f17 [0,4194304] 0 2024-02-16T19:24:25.091 INFO:tasks.workunit.client.1.smithi163.stdout:6/292: dwrite d0/d6/d9/d2e/f4f [0,4194304] 0 2024-02-16T19:24:25.092 INFO:tasks.workunit.client.1.smithi163.stdout:0/545: dwrite f8 [0,4194304] 0 2024-02-16T19:24:25.093 INFO:tasks.workunit.client.1.smithi163.stdout:3/360: truncate d0/dc/d50/f58 328019 0 2024-02-16T19:24:25.094 INFO:tasks.workunit.client.1.smithi163.stdout:9/494: symlink d5/de/d30/d3d/d49/d6d/lb7 0 2024-02-16T19:24:25.097 INFO:tasks.workunit.client.1.smithi163.stdout:1/307: mknod d7/d20/d28/c6f 0 2024-02-16T19:24:25.099 INFO:tasks.workunit.client.1.smithi163.stdout:2/472: dread d2/ff [8388608,4194304] 0 2024-02-16T19:24:25.099 INFO:tasks.workunit.client.1.smithi163.stdout:2/473: truncate d2/d2a/d39/d3a/f5b 917591 0 2024-02-16T19:24:25.099 INFO:tasks.workunit.client.1.smithi163.stdout:2/474: dread - d2/d2a/d39/d66/f9a zero size 2024-02-16T19:24:25.099 INFO:tasks.workunit.client.1.smithi163.stdout:2/475: stat d2/d2a/d39 0 2024-02-16T19:24:25.099 INFO:tasks.workunit.client.1.smithi163.stdout:2/476: stat d2/d2a/d2d/f6e 0 2024-02-16T19:24:25.101 INFO:tasks.workunit.client.0.smithi040.stdout:2/865: creat d5/d7/d10/d10e/f124 x:0 0 0 2024-02-16T19:24:25.102 INFO:tasks.workunit.client.1.smithi163.stdout:4/280: truncate d11/d18/f24 452428 0 2024-02-16T19:24:25.102 INFO:tasks.workunit.client.1.smithi163.stdout:4/281: chown d11/d18/c54 1655731 1 2024-02-16T19:24:25.104 INFO:tasks.workunit.client.0.smithi040.stdout:9/878: dwrite d3/d4/d11/d29/d36/f75 [0,4194304] 0 2024-02-16T19:24:25.104 INFO:tasks.workunit.client.1.smithi163.stdout:4/282: dread d11/f26 [0,4194304] 0 2024-02-16T19:24:25.115 INFO:tasks.workunit.client.1.smithi163.stdout:4/283: dread d11/d18/d1f/d29/f31 [0,4194304] 0 2024-02-16T19:24:25.115 INFO:tasks.workunit.client.1.smithi163.stdout:6/293: truncate d0/d6/df/d20/d40/d1e/f52 149847 0 2024-02-16T19:24:25.115 INFO:tasks.workunit.client.1.smithi163.stdout:6/294: read - d0/d6/df/d20/d40/d17/f4b zero size 2024-02-16T19:24:25.115 INFO:tasks.workunit.client.1.smithi163.stdout:6/295: rename d0/d6/d9/d2e to d0/d6/d9/d2e/d33/d55 22 2024-02-16T19:24:25.120 INFO:tasks.workunit.client.0.smithi040.stdout:2/866: dwrite d5/d5c/d65/f60 [4194304,4194304] 0 2024-02-16T19:24:25.120 INFO:tasks.workunit.client.0.smithi040.stdout:2/867: fdatasync d5/d7/d1e/d55/f6f 0 2024-02-16T19:24:25.121 INFO:tasks.workunit.client.0.smithi040.stdout:1/902: mknod d17/d29/c129 0 2024-02-16T19:24:25.131 INFO:tasks.workunit.client.0.smithi040.stdout:9/879: mkdir d3/d4/d4a/d114 0 2024-02-16T19:24:25.132 INFO:tasks.workunit.client.1.smithi163.stdout:6/296: dwrite d0/d6/f31 [4194304,4194304] 0 2024-02-16T19:24:25.133 INFO:tasks.workunit.client.0.smithi040.stdout:2/868: link d5/d7/d10/df3/d42/fc8 d5/d7/d1e/f125 0 2024-02-16T19:24:25.135 INFO:tasks.workunit.client.0.smithi040.stdout:1/903: getdents d17/d26 0 2024-02-16T19:24:25.135 INFO:tasks.workunit.client.0.smithi040.stdout:1/904: readlink d17/d26/d3d/ld7 0 2024-02-16T19:24:25.137 INFO:tasks.workunit.client.1.smithi163.stdout:3/361: creat d0/dc/d1d/d29/d5d/f81 x:0 0 0 2024-02-16T19:24:25.137 INFO:tasks.workunit.client.1.smithi163.stdout:3/362: write d0/dc/d1c/d21/f2d [9139,61266] 0 2024-02-16T19:24:25.139 INFO:tasks.workunit.client.1.smithi163.stdout:9/495: unlink d5/de/d30/d31/l42 0 2024-02-16T19:24:25.139 INFO:tasks.workunit.client.1.smithi163.stdout:9/496: fsync d5/de/d30/d3d/d49/d6d/f96 0 2024-02-16T19:24:25.140 INFO:tasks.workunit.client.1.smithi163.stdout:1/308: symlink d7/dc/d2d/l70 0 2024-02-16T19:24:25.141 INFO:tasks.workunit.client.0.smithi040.stdout:5/909: rmdir d1/da/d1c/d21 39 2024-02-16T19:24:25.143 INFO:tasks.workunit.client.1.smithi163.stdout:2/477: creat d2/d2a/d2d/d73/fa2 x:0 0 0 2024-02-16T19:24:25.143 INFO:tasks.workunit.client.1.smithi163.stdout:2/478: truncate d2/d2a/f5f 1165640 0 2024-02-16T19:24:25.145 INFO:tasks.workunit.client.0.smithi040.stdout:9/880: link d3/l37 d3/d1c/d73/l115 0 2024-02-16T19:24:25.147 INFO:tasks.workunit.client.0.smithi040.stdout:2/869: creat d5/d7/d1e/d4c/d52/d81/f126 x:0 0 0 2024-02-16T19:24:25.148 INFO:tasks.workunit.client.1.smithi163.stdout:4/284: stat d11/f4b 0 2024-02-16T19:24:25.148 INFO:tasks.workunit.client.1.smithi163.stdout:4/285: readlink d11/d18/l48 0 2024-02-16T19:24:25.148 INFO:tasks.workunit.client.1.smithi163.stdout:4/286: dread - d11/d18/d1f/d23/d3a/f3f zero size 2024-02-16T19:24:25.150 INFO:tasks.workunit.client.0.smithi040.stdout:5/910: rename d1/l116 to d1/da/dd5/dfb/l11e 0 2024-02-16T19:24:25.152 INFO:tasks.workunit.client.1.smithi163.stdout:6/297: creat d0/d6/d9/d15/d1c/f56 x:0 0 0 2024-02-16T19:24:25.154 INFO:tasks.workunit.client.0.smithi040.stdout:9/881: creat d3/d4/d11/d71/de8/f116 x:0 0 0 2024-02-16T19:24:25.154 INFO:tasks.workunit.client.0.smithi040.stdout:9/882: chown d3/d4/d11/d71/da8/fd2 212772063 1 2024-02-16T19:24:25.154 INFO:tasks.workunit.client.0.smithi040.stdout:2/870: dread d5/d7/d1e/d4c/d52/f9c [0,4194304] 0 2024-02-16T19:24:25.154 INFO:tasks.workunit.client.0.smithi040.stdout:2/871: dread - d5/d7/d1e/d31/d4f/d63/d8b/f108 zero size 2024-02-16T19:24:25.156 INFO:tasks.workunit.client.0.smithi040.stdout:5/911: mkdir d1/da/d1c/dba/db1/dde/d11f 0 2024-02-16T19:24:25.156 INFO:tasks.workunit.client.0.smithi040.stdout:7/792: sync 2024-02-16T19:24:25.160 INFO:tasks.workunit.client.0.smithi040.stdout:5/912: mkdir d1/da/d1c/dba/db1/d120 0 2024-02-16T19:24:25.161 INFO:tasks.workunit.client.0.smithi040.stdout:9/883: write d3/d4/f57 [2056782,5017] 0 2024-02-16T19:24:25.161 INFO:tasks.workunit.client.0.smithi040.stdout:9/884: truncate d3/d4/d4a/f62 672377 0 2024-02-16T19:24:25.161 INFO:tasks.workunit.client.0.smithi040.stdout:9/885: chown d3/d4/d4a/db4/df7/daf/fd1 1384 1 2024-02-16T19:24:25.161 INFO:tasks.workunit.client.0.smithi040.stdout:9/886: dread - d3/d1c/f64 zero size 2024-02-16T19:24:25.162 INFO:tasks.workunit.client.0.smithi040.stdout:7/793: mkdir d21/d8b/d9c/d10e 0 2024-02-16T19:24:25.163 INFO:tasks.workunit.client.0.smithi040.stdout:2/872: getdents d5/d7/d12/d1b/dae 0 2024-02-16T19:24:25.164 INFO:tasks.workunit.client.1.smithi163.stdout:0/546: rmdir d11/d21 39 2024-02-16T19:24:25.165 INFO:tasks.workunit.client.0.smithi040.stdout:5/913: creat d1/d45/dab/f121 x:0 0 0 2024-02-16T19:24:25.166 INFO:tasks.workunit.client.1.smithi163.stdout:3/363: mknod d0/dc/d1d/d29/d5d/c82 0 2024-02-16T19:24:25.168 INFO:tasks.workunit.client.0.smithi040.stdout:9/887: creat d3/d4/d4a/db4/df7/f117 x:0 0 0 2024-02-16T19:24:25.168 INFO:tasks.workunit.client.0.smithi040.stdout:9/888: chown d3/d4/d11/d29/d38/d46/d6b/d85/d112/d63/fb0 133 1 2024-02-16T19:24:25.169 INFO:tasks.workunit.client.1.smithi163.stdout:9/497: mkdir d5/de/d30/d3d/d49/d6d/db8 0 2024-02-16T19:24:25.169 INFO:tasks.workunit.client.0.smithi040.stdout:3/964: sync 2024-02-16T19:24:25.169 INFO:tasks.workunit.client.0.smithi040.stdout:4/717: sync 2024-02-16T19:24:25.170 INFO:tasks.workunit.client.0.smithi040.stdout:3/965: fsync d1/dc/d42/d5c/f6d 0 2024-02-16T19:24:25.170 INFO:tasks.workunit.client.0.smithi040.stdout:7/794: rmdir d21/d4d/d77/d87/d9b/df2/d100 0 2024-02-16T19:24:25.171 INFO:tasks.workunit.client.1.smithi163.stdout:1/309: mknod d7/dc/c71 0 2024-02-16T19:24:25.172 INFO:tasks.workunit.client.0.smithi040.stdout:2/873: creat d5/d7/d12/dca/f127 x:0 0 0 2024-02-16T19:24:25.175 INFO:tasks.workunit.client.0.smithi040.stdout:5/914: symlink d1/da/d1c/d21/l122 0 2024-02-16T19:24:25.177 INFO:tasks.workunit.client.1.smithi163.stdout:9/498: read d5/de/f5f [3971753,100846] 0 2024-02-16T19:24:25.177 INFO:tasks.workunit.client.1.smithi163.stdout:9/499: chown d5/de/c24 190586618 1 2024-02-16T19:24:25.180 INFO:tasks.workunit.client.0.smithi040.stdout:3/966: truncate d1/d4/d6e/dc2/dce/fd8 682497 0 2024-02-16T19:24:25.183 INFO:tasks.workunit.client.1.smithi163.stdout:2/479: rename d2/d2a/d39/d66/d6c/f70 to d2/d26/d54/d58/d98/fa3 0 2024-02-16T19:24:25.187 INFO:tasks.workunit.client.0.smithi040.stdout:4/718: rmdir dd/de/d10 39 2024-02-16T19:24:25.188 INFO:tasks.workunit.client.0.smithi040.stdout:5/915: mkdir d1/d62/dbb/d123 0 2024-02-16T19:24:25.190 INFO:tasks.workunit.client.1.smithi163.stdout:6/298: getdents d0/d6/df/d20/d40/d1e 0 2024-02-16T19:24:25.193 INFO:tasks.workunit.client.0.smithi040.stdout:3/967: rename d1/le to d1/dc/d42/d58/de8/l132 0 2024-02-16T19:24:25.193 INFO:tasks.workunit.client.0.smithi040.stdout:3/968: chown d1/dc/d42/d58/f76 3707 1 2024-02-16T19:24:25.195 INFO:tasks.workunit.client.0.smithi040.stdout:7/795: dwrite d21/d8b/d9c/fff [0,4194304] 0 2024-02-16T19:24:25.195 INFO:tasks.workunit.client.0.smithi040.stdout:2/874: dwrite d5/d7/d10/df3/f69 [0,4194304] 0 2024-02-16T19:24:25.195 INFO:tasks.workunit.client.0.smithi040.stdout:2/875: chown d5/d7/d1e/d31/fb8 45 1 2024-02-16T19:24:25.196 INFO:tasks.workunit.client.0.smithi040.stdout:4/719: truncate fb 2757828 0 2024-02-16T19:24:25.201 INFO:tasks.workunit.client.0.smithi040.stdout:5/916: creat d1/da/d7e/f124 x:0 0 0 2024-02-16T19:24:25.201 INFO:tasks.workunit.client.0.smithi040.stdout:5/917: fsync d1/da/d1c/da9/fd7 0 2024-02-16T19:24:25.201 INFO:tasks.workunit.client.0.smithi040.stdout:5/918: creat d1/d45/f125 x:0 0 0 2024-02-16T19:24:25.203 INFO:tasks.workunit.client.0.smithi040.stdout:3/969: symlink d1/d7/d33/l133 0 2024-02-16T19:24:25.204 INFO:tasks.workunit.client.1.smithi163.stdout:0/547: symlink d11/d4f/lc2 0 2024-02-16T19:24:25.204 INFO:tasks.workunit.client.1.smithi163.stdout:0/548: chown d11/d22/d3e/c43 27967415 1 2024-02-16T19:24:25.206 INFO:tasks.workunit.client.1.smithi163.stdout:1/310: symlink d7/dc/d2d/d32/d3b/l72 0 2024-02-16T19:24:25.206 INFO:tasks.workunit.client.1.smithi163.stdout:1/311: chown d7/dc/d2d/d32/d3b/f50 119366257 1 2024-02-16T19:24:25.211 INFO:tasks.workunit.client.0.smithi040.stdout:7/796: mkdir d21/d7e/df8/db7/de7/d10f 0 2024-02-16T19:24:25.211 INFO:tasks.workunit.client.0.smithi040.stdout:7/797: write d21/d29/d79/fc4 [966061,21062] 0 2024-02-16T19:24:25.211 INFO:tasks.workunit.client.0.smithi040.stdout:7/798: creat d21/d29/dad/f110 x:0 0 0 2024-02-16T19:24:25.215 INFO:tasks.workunit.client.1.smithi163.stdout:3/364: rename d0/dc/d1d/d29/d38/d46/d74 to d0/d83 0 2024-02-16T19:24:25.216 INFO:tasks.workunit.client.1.smithi163.stdout:4/287: dwrite d11/f2f [0,4194304] 0 2024-02-16T19:24:25.217 INFO:tasks.workunit.client.1.smithi163.stdout:0/549: symlink d11/d22/d61/lc3 0 2024-02-16T19:24:25.217 INFO:tasks.workunit.client.1.smithi163.stdout:0/550: stat d11/d21/f46 0 2024-02-16T19:24:25.217 INFO:tasks.workunit.client.1.smithi163.stdout:5/316: sync 2024-02-16T19:24:25.218 INFO:tasks.workunit.client.0.smithi040.stdout:2/876: getdents d5/d7/d1e/d4c 0 2024-02-16T19:24:25.218 INFO:tasks.workunit.client.0.smithi040.stdout:2/877: creat d5/d7/d1e/d31/d4f/d63/d8b/f128 x:0 0 0 2024-02-16T19:24:25.219 INFO:tasks.workunit.client.1.smithi163.stdout:1/312: symlink d7/dc/d2d/l73 0 2024-02-16T19:24:25.220 INFO:tasks.workunit.client.1.smithi163.stdout:6/299: dwrite d0/d6/df/f2a [0,4194304] 0 2024-02-16T19:24:25.220 INFO:tasks.workunit.client.1.smithi163.stdout:6/300: write d0/d6/df/d20/d40/d1e/f52 [914968,89076] 0 2024-02-16T19:24:25.220 INFO:tasks.workunit.client.1.smithi163.stdout:6/301: truncate d0/d6/df/d20/d40/d1e/f2c 168822 0 2024-02-16T19:24:25.222 INFO:tasks.workunit.client.1.smithi163.stdout:9/500: rmdir d5/de/d16/d46 39 2024-02-16T19:24:25.224 INFO:tasks.workunit.client.0.smithi040.stdout:4/720: link dd/de/d10/d36/d9e/c4b dd/de/daa/cde 0 2024-02-16T19:24:25.224 INFO:tasks.workunit.client.0.smithi040.stdout:4/721: read - dd/de/d10/d36/f6c zero size 2024-02-16T19:24:25.224 INFO:tasks.workunit.client.0.smithi040.stdout:4/722: dread - dd/de/d71/fa3 zero size 2024-02-16T19:24:25.225 INFO:tasks.workunit.client.0.smithi040.stdout:4/723: chown dd/de/d10/d36/dad/fb4 300 1 2024-02-16T19:24:25.225 INFO:tasks.workunit.client.0.smithi040.stdout:5/919: dwrite d1/da/d1c/f25 [0,4194304] 0 2024-02-16T19:24:25.225 INFO:tasks.workunit.client.0.smithi040.stdout:5/920: truncate d1/da/d1c/f94 1037200 0 2024-02-16T19:24:25.225 INFO:tasks.workunit.client.0.smithi040.stdout:5/921: stat d1/d62/d9a/fa2 0 2024-02-16T19:24:25.225 INFO:tasks.workunit.client.0.smithi040.stdout:5/922: stat d1/da/l2d 0 2024-02-16T19:24:25.227 INFO:tasks.workunit.client.1.smithi163.stdout:2/480: rename d2/d2a/d2d/f6b to d2/d26/d54/d58/d98/fa4 0 2024-02-16T19:24:25.227 INFO:tasks.workunit.client.1.smithi163.stdout:2/481: dread - d2/d26/d54/f9b zero size 2024-02-16T19:24:25.227 INFO:tasks.workunit.client.0.smithi040.stdout:5/923: dread d1/da/d1c/d73/f104 [0,4194304] 0 2024-02-16T19:24:25.229 INFO:tasks.workunit.client.1.smithi163.stdout:8/308: sync 2024-02-16T19:24:25.229 INFO:tasks.workunit.client.1.smithi163.stdout:7/318: sync 2024-02-16T19:24:25.229 INFO:tasks.workunit.client.1.smithi163.stdout:7/319: fdatasync d11/d24/d25/d4b/d66/f70 0 2024-02-16T19:24:25.229 INFO:tasks.workunit.client.1.smithi163.stdout:8/309: truncate d0/d7/d18/d2e/f5c 8748955 0 2024-02-16T19:24:25.229 INFO:tasks.workunit.client.1.smithi163.stdout:7/320: stat d11/l16 0 2024-02-16T19:24:25.229 INFO:tasks.workunit.client.1.smithi163.stdout:8/310: chown d0/d24/d3c/d3d/l1d 65145625 1 2024-02-16T19:24:25.231 INFO:tasks.workunit.client.0.smithi040.stdout:3/970: getdents d1/dc/d42/d58/de8 0 2024-02-16T19:24:25.231 INFO:tasks.workunit.client.0.smithi040.stdout:3/971: read - d1/dc/d42/d5c/f101 zero size 2024-02-16T19:24:25.232 INFO:tasks.workunit.client.1.smithi163.stdout:8/311: dread d0/d7/d18/d2e/f5f [0,4194304] 0 2024-02-16T19:24:25.232 INFO:tasks.workunit.client.1.smithi163.stdout:4/288: mknod d11/d18/d1f/c56 0 2024-02-16T19:24:25.232 INFO:tasks.workunit.client.1.smithi163.stdout:4/289: chown d11/f2f 0 1 2024-02-16T19:24:25.232 INFO:tasks.workunit.client.1.smithi163.stdout:4/290: write f10 [4906764,117081] 0 2024-02-16T19:24:25.233 INFO:tasks.workunit.client.1.smithi163.stdout:0/551: creat d11/d4f/fc4 x:0 0 0 2024-02-16T19:24:25.234 INFO:tasks.workunit.client.1.smithi163.stdout:0/552: truncate d11/d22/d7f/f83 1784563 0 2024-02-16T19:24:25.234 INFO:tasks.workunit.client.1.smithi163.stdout:0/553: write d11/d21/d50/d58/d74/f8f [234243,82321] 0 2024-02-16T19:24:25.236 INFO:tasks.workunit.client.1.smithi163.stdout:5/317: read d0/d22/f35 [774919,21137] 0 2024-02-16T19:24:25.239 INFO:tasks.workunit.client.1.smithi163.stdout:3/365: dwrite d0/dc/f3c [0,4194304] 0 2024-02-16T19:24:25.239 INFO:tasks.workunit.client.1.smithi163.stdout:1/313: truncate f6 2452639 0 2024-02-16T19:24:25.239 INFO:tasks.workunit.client.1.smithi163.stdout:1/314: chown d7/dc/l13 3736 1 2024-02-16T19:24:25.239 INFO:tasks.workunit.client.1.smithi163.stdout:1/315: dread - d7/d20/d21/f55 zero size 2024-02-16T19:24:25.241 INFO:tasks.workunit.client.1.smithi163.stdout:3/366: dread d0/dc/d1c/f23 [0,4194304] 0 2024-02-16T19:24:25.243 INFO:tasks.workunit.client.0.smithi040.stdout:2/878: mknod d5/d7/d1e/d4c/d52/d81/c129 0 2024-02-16T19:24:25.243 INFO:tasks.workunit.client.0.smithi040.stdout:2/879: getdents d5/d7/d1e/d55/d102/d10f 0 2024-02-16T19:24:25.243 INFO:tasks.workunit.client.0.smithi040.stdout:2/880: fsync d5/d7/d1e/d31/d4f/d63/f8e 0 2024-02-16T19:24:25.243 INFO:tasks.workunit.client.0.smithi040.stdout:2/881: creat d5/d7/d1e/d55/da3/f12a x:0 0 0 2024-02-16T19:24:25.245 INFO:tasks.workunit.client.0.smithi040.stdout:4/724: truncate dd/de/d10/fb6 3063183 0 2024-02-16T19:24:25.247 INFO:tasks.workunit.client.1.smithi163.stdout:6/302: mknod d0/d6/df/d20/d40/d53/c57 0 2024-02-16T19:24:25.247 INFO:tasks.workunit.client.1.smithi163.stdout:6/303: dread - d0/d6/d9/d2e/d33/f38 zero size 2024-02-16T19:24:25.247 INFO:tasks.workunit.client.1.smithi163.stdout:6/304: chown d0/l3f 554 1 2024-02-16T19:24:25.248 INFO:tasks.workunit.client.0.smithi040.stdout:5/924: rmdir d1/d62/d9a 39 2024-02-16T19:24:25.248 INFO:tasks.workunit.client.0.smithi040.stdout:5/925: dread - d1/da/d1c/dba/db1/dde/ddf/fe0 zero size 2024-02-16T19:24:25.255 INFO:tasks.workunit.client.1.smithi163.stdout:1/316: dwrite d7/dc/d2d/d32/f51 [0,4194304] 0 2024-02-16T19:24:25.255 INFO:tasks.workunit.client.1.smithi163.stdout:1/317: chown d7/dc/f5a 5439132 1 2024-02-16T19:24:25.256 INFO:tasks.workunit.client.1.smithi163.stdout:1/318: write d7/dc/d2d/d32/d3b/f58 [311971,102650] 0 2024-02-16T19:24:25.256 INFO:tasks.workunit.client.1.smithi163.stdout:1/319: dread - d7/dc/d2d/d32/d3b/f4d zero size 2024-02-16T19:24:25.256 INFO:tasks.workunit.client.1.smithi163.stdout:4/291: mkdir d11/d18/d1f/d29/d52/d57 0 2024-02-16T19:24:25.257 INFO:tasks.workunit.client.1.smithi163.stdout:9/501: dwrite d5/f2c [0,4194304] 0 2024-02-16T19:24:25.257 INFO:tasks.workunit.client.1.smithi163.stdout:0/554: rename d11/d12/d20/f30 to d11/d21/d7c/fc5 0 2024-02-16T19:24:25.258 INFO:tasks.workunit.client.1.smithi163.stdout:4/292: dread f2 [0,4194304] 0 2024-02-16T19:24:25.258 INFO:tasks.workunit.client.1.smithi163.stdout:5/318: getdents d0/d2 0 2024-02-16T19:24:25.258 INFO:tasks.workunit.client.1.smithi163.stdout:4/293: dread - d11/d2d/d4f/f55 zero size 2024-02-16T19:24:25.258 INFO:tasks.workunit.client.1.smithi163.stdout:5/319: stat d0/d1c/l26 0 2024-02-16T19:24:25.260 INFO:tasks.workunit.client.1.smithi163.stdout:3/367: symlink d0/dc/d1d/d29/d5d/l84 0 2024-02-16T19:24:25.264 INFO:tasks.workunit.client.1.smithi163.stdout:6/305: unlink d0/d6/df/d20/d40/d1e/f2c 0 2024-02-16T19:24:25.266 INFO:tasks.workunit.client.0.smithi040.stdout:7/799: getdents d21/d7e 0 2024-02-16T19:24:25.268 INFO:tasks.workunit.client.1.smithi163.stdout:2/482: getdents d2 0 2024-02-16T19:24:25.268 INFO:tasks.workunit.client.1.smithi163.stdout:2/483: dread - d2/d26/d54/d58/f87 zero size 2024-02-16T19:24:25.270 INFO:tasks.workunit.client.1.smithi163.stdout:7/321: getdents d11/d26 0 2024-02-16T19:24:25.270 INFO:tasks.workunit.client.1.smithi163.stdout:7/322: chown d11/d24/d25/f39 1852 1 2024-02-16T19:24:25.270 INFO:tasks.workunit.client.1.smithi163.stdout:7/323: dread - d11/d24/d49/f60 zero size 2024-02-16T19:24:25.271 INFO:tasks.workunit.client.1.smithi163.stdout:3/368: write d0/dc/d1c/f20 [7285860,93352] 0 2024-02-16T19:24:25.272 INFO:tasks.workunit.client.1.smithi163.stdout:3/369: write d0/dc/d1d/f1f [1617139,23735] 0 2024-02-16T19:24:25.272 INFO:tasks.workunit.client.0.smithi040.stdout:2/882: mknod d5/d7/d10/c12b 0 2024-02-16T19:24:25.274 INFO:tasks.workunit.client.1.smithi163.stdout:8/312: getdents d0/d24/d2c 0 2024-02-16T19:24:25.275 INFO:tasks.workunit.client.1.smithi163.stdout:8/313: fsync d0/dc/f22 0 2024-02-16T19:24:25.275 INFO:tasks.workunit.client.0.smithi040.stdout:5/926: dwrite d1/d45/fce [0,4194304] 0 2024-02-16T19:24:25.275 INFO:tasks.workunit.client.0.smithi040.stdout:5/927: write d1/d3c/f72 [3417861,53010] 0 2024-02-16T19:24:25.277 INFO:tasks.workunit.client.0.smithi040.stdout:7/800: mknod d21/c111 0 2024-02-16T19:24:25.277 INFO:tasks.workunit.client.0.smithi040.stdout:7/801: dread - d21/d29/dad/ff5 zero size 2024-02-16T19:24:25.279 INFO:tasks.workunit.client.1.smithi163.stdout:1/320: unlink d7/dc/d2d/d32/d3b/c53 0 2024-02-16T19:24:25.279 INFO:tasks.workunit.client.1.smithi163.stdout:1/321: dread - d7/d5b/f6a zero size 2024-02-16T19:24:25.280 INFO:tasks.workunit.client.0.smithi040.stdout:2/883: creat d5/d7/d1e/d31/d4f/f12c x:0 0 0 2024-02-16T19:24:25.280 INFO:tasks.workunit.client.0.smithi040.stdout:2/884: fdatasync d5/d7/d12/d1b/dae/fc4 0 2024-02-16T19:24:25.283 INFO:tasks.workunit.client.1.smithi163.stdout:0/555: mkdir d11/d12/dc6 0 2024-02-16T19:24:25.284 INFO:tasks.workunit.client.1.smithi163.stdout:0/556: write d11/d21/d50/d58/d74/f8f [485198,90795] 0 2024-02-16T19:24:25.284 INFO:tasks.workunit.client.1.smithi163.stdout:0/557: write d11/f33 [109880,116863] 0 2024-02-16T19:24:25.286 INFO:tasks.workunit.client.1.smithi163.stdout:9/502: dwrite d5/de/f5b [0,4194304] 0 2024-02-16T19:24:25.288 INFO:tasks.workunit.client.1.smithi163.stdout:6/306: unlink d0/dd/f23 0 2024-02-16T19:24:25.288 INFO:tasks.workunit.client.1.smithi163.stdout:6/307: creat d0/d18/f58 x:0 0 0 2024-02-16T19:24:25.288 INFO:tasks.workunit.client.1.smithi163.stdout:6/308: dread - d0/d6/df/d20/d40/d17/f4b zero size 2024-02-16T19:24:25.290 INFO:tasks.workunit.client.0.smithi040.stdout:5/928: mkdir d1/da/d1c/dba/dd6/d126 0 2024-02-16T19:24:25.290 INFO:tasks.workunit.client.0.smithi040.stdout:5/929: write d1/da/d51/ffd [639043,71810] 0 2024-02-16T19:24:25.290 INFO:tasks.workunit.client.0.smithi040.stdout:5/930: dread - d1/d3c/f5d zero size 2024-02-16T19:24:25.290 INFO:tasks.workunit.client.0.smithi040.stdout:5/931: fdatasync d1/da/d7e/f124 0 2024-02-16T19:24:25.291 INFO:tasks.workunit.client.1.smithi163.stdout:2/484: symlink d2/d2a/d9f/la5 0 2024-02-16T19:24:25.294 INFO:tasks.workunit.client.0.smithi040.stdout:7/802: mkdir d21/d4d/d77/d87/d9b/d112 0 2024-02-16T19:24:25.294 INFO:tasks.workunit.client.1.smithi163.stdout:7/324: dwrite d11/d24/d25/f39 [0,4194304] 0 2024-02-16T19:24:25.295 INFO:tasks.workunit.client.0.smithi040.stdout:7/803: chown d21/d29/c55 1056166645 1 2024-02-16T19:24:25.297 INFO:tasks.workunit.client.0.smithi040.stdout:2/885: creat d5/d7/d10/df3/d42/f12d x:0 0 0 2024-02-16T19:24:25.298 INFO:tasks.workunit.client.1.smithi163.stdout:6/309: dread d0/dd/f13 [0,4194304] 0 2024-02-16T19:24:25.298 INFO:tasks.workunit.client.1.smithi163.stdout:6/310: read d0/d6/f1a [2883852,105804] 0 2024-02-16T19:24:25.298 INFO:tasks.workunit.client.1.smithi163.stdout:6/311: write d0/d6/d9/d2e/d33/f38 [170721,75922] 0 2024-02-16T19:24:25.298 INFO:tasks.workunit.client.1.smithi163.stdout:6/312: dread - d0/d6/f45 zero size 2024-02-16T19:24:25.299 INFO:tasks.workunit.client.1.smithi163.stdout:3/370: truncate d0/dc/d1c/d32/f36 309024 0 2024-02-16T19:24:25.301 INFO:tasks.workunit.client.1.smithi163.stdout:8/314: rename d0/d7/d18/d2e/d46 to d0/d6f 0 2024-02-16T19:24:25.302 INFO:tasks.workunit.client.0.smithi040.stdout:2/886: link d5/d7/d1e/d31/d4f/l11d d5/d7/d1e/d55/d8d/l12e 0 2024-02-16T19:24:25.305 INFO:tasks.workunit.client.1.smithi163.stdout:9/503: write d5/f19 [7157450,12075] 0 2024-02-16T19:24:25.305 INFO:tasks.workunit.client.1.smithi163.stdout:5/320: dwrite d0/d22/f35 [0,4194304] 0 2024-02-16T19:24:25.309 INFO:tasks.workunit.client.1.smithi163.stdout:1/322: dwrite d7/d5b/f6a [0,4194304] 0 2024-02-16T19:24:25.311 INFO:tasks.workunit.client.0.smithi040.stdout:7/804: dread d21/d4d/d77/d87/dc9/dcb/f106 [0,4194304] 0 2024-02-16T19:24:25.312 INFO:tasks.workunit.client.0.smithi040.stdout:2/887: unlink d5/d7/d1e/d4c/d52/daa/dcd/ldb 0 2024-02-16T19:24:25.316 INFO:tasks.workunit.client.0.smithi040.stdout:7/805: mkdir d21/d113 0 2024-02-16T19:24:25.316 INFO:tasks.workunit.client.1.smithi163.stdout:3/371: mknod d0/dc/d1d/d29/d5d/d71/c85 0 2024-02-16T19:24:25.316 INFO:tasks.workunit.client.1.smithi163.stdout:3/372: creat d0/dc/d1d/d29/d5d/f86 x:0 0 0 2024-02-16T19:24:25.316 INFO:tasks.workunit.client.1.smithi163.stdout:3/373: stat d0/dc/d1c/d32/d62 0 2024-02-16T19:24:25.317 INFO:tasks.workunit.client.1.smithi163.stdout:7/325: rename d11/c1a to d11/d26/d36/c72 0 2024-02-16T19:24:25.318 INFO:tasks.workunit.client.0.smithi040.stdout:2/888: mknod d5/d7/d10/c12f 0 2024-02-16T19:24:25.319 INFO:tasks.workunit.client.0.smithi040.stdout:2/889: dread - d5/d7/d10/d40/f103 zero size 2024-02-16T19:24:25.319 INFO:tasks.workunit.client.0.smithi040.stdout:2/890: creat d5/d5c/d65/d9d/f130 x:0 0 0 2024-02-16T19:24:25.319 INFO:tasks.workunit.client.1.smithi163.stdout:8/315: rmdir d0/d7/d18 39 2024-02-16T19:24:25.322 INFO:tasks.workunit.client.0.smithi040.stdout:2/891: mkdir d5/d5c/d65/d131 0 2024-02-16T19:24:25.324 INFO:tasks.workunit.client.0.smithi040.stdout:2/892: creat d5/d7/d12/d1b/dae/f132 x:0 0 0 2024-02-16T19:24:25.326 INFO:tasks.workunit.client.0.smithi040.stdout:2/893: mkdir d5/d7/d1e/d55/d133 0 2024-02-16T19:24:25.328 INFO:tasks.workunit.client.0.smithi040.stdout:2/894: rename d5/d7/d12/da6/d114/f118 to d5/d7/d10/d40/f134 0 2024-02-16T19:24:25.329 INFO:tasks.workunit.client.0.smithi040.stdout:7/806: write d21/d29/f4c [3306346,68581] 0 2024-02-16T19:24:25.330 INFO:tasks.workunit.client.1.smithi163.stdout:0/558: rmdir d11 39 2024-02-16T19:24:25.330 INFO:tasks.workunit.client.1.smithi163.stdout:0/559: chown d11/d21/d7c 24311 1 2024-02-16T19:24:25.330 INFO:tasks.workunit.client.1.smithi163.stdout:2/485: dwrite d2/ff [8388608,4194304] 0 2024-02-16T19:24:25.330 INFO:tasks.workunit.client.1.smithi163.stdout:4/294: dwrite d11/f30 [0,4194304] 0 2024-02-16T19:24:25.330 INFO:tasks.workunit.client.1.smithi163.stdout:4/295: dread - d11/d18/d1f/d29/f36 zero size 2024-02-16T19:24:25.331 INFO:tasks.workunit.client.0.smithi040.stdout:5/932: dwrite d1/f1d [4194304,4194304] 0 2024-02-16T19:24:25.331 INFO:tasks.workunit.client.0.smithi040.stdout:5/933: chown d1/d3c 53 1 2024-02-16T19:24:25.331 INFO:tasks.workunit.client.1.smithi163.stdout:5/321: truncate d0/d2/f16 7299542 0 2024-02-16T19:24:25.331 INFO:tasks.workunit.client.1.smithi163.stdout:5/322: write d0/d22/f3a [1370193,19939] 0 2024-02-16T19:24:25.331 INFO:tasks.workunit.client.1.smithi163.stdout:5/323: stat d0/d22/d28 0 2024-02-16T19:24:25.331 INFO:tasks.workunit.client.1.smithi163.stdout:6/313: dwrite d0/d18/f58 [0,4194304] 0 2024-02-16T19:24:25.332 INFO:tasks.workunit.client.1.smithi163.stdout:6/314: write d0/d6/d9/f2d [1585304,126333] 0 2024-02-16T19:24:25.332 INFO:tasks.workunit.client.1.smithi163.stdout:6/315: truncate d0/d6/d9/f4c 606254 0 2024-02-16T19:24:25.335 INFO:tasks.workunit.client.0.smithi040.stdout:7/807: unlink d21/f9f 0 2024-02-16T19:24:25.337 INFO:tasks.workunit.client.0.smithi040.stdout:5/934: mkdir d1/da/dd5/d127 0 2024-02-16T19:24:25.339 INFO:tasks.workunit.client.0.smithi040.stdout:2/895: write d5/d7/d1e/d31/fb8 [973488,22496] 0 2024-02-16T19:24:25.340 INFO:tasks.workunit.client.1.smithi163.stdout:3/374: creat d0/dc/d1d/d29/d5d/d71/f87 x:0 0 0 2024-02-16T19:24:25.343 INFO:tasks.workunit.client.0.smithi040.stdout:7/808: unlink d21/d8b/fb5 0 2024-02-16T19:24:25.343 INFO:tasks.workunit.client.0.smithi040.stdout:7/809: read - d21/d4d/f6e zero size 2024-02-16T19:24:25.343 INFO:tasks.workunit.client.0.smithi040.stdout:7/810: dread - d21/d29/dad/ffa zero size 2024-02-16T19:24:25.343 INFO:tasks.workunit.client.0.smithi040.stdout:7/811: stat d21/d4d/d77/d7a/dc6/f101 0 2024-02-16T19:24:25.344 INFO:tasks.workunit.client.1.smithi163.stdout:9/504: rename d5/de/d30/f6c to d5/de/d30/d3d/d49/d6e/fb9 0 2024-02-16T19:24:25.346 INFO:tasks.workunit.client.0.smithi040.stdout:5/935: mkdir d1/da/d7e/d128 0 2024-02-16T19:24:25.347 INFO:tasks.workunit.client.1.smithi163.stdout:7/326: mknod d11/c73 0 2024-02-16T19:24:25.349 INFO:tasks.workunit.client.1.smithi163.stdout:8/316: truncate d0/d24/d3c/f4c 160792 0 2024-02-16T19:24:25.351 INFO:tasks.workunit.client.0.smithi040.stdout:2/896: creat d5/d7/d10/df3/f135 x:0 0 0 2024-02-16T19:24:25.351 INFO:tasks.workunit.client.0.smithi040.stdout:2/897: chown d5/d7/d10/d40/f120 134653 1 2024-02-16T19:24:25.353 INFO:tasks.workunit.client.1.smithi163.stdout:0/560: mknod d11/d12/d67/d6a/dae/cc7 0 2024-02-16T19:24:25.353 INFO:tasks.workunit.client.1.smithi163.stdout:0/561: chown d11/d21/l82 4146 1 2024-02-16T19:24:25.353 INFO:tasks.workunit.client.1.smithi163.stdout:0/562: readlink d11/d4f/lc2 0 2024-02-16T19:24:25.353 INFO:tasks.workunit.client.0.smithi040.stdout:7/812: symlink d21/df9/dfe/l114 0 2024-02-16T19:24:25.357 INFO:tasks.workunit.client.1.smithi163.stdout:2/486: mkdir d2/d26/d54/d58/da6 0 2024-02-16T19:24:25.360 INFO:tasks.workunit.client.0.smithi040.stdout:2/898: rmdir d5/d7/d10/d10e/d11e 0 2024-02-16T19:24:25.360 INFO:tasks.workunit.client.0.smithi040.stdout:2/899: chown d5/d5c/d65/d9d/l107 144326 1 2024-02-16T19:24:25.361 INFO:tasks.workunit.client.0.smithi040.stdout:5/936: dwrite d1/da/d1c/d73/fbe [0,4194304] 0 2024-02-16T19:24:25.361 INFO:tasks.workunit.client.0.smithi040.stdout:5/937: fdatasync d1/da/d1c/f26 0 2024-02-16T19:24:25.363 INFO:tasks.workunit.client.0.smithi040.stdout:2/900: mkdir d5/d7/d12/d1b/d136 0 2024-02-16T19:24:25.363 INFO:tasks.workunit.client.0.smithi040.stdout:2/901: readlink d5/d7/lc9 0 2024-02-16T19:24:25.366 INFO:tasks.workunit.client.1.smithi163.stdout:5/324: stat d0/d2/d34/l3d 0 2024-02-16T19:24:25.367 INFO:tasks.workunit.client.0.smithi040.stdout:5/938: symlink d1/da/d51/l129 0 2024-02-16T19:24:25.370 INFO:tasks.workunit.client.0.smithi040.stdout:2/902: symlink d5/d7/d1e/d31/d4f/d63/d8b/l137 0 2024-02-16T19:24:25.379 INFO:tasks.workunit.client.0.smithi040.stdout:5/939: read d1/d45/fce [749175,15515] 0 2024-02-16T19:24:25.379 INFO:tasks.workunit.client.0.smithi040.stdout:7/813: dwrite d21/d4d/f5b [0,4194304] 0 2024-02-16T19:24:25.379 INFO:tasks.workunit.client.0.smithi040.stdout:5/940: chown d1/da/d1c/f25 0 1 2024-02-16T19:24:25.379 INFO:tasks.workunit.client.0.smithi040.stdout:7/814: readlink l5 0 2024-02-16T19:24:25.379 INFO:tasks.workunit.client.0.smithi040.stdout:5/941: dread - d1/da/d51/f103 zero size 2024-02-16T19:24:25.380 INFO:tasks.workunit.client.1.smithi163.stdout:6/316: creat d0/d6/df/d20/d40/f59 x:0 0 0 2024-02-16T19:24:25.380 INFO:tasks.workunit.client.1.smithi163.stdout:6/317: write d0/d6/d9/d2e/d33/f38 [270386,123686] 0 2024-02-16T19:24:25.385 INFO:tasks.workunit.client.0.smithi040.stdout:7/815: symlink d21/d29/dad/l115 0 2024-02-16T19:24:25.385 INFO:tasks.workunit.client.0.smithi040.stdout:7/816: chown d21/d29/dad/l115 129 1 2024-02-16T19:24:25.385 INFO:tasks.workunit.client.0.smithi040.stdout:7/817: truncate d21/d29/dad/ffa 382347 0 2024-02-16T19:24:25.387 INFO:tasks.workunit.client.0.smithi040.stdout:5/942: creat d1/da/d1c/dba/db1/d120/f12a x:0 0 0 2024-02-16T19:24:25.389 INFO:tasks.workunit.client.0.smithi040.stdout:7/818: creat d21/d29/d79/f116 x:0 0 0 2024-02-16T19:24:25.397 INFO:tasks.workunit.client.1.smithi163.stdout:3/375: mknod d0/dc/d50/d66/c88 0 2024-02-16T19:24:25.400 INFO:tasks.workunit.client.1.smithi163.stdout:9/505: unlink d5/de/d16/f95 0 2024-02-16T19:24:25.400 INFO:tasks.workunit.client.1.smithi163.stdout:9/506: stat d5/de/d30/d3d/d49/d6d 0 2024-02-16T19:24:25.400 INFO:tasks.workunit.client.1.smithi163.stdout:9/507: write d5/de/d30/d3d/d49/d6e/f73 [752125,39271] 0 2024-02-16T19:24:25.400 INFO:tasks.workunit.client.1.smithi163.stdout:9/508: write d5/de/d30/d3d/d49/d6d/fac [56100,84253] 0 2024-02-16T19:24:25.400 INFO:tasks.workunit.client.1.smithi163.stdout:9/509: write d5/de/d30/d3d/d49/d50/f94 [11915,86706] 0 2024-02-16T19:24:25.400 INFO:tasks.workunit.client.1.smithi163.stdout:7/327: mkdir d11/d24/d25/d4b/d74 0 2024-02-16T19:24:25.402 INFO:tasks.workunit.client.1.smithi163.stdout:8/317: mkdir d0/d7/d19/d1f/d32/d70 0 2024-02-16T19:24:25.403 INFO:tasks.workunit.client.1.smithi163.stdout:0/563: creat d11/d22/d3e/fc8 x:0 0 0 2024-02-16T19:24:25.405 INFO:tasks.workunit.client.0.smithi040.stdout:2/903: write d5/d7/d1e/f80 [613460,54238] 0 2024-02-16T19:24:25.405 INFO:tasks.workunit.client.1.smithi163.stdout:2/487: symlink d2/d2a/d2d/d73/d84/la7 0 2024-02-16T19:24:25.406 INFO:tasks.workunit.client.1.smithi163.stdout:5/325: mkdir d0/d1c/d60 0 2024-02-16T19:24:25.406 INFO:tasks.workunit.client.1.smithi163.stdout:5/326: chown d0/d2/d55 129550676 1 2024-02-16T19:24:25.406 INFO:tasks.workunit.client.1.smithi163.stdout:5/327: readlink d0/l47 0 2024-02-16T19:24:25.406 INFO:tasks.workunit.client.1.smithi163.stdout:5/328: creat d0/d22/d28/d36/d3c/d4f/f61 x:0 0 0 2024-02-16T19:24:25.409 INFO:tasks.workunit.client.1.smithi163.stdout:3/376: dread d0/dc/d1c/d32/f3f [0,4194304] 0 2024-02-16T19:24:25.418 INFO:tasks.workunit.client.1.smithi163.stdout:9/510: mknod d5/de/d30/d3d/d49/d50/cba 0 2024-02-16T19:24:25.419 INFO:tasks.workunit.client.1.smithi163.stdout:9/511: creat d5/fbb x:0 0 0 2024-02-16T19:24:25.419 INFO:tasks.workunit.client.0.smithi040.stdout:2/904: truncate d5/d7/d1e/f6d 2869041 0 2024-02-16T19:24:25.419 INFO:tasks.workunit.client.0.smithi040.stdout:2/905: truncate d5/d7/d10/d40/f91 792377 0 2024-02-16T19:24:25.420 INFO:tasks.workunit.client.0.smithi040.stdout:2/906: creat d5/d7/f138 x:0 0 0 2024-02-16T19:24:25.420 INFO:tasks.workunit.client.0.smithi040.stdout:2/907: write d5/d7/d1e/d31/fd8 [735534,2872] 0 2024-02-16T19:24:25.420 INFO:tasks.workunit.client.0.smithi040.stdout:2/908: dread - d5/d5c/d65/d9d/ffd zero size 2024-02-16T19:24:25.420 INFO:tasks.workunit.client.1.smithi163.stdout:2/488: write d2/f1a [1452638,51332] 0 2024-02-16T19:24:25.420 INFO:tasks.workunit.client.1.smithi163.stdout:2/489: dread - d2/d2a/d39/f75 zero size 2024-02-16T19:24:25.421 INFO:tasks.workunit.client.1.smithi163.stdout:7/328: rmdir d11/d26/d36/d67/d71 0 2024-02-16T19:24:25.421 INFO:tasks.workunit.client.1.smithi163.stdout:7/329: readlink d11/d50/l57 0 2024-02-16T19:24:25.421 INFO:tasks.workunit.client.1.smithi163.stdout:7/330: stat l10 0 2024-02-16T19:24:25.421 INFO:tasks.workunit.client.1.smithi163.stdout:7/331: dread - d11/d24/d49/f60 zero size 2024-02-16T19:24:25.422 INFO:tasks.workunit.client.1.smithi163.stdout:7/332: chown d11/f15 827471 1 2024-02-16T19:24:25.422 INFO:tasks.workunit.client.1.smithi163.stdout:7/333: write f2 [2052541,99297] 0 2024-02-16T19:24:25.422 INFO:tasks.workunit.client.1.smithi163.stdout:0/564: unlink f8 0 2024-02-16T19:24:25.424 INFO:tasks.workunit.client.1.smithi163.stdout:0/565: dread d11/d21/d7c/fc5 [0,4194304] 0 2024-02-16T19:24:25.424 INFO:tasks.workunit.client.0.smithi040.stdout:2/909: mkdir d5/d7/d1e/d31/d4f/d139 0 2024-02-16T19:24:25.426 INFO:tasks.workunit.client.1.smithi163.stdout:5/329: link d0/d22/d28/f38 d0/d22/d28/d36/d3c/d4f/f62 0 2024-02-16T19:24:25.428 INFO:tasks.workunit.client.0.smithi040.stdout:5/943: dwrite d1/da/d1c/d21/f8f [0,4194304] 0 2024-02-16T19:24:25.428 INFO:tasks.workunit.client.0.smithi040.stdout:2/910: rename d5/d7/d12/dca/dea/d123 to d5/d7/d12/d1b/d13a 0 2024-02-16T19:24:25.429 INFO:tasks.workunit.client.0.smithi040.stdout:2/911: chown d5/d7/d12/d1b/dae/cec 926 1 2024-02-16T19:24:25.429 INFO:tasks.workunit.client.1.smithi163.stdout:9/512: getdents d5/de/d16/d46/d9b 0 2024-02-16T19:24:25.431 INFO:tasks.workunit.client.1.smithi163.stdout:2/490: symlink d2/d2a/d2d/la8 0 2024-02-16T19:24:25.433 INFO:tasks.workunit.client.1.smithi163.stdout:7/334: truncate d11/d24/d25/f5a 736028 0 2024-02-16T19:24:25.433 INFO:tasks.workunit.client.1.smithi163.stdout:7/335: readlink d11/l52 0 2024-02-16T19:24:25.433 INFO:tasks.workunit.client.0.smithi040.stdout:5/944: symlink d1/d62/d9a/l12b 0 2024-02-16T19:24:25.433 INFO:tasks.workunit.client.0.smithi040.stdout:5/945: creat d1/d45/f12c x:0 0 0 2024-02-16T19:24:25.434 INFO:tasks.workunit.client.1.smithi163.stdout:0/566: truncate f10 1367873 0 2024-02-16T19:24:25.435 INFO:tasks.workunit.client.0.smithi040.stdout:1/905: sync 2024-02-16T19:24:25.435 INFO:tasks.workunit.client.0.smithi040.stdout:9/889: sync 2024-02-16T19:24:25.435 INFO:tasks.workunit.client.0.smithi040.stdout:9/890: write d3/d4/d11/f67 [1397671,8690] 0 2024-02-16T19:24:25.436 INFO:tasks.workunit.client.0.smithi040.stdout:7/819: dwrite d21/d4d/d77/d7a/f82 [0,4194304] 0 2024-02-16T19:24:25.436 INFO:tasks.workunit.client.1.smithi163.stdout:5/330: mkdir d0/d2/d34/d44/d63 0 2024-02-16T19:24:25.436 INFO:tasks.workunit.client.1.smithi163.stdout:8/318: dwrite d0/d7/f6b [0,4194304] 0 2024-02-16T19:24:25.436 INFO:tasks.workunit.client.1.smithi163.stdout:8/319: chown d0/d24/d3c/d3d/l2b 8511657 1 2024-02-16T19:24:25.437 INFO:tasks.workunit.client.1.smithi163.stdout:3/377: dwrite d0/dc/f24 [0,4194304] 0 2024-02-16T19:24:25.439 INFO:tasks.workunit.client.0.smithi040.stdout:5/946: creat d1/da/d1c/dba/db1/dde/f12d x:0 0 0 2024-02-16T19:24:25.440 INFO:tasks.workunit.client.1.smithi163.stdout:7/336: mknod d11/d26/d36/d67/c75 0 2024-02-16T19:24:25.441 INFO:tasks.workunit.client.1.smithi163.stdout:7/337: chown d11/d30/f3e 443 1 2024-02-16T19:24:25.441 INFO:tasks.workunit.client.1.smithi163.stdout:7/338: chown d11/c1e 8081315 1 2024-02-16T19:24:25.442 INFO:tasks.workunit.client.0.smithi040.stdout:1/906: creat d17/d29/d8b/d10e/d112/f12a x:0 0 0 2024-02-16T19:24:25.444 INFO:tasks.workunit.client.1.smithi163.stdout:2/491: dread d2/d2a/d39/d66/d6c/f6d [0,4194304] 0 2024-02-16T19:24:25.444 INFO:tasks.workunit.client.1.smithi163.stdout:0/567: symlink d11/d22/d7f/lc9 0 2024-02-16T19:24:25.446 INFO:tasks.workunit.client.1.smithi163.stdout:8/320: dread d0/d24/d27/d45/f4a [0,4194304] 0 2024-02-16T19:24:25.447 INFO:tasks.workunit.client.1.smithi163.stdout:8/321: fsync d0/dc/f16 0 2024-02-16T19:24:25.450 INFO:tasks.workunit.client.1.smithi163.stdout:7/339: dread d11/d24/d25/f39 [0,4194304] 0 2024-02-16T19:24:25.450 INFO:tasks.workunit.client.1.smithi163.stdout:5/331: mkdir d0/d2/d34/d44/d64 0 2024-02-16T19:24:25.450 INFO:tasks.workunit.client.1.smithi163.stdout:5/332: chown d0/l5b 605697 1 2024-02-16T19:24:25.455 INFO:tasks.workunit.client.1.smithi163.stdout:9/513: mknod d5/de/d30/d3d/cbc 0 2024-02-16T19:24:25.456 INFO:tasks.workunit.client.0.smithi040.stdout:2/912: rename d5/d5c to d5/d7/d12/dca/d13b 0 2024-02-16T19:24:25.458 INFO:tasks.workunit.client.1.smithi163.stdout:2/492: rename d2/l49 to d2/d2a/d2d/d3c/la9 0 2024-02-16T19:24:25.459 INFO:tasks.workunit.client.1.smithi163.stdout:3/378: dwrite d0/f2 [0,4194304] 0 2024-02-16T19:24:25.459 INFO:tasks.workunit.client.1.smithi163.stdout:3/379: read d0/f1b [289247,68570] 0 2024-02-16T19:24:25.459 INFO:tasks.workunit.client.0.smithi040.stdout:9/891: rmdir d3/d4 39 2024-02-16T19:24:25.461 INFO:tasks.workunit.client.1.smithi163.stdout:0/568: mknod d11/d22/d61/cca 0 2024-02-16T19:24:25.463 INFO:tasks.workunit.client.0.smithi040.stdout:7/820: creat d21/d4d/f117 x:0 0 0 2024-02-16T19:24:25.463 INFO:tasks.workunit.client.1.smithi163.stdout:5/333: read d0/d22/d3b/f4c [2542403,110915] 0 2024-02-16T19:24:25.464 INFO:tasks.workunit.client.1.smithi163.stdout:8/322: creat d0/d24/d2c/f71 x:0 0 0 2024-02-16T19:24:25.466 INFO:tasks.workunit.client.1.smithi163.stdout:9/514: mknod d5/de/d16/d47/cbd 0 2024-02-16T19:24:25.467 INFO:tasks.workunit.client.1.smithi163.stdout:3/380: dread d0/dc/d1d/d49/f59 [4194304,4194304] 0 2024-02-16T19:24:25.467 INFO:tasks.workunit.client.1.smithi163.stdout:3/381: creat d0/dc/d1d/d29/d5d/d71/f89 x:0 0 0 2024-02-16T19:24:25.473 INFO:tasks.workunit.client.0.smithi040.stdout:5/947: creat d1/d3c/dea/f12e x:0 0 0 2024-02-16T19:24:25.475 INFO:tasks.workunit.client.0.smithi040.stdout:1/907: link d17/d26/d9a/dfb/f5d d17/d3c/d74/d126/f12b 0 2024-02-16T19:24:25.477 INFO:tasks.workunit.client.1.smithi163.stdout:7/340: rename d11/c1d to d11/d24/d25/c76 0 2024-02-16T19:24:25.478 INFO:tasks.workunit.client.0.smithi040.stdout:2/913: rename d5/d7/d1e/d31/d4f/d63/d8b/f128 to d5/d7/d10/df3/dbe/f13c 0 2024-02-16T19:24:25.479 INFO:tasks.workunit.client.1.smithi163.stdout:2/493: creat d2/d26/d54/d58/d8c/faa x:0 0 0 2024-02-16T19:24:25.480 INFO:tasks.workunit.client.1.smithi163.stdout:0/569: dwrite d11/d22/f3f [0,4194304] 0 2024-02-16T19:24:25.480 INFO:tasks.workunit.client.0.smithi040.stdout:9/892: mkdir d3/d4/d11/d29/d38/d46/d6b/d85/d118 0 2024-02-16T19:24:25.481 INFO:tasks.workunit.client.0.smithi040.stdout:9/893: readlink d3/d4/d11/d29/d38/d46/d6b/d85/d112/l41 0 2024-02-16T19:24:25.484 INFO:tasks.workunit.client.0.smithi040.stdout:7/821: mkdir d21/d7e/df8/db7/de7/d118 0 2024-02-16T19:24:25.484 INFO:tasks.workunit.client.0.smithi040.stdout:7/822: stat d21/d29/f38 0 2024-02-16T19:24:25.484 INFO:tasks.workunit.client.0.smithi040.stdout:7/823: truncate d21/d29/f5c 904199 0 2024-02-16T19:24:25.485 INFO:tasks.workunit.client.0.smithi040.stdout:1/908: write d17/f61 [2709362,39672] 0 2024-02-16T19:24:25.485 INFO:tasks.workunit.client.0.smithi040.stdout:1/909: truncate d17/d26/d5f/d78/d7e/ddb/f103 671272 0 2024-02-16T19:24:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:25 smithi163 ceph-mon[36452]: pgmap v251: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 142 MiB/s rd, 180 MiB/s wr, 251 op/s 2024-02-16T19:24:25.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:25 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:25.489 INFO:tasks.workunit.client.0.smithi040.stdout:5/948: symlink d1/da/dd5/dfb/l12f 0 2024-02-16T19:24:25.489 INFO:tasks.workunit.client.0.smithi040.stdout:5/949: chown d1/da/d1c/d21/daa/cae 3958 1 2024-02-16T19:24:25.496 INFO:tasks.workunit.client.1.smithi163.stdout:5/334: symlink d0/d2/d5c/l65 0 2024-02-16T19:24:25.498 INFO:tasks.workunit.client.1.smithi163.stdout:8/323: mkdir d0/d24/d27/d45/d72 0 2024-02-16T19:24:25.498 INFO:tasks.workunit.client.0.smithi040.stdout:2/914: creat d5/f13d x:0 0 0 2024-02-16T19:24:25.498 INFO:tasks.workunit.client.0.smithi040.stdout:2/915: chown d5/d7/d10/fb6 2 1 2024-02-16T19:24:25.500 INFO:tasks.workunit.client.0.smithi040.stdout:9/894: getdents d3/d4/d4a 0 2024-02-16T19:24:25.500 INFO:tasks.workunit.client.1.smithi163.stdout:9/515: symlink d5/de/d30/d31/lbe 0 2024-02-16T19:24:25.500 INFO:tasks.workunit.client.1.smithi163.stdout:9/516: creat d5/de/d16/fbf x:0 0 0 2024-02-16T19:24:25.500 INFO:tasks.workunit.client.1.smithi163.stdout:9/517: write d5/de/d30/d3d/d49/d6e/f73 [1276761,38362] 0 2024-02-16T19:24:25.501 INFO:tasks.workunit.client.1.smithi163.stdout:9/518: chown d5/de/d30/d43/d8f 0 1 2024-02-16T19:24:25.503 INFO:tasks.workunit.client.1.smithi163.stdout:5/335: dread d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:25.504 INFO:tasks.workunit.client.1.smithi163.stdout:5/336: dread d0/d22/d3b/f42 [0,4194304] 0 2024-02-16T19:24:25.505 INFO:tasks.workunit.client.0.smithi040.stdout:2/916: link d5/d7/d1e/d4c/d52/d81/f126 d5/d7/d1e/d4c/f13e 0 2024-02-16T19:24:25.506 INFO:tasks.workunit.client.0.smithi040.stdout:2/917: dread - d5/d7/f68 zero size 2024-02-16T19:24:25.506 INFO:tasks.workunit.client.1.smithi163.stdout:5/337: write d0/d22/f3a [336030,4033] 0 2024-02-16T19:24:25.506 INFO:tasks.workunit.client.1.smithi163.stdout:5/338: chown d0/d1c/c20 26064230 1 2024-02-16T19:24:25.508 INFO:tasks.workunit.client.1.smithi163.stdout:7/341: rmdir d11/d24/d25/d4b/d74 0 2024-02-16T19:24:25.510 INFO:tasks.workunit.client.1.smithi163.stdout:2/494: creat d2/d26/d54/d58/d98/fab x:0 0 0 2024-02-16T19:24:25.510 INFO:tasks.workunit.client.1.smithi163.stdout:2/495: creat d2/d26/d54/d58/fac x:0 0 0 2024-02-16T19:24:25.511 INFO:tasks.workunit.client.0.smithi040.stdout:2/918: mknod d5/d7/d10/d10e/c13f 0 2024-02-16T19:24:25.515 INFO:tasks.workunit.client.1.smithi163.stdout:0/570: creat d11/d12/d67/d6a/d6d/fcb x:0 0 0 2024-02-16T19:24:25.516 INFO:tasks.workunit.client.1.smithi163.stdout:8/324: write d0/d7/d19/d1f/d30/f47 [2107725,77686] 0 2024-02-16T19:24:25.518 INFO:tasks.workunit.client.0.smithi040.stdout:2/919: dread d5/d7/d1e/d31/d4f/fd3 [0,4194304] 0 2024-02-16T19:24:25.518 INFO:tasks.workunit.client.1.smithi163.stdout:3/382: getdents d0/dc/d50 0 2024-02-16T19:24:25.518 INFO:tasks.workunit.client.1.smithi163.stdout:3/383: creat d0/f8a x:0 0 0 2024-02-16T19:24:25.518 INFO:tasks.workunit.client.1.smithi163.stdout:3/384: fsync d0/f8 0 2024-02-16T19:24:25.519 INFO:tasks.workunit.client.1.smithi163.stdout:3/385: read d0/dc/d1d/d29/f45 [3609287,33184] 0 2024-02-16T19:24:25.520 INFO:tasks.workunit.client.0.smithi040.stdout:2/920: creat d5/d7/d12/dca/d13b/d65/f140 x:0 0 0 2024-02-16T19:24:25.522 INFO:tasks.workunit.client.1.smithi163.stdout:5/339: link d0/d2/f7 d0/d22/d3b/f66 0 2024-02-16T19:24:25.524 INFO:tasks.workunit.client.1.smithi163.stdout:7/342: mkdir d11/d24/d34/d77 0 2024-02-16T19:24:25.524 INFO:tasks.workunit.client.1.smithi163.stdout:0/571: write d11/d21/f46 [185822,16771] 0 2024-02-16T19:24:25.528 INFO:tasks.workunit.client.1.smithi163.stdout:2/496: mknod d2/d26/d54/cad 0 2024-02-16T19:24:25.529 INFO:tasks.workunit.client.1.smithi163.stdout:8/325: stat d0/d7/d19/l6e 0 2024-02-16T19:24:25.530 INFO:tasks.workunit.client.1.smithi163.stdout:8/326: read - d0/d24/d2c/f71 zero size 2024-02-16T19:24:25.531 INFO:tasks.workunit.client.1.smithi163.stdout:3/386: getdents d0/dc/d1d/d29/d38/d46 0 2024-02-16T19:24:25.533 INFO:tasks.workunit.client.1.smithi163.stdout:5/340: creat d0/d2/f67 x:0 0 0 2024-02-16T19:24:25.535 INFO:tasks.workunit.client.1.smithi163.stdout:7/343: unlink d11/d24/d49/l68 0 2024-02-16T19:24:25.535 INFO:tasks.workunit.client.1.smithi163.stdout:7/344: dread - d11/d24/d49/f69 zero size 2024-02-16T19:24:25.535 INFO:tasks.workunit.client.1.smithi163.stdout:7/345: fdatasync d11/d24/d49/f69 0 2024-02-16T19:24:25.537 INFO:tasks.workunit.client.1.smithi163.stdout:0/572: mkdir d11/d12/d67/d6a/dae/dcc 0 2024-02-16T19:24:25.539 INFO:tasks.workunit.client.1.smithi163.stdout:2/497: creat d2/d2a/d39/fae x:0 0 0 2024-02-16T19:24:25.539 INFO:tasks.workunit.client.0.smithi040.stdout:3/972: sync 2024-02-16T19:24:25.539 INFO:tasks.workunit.client.0.smithi040.stdout:3/973: readlink d1/dc/d35/l46 0 2024-02-16T19:24:25.539 INFO:tasks.workunit.client.0.smithi040.stdout:3/974: fdatasync d1/d7/d30/fa8 0 2024-02-16T19:24:25.539 INFO:tasks.workunit.client.0.smithi040.stdout:3/975: write d1/d4/d6e/dc2/f130 [96916,7871] 0 2024-02-16T19:24:25.542 INFO:tasks.workunit.client.0.smithi040.stdout:3/976: rename d1/dc/d35/l49 to d1/d7/d33/l134 0 2024-02-16T19:24:25.543 INFO:tasks.workunit.client.1.smithi163.stdout:3/387: rename d0/dc/d1c/f3b to d0/dc/d50/f8b 0 2024-02-16T19:24:25.543 INFO:tasks.workunit.client.1.smithi163.stdout:3/388: readlink d0/dc/d1d/d49/l7c 0 2024-02-16T19:24:25.543 INFO:tasks.workunit.client.1.smithi163.stdout:3/389: creat d0/dc/f8c x:0 0 0 2024-02-16T19:24:25.543 INFO:tasks.workunit.client.1.smithi163.stdout:2/498: mkdir d2/d26/d54/d58/d7d/daf 0 2024-02-16T19:24:25.544 INFO:tasks.workunit.client.0.smithi040.stdout:1/910: dwrite d17/d26/d5f/d63/f66 [0,4194304] 0 2024-02-16T19:24:25.545 INFO:tasks.workunit.client.0.smithi040.stdout:7/824: dwrite d21/d29/f30 [0,4194304] 0 2024-02-16T19:24:25.545 INFO:tasks.workunit.client.0.smithi040.stdout:7/825: stat d21/d4d/d77/d87/dc9/ddd/c10b 0 2024-02-16T19:24:25.545 INFO:tasks.workunit.client.1.smithi163.stdout:7/346: rename d11/d24/d49/f60 to d11/d24/f78 0 2024-02-16T19:24:25.546 INFO:tasks.workunit.client.0.smithi040.stdout:5/950: dwrite d1/f4c [0,4194304] 0 2024-02-16T19:24:25.548 INFO:tasks.workunit.client.0.smithi040.stdout:9/895: dwrite d3/f14 [0,4194304] 0 2024-02-16T19:24:25.549 INFO:tasks.workunit.client.1.smithi163.stdout:7/347: dread d11/d30/f56 [0,4194304] 0 2024-02-16T19:24:25.552 INFO:tasks.workunit.client.1.smithi163.stdout:3/390: symlink d0/dc/d50/d66/l8d 0 2024-02-16T19:24:25.553 INFO:tasks.workunit.client.0.smithi040.stdout:1/911: symlink d17/d3c/l12c 0 2024-02-16T19:24:25.553 INFO:tasks.workunit.client.0.smithi040.stdout:1/912: truncate d17/d26/d5f/d63/f66 5133252 0 2024-02-16T19:24:25.555 INFO:tasks.workunit.client.1.smithi163.stdout:9/519: dwrite d5/f2f [0,4194304] 0 2024-02-16T19:24:25.555 INFO:tasks.workunit.client.1.smithi163.stdout:5/341: dread d0/f2c [0,4194304] 0 2024-02-16T19:24:25.556 INFO:tasks.workunit.client.1.smithi163.stdout:3/391: link d0/dc/f8c d0/f8e 0 2024-02-16T19:24:25.556 INFO:tasks.workunit.client.1.smithi163.stdout:3/392: read d0/dc/d1d/d49/f59 [118293,63527] 0 2024-02-16T19:24:25.556 INFO:tasks.workunit.client.1.smithi163.stdout:3/393: write d0/dc/d1c/f23 [4012950,47873] 0 2024-02-16T19:24:25.558 INFO:tasks.workunit.client.0.smithi040.stdout:7/826: creat d21/d4d/d7b/f119 x:0 0 0 2024-02-16T19:24:25.561 INFO:tasks.workunit.client.0.smithi040.stdout:2/921: dwrite d5/d7/d12/d1b/f121 [0,4194304] 0 2024-02-16T19:24:25.561 INFO:tasks.workunit.client.1.smithi163.stdout:8/327: dwrite d0/d24/d2c/f4d [0,4194304] 0 2024-02-16T19:24:25.562 INFO:tasks.workunit.client.1.smithi163.stdout:0/573: dwrite d11/d12/d20/f39 [0,4194304] 0 2024-02-16T19:24:25.568 INFO:tasks.workunit.client.0.smithi040.stdout:3/977: dwrite d1/d8c/d8e/fd3 [0,4194304] 0 2024-02-16T19:24:25.569 INFO:tasks.workunit.client.1.smithi163.stdout:9/520: mkdir d5/dc0 0 2024-02-16T19:24:25.570 INFO:tasks.workunit.client.1.smithi163.stdout:9/521: fdatasync d5/de/d30/d3d/d49/f62 0 2024-02-16T19:24:25.571 INFO:tasks.workunit.client.1.smithi163.stdout:5/342: creat d0/d2/d34/d44/d53/f68 x:0 0 0 2024-02-16T19:24:25.572 INFO:tasks.workunit.client.1.smithi163.stdout:5/343: truncate d0/d2/d34/d44/d53/f68 456311 0 2024-02-16T19:24:25.573 INFO:tasks.workunit.client.0.smithi040.stdout:5/951: rename d1/f11d to d1/da/d1c/da9/f130 0 2024-02-16T19:24:25.575 INFO:tasks.workunit.client.0.smithi040.stdout:1/913: dwrite d17/d29/d8b/dd2/fe7 [0,4194304] 0 2024-02-16T19:24:25.575 INFO:tasks.workunit.client.1.smithi163.stdout:3/394: link d0/l15 d0/dc/l8f 0 2024-02-16T19:24:25.576 INFO:tasks.workunit.client.1.smithi163.stdout:7/348: dwrite d11/d26/d36/f4e [4194304,4194304] 0 2024-02-16T19:24:25.576 INFO:tasks.workunit.client.0.smithi040.stdout:9/896: mknod d3/d4/d11/d71/da8/c119 0 2024-02-16T19:24:25.579 INFO:tasks.workunit.client.0.smithi040.stdout:7/827: mknod d21/d7e/df8/c11a 0 2024-02-16T19:24:25.588 INFO:tasks.workunit.client.0.smithi040.stdout:2/922: mknod d5/d7/d1e/d55/c141 0 2024-02-16T19:24:25.592 INFO:tasks.workunit.client.1.smithi163.stdout:8/328: link d0/d7/c12 d0/d24/d3c/d3d/d41/c73 0 2024-02-16T19:24:25.595 INFO:tasks.workunit.client.1.smithi163.stdout:7/349: dwrite d11/d30/f56 [0,4194304] 0 2024-02-16T19:24:25.595 INFO:tasks.workunit.client.0.smithi040.stdout:7/828: dwrite f9 [0,4194304] 0 2024-02-16T19:24:25.596 INFO:tasks.workunit.client.1.smithi163.stdout:0/574: chown d11/d4f/fb0 110 1 2024-02-16T19:24:25.596 INFO:tasks.workunit.client.0.smithi040.stdout:3/978: read d1/d8c/fc3 [259828,72023] 0 2024-02-16T19:24:25.597 INFO:tasks.workunit.client.0.smithi040.stdout:3/979: creat d1/dc/d42/d5c/df3/f135 x:0 0 0 2024-02-16T19:24:25.598 INFO:tasks.workunit.client.1.smithi163.stdout:8/329: write d0/dc/f22 [268266,86996] 0 2024-02-16T19:24:25.598 INFO:tasks.workunit.client.1.smithi163.stdout:9/522: rename d5/de/d30/d3d/cbc to d5/de/d30/d3d/d93/cc1 0 2024-02-16T19:24:25.598 INFO:tasks.workunit.client.1.smithi163.stdout:8/330: chown d0/f44 27 1 2024-02-16T19:24:25.598 INFO:tasks.workunit.client.1.smithi163.stdout:9/523: write d5/de/d16/d47/f87 [999991,46062] 0 2024-02-16T19:24:25.598 INFO:tasks.workunit.client.0.smithi040.stdout:5/952: mkdir d1/da/d1c/da7/d131 0 2024-02-16T19:24:25.600 INFO:tasks.workunit.client.0.smithi040.stdout:1/914: mknod d17/d3c/d74/d97/c12d 0 2024-02-16T19:24:25.603 INFO:tasks.workunit.client.1.smithi163.stdout:9/524: dread d5/de/d30/d31/d52/f81 [0,4194304] 0 2024-02-16T19:24:25.603 INFO:tasks.workunit.client.1.smithi163.stdout:9/525: chown d5/de/d30/d3d/d49/d50 3554 1 2024-02-16T19:24:25.603 INFO:tasks.workunit.client.1.smithi163.stdout:1/323: sync 2024-02-16T19:24:25.603 INFO:tasks.workunit.client.1.smithi163.stdout:1/324: write d7/d5b/f6a [4472223,11897] 0 2024-02-16T19:24:25.605 INFO:tasks.workunit.client.0.smithi040.stdout:2/923: creat d5/d7/d1e/d4c/d52/f142 x:0 0 0 2024-02-16T19:24:25.605 INFO:tasks.workunit.client.0.smithi040.stdout:4/725: sync 2024-02-16T19:24:25.605 INFO:tasks.workunit.client.0.smithi040.stdout:4/726: fsync dd/d58/d5e/f7f 0 2024-02-16T19:24:25.607 INFO:tasks.workunit.client.0.smithi040.stdout:7/829: mkdir d21/d7e/df8/db7/de7/d118/d11b 0 2024-02-16T19:24:25.607 INFO:tasks.workunit.client.0.smithi040.stdout:7/830: chown d21/d7e/fe4 1426185300 1 2024-02-16T19:24:25.610 INFO:tasks.workunit.client.1.smithi163.stdout:7/350: write d11/d30/f56 [615956,117724] 0 2024-02-16T19:24:25.611 INFO:tasks.workunit.client.1.smithi163.stdout:0/575: dread d11/d21/d50/f79 [0,4194304] 0 2024-02-16T19:24:25.613 INFO:tasks.workunit.client.0.smithi040.stdout:3/980: dread d1/d4/d6e/fc9 [4194304,4194304] 0 2024-02-16T19:24:25.613 INFO:tasks.workunit.client.0.smithi040.stdout:5/953: link d1/da/d1c/d21/l5b d1/da/dd5/dfb/l132 0 2024-02-16T19:24:25.616 INFO:tasks.workunit.client.1.smithi163.stdout:8/331: rmdir d0/d24/d3c/d3d/d41/d54 0 2024-02-16T19:24:25.616 INFO:tasks.workunit.client.0.smithi040.stdout:1/915: stat d17/d26/d5f/d63/cae 0 2024-02-16T19:24:25.617 INFO:tasks.workunit.client.1.smithi163.stdout:4/296: sync 2024-02-16T19:24:25.617 INFO:tasks.workunit.client.1.smithi163.stdout:6/318: sync 2024-02-16T19:24:25.619 INFO:tasks.workunit.client.1.smithi163.stdout:5/344: rmdir d0/d22/d28/d36/d3c 39 2024-02-16T19:24:25.619 INFO:tasks.workunit.client.1.smithi163.stdout:5/345: creat d0/d22/d28/d36/f69 x:0 0 0 2024-02-16T19:24:25.620 INFO:tasks.workunit.client.0.smithi040.stdout:7/831: read d21/d4d/f56 [2175945,102887] 0 2024-02-16T19:24:25.621 INFO:tasks.workunit.client.0.smithi040.stdout:2/924: link d5/d7/d1e/d4c/f13e d5/d7/d1e/d31/d4f/d139/f143 0 2024-02-16T19:24:25.621 INFO:tasks.workunit.client.0.smithi040.stdout:4/727: dwrite dd/de/d10/d36/f6c [0,4194304] 0 2024-02-16T19:24:25.621 INFO:tasks.workunit.client.0.smithi040.stdout:4/728: chown dd/d58/d5e/dd1/d9d/dd6 75776878 1 2024-02-16T19:24:25.622 INFO:tasks.workunit.client.1.smithi163.stdout:5/346: dread d0/d22/d28/d36/d3c/f43 [0,4194304] 0 2024-02-16T19:24:25.622 INFO:tasks.workunit.client.1.smithi163.stdout:5/347: fsync d0/d22/d3b/f66 0 2024-02-16T19:24:25.622 INFO:tasks.workunit.client.1.smithi163.stdout:3/395: rmdir d0/d3e 39 2024-02-16T19:24:25.622 INFO:tasks.workunit.client.0.smithi040.stdout:5/954: mkdir d1/d62/d133 0 2024-02-16T19:24:25.622 INFO:tasks.workunit.client.0.smithi040.stdout:5/955: creat d1/da/d1c/dba/db1/f134 x:0 0 0 2024-02-16T19:24:25.624 INFO:tasks.workunit.client.1.smithi163.stdout:9/526: rename d5/de/d30/d31/c34 to d5/de/d16/d47/cc2 0 2024-02-16T19:24:25.624 INFO:tasks.workunit.client.1.smithi163.stdout:9/527: write d5/de/d30/d3d/f88 [3607138,50699] 0 2024-02-16T19:24:25.625 INFO:tasks.workunit.client.0.smithi040.stdout:1/916: getdents d17/d26/d9a/dfb/d92/daa 0 2024-02-16T19:24:25.626 INFO:tasks.workunit.client.0.smithi040.stdout:7/832: mknod d21/d4d/d77/d7a/dc6/c11c 0 2024-02-16T19:24:25.627 INFO:tasks.workunit.client.1.smithi163.stdout:4/297: dread d11/d18/d1f/f27 [4194304,4194304] 0 2024-02-16T19:24:25.627 INFO:tasks.workunit.client.1.smithi163.stdout:4/298: creat d11/d18/d1f/d23/d3a/f58 x:0 0 0 2024-02-16T19:24:25.628 INFO:tasks.workunit.client.1.smithi163.stdout:1/325: mkdir d7/d20/d21/d74 0 2024-02-16T19:24:25.628 INFO:tasks.workunit.client.1.smithi163.stdout:1/326: creat d7/dc/d2d/d32/f75 x:0 0 0 2024-02-16T19:24:25.631 INFO:tasks.workunit.client.1.smithi163.stdout:7/351: getdents d11/d26/d36 0 2024-02-16T19:24:25.631 INFO:tasks.workunit.client.1.smithi163.stdout:7/352: readlink d11/d50/l55 0 2024-02-16T19:24:25.631 INFO:tasks.workunit.client.1.smithi163.stdout:7/353: creat d11/d24/f79 x:0 0 0 2024-02-16T19:24:25.631 INFO:tasks.workunit.client.1.smithi163.stdout:7/354: dread - d11/d24/d25/f48 zero size 2024-02-16T19:24:25.631 INFO:tasks.workunit.client.1.smithi163.stdout:4/299: dread d11/f14 [0,4194304] 0 2024-02-16T19:24:25.635 INFO:tasks.workunit.client.1.smithi163.stdout:0/576: getdents d11/d12 0 2024-02-16T19:24:25.637 INFO:tasks.workunit.client.1.smithi163.stdout:9/528: write d5/de/d16/d47/f4c [1845001,117497] 0 2024-02-16T19:24:25.637 INFO:tasks.workunit.client.1.smithi163.stdout:8/332: mkdir d0/d24/d2c/d74 0 2024-02-16T19:24:25.637 INFO:tasks.workunit.client.1.smithi163.stdout:8/333: truncate d0/d7/d18/d2e/f5d 5205719 0 2024-02-16T19:24:25.640 INFO:tasks.workunit.client.1.smithi163.stdout:6/319: creat d0/d6/df/d30/f5a x:0 0 0 2024-02-16T19:24:25.642 INFO:tasks.workunit.client.0.smithi040.stdout:4/729: truncate dd/de/d10/d36/d9e/d77/f7b 1850419 0 2024-02-16T19:24:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:25 smithi040 ceph-mon[29201]: pgmap v251: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 142 MiB/s rd, 180 MiB/s wr, 251 op/s 2024-02-16T19:24:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:25 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:25.644 INFO:tasks.workunit.client.1.smithi163.stdout:6/320: dread d0/d6/d9/f4c [0,4194304] 0 2024-02-16T19:24:25.645 INFO:tasks.workunit.client.1.smithi163.stdout:8/334: dread d0/d7/d18/d2e/f5f [0,4194304] 0 2024-02-16T19:24:25.645 INFO:tasks.workunit.client.1.smithi163.stdout:8/335: creat d0/d24/d27/d45/f75 x:0 0 0 2024-02-16T19:24:25.645 INFO:tasks.workunit.client.1.smithi163.stdout:8/336: fsync d0/d24/f60 0 2024-02-16T19:24:25.645 INFO:tasks.workunit.client.1.smithi163.stdout:8/337: chown d0/d7/d19 2528 1 2024-02-16T19:24:25.648 INFO:tasks.workunit.client.1.smithi163.stdout:8/338: dread d0/d7/d18/d2e/f5f [0,4194304] 0 2024-02-16T19:24:25.648 INFO:tasks.workunit.client.1.smithi163.stdout:3/396: dwrite d0/dc/d1c/d21/f7a [0,4194304] 0 2024-02-16T19:24:25.650 INFO:tasks.workunit.client.0.smithi040.stdout:5/956: rename d1/f24 to d1/d3c/dea/f135 0 2024-02-16T19:24:25.653 INFO:tasks.workunit.client.0.smithi040.stdout:7/833: symlink d21/d4d/d77/d87/l11d 0 2024-02-16T19:24:25.654 INFO:tasks.workunit.client.1.smithi163.stdout:1/327: creat d7/d20/d21/d74/f76 x:0 0 0 2024-02-16T19:24:25.654 INFO:tasks.workunit.client.1.smithi163.stdout:1/328: write d7/f49 [1736728,83207] 0 2024-02-16T19:24:25.654 INFO:tasks.workunit.client.1.smithi163.stdout:1/329: readlink d7/dc/l13 0 2024-02-16T19:24:25.654 INFO:tasks.workunit.client.1.smithi163.stdout:1/330: creat d7/dc/d2d/d3f/f77 x:0 0 0 2024-02-16T19:24:25.654 INFO:tasks.workunit.client.1.smithi163.stdout:1/331: chown d7/d20/c23 1523 1 2024-02-16T19:24:25.656 INFO:tasks.workunit.client.1.smithi163.stdout:7/355: symlink d11/d26/d36/l7a 0 2024-02-16T19:24:25.658 INFO:tasks.workunit.client.0.smithi040.stdout:5/957: mknod d1/d62/d9a/dc5/c136 0 2024-02-16T19:24:25.658 INFO:tasks.workunit.client.0.smithi040.stdout:4/730: dread dd/de/d10/d36/d9e/d32/f64 [0,4194304] 0 2024-02-16T19:24:25.658 INFO:tasks.workunit.client.0.smithi040.stdout:4/731: stat dd/de/daa/dd8/cd5 0 2024-02-16T19:24:25.660 INFO:tasks.workunit.client.0.smithi040.stdout:5/958: link d1/da/d1c/d73/f104 d1/da/d51/f137 0 2024-02-16T19:24:25.660 INFO:tasks.workunit.client.0.smithi040.stdout:5/959: chown d1/da/d7e/f86 255054045 1 2024-02-16T19:24:25.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/732: creat dd/de/d10/d36/dc3/fdf x:0 0 0 2024-02-16T19:24:25.663 INFO:tasks.workunit.client.1.smithi163.stdout:5/348: getdents d0/d22/d3b 0 2024-02-16T19:24:25.665 INFO:tasks.workunit.client.0.smithi040.stdout:5/960: rmdir d1/d62/dbb 39 2024-02-16T19:24:25.667 INFO:tasks.workunit.client.1.smithi163.stdout:8/339: rename d0/d7/d19/d1f/d32/l69 to d0/d24/d27/d45/d72/l76 0 2024-02-16T19:24:25.668 INFO:tasks.workunit.client.1.smithi163.stdout:8/340: dread - d0/d7/f68 zero size 2024-02-16T19:24:25.668 INFO:tasks.workunit.client.0.smithi040.stdout:1/917: dwrite d17/db5/ff2 [0,4194304] 0 2024-02-16T19:24:25.669 INFO:tasks.workunit.client.1.smithi163.stdout:3/397: unlink d0/l15 0 2024-02-16T19:24:25.670 INFO:tasks.workunit.client.0.smithi040.stdout:1/918: dread d17/d26/d9a/dfb/f21 [0,4194304] 0 2024-02-16T19:24:25.671 INFO:tasks.workunit.client.1.smithi163.stdout:7/356: rmdir d11/d26/d36/d67 39 2024-02-16T19:24:25.671 INFO:tasks.workunit.client.1.smithi163.stdout:7/357: chown d11/d24/l65 1879994630 1 2024-02-16T19:24:25.675 INFO:tasks.workunit.client.0.smithi040.stdout:1/919: mknod d17/d26/d9a/dfb/d92/daa/c12e 0 2024-02-16T19:24:25.675 INFO:tasks.workunit.client.0.smithi040.stdout:1/920: write d17/d26/d5f/d63/fa7 [632045,41735] 0 2024-02-16T19:24:25.675 INFO:tasks.workunit.client.0.smithi040.stdout:1/921: write d17/d29/d8b/d93/f99 [267716,125508] 0 2024-02-16T19:24:25.675 INFO:tasks.workunit.client.0.smithi040.stdout:1/922: creat d17/db5/f12f x:0 0 0 2024-02-16T19:24:25.676 INFO:tasks.workunit.client.1.smithi163.stdout:3/398: write d0/dc/d1c/d32/f3f [2872725,4331] 0 2024-02-16T19:24:25.678 INFO:tasks.workunit.client.1.smithi163.stdout:5/349: mknod d0/d22/c6a 0 2024-02-16T19:24:25.678 INFO:tasks.workunit.client.1.smithi163.stdout:5/350: chown d0/d2/d55/l5f 1494 1 2024-02-16T19:24:25.678 INFO:tasks.workunit.client.1.smithi163.stdout:5/351: chown d0/d2/f16 7 1 2024-02-16T19:24:25.678 INFO:tasks.workunit.client.1.smithi163.stdout:5/352: creat d0/d22/d28/d36/f6b x:0 0 0 2024-02-16T19:24:25.679 INFO:tasks.workunit.client.1.smithi163.stdout:5/353: write d0/d22/d28/d36/d3c/d4f/f50 [773780,51325] 0 2024-02-16T19:24:25.679 INFO:tasks.workunit.client.1.smithi163.stdout:5/354: chown d0/d2/d5c/l65 14055 1 2024-02-16T19:24:25.681 INFO:tasks.workunit.client.1.smithi163.stdout:8/341: symlink d0/d7/d18/l77 0 2024-02-16T19:24:25.682 INFO:tasks.workunit.client.1.smithi163.stdout:7/358: symlink d11/d24/d49/l7b 0 2024-02-16T19:24:25.686 INFO:tasks.workunit.client.1.smithi163.stdout:5/355: dread d0/d2/f21 [0,4194304] 0 2024-02-16T19:24:25.686 INFO:tasks.workunit.client.1.smithi163.stdout:3/399: link d0/dc/d1c/f2c d0/dc/d1d/d29/d38/d46/f90 0 2024-02-16T19:24:25.687 INFO:tasks.workunit.client.1.smithi163.stdout:3/400: read d0/f2 [4784561,53889] 0 2024-02-16T19:24:25.687 INFO:tasks.workunit.client.0.smithi040.stdout:5/961: dread d1/da/d1c/f26 [0,4194304] 0 2024-02-16T19:24:25.690 INFO:tasks.workunit.client.0.smithi040.stdout:5/962: getdents d1/d62/d9a 0 2024-02-16T19:24:25.690 INFO:tasks.workunit.client.0.smithi040.stdout:1/923: write d17/d29/d8b/fd6 [1125569,19476] 0 2024-02-16T19:24:25.691 INFO:tasks.workunit.client.1.smithi163.stdout:4/300: dwrite d11/d18/d1f/d29/f3e [0,4194304] 0 2024-02-16T19:24:25.691 INFO:tasks.workunit.client.0.smithi040.stdout:5/963: dread d1/da/d7e/f86 [0,4194304] 0 2024-02-16T19:24:25.692 INFO:tasks.workunit.client.1.smithi163.stdout:7/359: creat d11/d50/f7c x:0 0 0 2024-02-16T19:24:25.692 INFO:tasks.workunit.client.1.smithi163.stdout:7/360: write d11/d24/d25/d4b/d66/f6f [28505,13235] 0 2024-02-16T19:24:25.693 INFO:tasks.workunit.client.1.smithi163.stdout:4/301: dread d11/f26 [0,4194304] 0 2024-02-16T19:24:25.694 INFO:tasks.workunit.client.0.smithi040.stdout:5/964: mkdir d1/da/d1c/dba/db1/dde/ddf/d138 0 2024-02-16T19:24:25.698 INFO:tasks.workunit.client.0.smithi040.stdout:5/965: mknod d1/da/d1c/da7/c139 0 2024-02-16T19:24:25.699 INFO:tasks.workunit.client.1.smithi163.stdout:0/577: dwrite d11/d21/d50/d58/d74/f7e [0,4194304] 0 2024-02-16T19:24:25.699 INFO:tasks.workunit.client.1.smithi163.stdout:0/578: dread - d11/d12/d67/d6a/d6d/f65 zero size 2024-02-16T19:24:25.700 INFO:tasks.workunit.client.0.smithi040.stdout:9/897: dwrite d3/d4/d11/d29/d38/d46/d6b/d85/d112/d63/fd4 [0,4194304] 0 2024-02-16T19:24:25.703 INFO:tasks.workunit.client.0.smithi040.stdout:5/966: rmdir d1/da/d1c/da7/d131 0 2024-02-16T19:24:25.703 INFO:tasks.workunit.client.0.smithi040.stdout:5/967: chown d1/da/d1c/c11c 9165598 1 2024-02-16T19:24:25.703 INFO:tasks.workunit.client.0.smithi040.stdout:7/834: dwrite d21/d29/d79/fee [0,4194304] 0 2024-02-16T19:24:25.703 INFO:tasks.workunit.client.0.smithi040.stdout:7/835: truncate d21/d4d/d7b/fa6 622830 0 2024-02-16T19:24:25.703 INFO:tasks.workunit.client.0.smithi040.stdout:4/733: dwrite dd/d58/d8c/fbe [0,4194304] 0 2024-02-16T19:24:25.703 INFO:tasks.workunit.client.0.smithi040.stdout:7/836: read d21/d29/f5c [41804,60618] 0 2024-02-16T19:24:25.704 INFO:tasks.workunit.client.1.smithi163.stdout:9/529: dwrite d5/f19 [8388608,4194304] 0 2024-02-16T19:24:25.704 INFO:tasks.workunit.client.1.smithi163.stdout:6/321: dwrite d0/d6/d9/d2e/d33/f3d [0,4194304] 0 2024-02-16T19:24:25.704 INFO:tasks.workunit.client.1.smithi163.stdout:9/530: readlink d5/de/l22 0 2024-02-16T19:24:25.705 INFO:tasks.workunit.client.1.smithi163.stdout:9/531: chown d5/de/d16/d46/d9b 3 1 2024-02-16T19:24:25.705 INFO:tasks.workunit.client.1.smithi163.stdout:1/332: dwrite d7/d20/d28/f3d [0,4194304] 0 2024-02-16T19:24:25.705 INFO:tasks.workunit.client.1.smithi163.stdout:9/532: write d5/de/d30/d3d/d49/d6e/f73 [1986326,94556] 0 2024-02-16T19:24:25.705 INFO:tasks.workunit.client.1.smithi163.stdout:9/533: fdatasync d5/de/d16/d47/f87 0 2024-02-16T19:24:25.705 INFO:tasks.workunit.client.1.smithi163.stdout:1/333: getdents d7/d5b/d63 0 2024-02-16T19:24:25.706 INFO:tasks.workunit.client.0.smithi040.stdout:1/924: dwrite d17/d29/d8b/f115 [0,4194304] 0 2024-02-16T19:24:25.706 INFO:tasks.workunit.client.0.smithi040.stdout:1/925: fsync d17/d26/d9a/dfb/d92/f8c 0 2024-02-16T19:24:25.709 INFO:tasks.workunit.client.1.smithi163.stdout:9/534: dread d5/de/d30/d3d/f6f [0,4194304] 0 2024-02-16T19:24:25.709 INFO:tasks.workunit.client.1.smithi163.stdout:9/535: fdatasync d5/de/d30/d3d/f6f 0 2024-02-16T19:24:25.709 INFO:tasks.workunit.client.1.smithi163.stdout:9/536: creat d5/de/d30/d3d/d49/d6d/fc3 x:0 0 0 2024-02-16T19:24:25.711 INFO:tasks.workunit.client.0.smithi040.stdout:9/898: rename d3/d4/d11/d71/fc2 to d3/d4/d11/d29/df1/f11a 0 2024-02-16T19:24:25.713 INFO:tasks.workunit.client.0.smithi040.stdout:2/925: write d5/d7/d1e/d31/d4f/ff2 [788250,17509] 0 2024-02-16T19:24:25.713 INFO:tasks.workunit.client.1.smithi163.stdout:7/361: mknod d11/d30/c7d 0 2024-02-16T19:24:25.713 INFO:tasks.workunit.client.1.smithi163.stdout:7/362: fdatasync d11/d24/d25/f2c 0 2024-02-16T19:24:25.713 INFO:tasks.workunit.client.1.smithi163.stdout:7/363: fdatasync d11/d24/f78 0 2024-02-16T19:24:25.713 INFO:tasks.workunit.client.1.smithi163.stdout:7/364: write ff [1304619,18921] 0 2024-02-16T19:24:25.714 INFO:tasks.workunit.client.1.smithi163.stdout:7/365: write d11/d26/f6d [724476,11363] 0 2024-02-16T19:24:25.714 INFO:tasks.workunit.client.0.smithi040.stdout:5/968: mkdir d1/da/d1c/dba/db1/dde/ddf/d138/d13a 0 2024-02-16T19:24:25.714 INFO:tasks.workunit.client.0.smithi040.stdout:5/969: fdatasync d1/da/f2e 0 2024-02-16T19:24:25.714 INFO:tasks.workunit.client.0.smithi040.stdout:5/970: read d1/da/d51/f58 [463164,108048] 0 2024-02-16T19:24:25.716 INFO:tasks.workunit.client.1.smithi163.stdout:4/302: symlink d11/d2d/l59 0 2024-02-16T19:24:25.719 INFO:tasks.workunit.client.1.smithi163.stdout:6/322: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:25.724 INFO:tasks.workunit.client.0.smithi040.stdout:9/899: dread d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/f113 [0,4194304] 0 2024-02-16T19:24:25.727 INFO:tasks.workunit.client.1.smithi163.stdout:8/342: rename d0/d24/d27/d45/d72 to d0/d24/d78 0 2024-02-16T19:24:25.729 INFO:tasks.workunit.client.0.smithi040.stdout:4/734: creat dd/de/d10/fe0 x:0 0 0 2024-02-16T19:24:25.734 INFO:tasks.workunit.client.1.smithi163.stdout:4/303: write d11/d18/d1f/d34/f51 [1778448,24891] 0 2024-02-16T19:24:25.740 INFO:tasks.workunit.client.0.smithi040.stdout:9/900: unlink d3/d4/f57 0 2024-02-16T19:24:25.740 INFO:tasks.workunit.client.0.smithi040.stdout:9/901: fdatasync d3/d4/d11/d29/d38/d46/d6b/f97 0 2024-02-16T19:24:25.741 INFO:tasks.workunit.client.0.smithi040.stdout:9/902: write d3/d4/d11/d29/d38/d46/d6b/d85/d112/d8a/d99/fb2 [1016645,115747] 0 2024-02-16T19:24:25.741 INFO:tasks.workunit.client.1.smithi163.stdout:3/401: rmdir d0 39 2024-02-16T19:24:25.742 INFO:tasks.workunit.client.0.smithi040.stdout:1/926: rmdir d17/d29/d8b/d10e/d121 39 2024-02-16T19:24:25.743 INFO:tasks.workunit.client.1.smithi163.stdout:3/402: read d0/f6d [295897,120600] 0 2024-02-16T19:24:25.743 INFO:tasks.workunit.client.0.smithi040.stdout:4/735: mknod dd/d58/ce1 0 2024-02-16T19:24:25.744 INFO:tasks.workunit.client.0.smithi040.stdout:4/736: write dd/d58/fdc [871203,107725] 0 2024-02-16T19:24:25.748 INFO:tasks.workunit.client.0.smithi040.stdout:9/903: creat d3/d4/dbe/de9/f11b x:0 0 0 2024-02-16T19:24:25.749 INFO:tasks.workunit.client.1.smithi163.stdout:1/334: mkdir d7/dc/d78 0 2024-02-16T19:24:25.749 INFO:tasks.workunit.client.1.smithi163.stdout:1/335: creat d7/d20/d21/d4e/f79 x:0 0 0 2024-02-16T19:24:25.750 INFO:tasks.workunit.client.0.smithi040.stdout:1/927: unlink d17/c4f 0 2024-02-16T19:24:25.751 INFO:tasks.workunit.client.0.smithi040.stdout:4/737: dread dd/de/f5d [0,4194304] 0 2024-02-16T19:24:25.751 INFO:tasks.workunit.client.0.smithi040.stdout:7/837: dwrite d21/d8b/fe0 [0,4194304] 0 2024-02-16T19:24:25.751 INFO:tasks.workunit.client.0.smithi040.stdout:7/838: chown d21/d4d/f74 3 1 2024-02-16T19:24:25.751 INFO:tasks.workunit.client.1.smithi163.stdout:7/366: dwrite d11/d30/f32 [0,4194304] 0 2024-02-16T19:24:25.752 INFO:tasks.workunit.client.1.smithi163.stdout:7/367: creat d11/f7e x:0 0 0 2024-02-16T19:24:25.752 INFO:tasks.workunit.client.1.smithi163.stdout:7/368: creat d11/d24/d25/f7f x:0 0 0 2024-02-16T19:24:25.752 INFO:tasks.workunit.client.1.smithi163.stdout:7/369: readlink d11/d30/l47 0 2024-02-16T19:24:25.752 INFO:tasks.workunit.client.0.smithi040.stdout:2/926: dwrite d5/d7/d10/dbc/fe3 [0,4194304] 0 2024-02-16T19:24:25.752 INFO:tasks.workunit.client.0.smithi040.stdout:5/971: dwrite d1/da/d7e/f86 [0,4194304] 0 2024-02-16T19:24:25.753 INFO:tasks.workunit.client.0.smithi040.stdout:9/904: creat d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/f11c x:0 0 0 2024-02-16T19:24:25.755 INFO:tasks.workunit.client.1.smithi163.stdout:9/537: symlink d5/de/d30/d8a/lc4 0 2024-02-16T19:24:25.756 INFO:tasks.workunit.client.0.smithi040.stdout:2/927: dread d5/d7/d1e/d4c/d52/daa/dcd/ff4 [0,4194304] 0 2024-02-16T19:24:25.756 INFO:tasks.workunit.client.0.smithi040.stdout:2/928: write d5/d7/d10/df3/d42/f12d [440629,96678] 0 2024-02-16T19:24:25.756 INFO:tasks.workunit.client.0.smithi040.stdout:2/929: dread - d5/d7/d12/da6/d114/f119 zero size 2024-02-16T19:24:25.760 INFO:tasks.workunit.client.0.smithi040.stdout:4/738: write dd/de/d10/d36/d9e/d77/f96 [2059731,61397] 0 2024-02-16T19:24:25.769 INFO:tasks.workunit.client.1.smithi163.stdout:7/370: dwrite d11/d30/f56 [0,4194304] 0 2024-02-16T19:24:25.769 INFO:tasks.workunit.client.1.smithi163.stdout:7/371: write fb [2563571,97622] 0 2024-02-16T19:24:25.770 INFO:tasks.workunit.client.1.smithi163.stdout:9/538: dread d5/de/d30/d3d/d49/d6e/fb9 [0,4194304] 0 2024-02-16T19:24:25.771 INFO:tasks.workunit.client.0.smithi040.stdout:7/839: rename d21/d4d/d77/d87/dc9/ddd/dbe/fe3 to d21/d8b/f11e 0 2024-02-16T19:24:25.771 INFO:tasks.workunit.client.0.smithi040.stdout:7/840: write d21/d4d/d77/d7a/dc6/f101 [359364,61439] 0 2024-02-16T19:24:25.771 INFO:tasks.workunit.client.0.smithi040.stdout:7/841: creat d21/d8b/d9c/f11f x:0 0 0 2024-02-16T19:24:25.771 INFO:tasks.workunit.client.0.smithi040.stdout:7/842: chown d21/f62 117913 1 2024-02-16T19:24:25.771 INFO:tasks.workunit.client.0.smithi040.stdout:7/843: dread - d21/d7e/fe1 zero size 2024-02-16T19:24:25.771 INFO:tasks.workunit.client.0.smithi040.stdout:7/844: chown d21/d7e/df8/c11a 56 1 2024-02-16T19:24:25.774 INFO:tasks.workunit.client.1.smithi163.stdout:6/323: symlink d0/dd/l5b 0 2024-02-16T19:24:25.774 INFO:tasks.workunit.client.1.smithi163.stdout:6/324: creat d0/d6/df/d30/f5c x:0 0 0 2024-02-16T19:24:25.775 INFO:tasks.workunit.client.1.smithi163.stdout:6/325: write d0/d6/f31 [8809472,119323] 0 2024-02-16T19:24:25.775 INFO:tasks.workunit.client.1.smithi163.stdout:6/326: write d0/d6/df/d20/d40/d17/f4b [187308,67113] 0 2024-02-16T19:24:25.777 INFO:tasks.workunit.client.1.smithi163.stdout:5/356: rename d0/d22/d28/d36/d3c/l58 to d0/d22/d3b/l6c 0 2024-02-16T19:24:25.780 INFO:tasks.workunit.client.1.smithi163.stdout:4/304: creat d11/d18/d1f/d29/d52/f5a x:0 0 0 2024-02-16T19:24:25.786 INFO:tasks.workunit.client.0.smithi040.stdout:1/928: dwrite d17/d2d/f73 [0,4194304] 0 2024-02-16T19:24:25.788 INFO:tasks.workunit.client.1.smithi163.stdout:1/336: mknod d7/d20/d2a/c7a 0 2024-02-16T19:24:25.789 INFO:tasks.workunit.client.0.smithi040.stdout:5/972: dwrite d1/d45/f125 [0,4194304] 0 2024-02-16T19:24:25.789 INFO:tasks.workunit.client.0.smithi040.stdout:7/845: dwrite d21/d8b/d9c/ff7 [0,4194304] 0 2024-02-16T19:24:25.797 INFO:tasks.workunit.client.0.smithi040.stdout:5/973: write d1/da/d1c/d73/fbe [3577630,105326] 0 2024-02-16T19:24:25.797 INFO:tasks.workunit.client.0.smithi040.stdout:5/974: write d1/da/da4/fa6 [3054995,129532] 0 2024-02-16T19:24:25.799 INFO:tasks.workunit.client.0.smithi040.stdout:9/905: symlink d3/d4/d4a/db4/df7/daf/l11d 0 2024-02-16T19:24:25.801 INFO:tasks.workunit.client.1.smithi163.stdout:7/372: mknod d11/c80 0 2024-02-16T19:24:25.801 INFO:tasks.workunit.client.1.smithi163.stdout:7/373: creat d11/d24/d49/f81 x:0 0 0 2024-02-16T19:24:25.801 INFO:tasks.workunit.client.1.smithi163.stdout:7/374: chown d11/d26/c64 127466 1 2024-02-16T19:24:25.801 INFO:tasks.workunit.client.1.smithi163.stdout:7/375: write d11/d30/f3e [962774,119779] 0 2024-02-16T19:24:25.803 INFO:tasks.workunit.client.0.smithi040.stdout:2/930: symlink d5/d7/d12/d1b/dae/l144 0 2024-02-16T19:24:25.804 INFO:tasks.workunit.client.1.smithi163.stdout:9/539: creat d5/dc0/fc5 x:0 0 0 2024-02-16T19:24:25.808 INFO:tasks.workunit.client.0.smithi040.stdout:5/975: dread d1/da/d1c/d21/f42 [0,4194304] 0 2024-02-16T19:24:25.808 INFO:tasks.workunit.client.0.smithi040.stdout:4/739: creat dd/d58/d5e/dd1/d9d/fe2 x:0 0 0 2024-02-16T19:24:25.808 INFO:tasks.workunit.client.0.smithi040.stdout:4/740: write dd/de/d10/fa2 [186691,15492] 0 2024-02-16T19:24:25.808 INFO:tasks.workunit.client.0.smithi040.stdout:4/741: write dd/d58/fc5 [153532,98265] 0 2024-02-16T19:24:25.810 INFO:tasks.workunit.client.1.smithi163.stdout:6/327: unlink d0/d6/d9/cb 0 2024-02-16T19:24:25.812 INFO:tasks.workunit.client.1.smithi163.stdout:0/579: rename f6 to d11/d21/d50/d58/d6b/fcd 0 2024-02-16T19:24:25.812 INFO:tasks.workunit.client.1.smithi163.stdout:0/580: write d11/d21/d50/d58/d6b/f7a [5056651,3891] 0 2024-02-16T19:24:25.812 INFO:tasks.workunit.client.1.smithi163.stdout:0/581: chown d11/d12/d67/d6a/d6d 896 1 2024-02-16T19:24:25.812 INFO:tasks.workunit.client.0.smithi040.stdout:1/929: getdents d17/d3c/dde 0 2024-02-16T19:24:25.813 INFO:tasks.workunit.client.0.smithi040.stdout:9/906: dwrite d3/d4/d11/d29/d38/d46/d6b/d85/d112/ff4 [0,4194304] 0 2024-02-16T19:24:25.814 INFO:tasks.workunit.client.1.smithi163.stdout:5/357: creat d0/d2/d34/f6d x:0 0 0 2024-02-16T19:24:25.814 INFO:tasks.workunit.client.1.smithi163.stdout:5/358: read d0/d22/d28/d36/d3c/d4f/f50 [239912,29229] 0 2024-02-16T19:24:25.814 INFO:tasks.workunit.client.1.smithi163.stdout:5/359: write d0/d22/d28/d36/f6b [905056,3546] 0 2024-02-16T19:24:25.818 INFO:tasks.workunit.client.1.smithi163.stdout:1/337: creat d7/d5b/d63/f7b x:0 0 0 2024-02-16T19:24:25.820 INFO:tasks.workunit.client.1.smithi163.stdout:5/360: write d0/d2/f16 [2719430,69640] 0 2024-02-16T19:24:25.820 INFO:tasks.workunit.client.1.smithi163.stdout:5/361: write d0/d2/fb [1827304,28430] 0 2024-02-16T19:24:25.821 INFO:tasks.workunit.client.0.smithi040.stdout:7/846: truncate d21/d7e/fe9 2924381 0 2024-02-16T19:24:25.824 INFO:tasks.workunit.client.1.smithi163.stdout:9/540: creat d5/de/d16/d46/d9b/fc6 x:0 0 0 2024-02-16T19:24:25.826 INFO:tasks.workunit.client.0.smithi040.stdout:2/931: unlink d5/c29 0 2024-02-16T19:24:25.827 INFO:tasks.workunit.client.1.smithi163.stdout:6/328: symlink d0/d6/d9/d2e/l5d 0 2024-02-16T19:24:25.827 INFO:tasks.workunit.client.1.smithi163.stdout:6/329: creat d0/d6/df/d20/d40/d17/f5e x:0 0 0 2024-02-16T19:24:25.828 INFO:tasks.workunit.client.0.smithi040.stdout:5/976: mknod d1/da/d1c/dba/db1/dde/d11f/c13b 0 2024-02-16T19:24:25.829 INFO:tasks.workunit.client.1.smithi163.stdout:9/541: dread d5/de/d30/f36 [0,4194304] 0 2024-02-16T19:24:25.829 INFO:tasks.workunit.client.0.smithi040.stdout:4/742: symlink dd/d58/d5e/dd1/dc6/le3 0 2024-02-16T19:24:25.830 INFO:tasks.workunit.client.0.smithi040.stdout:4/743: write dd/de/d10/f2e [1491896,11574] 0 2024-02-16T19:24:25.830 INFO:tasks.workunit.client.1.smithi163.stdout:0/582: creat d11/d21/d7c/fce x:0 0 0 2024-02-16T19:24:25.832 INFO:tasks.workunit.client.1.smithi163.stdout:4/305: mkdir d11/d5b 0 2024-02-16T19:24:25.832 INFO:tasks.workunit.client.1.smithi163.stdout:0/583: dread d11/d22/d7f/fb6 [0,4194304] 0 2024-02-16T19:24:25.835 INFO:tasks.workunit.client.1.smithi163.stdout:1/338: write d7/f26 [1845004,3158] 0 2024-02-16T19:24:25.836 INFO:tasks.workunit.client.1.smithi163.stdout:5/362: truncate d0/d22/d3b/f66 912336 0 2024-02-16T19:24:25.836 INFO:tasks.workunit.client.1.smithi163.stdout:5/363: write d0/d1c/f40 [3364240,45322] 0 2024-02-16T19:24:25.836 INFO:tasks.workunit.client.1.smithi163.stdout:5/364: creat d0/d2/d34/f6e x:0 0 0 2024-02-16T19:24:25.836 INFO:tasks.workunit.client.1.smithi163.stdout:5/365: fsync d0/d2/f21 0 2024-02-16T19:24:25.837 INFO:tasks.workunit.client.0.smithi040.stdout:9/907: creat d3/d4/d11/d29/d38/f11e x:0 0 0 2024-02-16T19:24:25.838 INFO:tasks.workunit.client.1.smithi163.stdout:7/376: getdents d11/d24/d25/d4b/d66 0 2024-02-16T19:24:25.841 INFO:tasks.workunit.client.0.smithi040.stdout:9/908: write d3/d4/d11/d29/d38/d46/d6b/d85/d112/d70/dcc/ffa [1260518,37499] 0 2024-02-16T19:24:25.842 INFO:tasks.workunit.client.0.smithi040.stdout:7/847: rmdir d21/d4d/d7d 39 2024-02-16T19:24:25.844 INFO:tasks.workunit.client.1.smithi163.stdout:9/542: mkdir d5/de/d16/d46/d9b/dc7 0 2024-02-16T19:24:25.844 INFO:tasks.workunit.client.1.smithi163.stdout:9/543: creat d5/fc8 x:0 0 0 2024-02-16T19:24:25.844 INFO:tasks.workunit.client.1.smithi163.stdout:9/544: write d5/de/d72/fb0 [105800,66537] 0 2024-02-16T19:24:25.844 INFO:tasks.workunit.client.1.smithi163.stdout:9/545: fdatasync d5/f2f 0 2024-02-16T19:24:25.845 INFO:tasks.workunit.client.0.smithi040.stdout:7/848: dread d21/f28 [4194304,4194304] 0 2024-02-16T19:24:25.847 INFO:tasks.workunit.client.1.smithi163.stdout:4/306: symlink d11/d18/d1f/d29/d52/d57/l5c 0 2024-02-16T19:24:25.848 INFO:tasks.workunit.client.0.smithi040.stdout:5/977: mknod d1/da/dd5/dfb/c13c 0 2024-02-16T19:24:25.850 INFO:tasks.workunit.client.1.smithi163.stdout:0/584: mkdir d11/d22/d7f/dcf 0 2024-02-16T19:24:25.852 INFO:tasks.workunit.client.1.smithi163.stdout:7/377: dwrite d11/d24/d49/f69 [0,4194304] 0 2024-02-16T19:24:25.855 INFO:tasks.workunit.client.0.smithi040.stdout:4/744: getdents dd/d58/d68 0 2024-02-16T19:24:25.855 INFO:tasks.workunit.client.0.smithi040.stdout:4/745: write dd/de/d10/d36/d9e/d77/f96 [4183396,93341] 0 2024-02-16T19:24:25.858 INFO:tasks.workunit.client.0.smithi040.stdout:1/930: mkdir d17/d26/d3d/d130 0 2024-02-16T19:24:25.864 INFO:tasks.workunit.client.1.smithi163.stdout:5/366: creat d0/d1c/d60/f6f x:0 0 0 2024-02-16T19:24:25.864 INFO:tasks.workunit.client.1.smithi163.stdout:5/367: chown d0/d22/d3b/f4c 98 1 2024-02-16T19:24:25.864 INFO:tasks.workunit.client.1.smithi163.stdout:5/368: creat d0/d22/d28/f70 x:0 0 0 2024-02-16T19:24:25.866 INFO:tasks.workunit.client.1.smithi163.stdout:6/330: getdents d0/d18 0 2024-02-16T19:24:25.868 INFO:tasks.workunit.client.1.smithi163.stdout:2/499: sync 2024-02-16T19:24:25.871 INFO:tasks.workunit.client.1.smithi163.stdout:9/546: mknod d5/de/cc9 0 2024-02-16T19:24:25.871 INFO:tasks.workunit.client.1.smithi163.stdout:9/547: creat d5/de/d16/d46/d9b/fca x:0 0 0 2024-02-16T19:24:25.878 INFO:tasks.workunit.client.0.smithi040.stdout:9/909: rename d3/d4/f96 to d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/d110/f11f 0 2024-02-16T19:24:25.880 INFO:tasks.workunit.client.0.smithi040.stdout:2/932: rmdir d5/d7/d1e/d31/dc3 39 2024-02-16T19:24:25.883 INFO:tasks.workunit.client.1.smithi163.stdout:9/548: write d5/f2c [2142737,25424] 0 2024-02-16T19:24:25.883 INFO:tasks.workunit.client.1.smithi163.stdout:4/307: dwrite d11/d18/f24 [0,4194304] 0 2024-02-16T19:24:25.886 INFO:tasks.workunit.client.0.smithi040.stdout:5/978: symlink d1/da/d1c/dba/db1/dde/l13d 0 2024-02-16T19:24:25.887 INFO:tasks.workunit.client.1.smithi163.stdout:0/585: symlink d11/d4f/ld0 0 2024-02-16T19:24:25.889 INFO:tasks.workunit.client.1.smithi163.stdout:1/339: rename d7/d20/d28 to d7/dc/d78/d7c 0 2024-02-16T19:24:25.889 INFO:tasks.workunit.client.1.smithi163.stdout:1/340: write d7/d20/d21/d4e/f68 [192795,88191] 0 2024-02-16T19:24:25.890 INFO:tasks.workunit.client.0.smithi040.stdout:9/910: mkdir d3/d4/d11/d29/d38/d46/d6b/d85/d112/dc3/d120 0 2024-02-16T19:24:25.891 INFO:tasks.workunit.client.0.smithi040.stdout:7/849: rmdir d21/d4d/d77/d87/d9b/df2 39 2024-02-16T19:24:25.894 INFO:tasks.workunit.client.0.smithi040.stdout:5/979: unlink d1/da/d1c/d21/f42 0 2024-02-16T19:24:25.896 INFO:tasks.workunit.client.1.smithi163.stdout:4/308: chown d11/d18/c28 68692 1 2024-02-16T19:24:25.897 INFO:tasks.workunit.client.0.smithi040.stdout:7/850: mknod d21/d29/d79/d10c/c120 0 2024-02-16T19:24:25.897 INFO:tasks.workunit.client.0.smithi040.stdout:7/851: chown d21/d4d/d77/d87/dc9/ddd/dbe/cc1 285 1 2024-02-16T19:24:25.898 INFO:tasks.workunit.client.1.smithi163.stdout:0/586: symlink d11/d12/d67/d6a/dae/ld1 0 2024-02-16T19:24:25.898 INFO:tasks.workunit.client.1.smithi163.stdout:9/549: write d5/f19 [11057042,40632] 0 2024-02-16T19:24:25.898 INFO:tasks.workunit.client.1.smithi163.stdout:9/550: readlink d5/de/d30/d3d/d49/la5 0 2024-02-16T19:24:25.898 INFO:tasks.workunit.client.1.smithi163.stdout:9/551: stat d5/de/d16/fbf 0 2024-02-16T19:24:25.902 INFO:tasks.workunit.client.1.smithi163.stdout:5/369: getdents d0/d1c/d60 0 2024-02-16T19:24:25.907 INFO:tasks.workunit.client.1.smithi163.stdout:0/587: read - d11/d4f/fc4 zero size 2024-02-16T19:24:25.907 INFO:tasks.workunit.client.1.smithi163.stdout:7/378: dwrite d11/d30/f32 [0,4194304] 0 2024-02-16T19:24:25.909 INFO:tasks.workunit.client.0.smithi040.stdout:1/931: dwrite d17/d26/d9a/dfb/d92/dca/fda [4194304,4194304] 0 2024-02-16T19:24:25.911 INFO:tasks.workunit.client.0.smithi040.stdout:3/981: sync 2024-02-16T19:24:25.911 INFO:tasks.workunit.client.1.smithi163.stdout:6/331: dwrite d0/dd/f13 [0,4194304] 0 2024-02-16T19:24:25.912 INFO:tasks.workunit.client.1.smithi163.stdout:6/332: write d0/d6/d9/d15/f4d [1037022,26184] 0 2024-02-16T19:24:25.912 INFO:tasks.workunit.client.1.smithi163.stdout:6/333: creat d0/d6/d9/d2e/d33/f5f x:0 0 0 2024-02-16T19:24:25.912 INFO:tasks.workunit.client.1.smithi163.stdout:4/309: symlink d11/d3d/l5d 0 2024-02-16T19:24:25.914 INFO:tasks.workunit.client.1.smithi163.stdout:5/370: truncate d0/f32 807348 0 2024-02-16T19:24:25.919 INFO:tasks.workunit.client.0.smithi040.stdout:4/746: dwrite dd/de/d45/f74 [0,4194304] 0 2024-02-16T19:24:25.919 INFO:tasks.workunit.client.0.smithi040.stdout:7/852: dread d21/d8b/d9c/ff7 [0,4194304] 0 2024-02-16T19:24:25.921 INFO:tasks.workunit.client.1.smithi163.stdout:7/379: rename d11/d26/l28 to d11/l82 0 2024-02-16T19:24:25.924 INFO:tasks.workunit.client.0.smithi040.stdout:3/982: dread d1/f34 [0,4194304] 0 2024-02-16T19:24:25.930 INFO:tasks.workunit.client.0.smithi040.stdout:1/932: dread d17/d26/d3d/f56 [4194304,4194304] 0 2024-02-16T19:24:25.930 INFO:tasks.workunit.client.0.smithi040.stdout:1/933: creat d17/d3c/d6b/de8/f131 x:0 0 0 2024-02-16T19:24:25.930 INFO:tasks.workunit.client.0.smithi040.stdout:1/934: chown d17/d3c/d6b/de8 508406399 1 2024-02-16T19:24:25.930 INFO:tasks.workunit.client.1.smithi163.stdout:9/552: dwrite d5/f18 [0,4194304] 0 2024-02-16T19:24:25.931 INFO:tasks.workunit.client.1.smithi163.stdout:9/553: fsync d5/de/d30/d31/f7f 0 2024-02-16T19:24:25.931 INFO:tasks.workunit.client.1.smithi163.stdout:9/554: chown d5/de/cc9 148 1 2024-02-16T19:24:25.933 INFO:tasks.workunit.client.0.smithi040.stdout:9/911: rmdir d3/d4 39 2024-02-16T19:24:25.937 INFO:tasks.workunit.client.0.smithi040.stdout:7/853: write d21/f28 [3626895,119301] 0 2024-02-16T19:24:25.937 INFO:tasks.workunit.client.0.smithi040.stdout:7/854: chown d21/d4d/d77/d7a/dc6/l108 41614087 1 2024-02-16T19:24:25.946 INFO:tasks.workunit.client.1.smithi163.stdout:6/334: dwrite d0/d6/df/f32 [0,4194304] 0 2024-02-16T19:24:25.946 INFO:tasks.workunit.client.1.smithi163.stdout:6/335: creat d0/d6/d9/d2e/d33/f60 x:0 0 0 2024-02-16T19:24:25.946 INFO:tasks.workunit.client.1.smithi163.stdout:6/336: creat d0/d6/d9/d2e/f61 x:0 0 0 2024-02-16T19:24:25.947 INFO:tasks.workunit.client.1.smithi163.stdout:0/588: dwrite d11/d22/f3f [0,4194304] 0 2024-02-16T19:24:25.947 INFO:tasks.workunit.client.0.smithi040.stdout:1/935: creat d17/d26/d9a/dfb/f132 x:0 0 0 2024-02-16T19:24:25.947 INFO:tasks.workunit.client.0.smithi040.stdout:1/936: dread - d17/d26/d9a/dfb/fdc zero size 2024-02-16T19:24:25.947 INFO:tasks.workunit.client.0.smithi040.stdout:1/937: dread d17/d3c/d74/d97/fc5 [0,4194304] 0 2024-02-16T19:24:25.950 INFO:tasks.workunit.client.0.smithi040.stdout:9/912: unlink d3/d4/d11/d29/d38/d46/d6b/d85/d112/d8a/d99/fb2 0 2024-02-16T19:24:25.955 INFO:tasks.workunit.client.0.smithi040.stdout:7/855: truncate d21/d29/f73 1336285 0 2024-02-16T19:24:25.956 INFO:tasks.workunit.client.0.smithi040.stdout:9/913: symlink d3/d8b/l121 0 2024-02-16T19:24:25.956 INFO:tasks.workunit.client.0.smithi040.stdout:9/914: readlink d3/d8b/l121 0 2024-02-16T19:24:25.957 INFO:tasks.workunit.client.0.smithi040.stdout:7/856: rmdir d21/d4d/d77/d7a 39 2024-02-16T19:24:25.959 INFO:tasks.workunit.client.1.smithi163.stdout:5/371: mknod d0/d2/d34/d44/d63/c71 0 2024-02-16T19:24:25.959 INFO:tasks.workunit.client.1.smithi163.stdout:5/372: creat d0/d2/d34/d44/f72 x:0 0 0 2024-02-16T19:24:25.959 INFO:tasks.workunit.client.1.smithi163.stdout:9/555: truncate d5/de/d30/d31/d52/f81 125361 0 2024-02-16T19:24:25.962 INFO:tasks.workunit.client.1.smithi163.stdout:9/556: dread d5/de/d30/d31/f69 [0,4194304] 0 2024-02-16T19:24:25.962 INFO:tasks.workunit.client.0.smithi040.stdout:7/857: dread d21/d8b/d9c/ff7 [0,4194304] 0 2024-02-16T19:24:25.963 INFO:tasks.workunit.client.1.smithi163.stdout:6/337: rename d0/dd/l5b to d0/l62 0 2024-02-16T19:24:25.964 INFO:tasks.workunit.client.0.smithi040.stdout:7/858: mkdir d21/d121 0 2024-02-16T19:24:25.964 INFO:tasks.workunit.client.0.smithi040.stdout:7/859: creat d21/d29/d79/f122 x:0 0 0 2024-02-16T19:24:25.965 INFO:tasks.workunit.client.1.smithi163.stdout:7/380: dwrite d11/f1b [0,4194304] 0 2024-02-16T19:24:25.965 INFO:tasks.workunit.client.0.smithi040.stdout:4/747: dwrite dd/de/daa/dd8/fc9 [0,4194304] 0 2024-02-16T19:24:25.966 INFO:tasks.workunit.client.1.smithi163.stdout:0/589: creat d11/d12/d67/fd2 x:0 0 0 2024-02-16T19:24:25.966 INFO:tasks.workunit.client.1.smithi163.stdout:0/590: write d11/f98 [1162842,799] 0 2024-02-16T19:24:25.967 INFO:tasks.workunit.client.0.smithi040.stdout:9/915: dread d3/d4/d11/d29/d38/d46/d6b/d85/d112/d8a/f8f [0,4194304] 0 2024-02-16T19:24:25.968 INFO:tasks.workunit.client.1.smithi163.stdout:4/310: symlink d11/l5e 0 2024-02-16T19:24:25.969 INFO:tasks.workunit.client.0.smithi040.stdout:3/983: dwrite d1/d11a/f122 [0,4194304] 0 2024-02-16T19:24:25.971 INFO:tasks.workunit.client.0.smithi040.stdout:1/938: dwrite d17/d26/d9a/dfb/f21 [0,4194304] 0 2024-02-16T19:24:25.971 INFO:tasks.workunit.client.0.smithi040.stdout:1/939: dread - d17/d3c/d6b/de8/f110 zero size 2024-02-16T19:24:25.971 INFO:tasks.workunit.client.0.smithi040.stdout:1/940: creat d17/db5/f133 x:0 0 0 2024-02-16T19:24:25.974 INFO:tasks.workunit.client.0.smithi040.stdout:7/860: link d21/d4d/d77/d7a/dc6/fe6 d21/d4d/d77/d7a/f123 0 2024-02-16T19:24:25.974 INFO:tasks.workunit.client.0.smithi040.stdout:7/861: readlink d21/d29/l39 0 2024-02-16T19:24:25.976 INFO:tasks.workunit.client.1.smithi163.stdout:5/373: rename d0/d2/l1f to d0/d1c/l73 0 2024-02-16T19:24:25.979 INFO:tasks.workunit.client.1.smithi163.stdout:6/338: getdents d0/d6/df/d20 0 2024-02-16T19:24:25.982 INFO:tasks.workunit.client.1.smithi163.stdout:7/381: unlink d11/l16 0 2024-02-16T19:24:25.983 INFO:tasks.workunit.client.0.smithi040.stdout:4/748: symlink dd/de/daa/dd8/le4 0 2024-02-16T19:24:25.984 INFO:tasks.workunit.client.1.smithi163.stdout:5/374: dread d0/f32 [0,4194304] 0 2024-02-16T19:24:25.989 INFO:tasks.workunit.client.0.smithi040.stdout:9/916: rename d3/dd3 to d3/d4/d11/d29/d38/d46/d6b/d85/d122 0 2024-02-16T19:24:25.989 INFO:tasks.workunit.client.1.smithi163.stdout:9/557: dwrite d5/de/d16/f2a [0,4194304] 0 2024-02-16T19:24:25.997 INFO:tasks.workunit.client.1.smithi163.stdout:7/382: write d11/d26/d36/f4e [8289379,107323] 0 2024-02-16T19:24:25.997 INFO:tasks.workunit.client.0.smithi040.stdout:3/984: symlink d1/d4/d6e/dc2/l136 0 2024-02-16T19:24:25.998 INFO:tasks.workunit.client.1.smithi163.stdout:6/339: mkdir d0/d18/d51/d63 0 2024-02-16T19:24:26.001 INFO:tasks.workunit.client.1.smithi163.stdout:5/375: link d0/d2/d34/f6d d0/d1c/f74 0 2024-02-16T19:24:26.002 INFO:tasks.workunit.client.1.smithi163.stdout:5/376: read d0/f4d [367630,42601] 0 2024-02-16T19:24:26.002 INFO:tasks.workunit.client.1.smithi163.stdout:5/377: write d0/d22/f33 [3973301,39128] 0 2024-02-16T19:24:26.002 INFO:tasks.workunit.client.1.smithi163.stdout:5/378: creat d0/d2/d5c/f75 x:0 0 0 2024-02-16T19:24:26.004 INFO:tasks.workunit.client.1.smithi163.stdout:4/311: getdents d11/d18/d1f/d29/d52 0 2024-02-16T19:24:26.007 INFO:tasks.workunit.client.0.smithi040.stdout:1/941: mkdir d17/d26/d5f/d78/d134 0 2024-02-16T19:24:26.007 INFO:tasks.workunit.client.0.smithi040.stdout:7/862: dwrite d21/f3c [4194304,4194304] 0 2024-02-16T19:24:26.008 INFO:tasks.workunit.client.1.smithi163.stdout:0/591: dwrite d11/d22/d7f/fb6 [0,4194304] 0 2024-02-16T19:24:26.009 INFO:tasks.workunit.client.1.smithi163.stdout:0/592: readlink d11/d21/d50/d58/l84 0 2024-02-16T19:24:26.009 INFO:tasks.workunit.client.0.smithi040.stdout:4/749: dwrite dd/de/db0/fb2 [0,4194304] 0 2024-02-16T19:24:26.009 INFO:tasks.workunit.client.0.smithi040.stdout:3/985: dread d1/d7/d36/f7c [0,4194304] 0 2024-02-16T19:24:26.010 INFO:tasks.workunit.client.0.smithi040.stdout:4/750: write dd/d58/fdc [432659,95290] 0 2024-02-16T19:24:26.011 INFO:tasks.workunit.client.1.smithi163.stdout:9/558: link d5/de/d16/f2a d5/de/d30/d8a/da8/fcb 0 2024-02-16T19:24:26.012 INFO:tasks.workunit.client.0.smithi040.stdout:4/751: dread dd/de/d10/d36/f86 [0,4194304] 0 2024-02-16T19:24:26.014 INFO:tasks.workunit.client.1.smithi163.stdout:7/383: rmdir d11/d24/d34/d77 0 2024-02-16T19:24:26.016 INFO:tasks.workunit.client.0.smithi040.stdout:9/917: creat d3/d4/d11/f123 x:0 0 0 2024-02-16T19:24:26.018 INFO:tasks.workunit.client.1.smithi163.stdout:0/593: write d11/d22/d7f/d8e/fa4 [3109956,66778] 0 2024-02-16T19:24:26.018 INFO:tasks.workunit.client.1.smithi163.stdout:0/594: dread - d11/fb4 zero size 2024-02-16T19:24:26.019 INFO:tasks.workunit.client.1.smithi163.stdout:0/595: write d11/d22/d3e/f6f [4563755,109655] 0 2024-02-16T19:24:26.020 INFO:tasks.workunit.client.0.smithi040.stdout:1/942: dread f5 [0,4194304] 0 2024-02-16T19:24:26.021 INFO:tasks.workunit.client.0.smithi040.stdout:1/943: dread - d17/d3c/d74/d97/fe5 zero size 2024-02-16T19:24:26.021 INFO:tasks.workunit.client.0.smithi040.stdout:9/918: dread d3/d4/d11/f67 [0,4194304] 0 2024-02-16T19:24:26.022 INFO:tasks.workunit.client.0.smithi040.stdout:7/863: link d21/c111 d21/c124 0 2024-02-16T19:24:26.022 INFO:tasks.workunit.client.0.smithi040.stdout:7/864: dread - d21/d29/dad/ff5 zero size 2024-02-16T19:24:26.023 INFO:tasks.workunit.client.0.smithi040.stdout:9/919: write d3/d4/d11/d29/f52 [50258,79062] 0 2024-02-16T19:24:26.024 INFO:tasks.workunit.client.1.smithi163.stdout:5/379: symlink d0/d2/d5c/l76 0 2024-02-16T19:24:26.029 INFO:tasks.workunit.client.1.smithi163.stdout:4/312: unlink d11/d2d/l59 0 2024-02-16T19:24:26.031 INFO:tasks.workunit.client.0.smithi040.stdout:3/986: mknod d1/d4/d6e/c137 0 2024-02-16T19:24:26.032 INFO:tasks.workunit.client.1.smithi163.stdout:5/380: dread d0/f39 [0,4194304] 0 2024-02-16T19:24:26.033 INFO:tasks.workunit.client.0.smithi040.stdout:3/987: write d1/dc/d42/d58/f93 [213521,54983] 0 2024-02-16T19:24:26.033 INFO:tasks.workunit.client.0.smithi040.stdout:4/752: symlink dd/de/le5 0 2024-02-16T19:24:26.033 INFO:tasks.workunit.client.0.smithi040.stdout:4/753: truncate dd/d58/d5e/dd1/d9d/fe2 629866 0 2024-02-16T19:24:26.034 INFO:tasks.workunit.client.1.smithi163.stdout:7/384: dwrite d11/d26/d36/f3c [0,4194304] 0 2024-02-16T19:24:26.034 INFO:tasks.workunit.client.1.smithi163.stdout:7/385: chown d11/l13 35 1 2024-02-16T19:24:26.036 INFO:tasks.workunit.client.0.smithi040.stdout:1/944: creat d17/d3c/dde/f135 x:0 0 0 2024-02-16T19:24:26.037 INFO:tasks.workunit.client.0.smithi040.stdout:1/945: read d17/d26/d5f/fc3 [153694,93659] 0 2024-02-16T19:24:26.041 INFO:tasks.workunit.client.0.smithi040.stdout:3/988: mknod d1/dc/d35/d10a/c138 0 2024-02-16T19:24:26.046 INFO:tasks.workunit.client.1.smithi163.stdout:8/343: sync 2024-02-16T19:24:26.046 INFO:tasks.workunit.client.1.smithi163.stdout:8/344: readlink d0/dc/l4f 0 2024-02-16T19:24:26.047 INFO:tasks.workunit.client.0.smithi040.stdout:3/989: mknod d1/d7/d30/c139 0 2024-02-16T19:24:26.047 INFO:tasks.workunit.client.0.smithi040.stdout:3/990: write d1/d4/fa [6693794,120489] 0 2024-02-16T19:24:26.050 INFO:tasks.workunit.client.1.smithi163.stdout:9/559: rename d5/de/d30/d31/f7f to d5/dc0/fcc 0 2024-02-16T19:24:26.052 INFO:tasks.workunit.client.1.smithi163.stdout:6/340: getdents d0/d6/d9/d2e 0 2024-02-16T19:24:26.052 INFO:tasks.workunit.client.1.smithi163.stdout:6/341: fsync d0/d6/d9/d2e/d33/f38 0 2024-02-16T19:24:26.053 INFO:tasks.workunit.client.0.smithi040.stdout:1/946: rename d17/d3c/d74/d97/da9/d10b to d17/d26/d9a/dfb/d92/dca/d136 0 2024-02-16T19:24:26.054 INFO:tasks.workunit.client.0.smithi040.stdout:9/920: dwrite d3/d4/d4a/db4/df7/f117 [0,4194304] 0 2024-02-16T19:24:26.054 INFO:tasks.workunit.client.0.smithi040.stdout:9/921: chown d3/d4/d11/d71/dc9/dca/f104 1105688 1 2024-02-16T19:24:26.056 INFO:tasks.workunit.client.1.smithi163.stdout:8/345: dread d0/d7/d18/d2e/f5d [0,4194304] 0 2024-02-16T19:24:26.056 INFO:tasks.workunit.client.1.smithi163.stdout:8/346: write d0/d7/d19/d1f/d32/f59 [824970,104118] 0 2024-02-16T19:24:26.057 INFO:tasks.workunit.client.0.smithi040.stdout:4/754: dwrite dd/de/d45/f74 [4194304,4194304] 0 2024-02-16T19:24:26.057 INFO:tasks.workunit.client.0.smithi040.stdout:1/947: rename d17/d26/d3d/f10a to d17/f137 0 2024-02-16T19:24:26.059 INFO:tasks.workunit.client.1.smithi163.stdout:4/313: mkdir d11/d2d/d4f/d5f 0 2024-02-16T19:24:26.061 INFO:tasks.workunit.client.1.smithi163.stdout:5/381: symlink d0/d2/d34/d44/d64/l77 0 2024-02-16T19:24:26.061 INFO:tasks.workunit.client.1.smithi163.stdout:5/382: write d0/d22/d28/d36/d3c/d4f/f50 [1804504,110890] 0 2024-02-16T19:24:26.062 INFO:tasks.workunit.client.1.smithi163.stdout:5/383: truncate d0/d2/f4 1581283 0 2024-02-16T19:24:26.062 INFO:tasks.workunit.client.1.smithi163.stdout:5/384: read d0/d2/fc [860406,124226] 0 2024-02-16T19:24:26.062 INFO:tasks.workunit.client.1.smithi163.stdout:5/385: truncate d0/d22/f23 795336 0 2024-02-16T19:24:26.063 INFO:tasks.workunit.client.1.smithi163.stdout:7/386: dwrite d11/d50/f7c [0,4194304] 0 2024-02-16T19:24:26.069 INFO:tasks.workunit.client.0.smithi040.stdout:9/922: dread d3/d4/d11/d29/d38/d46/d6b/d85/d112/f2e [0,4194304] 0 2024-02-16T19:24:26.071 INFO:tasks.workunit.client.1.smithi163.stdout:9/560: rmdir d5/de/d30/d3d/d49/d6d/db8 0 2024-02-16T19:24:26.071 INFO:tasks.workunit.client.1.smithi163.stdout:9/561: write d5/de/d30/d3d/d49/d6e/f92 [492568,129433] 0 2024-02-16T19:24:26.072 INFO:tasks.workunit.client.1.smithi163.stdout:3/403: sync 2024-02-16T19:24:26.072 INFO:tasks.workunit.client.1.smithi163.stdout:2/500: sync 2024-02-16T19:24:26.073 INFO:tasks.workunit.client.1.smithi163.stdout:1/341: sync 2024-02-16T19:24:26.073 INFO:tasks.workunit.client.1.smithi163.stdout:1/342: stat d7/d20/d2a 0 2024-02-16T19:24:26.073 INFO:tasks.workunit.client.1.smithi163.stdout:2/501: dread - d2/d2a/d2d/d73/d84/f8e zero size 2024-02-16T19:24:26.073 INFO:tasks.workunit.client.1.smithi163.stdout:1/343: write d7/dc/d2d/d32/d3b/f4d [553737,112723] 0 2024-02-16T19:24:26.074 INFO:tasks.workunit.client.1.smithi163.stdout:6/342: mknod d0/d18/d51/c64 0 2024-02-16T19:24:26.075 INFO:tasks.workunit.client.0.smithi040.stdout:4/755: mknod dd/d58/d5e/dd1/d9d/dd6/ce6 0 2024-02-16T19:24:26.075 INFO:tasks.workunit.client.0.smithi040.stdout:4/756: stat dd/de/d10/d36/d9e/d32/c35 0 2024-02-16T19:24:26.075 INFO:tasks.workunit.client.1.smithi163.stdout:7/387: dread f5 [0,4194304] 0 2024-02-16T19:24:26.076 INFO:tasks.workunit.client.1.smithi163.stdout:0/596: mknod d11/d4f/cd3 0 2024-02-16T19:24:26.078 INFO:tasks.workunit.client.0.smithi040.stdout:1/948: rmdir d17/d29/d8b/d10e/d121/d122/dab/dd3/d111 0 2024-02-16T19:24:26.080 INFO:tasks.workunit.client.1.smithi163.stdout:8/347: dwrite d0/dc/f16 [0,4194304] 0 2024-02-16T19:24:26.080 INFO:tasks.workunit.client.1.smithi163.stdout:8/348: fsync d0/d7/f68 0 2024-02-16T19:24:26.080 INFO:tasks.workunit.client.1.smithi163.stdout:5/386: symlink d0/d2/d55/l78 0 2024-02-16T19:24:26.081 INFO:tasks.workunit.client.0.smithi040.stdout:9/923: dread d3/d4/d11/d29/d38/d46/d6b/dac/ff3 [0,4194304] 0 2024-02-16T19:24:26.085 INFO:tasks.workunit.client.0.smithi040.stdout:9/924: write d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/f113 [2134244,106811] 0 2024-02-16T19:24:26.085 INFO:tasks.workunit.client.0.smithi040.stdout:4/757: creat dd/d58/d5e/fe7 x:0 0 0 2024-02-16T19:24:26.088 INFO:tasks.workunit.client.0.smithi040.stdout:1/949: creat d17/d26/d54/f138 x:0 0 0 2024-02-16T19:24:26.089 INFO:tasks.workunit.client.1.smithi163.stdout:5/387: write d0/d2/f30 [1901637,57748] 0 2024-02-16T19:24:26.092 INFO:tasks.workunit.client.1.smithi163.stdout:9/562: symlink d5/de/d30/d43/lcd 0 2024-02-16T19:24:26.095 INFO:tasks.workunit.client.1.smithi163.stdout:3/404: fdatasync d0/dc/d1d/d29/f33 0 2024-02-16T19:24:26.096 INFO:tasks.workunit.client.0.smithi040.stdout:4/758: unlink dd/de/d10/l2c 0 2024-02-16T19:24:26.097 INFO:tasks.workunit.client.1.smithi163.stdout:2/502: symlink d2/d26/d54/d58/d98/lb0 0 2024-02-16T19:24:26.099 INFO:tasks.workunit.client.0.smithi040.stdout:9/925: read d3/d4/d11/d71/da8/f43 [629683,112344] 0 2024-02-16T19:24:26.099 INFO:tasks.workunit.client.0.smithi040.stdout:9/926: write d3/d4/d11/d29/f24 [4342636,54295] 0 2024-02-16T19:24:26.102 INFO:tasks.workunit.client.0.smithi040.stdout:4/759: symlink dd/de/daa/le8 0 2024-02-16T19:24:26.102 INFO:tasks.workunit.client.0.smithi040.stdout:4/760: chown dd/de/d10/f4a 2730080 1 2024-02-16T19:24:26.105 INFO:tasks.workunit.client.0.smithi040.stdout:1/950: dwrite d17/d3c/d74/d97/fc5 [0,4194304] 0 2024-02-16T19:24:26.106 INFO:tasks.workunit.client.1.smithi163.stdout:2/503: dread d2/f15 [0,4194304] 0 2024-02-16T19:24:26.107 INFO:tasks.workunit.client.0.smithi040.stdout:9/927: creat d3/d4/d11/d71/dc9/dca/f124 x:0 0 0 2024-02-16T19:24:26.109 INFO:tasks.workunit.client.0.smithi040.stdout:4/761: mknod dd/de/d10/d36/dc3/ce9 0 2024-02-16T19:24:26.115 INFO:tasks.workunit.client.0.smithi040.stdout:1/951: mkdir d17/d26/d9a/dfb/d92/dca/d139 0 2024-02-16T19:24:26.117 INFO:tasks.workunit.client.1.smithi163.stdout:0/597: mkdir d11/d12/d67/dd4 0 2024-02-16T19:24:26.117 INFO:tasks.workunit.client.1.smithi163.stdout:0/598: stat d11/d21/d50/d58/l73 0 2024-02-16T19:24:26.118 INFO:tasks.workunit.client.0.smithi040.stdout:9/928: getdents d3/d4/d11/d29/d38/d46/d6b/d85/d122 0 2024-02-16T19:24:26.118 INFO:tasks.workunit.client.0.smithi040.stdout:9/929: write d3/d4/d11/d29/d38/d46/d6b/f97 [1317217,55084] 0 2024-02-16T19:24:26.123 INFO:tasks.workunit.client.0.smithi040.stdout:4/762: creat dd/de/d10/d36/d9e/fea x:0 0 0 2024-02-16T19:24:26.127 INFO:tasks.workunit.client.1.smithi163.stdout:8/349: rename d0/d7/f6b to d0/d24/d2c/d74/f79 0 2024-02-16T19:24:26.128 INFO:tasks.workunit.client.1.smithi163.stdout:8/350: fsync d0/d24/d3c/f4c 0 2024-02-16T19:24:26.128 INFO:tasks.workunit.client.1.smithi163.stdout:0/599: dread d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:26.128 INFO:tasks.workunit.client.1.smithi163.stdout:0/600: readlink d11/d22/d3e/l4b 0 2024-02-16T19:24:26.129 INFO:tasks.workunit.client.0.smithi040.stdout:9/930: mknod d3/d4/dbe/c125 0 2024-02-16T19:24:26.130 INFO:tasks.workunit.client.1.smithi163.stdout:5/388: mkdir d0/d2/d34/d44/d63/d79 0 2024-02-16T19:24:26.131 INFO:tasks.workunit.client.1.smithi163.stdout:9/563: unlink d5/f14 0 2024-02-16T19:24:26.133 INFO:tasks.workunit.client.0.smithi040.stdout:9/931: truncate d3/d4/d11/d29/d38/d46/d6b/fc8 2609669 0 2024-02-16T19:24:26.133 INFO:tasks.workunit.client.0.smithi040.stdout:9/932: chown d3/d4/f2c 1 1 2024-02-16T19:24:26.139 INFO:tasks.workunit.client.0.smithi040.stdout:4/763: dwrite dd/de/d45/fd4 [0,4194304] 0 2024-02-16T19:24:26.141 INFO:tasks.workunit.client.1.smithi163.stdout:1/344: rmdir d7 39 2024-02-16T19:24:26.143 INFO:tasks.workunit.client.1.smithi163.stdout:6/343: getdents d0/d6/d9 0 2024-02-16T19:24:26.150 INFO:tasks.workunit.client.0.smithi040.stdout:4/764: symlink dd/de/d71/leb 0 2024-02-16T19:24:26.151 INFO:tasks.workunit.client.1.smithi163.stdout:7/388: getdents d11/d30 0 2024-02-16T19:24:26.153 INFO:tasks.workunit.client.0.smithi040.stdout:4/765: unlink dd/d58/l88 0 2024-02-16T19:24:26.153 INFO:tasks.workunit.client.0.smithi040.stdout:4/766: write dd/de/d10/f4a [313313,28098] 0 2024-02-16T19:24:26.153 INFO:tasks.workunit.client.0.smithi040.stdout:4/767: dread - dd/d58/d5e/dd1/d9d/fce zero size 2024-02-16T19:24:26.154 INFO:tasks.workunit.client.1.smithi163.stdout:8/351: truncate d0/d7/d18/d2e/f5f 318273 0 2024-02-16T19:24:26.156 INFO:tasks.workunit.client.0.smithi040.stdout:4/768: creat dd/d58/d5e/dd1/fec x:0 0 0 2024-02-16T19:24:26.156 INFO:tasks.workunit.client.0.smithi040.stdout:4/769: fsync dd/f66 0 2024-02-16T19:24:26.156 INFO:tasks.workunit.client.0.smithi040.stdout:4/770: fsync dd/d58/d5e/f7f 0 2024-02-16T19:24:26.156 INFO:tasks.workunit.client.0.smithi040.stdout:4/771: readlink dd/de/d10/l25 0 2024-02-16T19:24:26.156 INFO:tasks.workunit.client.0.smithi040.stdout:4/772: readlink dd/l78 0 2024-02-16T19:24:26.156 INFO:tasks.workunit.client.1.smithi163.stdout:0/601: symlink d11/d12/d67/dd4/ld5 0 2024-02-16T19:24:26.158 INFO:tasks.workunit.client.0.smithi040.stdout:4/773: creat dd/fed x:0 0 0 2024-02-16T19:24:26.161 INFO:tasks.workunit.client.0.smithi040.stdout:4/774: rename dd/d58/fdd to dd/de/fee 0 2024-02-16T19:24:26.161 INFO:tasks.workunit.client.1.smithi163.stdout:3/405: mkdir d0/dc/d1c/d91 0 2024-02-16T19:24:26.163 INFO:tasks.workunit.client.1.smithi163.stdout:6/344: rename d0/d18/d51 to d0/d6/d9/d15/d1c/d65 0 2024-02-16T19:24:26.164 INFO:tasks.workunit.client.1.smithi163.stdout:8/352: creat d0/d7/d18/d53/f7a x:0 0 0 2024-02-16T19:24:26.164 INFO:tasks.workunit.client.1.smithi163.stdout:8/353: fsync d0/dc/f6d 0 2024-02-16T19:24:26.165 INFO:tasks.workunit.client.1.smithi163.stdout:0/602: link d11/l25 d11/d12/d67/dd4/ld6 0 2024-02-16T19:24:26.165 INFO:tasks.workunit.client.1.smithi163.stdout:0/603: fdatasync d11/d22/fa7 0 2024-02-16T19:24:26.167 INFO:tasks.workunit.client.1.smithi163.stdout:0/604: write d11/d22/d3e/f66 [400921,49853] 0 2024-02-16T19:24:26.168 INFO:tasks.workunit.client.1.smithi163.stdout:0/605: write d11/d12/d67/fd2 [732979,47446] 0 2024-02-16T19:24:26.168 INFO:tasks.workunit.client.1.smithi163.stdout:0/606: readlink d11/l8a 0 2024-02-16T19:24:26.168 INFO:tasks.workunit.client.1.smithi163.stdout:3/406: dread d0/dc/d1c/d32/f3f [0,4194304] 0 2024-02-16T19:24:26.168 INFO:tasks.workunit.client.0.smithi040.stdout:4/775: dread dd/de/d10/d36/d9e/d77/f3b [0,4194304] 0 2024-02-16T19:24:26.168 INFO:tasks.workunit.client.1.smithi163.stdout:1/345: creat d7/f7d x:0 0 0 2024-02-16T19:24:26.169 INFO:tasks.workunit.client.1.smithi163.stdout:7/389: rename d11/d30/l47 to d11/d24/d25/d4b/d66/l83 0 2024-02-16T19:24:26.170 INFO:tasks.workunit.client.1.smithi163.stdout:6/345: symlink d0/d6/l66 0 2024-02-16T19:24:26.171 INFO:tasks.workunit.client.1.smithi163.stdout:6/346: write d0/d6/d9/f2d [4266230,42098] 0 2024-02-16T19:24:26.171 INFO:tasks.workunit.client.1.smithi163.stdout:6/347: write d0/d6/d9/d2e/d33/f60 [78749,7242] 0 2024-02-16T19:24:26.171 INFO:tasks.workunit.client.1.smithi163.stdout:6/348: chown d0/d6/d9/d15/d1c/d65/c64 65943813 1 2024-02-16T19:24:26.171 INFO:tasks.workunit.client.1.smithi163.stdout:6/349: fdatasync d0/d6/df/d20/d40/d17/f4b 0 2024-02-16T19:24:26.171 INFO:tasks.workunit.client.1.smithi163.stdout:0/607: creat d11/d22/fd7 x:0 0 0 2024-02-16T19:24:26.173 INFO:tasks.workunit.client.1.smithi163.stdout:0/608: truncate d11/d21/f46 1980137 0 2024-02-16T19:24:26.173 INFO:tasks.workunit.client.1.smithi163.stdout:0/609: write d11/d21/d7c/fce [445015,20375] 0 2024-02-16T19:24:26.175 INFO:tasks.workunit.client.1.smithi163.stdout:0/610: stat d11/d21/d50/d58/l84 0 2024-02-16T19:24:26.177 INFO:tasks.workunit.client.1.smithi163.stdout:8/354: write d0/d7/d18/f37 [4947525,97365] 0 2024-02-16T19:24:26.177 INFO:tasks.workunit.client.1.smithi163.stdout:8/355: mkdir d0/d7/d19/d1f/d30/d6a/d7b 0 2024-02-16T19:24:26.179 INFO:tasks.workunit.client.1.smithi163.stdout:8/356: truncate d0/d7/d18/d2e/f5c 4398585 0 2024-02-16T19:24:26.184 INFO:tasks.workunit.client.1.smithi163.stdout:1/346: write d7/dc/d2d/d32/d3b/f50 [982727,13043] 0 2024-02-16T19:24:26.186 INFO:tasks.workunit.client.1.smithi163.stdout:1/347: stat d7/d20/l47 0 2024-02-16T19:24:26.187 INFO:tasks.workunit.client.1.smithi163.stdout:1/348: chown d7/d20/c6e 376 1 2024-02-16T19:24:26.187 INFO:tasks.workunit.client.1.smithi163.stdout:1/349: write d7/dc/d2d/d32/d3b/f4d [1447754,74982] 0 2024-02-16T19:24:26.187 INFO:tasks.workunit.client.1.smithi163.stdout:3/407: write d0/dc/d50/f8b [2501785,42044] 0 2024-02-16T19:24:26.187 INFO:tasks.workunit.client.1.smithi163.stdout:3/408: dread - d0/dc/d1d/d29/f72 zero size 2024-02-16T19:24:26.192 INFO:tasks.workunit.client.1.smithi163.stdout:3/409: rename d0/dc/d50/f58 to d0/f92 0 2024-02-16T19:24:26.192 INFO:tasks.workunit.client.1.smithi163.stdout:3/410: fsync d0/f43 0 2024-02-16T19:24:26.194 INFO:tasks.workunit.client.1.smithi163.stdout:3/411: mkdir d0/dc/d50/d66/d93 0 2024-02-16T19:24:26.194 INFO:tasks.workunit.client.1.smithi163.stdout:3/412: chown d0/c3d 2964 1 2024-02-16T19:24:26.196 INFO:tasks.workunit.client.1.smithi163.stdout:1/350: dread d7/dc/d2d/d32/d3b/f58 [0,4194304] 0 2024-02-16T19:24:26.196 INFO:tasks.workunit.client.1.smithi163.stdout:1/351: stat d7/d5b/f6a 0 2024-02-16T19:24:26.197 INFO:tasks.workunit.client.1.smithi163.stdout:1/352: getdents d7/dc/d2d/d32 0 2024-02-16T19:24:26.199 INFO:tasks.workunit.client.1.smithi163.stdout:1/353: rename d7/dc/d78/d7c/c31 to d7/dc/d2d/d32/c7e 0 2024-02-16T19:24:26.200 INFO:tasks.workunit.client.1.smithi163.stdout:1/354: readlink d7/l8 0 2024-02-16T19:24:26.202 INFO:tasks.workunit.client.1.smithi163.stdout:1/355: mknod d7/dc/d78/d7c/c7f 0 2024-02-16T19:24:26.203 INFO:tasks.workunit.client.1.smithi163.stdout:1/356: mknod d7/dc/d78/c80 0 2024-02-16T19:24:26.205 INFO:tasks.workunit.client.0.smithi040.stdout:1/952: dwrite d17/d26/d9a/dfb/fdc [0,4194304] 0 2024-02-16T19:24:26.205 INFO:tasks.workunit.client.0.smithi040.stdout:1/953: getdents d17/d26/d3d/d130 0 2024-02-16T19:24:26.205 INFO:tasks.workunit.client.0.smithi040.stdout:1/954: chown d17/d26/d5f/d78/ce4 7309830 1 2024-02-16T19:24:26.205 INFO:tasks.workunit.client.0.smithi040.stdout:1/955: chown d17/d26/d9a/dfb/ffd 2330610 1 2024-02-16T19:24:26.206 INFO:tasks.workunit.client.1.smithi163.stdout:1/357: creat d7/dc/d78/d7c/f81 x:0 0 0 2024-02-16T19:24:26.206 INFO:tasks.workunit.client.1.smithi163.stdout:1/358: chown d7/dc/d2d/l73 28809 1 2024-02-16T19:24:26.206 INFO:tasks.workunit.client.1.smithi163.stdout:1/359: chown d7 10439326 1 2024-02-16T19:24:26.209 INFO:tasks.workunit.client.1.smithi163.stdout:6/350: dwrite d0/d6/d9/d2e/f61 [0,4194304] 0 2024-02-16T19:24:26.210 INFO:tasks.workunit.client.1.smithi163.stdout:7/390: dwrite d11/d24/d25/f4a [0,4194304] 0 2024-02-16T19:24:26.211 INFO:tasks.workunit.client.0.smithi040.stdout:4/776: fsync dd/de/d10/d36/d9e/fea 0 2024-02-16T19:24:26.211 INFO:tasks.workunit.client.0.smithi040.stdout:4/777: write dd/f43 [5171709,88984] 0 2024-02-16T19:24:26.212 INFO:tasks.workunit.client.1.smithi163.stdout:7/391: dread d11/d24/d25/f5a [0,4194304] 0 2024-02-16T19:24:26.212 INFO:tasks.workunit.client.1.smithi163.stdout:7/392: truncate d11/d24/d25/f48 3248 0 2024-02-16T19:24:26.212 INFO:tasks.workunit.client.1.smithi163.stdout:7/393: creat d11/d24/d49/f84 x:0 0 0 2024-02-16T19:24:26.212 INFO:tasks.workunit.client.1.smithi163.stdout:7/394: truncate d11/d26/d36/f3c 5257464 0 2024-02-16T19:24:26.214 INFO:tasks.workunit.client.1.smithi163.stdout:5/389: rmdir d0/d2 39 2024-02-16T19:24:26.220 INFO:tasks.workunit.client.1.smithi163.stdout:3/413: dread d0/f13 [0,4194304] 0 2024-02-16T19:24:26.222 INFO:tasks.workunit.client.0.smithi040.stdout:1/956: symlink d17/d29/d8b/dd2/l13a 0 2024-02-16T19:24:26.222 INFO:tasks.workunit.client.0.smithi040.stdout:1/957: write d17/d26/d5f/d78/dac/f11b [582751,26212] 0 2024-02-16T19:24:26.222 INFO:tasks.workunit.client.0.smithi040.stdout:1/958: stat d17/d3c 0 2024-02-16T19:24:26.224 INFO:tasks.workunit.client.0.smithi040.stdout:9/933: dwrite d3/d4/f2c [4194304,4194304] 0 2024-02-16T19:24:26.224 INFO:tasks.workunit.client.0.smithi040.stdout:9/934: fdatasync d3/d4/d11/d29/d38/d46/d6b/d85/d112/d63/f105 0 2024-02-16T19:24:26.225 INFO:tasks.workunit.client.0.smithi040.stdout:4/778: dread dd/d58/d5e/f7f [0,4194304] 0 2024-02-16T19:24:26.225 INFO:tasks.workunit.client.0.smithi040.stdout:4/779: chown dd/d58/d5e/ld3 400 1 2024-02-16T19:24:26.225 INFO:tasks.workunit.client.0.smithi040.stdout:1/959: getdents d17/d29/d8b/d10e/d121/d122/dab/dd3 0 2024-02-16T19:24:26.226 INFO:tasks.workunit.client.1.smithi163.stdout:0/611: dwrite d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:26.229 INFO:tasks.workunit.client.1.smithi163.stdout:8/357: dwrite d0/d24/d27/d45/f75 [0,4194304] 0 2024-02-16T19:24:26.230 INFO:tasks.workunit.client.1.smithi163.stdout:8/358: fsync d0/d7/d18/d2e/f5d 0 2024-02-16T19:24:26.230 INFO:tasks.workunit.client.1.smithi163.stdout:8/359: read - d0/d7/d19/d1f/d30/f52 zero size 2024-02-16T19:24:26.231 INFO:tasks.workunit.client.1.smithi163.stdout:7/395: link d11/d24/d25/f5a d11/d26/d36/d67/f85 0 2024-02-16T19:24:26.232 INFO:tasks.workunit.client.0.smithi040.stdout:1/960: dread d17/d26/f53 [0,4194304] 0 2024-02-16T19:24:26.233 INFO:tasks.workunit.client.0.smithi040.stdout:9/935: rename d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/l34 to d3/d4/d11/d29/d38/d86/l126 0 2024-02-16T19:24:26.233 INFO:tasks.workunit.client.0.smithi040.stdout:9/936: truncate d3/d4/d11/d71/da8/f58 4610726 0 2024-02-16T19:24:26.234 INFO:tasks.workunit.client.1.smithi163.stdout:5/390: creat d0/d2/d34/d44/d63/f7a x:0 0 0 2024-02-16T19:24:26.235 INFO:tasks.workunit.client.0.smithi040.stdout:1/961: creat d17/d26/d9a/dfb/d92/daa/f13b x:0 0 0 2024-02-16T19:24:26.235 INFO:tasks.workunit.client.0.smithi040.stdout:1/962: write d17/d29/d8b/dd2/f113 [314873,4274] 0 2024-02-16T19:24:26.236 INFO:tasks.workunit.client.0.smithi040.stdout:1/963: write d17/d26/d5f/da6/fc6 [348312,45671] 0 2024-02-16T19:24:26.238 INFO:tasks.workunit.client.0.smithi040.stdout:4/780: dread dd/de/d10/f30 [0,4194304] 0 2024-02-16T19:24:26.238 INFO:tasks.workunit.client.0.smithi040.stdout:4/781: write dd/de/d10/f2e [2877358,53523] 0 2024-02-16T19:24:26.238 INFO:tasks.workunit.client.0.smithi040.stdout:4/782: truncate dd/de/d10/d36/d9e/fea 632429 0 2024-02-16T19:24:26.238 INFO:tasks.workunit.client.0.smithi040.stdout:1/964: unlink d17/d29/d8b/dd2/f113 0 2024-02-16T19:24:26.242 INFO:tasks.workunit.client.1.smithi163.stdout:8/360: dread d0/d7/d19/d1f/d30/d48/f65 [0,4194304] 0 2024-02-16T19:24:26.242 INFO:tasks.workunit.client.1.smithi163.stdout:8/361: chown d0/d24/d2c/f4d 19711 1 2024-02-16T19:24:26.242 INFO:tasks.workunit.client.1.smithi163.stdout:8/362: chown d0/d6f 4 1 2024-02-16T19:24:26.243 INFO:tasks.workunit.client.0.smithi040.stdout:9/937: write d3/d4/d11/d29/d36/f74 [2988136,6868] 0 2024-02-16T19:24:26.243 INFO:tasks.workunit.client.0.smithi040.stdout:9/938: fdatasync d3/d4/d4a/fda 0 2024-02-16T19:24:26.243 INFO:tasks.workunit.client.1.smithi163.stdout:1/360: dwrite d7/dc/d2d/d32/f64 [0,4194304] 0 2024-02-16T19:24:26.252 INFO:tasks.workunit.client.0.smithi040.stdout:4/783: write dd/de/db0/fb2 [2905099,106009] 0 2024-02-16T19:24:26.256 INFO:tasks.workunit.client.1.smithi163.stdout:9/564: stat d5/de/d16/f53 0 2024-02-16T19:24:26.256 INFO:tasks.workunit.client.0.smithi040.stdout:9/939: write d3/d4/d11/d29/d36/f3a [2000754,15007] 0 2024-02-16T19:24:26.264 INFO:tasks.workunit.client.0.smithi040.stdout:4/784: write dd/de/d10/d36/d9e/f46 [116531,38344] 0 2024-02-16T19:24:26.269 INFO:tasks.workunit.client.0.smithi040.stdout:9/940: dread d3/d4/d4a/db4/df7/daf/fe1 [0,4194304] 0 2024-02-16T19:24:26.271 INFO:tasks.workunit.client.1.smithi163.stdout:5/391: dwrite d0/d22/f23 [0,4194304] 0 2024-02-16T19:24:26.272 INFO:tasks.workunit.client.1.smithi163.stdout:5/392: dread - d0/d2/d34/d44/f72 zero size 2024-02-16T19:24:26.273 INFO:tasks.workunit.client.0.smithi040.stdout:1/965: dwrite d17/d29/fb8 [0,4194304] 0 2024-02-16T19:24:26.273 INFO:tasks.workunit.client.0.smithi040.stdout:1/966: write d17/d29/d8b/d10e/d112/f12a [464634,72113] 0 2024-02-16T19:24:26.273 INFO:tasks.workunit.client.0.smithi040.stdout:4/785: symlink dd/de/d10/d36/dc3/lef 0 2024-02-16T19:24:26.275 INFO:tasks.workunit.client.0.smithi040.stdout:1/967: dread d17/d26/d5f/fe1 [0,4194304] 0 2024-02-16T19:24:26.275 INFO:tasks.workunit.client.0.smithi040.stdout:1/968: truncate d17/d29/fb8 5059645 0 2024-02-16T19:24:26.280 INFO:tasks.workunit.client.0.smithi040.stdout:9/941: link d3/d4/d11/d71/da8/dde/c10f d3/d4/d11/d29/d38/d46/d6b/d85/d112/d70/dcc/c127 0 2024-02-16T19:24:26.281 INFO:tasks.workunit.client.1.smithi163.stdout:0/612: unlink d11/d12/d67/d6a/laa 0 2024-02-16T19:24:26.284 INFO:tasks.workunit.client.1.smithi163.stdout:6/351: rmdir d0/d6 39 2024-02-16T19:24:26.289 INFO:tasks.workunit.client.0.smithi040.stdout:4/786: mkdir dd/d58/d5e/dd1/dc6/df0 0 2024-02-16T19:24:26.291 INFO:tasks.workunit.client.1.smithi163.stdout:7/396: truncate f2 487005 0 2024-02-16T19:24:26.291 INFO:tasks.workunit.client.1.smithi163.stdout:7/397: creat d11/d50/f86 x:0 0 0 2024-02-16T19:24:26.294 INFO:tasks.workunit.client.1.smithi163.stdout:8/363: symlink d0/l7c 0 2024-02-16T19:24:26.300 INFO:tasks.workunit.client.1.smithi163.stdout:1/361: mknod d7/d5b/d63/c82 0 2024-02-16T19:24:26.302 INFO:tasks.workunit.client.1.smithi163.stdout:3/414: mkdir d0/dc/d94 0 2024-02-16T19:24:26.304 INFO:tasks.workunit.client.1.smithi163.stdout:9/565: symlink d5/de/d30/d3d/d49/d6d/lce 0 2024-02-16T19:24:26.304 INFO:tasks.workunit.client.1.smithi163.stdout:9/566: stat d5/de/d30/d31/d52/c83 0 2024-02-16T19:24:26.304 INFO:tasks.workunit.client.0.smithi040.stdout:1/969: dwrite d17/d29/d8b/dd2/fe7 [0,4194304] 0 2024-02-16T19:24:26.307 INFO:tasks.workunit.client.0.smithi040.stdout:9/942: dwrite d3/d4/d4a/db4/df7/daf/fe1 [4194304,4194304] 0 2024-02-16T19:24:26.308 INFO:tasks.workunit.client.0.smithi040.stdout:1/970: creat d17/d26/d5f/d78/d134/f13c x:0 0 0 2024-02-16T19:24:26.308 INFO:tasks.workunit.client.0.smithi040.stdout:1/971: chown d17/d26/d3d/cf1 204 1 2024-02-16T19:24:26.311 INFO:tasks.workunit.client.0.smithi040.stdout:9/943: mknod d3/d4/d4a/d114/c128 0 2024-02-16T19:24:26.313 INFO:tasks.workunit.client.1.smithi163.stdout:0/613: mkdir d11/d21/d50/d58/d72/d9e/dd8 0 2024-02-16T19:24:26.314 INFO:tasks.workunit.client.1.smithi163.stdout:0/614: write d11/d12/d67/d6a/d6d/f93 [776462,51577] 0 2024-02-16T19:24:26.315 INFO:tasks.workunit.client.0.smithi040.stdout:9/944: mkdir d3/d1c/d73/d129 0 2024-02-16T19:24:26.316 INFO:tasks.workunit.client.1.smithi163.stdout:9/567: dread d5/f40 [0,4194304] 0 2024-02-16T19:24:26.316 INFO:tasks.workunit.client.1.smithi163.stdout:6/352: symlink d0/d6/l67 0 2024-02-16T19:24:26.318 INFO:tasks.workunit.client.1.smithi163.stdout:8/364: dwrite d0/d24/d2c/f3b [0,4194304] 0 2024-02-16T19:24:26.318 INFO:tasks.workunit.client.1.smithi163.stdout:8/365: write d0/dc/f16 [4346238,69942] 0 2024-02-16T19:24:26.318 INFO:tasks.workunit.client.1.smithi163.stdout:8/366: creat d0/dc/f7d x:0 0 0 2024-02-16T19:24:26.318 INFO:tasks.workunit.client.1.smithi163.stdout:8/367: write d0/d7/d19/f2f [358508,14095] 0 2024-02-16T19:24:26.318 INFO:tasks.workunit.client.0.smithi040.stdout:9/945: symlink d3/de5/l12a 0 2024-02-16T19:24:26.321 INFO:tasks.workunit.client.1.smithi163.stdout:7/398: symlink d11/d24/d25/l87 0 2024-02-16T19:24:26.321 INFO:tasks.workunit.client.1.smithi163.stdout:7/399: creat d11/d50/f88 x:0 0 0 2024-02-16T19:24:26.321 INFO:tasks.workunit.client.1.smithi163.stdout:7/400: write fc [343798,101057] 0 2024-02-16T19:24:26.323 INFO:tasks.workunit.client.0.smithi040.stdout:9/946: link d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/f35 d3/d4/d11/d29/d38/d46/d6b/d85/d112/d70/dcc/f12b 0 2024-02-16T19:24:26.327 INFO:tasks.workunit.client.0.smithi040.stdout:1/972: dread d17/d26/d9a/dfb/d92/f79 [0,4194304] 0 2024-02-16T19:24:26.332 INFO:tasks.workunit.client.1.smithi163.stdout:9/568: creat d5/de/d30/d3d/fcf x:0 0 0 2024-02-16T19:24:26.333 INFO:tasks.workunit.client.1.smithi163.stdout:6/353: creat d0/d6/df/d20/f68 x:0 0 0 2024-02-16T19:24:26.348 INFO:tasks.workunit.client.1.smithi163.stdout:1/362: symlink d7/dc/d2d/d32/d3b/l83 0 2024-02-16T19:24:26.349 INFO:tasks.workunit.client.1.smithi163.stdout:0/615: dwrite d11/d12/d67/d6a/d6d/f93 [4194304,4194304] 0 2024-02-16T19:24:26.349 INFO:tasks.workunit.client.1.smithi163.stdout:0/616: stat d11/d21/d50/d58/d72/d9e/fab 0 2024-02-16T19:24:26.349 INFO:tasks.workunit.client.1.smithi163.stdout:1/363: dread d7/d20/d21/f45 [0,4194304] 0 2024-02-16T19:24:26.349 INFO:tasks.workunit.client.1.smithi163.stdout:1/364: truncate d7/d20/d21/f6c 306752 0 2024-02-16T19:24:26.352 INFO:tasks.workunit.client.1.smithi163.stdout:7/401: rename d11/d50/d5d/f61 to d11/f89 0 2024-02-16T19:24:26.354 INFO:tasks.workunit.client.1.smithi163.stdout:9/569: symlink d5/de/d30/d31/d52/ld0 0 2024-02-16T19:24:26.354 INFO:tasks.workunit.client.1.smithi163.stdout:9/570: readlink d5/de/d30/d8a/lc4 0 2024-02-16T19:24:26.354 INFO:tasks.workunit.client.1.smithi163.stdout:9/571: read - d5/de/d30/d3d/d49/d6d/fc3 zero size 2024-02-16T19:24:26.356 INFO:tasks.workunit.client.1.smithi163.stdout:8/368: getdents d0/d7/d19/d1f/d32 0 2024-02-16T19:24:26.357 INFO:tasks.workunit.client.1.smithi163.stdout:0/617: mkdir d11/d12/d67/d6a/dae/dcc/dd9 0 2024-02-16T19:24:26.358 INFO:tasks.workunit.client.0.smithi040.stdout:1/973: dwrite d17/d3c/d71/f11f [4194304,4194304] 0 2024-02-16T19:24:26.358 INFO:tasks.workunit.client.0.smithi040.stdout:1/974: creat d17/d26/d5f/d78/d134/f13d x:0 0 0 2024-02-16T19:24:26.358 INFO:tasks.workunit.client.1.smithi163.stdout:6/354: dwrite d0/d6/df/d20/d40/d1e/f52 [0,4194304] 0 2024-02-16T19:24:26.362 INFO:tasks.workunit.client.1.smithi163.stdout:1/365: mknod d7/dc/d78/c84 0 2024-02-16T19:24:26.363 INFO:tasks.workunit.client.1.smithi163.stdout:1/366: dread - d7/d20/d21/d4e/f79 zero size 2024-02-16T19:24:26.363 INFO:tasks.workunit.client.1.smithi163.stdout:1/367: stat d7/d20/d21/f55 0 2024-02-16T19:24:26.363 INFO:tasks.workunit.client.1.smithi163.stdout:7/402: truncate fb 2948948 0 2024-02-16T19:24:26.366 INFO:tasks.workunit.client.1.smithi163.stdout:6/355: mkdir d0/d6/df/d20/d69 0 2024-02-16T19:24:26.367 INFO:tasks.workunit.client.0.smithi040.stdout:1/975: mkdir d17/d3c/d13e 0 2024-02-16T19:24:26.369 INFO:tasks.workunit.client.1.smithi163.stdout:1/368: mkdir d7/dc/d2d/d3f/d85 0 2024-02-16T19:24:26.369 INFO:tasks.workunit.client.1.smithi163.stdout:1/369: chown d7/d5b/f6a 115448 1 2024-02-16T19:24:26.370 INFO:tasks.workunit.client.1.smithi163.stdout:8/369: dread d0/d24/d27/d45/f4a [0,4194304] 0 2024-02-16T19:24:26.370 INFO:tasks.workunit.client.1.smithi163.stdout:6/356: creat d0/d6/df/d20/d40/d17/d47/f6a x:0 0 0 2024-02-16T19:24:26.370 INFO:tasks.workunit.client.1.smithi163.stdout:9/572: dwrite d5/de/d72/fb0 [0,4194304] 0 2024-02-16T19:24:26.371 INFO:tasks.workunit.client.0.smithi040.stdout:1/976: getdents d17/d29/d8b/d10e/d121/d122/dab 0 2024-02-16T19:24:26.372 INFO:tasks.workunit.client.1.smithi163.stdout:7/403: dread d11/d24/d25/f4a [0,4194304] 0 2024-02-16T19:24:26.373 INFO:tasks.workunit.client.1.smithi163.stdout:7/404: readlink d11/d24/d25/d4b/d66/l83 0 2024-02-16T19:24:26.373 INFO:tasks.workunit.client.1.smithi163.stdout:7/405: write d11/d30/f32 [4261890,44122] 0 2024-02-16T19:24:26.373 INFO:tasks.workunit.client.1.smithi163.stdout:1/370: symlink d7/dc/d2d/d3f/l86 0 2024-02-16T19:24:26.373 INFO:tasks.workunit.client.1.smithi163.stdout:1/371: chown d7/d20/d21/f6c 2022 1 2024-02-16T19:24:26.374 INFO:tasks.workunit.client.1.smithi163.stdout:1/372: chown d7/d20/d2a/c5d 5 1 2024-02-16T19:24:26.375 INFO:tasks.workunit.client.1.smithi163.stdout:8/370: creat d0/d24/d3c/f7e x:0 0 0 2024-02-16T19:24:26.376 INFO:tasks.workunit.client.1.smithi163.stdout:0/618: dread d11/d21/d50/d58/f92 [0,4194304] 0 2024-02-16T19:24:26.380 INFO:tasks.workunit.client.1.smithi163.stdout:9/573: rename d5/de/d30/d8a/lc4 to d5/de/d30/d3d/d93/ld1 0 2024-02-16T19:24:26.382 INFO:tasks.workunit.client.1.smithi163.stdout:7/406: getdents d11/d26/d36 0 2024-02-16T19:24:26.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/371: read d0/dc/f2d [2672556,114399] 0 2024-02-16T19:24:26.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/372: readlink d0/d24/d2c/l62 0 2024-02-16T19:24:26.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/373: readlink d0/d24/d2c/l62 0 2024-02-16T19:24:26.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/374: chown d0/d7/d19/d1f/d30/d6a/d7b 171420 1 2024-02-16T19:24:26.385 INFO:tasks.workunit.client.1.smithi163.stdout:8/375: readlink d0/l1c 0 2024-02-16T19:24:26.385 INFO:tasks.workunit.client.1.smithi163.stdout:0/619: rename d11/l90 to d11/d21/d50/d58/d72/d9e/dd8/lda 0 2024-02-16T19:24:26.386 INFO:tasks.workunit.client.1.smithi163.stdout:0/620: chown d11/d22 17933151 1 2024-02-16T19:24:26.386 INFO:tasks.workunit.client.1.smithi163.stdout:0/621: fsync d11/fb4 0 2024-02-16T19:24:26.386 INFO:tasks.workunit.client.1.smithi163.stdout:8/376: dread d0/dc/f6c [0,4194304] 0 2024-02-16T19:24:26.386 INFO:tasks.workunit.client.1.smithi163.stdout:8/377: write d0/d7/d19/f26 [3562669,88440] 0 2024-02-16T19:24:26.386 INFO:tasks.workunit.client.1.smithi163.stdout:8/378: stat d0/d7/d19/f2f 0 2024-02-16T19:24:26.386 INFO:tasks.workunit.client.1.smithi163.stdout:8/379: read d0/d7/d19/d1f/d32/f34 [1721176,51797] 0 2024-02-16T19:24:26.386 INFO:tasks.workunit.client.1.smithi163.stdout:1/373: mkdir d7/dc/d2d/d32/d87 0 2024-02-16T19:24:26.389 INFO:tasks.workunit.client.1.smithi163.stdout:9/574: dread d5/de/d30/d3d/f88 [0,4194304] 0 2024-02-16T19:24:26.390 INFO:tasks.workunit.client.1.smithi163.stdout:0/622: link d11/d22/d3e/fa5 d11/d21/fdb 0 2024-02-16T19:24:26.392 INFO:tasks.workunit.client.1.smithi163.stdout:8/380: link d0/d7/d19/f26 d0/d7/d18/d2e/f7f 0 2024-02-16T19:24:26.393 INFO:tasks.workunit.client.1.smithi163.stdout:1/374: creat d7/dc/d2d/d32/d87/f88 x:0 0 0 2024-02-16T19:24:26.394 INFO:tasks.workunit.client.1.smithi163.stdout:9/575: write d5/de/d16/f98 [427509,121166] 0 2024-02-16T19:24:26.395 INFO:tasks.workunit.client.1.smithi163.stdout:9/576: readlink d5/ld 0 2024-02-16T19:24:26.395 INFO:tasks.workunit.client.1.smithi163.stdout:9/577: dread - d5/fa0 zero size 2024-02-16T19:24:26.395 INFO:tasks.workunit.client.1.smithi163.stdout:7/407: write d11/d50/f7c [4142755,27568] 0 2024-02-16T19:24:26.395 INFO:tasks.workunit.client.1.smithi163.stdout:0/623: creat d11/d12/d67/d6a/dae/dcc/fdc x:0 0 0 2024-02-16T19:24:26.396 INFO:tasks.workunit.client.1.smithi163.stdout:8/381: link d0/d7/d18/f1e d0/d7/d55/f80 0 2024-02-16T19:24:26.398 INFO:tasks.workunit.client.1.smithi163.stdout:9/578: creat d5/de/d30/d8a/fd2 x:0 0 0 2024-02-16T19:24:26.398 INFO:tasks.workunit.client.1.smithi163.stdout:9/579: chown d5/de/d84/c9e 34 1 2024-02-16T19:24:26.400 INFO:tasks.workunit.client.0.smithi040.stdout:2/933: sync 2024-02-16T19:24:26.400 INFO:tasks.workunit.client.0.smithi040.stdout:5/980: sync 2024-02-16T19:24:26.400 INFO:tasks.workunit.client.0.smithi040.stdout:2/934: chown d5/d7/ce 29139 1 2024-02-16T19:24:26.401 INFO:tasks.workunit.client.0.smithi040.stdout:2/935: fsync d5/f79 0 2024-02-16T19:24:26.401 INFO:tasks.workunit.client.0.smithi040.stdout:2/936: chown d5/d7/d12/d1b/lf6 6108 1 2024-02-16T19:24:26.401 INFO:tasks.workunit.client.0.smithi040.stdout:2/937: chown d5/d7/d1e/lef 5 1 2024-02-16T19:24:26.402 INFO:tasks.workunit.client.1.smithi163.stdout:7/408: rename d11/d26/c64 to d11/d26/d36/d67/c8a 0 2024-02-16T19:24:26.402 INFO:tasks.workunit.client.1.smithi163.stdout:0/624: creat d11/d12/fdd x:0 0 0 2024-02-16T19:24:26.403 INFO:tasks.workunit.client.1.smithi163.stdout:7/409: dread - d11/d26/f46 zero size 2024-02-16T19:24:26.403 INFO:tasks.workunit.client.0.smithi040.stdout:5/981: symlink d1/da/d7e/l13e 0 2024-02-16T19:24:26.404 INFO:tasks.workunit.client.0.smithi040.stdout:1/977: dwrite d17/d29/f2b [0,4194304] 0 2024-02-16T19:24:26.404 INFO:tasks.workunit.client.1.smithi163.stdout:6/357: dwrite d0/d18/f48 [0,4194304] 0 2024-02-16T19:24:26.404 INFO:tasks.workunit.client.1.smithi163.stdout:6/358: fdatasync d0/d6/d9/d2e/d33/f38 0 2024-02-16T19:24:26.406 INFO:tasks.workunit.client.1.smithi163.stdout:8/382: creat d0/d7/d19/d1f/d32/f81 x:0 0 0 2024-02-16T19:24:26.408 INFO:tasks.workunit.client.0.smithi040.stdout:2/938: mkdir d5/d7/d10/df3/dbe/d145 0 2024-02-16T19:24:26.410 INFO:tasks.workunit.client.1.smithi163.stdout:1/375: symlink d7/dc/d2d/d32/l89 0 2024-02-16T19:24:26.410 INFO:tasks.workunit.client.1.smithi163.stdout:1/376: read d7/d20/f38 [408442,128067] 0 2024-02-16T19:24:26.410 INFO:tasks.workunit.client.1.smithi163.stdout:1/377: chown d7/dc/d2d/c62 493 1 2024-02-16T19:24:26.412 INFO:tasks.workunit.client.0.smithi040.stdout:5/982: rmdir d1/da/dd5 39 2024-02-16T19:24:26.414 INFO:tasks.workunit.client.1.smithi163.stdout:9/580: creat d5/de/d30/d3d/d49/fd3 x:0 0 0 2024-02-16T19:24:26.416 INFO:tasks.workunit.client.1.smithi163.stdout:6/359: dread d0/d6/d9/d2e/f4f [0,4194304] 0 2024-02-16T19:24:26.416 INFO:tasks.workunit.client.1.smithi163.stdout:8/383: dread d0/d7/d18/f37 [4194304,4194304] 0 2024-02-16T19:24:26.418 INFO:tasks.workunit.client.1.smithi163.stdout:6/360: read d0/d6/d9/d15/f4d [683574,27125] 0 2024-02-16T19:24:26.427 INFO:tasks.workunit.client.0.smithi040.stdout:5/983: read d1/da/d1c/d21/f8f [1998114,22280] 0 2024-02-16T19:24:26.427 INFO:tasks.workunit.client.0.smithi040.stdout:5/984: chown d1/l1b 343 1 2024-02-16T19:24:26.427 INFO:tasks.workunit.client.0.smithi040.stdout:5/985: chown d1/d62/d9a/l12b 406 1 2024-02-16T19:24:26.430 INFO:tasks.workunit.client.1.smithi163.stdout:1/378: mknod d7/dc/c8a 0 2024-02-16T19:24:26.430 INFO:tasks.workunit.client.1.smithi163.stdout:8/384: write d0/d24/d2c/f3b [686551,31674] 0 2024-02-16T19:24:26.430 INFO:tasks.workunit.client.1.smithi163.stdout:0/625: dwrite d11/d12/d20/fb9 [0,4194304] 0 2024-02-16T19:24:26.430 INFO:tasks.workunit.client.1.smithi163.stdout:8/385: read d0/d24/f60 [2473203,125450] 0 2024-02-16T19:24:26.430 INFO:tasks.workunit.client.1.smithi163.stdout:0/626: readlink d11/d22/l88 0 2024-02-16T19:24:26.432 INFO:tasks.workunit.client.1.smithi163.stdout:6/361: creat d0/d6/d9/d15/d1c/d65/d63/f6b x:0 0 0 2024-02-16T19:24:26.433 INFO:tasks.workunit.client.1.smithi163.stdout:1/379: mknod d7/d20/d21/d4e/c8b 0 2024-02-16T19:24:26.433 INFO:tasks.workunit.client.0.smithi040.stdout:1/978: dwrite d17/d26/d9a/dfb/fc9 [0,4194304] 0 2024-02-16T19:24:26.438 INFO:tasks.workunit.client.0.smithi040.stdout:3/991: sync 2024-02-16T19:24:26.438 INFO:tasks.workunit.client.0.smithi040.stdout:7/865: sync 2024-02-16T19:24:26.438 INFO:tasks.workunit.client.0.smithi040.stdout:3/992: creat d1/dc/f13a x:0 0 0 2024-02-16T19:24:26.438 INFO:tasks.workunit.client.0.smithi040.stdout:3/993: write d1/dc/d42/d58/f93 [968855,51551] 0 2024-02-16T19:24:26.438 INFO:tasks.workunit.client.0.smithi040.stdout:3/994: chown d1/d4/d6e/dc2/dce/ce4 84077 1 2024-02-16T19:24:26.442 INFO:tasks.workunit.client.1.smithi163.stdout:0/627: write d11/d22/d7f/f96 [1286555,56991] 0 2024-02-16T19:24:26.444 INFO:tasks.workunit.client.1.smithi163.stdout:6/362: read d0/d6/d9/d2e/d33/f3d [2820563,39559] 0 2024-02-16T19:24:26.444 INFO:tasks.workunit.client.1.smithi163.stdout:6/363: stat d0/d6/df/d20/d40/d17/c3a 0 2024-02-16T19:24:26.447 INFO:tasks.workunit.client.1.smithi163.stdout:6/364: write d0/f2 [822886,72934] 0 2024-02-16T19:24:26.447 INFO:tasks.workunit.client.1.smithi163.stdout:9/581: mknod d5/de/d30/d3d/cd4 0 2024-02-16T19:24:26.448 INFO:tasks.workunit.client.0.smithi040.stdout:3/995: dread d1/dc/d42/d5c/df3/f102 [0,4194304] 0 2024-02-16T19:24:26.448 INFO:tasks.workunit.client.0.smithi040.stdout:4/787: rmdir dd/de/d10/d36/dc3 39 2024-02-16T19:24:26.449 INFO:tasks.workunit.client.1.smithi163.stdout:1/380: symlink d7/dc/l8c 0 2024-02-16T19:24:26.449 INFO:tasks.workunit.client.0.smithi040.stdout:5/986: dwrite d1/da/d1c/dba/db1/dde/ddf/fe8 [0,4194304] 0 2024-02-16T19:24:26.449 INFO:tasks.workunit.client.0.smithi040.stdout:5/987: chown d1/da/d1c/dba/dd6 648436366 1 2024-02-16T19:24:26.450 INFO:tasks.workunit.client.0.smithi040.stdout:7/866: write d21/d4d/d77/d7a/f123 [3390025,100888] 0 2024-02-16T19:24:26.450 INFO:tasks.workunit.client.0.smithi040.stdout:7/867: fsync d21/d4d/d7d/fb4 0 2024-02-16T19:24:26.450 INFO:tasks.workunit.client.0.smithi040.stdout:7/868: readlink d21/l5f 0 2024-02-16T19:24:26.455 INFO:tasks.workunit.client.0.smithi040.stdout:1/979: rename d17/d2d/l3b to d17/d26/d5f/l13f 0 2024-02-16T19:24:26.458 INFO:tasks.workunit.client.1.smithi163.stdout:0/628: symlink d11/d12/d67/d6a/dae/dcc/lde 0 2024-02-16T19:24:26.458 INFO:tasks.workunit.client.1.smithi163.stdout:0/629: creat d11/d22/d3e/fdf x:0 0 0 2024-02-16T19:24:26.458 INFO:tasks.workunit.client.1.smithi163.stdout:0/630: stat d11/d22 0 2024-02-16T19:24:26.459 INFO:tasks.workunit.client.0.smithi040.stdout:3/996: mkdir d1/d4/d6e/dc2/dce/d11f/d13b 0 2024-02-16T19:24:26.461 INFO:tasks.workunit.client.0.smithi040.stdout:4/788: dread dd/d58/d5e/f61 [0,4194304] 0 2024-02-16T19:24:26.462 INFO:tasks.workunit.client.0.smithi040.stdout:4/789: chown dd/de/d10/d36/d9e/d32 15 1 2024-02-16T19:24:26.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/365: mknod d0/d6/d9/d15/c6c 0 2024-02-16T19:24:26.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/366: write d0/d6/d9/d15/d1c/f56 [769883,39621] 0 2024-02-16T19:24:26.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/367: truncate d0/d6/d9/d2e/d33/f60 630485 0 2024-02-16T19:24:26.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/368: creat d0/dd/f6d x:0 0 0 2024-02-16T19:24:26.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/369: write d0/d6/df/d20/f68 [40581,56876] 0 2024-02-16T19:24:26.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/370: write d0/d6/d9/d15/d1c/d65/d63/f6b [800381,6264] 0 2024-02-16T19:24:26.469 INFO:tasks.workunit.client.1.smithi163.stdout:1/381: mkdir d7/d20/d2a/d8d 0 2024-02-16T19:24:26.472 INFO:tasks.workunit.client.0.smithi040.stdout:5/988: link d1/da/db0/f118 d1/da/d1c/dba/db1/dde/ddf/f13f 0 2024-02-16T19:24:26.477 INFO:tasks.workunit.client.0.smithi040.stdout:3/997: dread d1/d7/d36/f7c [0,4194304] 0 2024-02-16T19:24:26.479 INFO:tasks.workunit.client.0.smithi040.stdout:7/869: rename d21/d8b/fb0 to d21/d4d/d77/d87/dc9/ddd/f125 0 2024-02-16T19:24:26.480 INFO:tasks.workunit.client.1.smithi163.stdout:0/631: truncate d11/d22/f34 2413445 0 2024-02-16T19:24:26.481 INFO:tasks.workunit.client.0.smithi040.stdout:1/980: dwrite d17/d26/d5f/da6/fc6 [0,4194304] 0 2024-02-16T19:24:26.484 INFO:tasks.workunit.client.0.smithi040.stdout:5/989: mkdir d1/da/dd5/dfb/d140 0 2024-02-16T19:24:26.485 INFO:tasks.workunit.client.0.smithi040.stdout:5/990: write d1/da/d1c/d73/f104 [35767,120883] 0 2024-02-16T19:24:26.485 INFO:tasks.workunit.client.0.smithi040.stdout:5/991: fdatasync d1/f96 0 2024-02-16T19:24:26.486 INFO:tasks.workunit.client.0.smithi040.stdout:5/992: write d1/da/d51/f103 [631157,14513] 0 2024-02-16T19:24:26.487 INFO:tasks.workunit.client.1.smithi163.stdout:1/382: read d7/d20/f24 [335240,22544] 0 2024-02-16T19:24:26.488 INFO:tasks.workunit.client.1.smithi163.stdout:0/632: mknod d11/d21/d50/d58/ce0 0 2024-02-16T19:24:26.491 INFO:tasks.workunit.client.1.smithi163.stdout:4/314: sync 2024-02-16T19:24:26.491 INFO:tasks.workunit.client.1.smithi163.stdout:2/504: sync 2024-02-16T19:24:26.493 INFO:tasks.workunit.client.1.smithi163.stdout:1/383: link d7/dc/d2d/d32/d3b/f58 d7/dc/d2d/f8e 0 2024-02-16T19:24:26.495 INFO:tasks.workunit.client.1.smithi163.stdout:4/315: getdents d11 0 2024-02-16T19:24:26.498 INFO:tasks.workunit.client.1.smithi163.stdout:2/505: creat d2/d26/d97/fb1 x:0 0 0 2024-02-16T19:24:26.498 INFO:tasks.workunit.client.1.smithi163.stdout:6/371: dwrite d0/d6/d9/d2e/d33/f5f [0,4194304] 0 2024-02-16T19:24:26.498 INFO:tasks.workunit.client.1.smithi163.stdout:6/372: truncate d0/d6/df/d20/d40/d17/f5e 315821 0 2024-02-16T19:24:26.500 INFO:tasks.workunit.client.1.smithi163.stdout:4/316: mkdir d11/d2d/d4f/d60 0 2024-02-16T19:24:26.500 INFO:tasks.workunit.client.1.smithi163.stdout:4/317: write d11/f14 [1272769,31634] 0 2024-02-16T19:24:26.501 INFO:tasks.workunit.client.1.smithi163.stdout:4/318: stat d11/d18/l25 0 2024-02-16T19:24:26.502 INFO:tasks.workunit.client.0.smithi040.stdout:3/998: rename d1/d7/d30 to d1/d4/d6e/dc2/d13c 0 2024-02-16T19:24:26.504 INFO:tasks.workunit.client.0.smithi040.stdout:4/790: dwrite dd/de/d71/f9a [0,4194304] 0 2024-02-16T19:24:26.504 INFO:tasks.workunit.client.0.smithi040.stdout:7/870: mkdir d21/d4d/d77/d87/d9b/df2/d126 0 2024-02-16T19:24:26.504 INFO:tasks.workunit.client.0.smithi040.stdout:7/871: truncate d21/d29/f73 2307829 0 2024-02-16T19:24:26.512 INFO:tasks.workunit.client.1.smithi163.stdout:6/373: symlink d0/d6/df/d30/l6e 0 2024-02-16T19:24:26.515 INFO:tasks.workunit.client.1.smithi163.stdout:6/374: rmdir d0/d6/df 39 2024-02-16T19:24:26.516 INFO:tasks.workunit.client.0.smithi040.stdout:3/999: truncate d1/dc/d42/d4a/f50 817652 0 2024-02-16T19:24:26.520 INFO:tasks.workunit.client.0.smithi040.stdout:4/791: creat dd/de/d10/d36/dc3/ff1 x:0 0 0 2024-02-16T19:24:26.522 INFO:tasks.workunit.client.1.smithi163.stdout:1/384: dwrite d7/d5b/d63/f7b [0,4194304] 0 2024-02-16T19:24:26.523 INFO:tasks.workunit.client.0.smithi040.stdout:7/872: mknod d21/d7e/df8/c127 0 2024-02-16T19:24:26.523 INFO:tasks.workunit.client.0.smithi040.stdout:5/993: dwrite d1/da/d1c/f82 [0,4194304] 0 2024-02-16T19:24:26.524 INFO:tasks.workunit.client.1.smithi163.stdout:2/506: dwrite d2/d2a/d2d/d3c/d7c/f99 [0,4194304] 0 2024-02-16T19:24:26.526 INFO:tasks.workunit.client.0.smithi040.stdout:1/981: symlink d17/d2d/l140 0 2024-02-16T19:24:26.532 INFO:tasks.workunit.client.0.smithi040.stdout:4/792: rename dd/de/d10/fb6 to dd/de/d71/ff2 0 2024-02-16T19:24:26.534 INFO:tasks.workunit.client.1.smithi163.stdout:1/385: write d7/dc/d78/d7c/f3d [780593,71537] 0 2024-02-16T19:24:26.535 INFO:tasks.workunit.client.0.smithi040.stdout:7/873: dread d21/f62 [0,4194304] 0 2024-02-16T19:24:26.535 INFO:tasks.workunit.client.0.smithi040.stdout:7/874: dread - d21/d7e/fe4 zero size 2024-02-16T19:24:26.537 INFO:tasks.workunit.client.1.smithi163.stdout:6/375: creat d0/d6/d9/f6f x:0 0 0 2024-02-16T19:24:26.540 INFO:tasks.workunit.client.0.smithi040.stdout:5/994: link d1/da/c10 d1/d3c/dea/c141 0 2024-02-16T19:24:26.540 INFO:tasks.workunit.client.1.smithi163.stdout:2/507: write d2/ff [1549663,104436] 0 2024-02-16T19:24:26.542 INFO:tasks.workunit.client.1.smithi163.stdout:7/410: rename d11/d26/d36 to d11/d50/d5d/d8b 0 2024-02-16T19:24:26.543 INFO:tasks.workunit.client.0.smithi040.stdout:1/982: symlink d17/d3c/d71/l141 0 2024-02-16T19:24:26.544 INFO:tasks.workunit.client.0.smithi040.stdout:1/983: creat d17/f142 x:0 0 0 2024-02-16T19:24:26.546 INFO:tasks.workunit.client.0.smithi040.stdout:7/875: getdents d21/d4d/d77 0 2024-02-16T19:24:26.546 INFO:tasks.workunit.client.0.smithi040.stdout:7/876: write d21/f24 [2673920,34815] 0 2024-02-16T19:24:26.551 INFO:tasks.workunit.client.0.smithi040.stdout:1/984: mkdir d17/db5/d143 0 2024-02-16T19:24:26.551 INFO:tasks.workunit.client.0.smithi040.stdout:1/985: creat d17/db5/f144 x:0 0 0 2024-02-16T19:24:26.551 INFO:tasks.workunit.client.0.smithi040.stdout:1/986: write d17/d26/d9a/dfb/fef [164234,130877] 0 2024-02-16T19:24:26.551 INFO:tasks.workunit.client.0.smithi040.stdout:1/987: write d17/f61 [1396627,107018] 0 2024-02-16T19:24:26.553 INFO:tasks.workunit.client.0.smithi040.stdout:5/995: dread d1/d3c/f77 [4194304,4194304] 0 2024-02-16T19:24:26.554 INFO:tasks.workunit.client.1.smithi163.stdout:1/386: creat d7/d20/d21/d4e/f8f x:0 0 0 2024-02-16T19:24:26.554 INFO:tasks.workunit.client.0.smithi040.stdout:4/793: dwrite dd/de/d10/d36/dad/fb4 [0,4194304] 0 2024-02-16T19:24:26.555 INFO:tasks.workunit.client.0.smithi040.stdout:7/877: creat d21/d29/f128 x:0 0 0 2024-02-16T19:24:26.555 INFO:tasks.workunit.client.0.smithi040.stdout:7/878: truncate d21/d29/ff1 809260 0 2024-02-16T19:24:26.556 INFO:tasks.workunit.client.1.smithi163.stdout:6/376: mkdir d0/d6/df/d20/d40/d17/d47/d70 0 2024-02-16T19:24:26.556 INFO:tasks.workunit.client.1.smithi163.stdout:6/377: readlink d0/d6/l67 0 2024-02-16T19:24:26.560 INFO:tasks.workunit.client.1.smithi163.stdout:2/508: link d2/f1f d2/d2a/fb2 0 2024-02-16T19:24:26.560 INFO:tasks.workunit.client.1.smithi163.stdout:2/509: chown d2/d26/d54 1 1 2024-02-16T19:24:26.560 INFO:tasks.workunit.client.1.smithi163.stdout:2/510: write d2/d26/f5d [485327,2788] 0 2024-02-16T19:24:26.561 INFO:tasks.workunit.client.0.smithi040.stdout:1/988: read d17/d26/f7a [4748899,113849] 0 2024-02-16T19:24:26.562 INFO:tasks.workunit.client.0.smithi040.stdout:5/996: mkdir d1/d142 0 2024-02-16T19:24:26.563 INFO:tasks.workunit.client.1.smithi163.stdout:7/411: mkdir d11/d24/d25/d4b/d8c 0 2024-02-16T19:24:26.564 INFO:tasks.workunit.client.0.smithi040.stdout:4/794: dread dd/d58/d8c/fbe [0,4194304] 0 2024-02-16T19:24:26.566 INFO:tasks.workunit.client.1.smithi163.stdout:0/633: rename d11/d12/c1b to d11/d21/d50/d58/d74/ce1 0 2024-02-16T19:24:26.569 INFO:tasks.workunit.client.1.smithi163.stdout:2/511: dread d2/f1c [0,4194304] 0 2024-02-16T19:24:26.569 INFO:tasks.workunit.client.0.smithi040.stdout:7/879: truncate d21/d4d/f66 1502315 0 2024-02-16T19:24:26.569 INFO:tasks.workunit.client.0.smithi040.stdout:7/880: dread - d21/d4d/d77/d87/fa9 zero size 2024-02-16T19:24:26.571 INFO:tasks.workunit.client.1.smithi163.stdout:1/387: mkdir d7/dc/d2d/d90 0 2024-02-16T19:24:26.571 INFO:tasks.workunit.client.0.smithi040.stdout:1/989: symlink d17/d3c/d74/d97/da9/l145 0 2024-02-16T19:24:26.573 INFO:tasks.workunit.client.1.smithi163.stdout:6/378: link d0/d6/d9/d2e/d33/f3d d0/d6/df/d20/d40/f71 0 2024-02-16T19:24:26.576 INFO:tasks.workunit.client.0.smithi040.stdout:5/997: creat d1/da/dd5/dfb/f143 x:0 0 0 2024-02-16T19:24:26.578 INFO:tasks.workunit.client.1.smithi163.stdout:7/412: mkdir d11/d24/d34/d8d 0 2024-02-16T19:24:26.579 INFO:tasks.workunit.client.1.smithi163.stdout:7/413: creat d11/d24/d34/f8e x:0 0 0 2024-02-16T19:24:26.582 INFO:tasks.workunit.client.0.smithi040.stdout:7/881: symlink d21/d7e/l129 0 2024-02-16T19:24:26.582 INFO:tasks.workunit.client.0.smithi040.stdout:7/882: chown d21/d29/f73 129609 1 2024-02-16T19:24:26.583 INFO:tasks.workunit.client.1.smithi163.stdout:6/379: creat d0/d6/df/d20/d40/d53/f72 x:0 0 0 2024-02-16T19:24:26.587 INFO:tasks.workunit.client.1.smithi163.stdout:1/388: dread d7/d20/f66 [0,4194304] 0 2024-02-16T19:24:26.588 INFO:tasks.workunit.client.0.smithi040.stdout:5/998: mknod d1/d3c/dea/c144 0 2024-02-16T19:24:26.589 INFO:tasks.workunit.client.1.smithi163.stdout:0/634: symlink d11/d4f/le2 0 2024-02-16T19:24:26.590 INFO:tasks.workunit.client.0.smithi040.stdout:4/795: mknod dd/de/daa/dd8/cf3 0 2024-02-16T19:24:26.590 INFO:tasks.workunit.client.0.smithi040.stdout:4/796: fdatasync dd/f43 0 2024-02-16T19:24:26.592 INFO:tasks.workunit.client.0.smithi040.stdout:7/883: unlink d21/d8b/cbb 0 2024-02-16T19:24:26.596 INFO:tasks.workunit.client.0.smithi040.stdout:1/990: getdents d17/d29/d8b/dd2 0 2024-02-16T19:24:26.596 INFO:tasks.workunit.client.1.smithi163.stdout:0/635: truncate d11/d22/d3e/fa5 944139 0 2024-02-16T19:24:26.596 INFO:tasks.workunit.client.1.smithi163.stdout:0/636: truncate d11/d12/fdd 365818 0 2024-02-16T19:24:26.596 INFO:tasks.workunit.client.1.smithi163.stdout:0/637: creat d11/d12/d67/d6a/d6d/fe3 x:0 0 0 2024-02-16T19:24:26.597 INFO:tasks.workunit.client.1.smithi163.stdout:1/389: dread d7/dc/d2d/d32/d3b/f4d [0,4194304] 0 2024-02-16T19:24:26.598 INFO:tasks.workunit.client.0.smithi040.stdout:4/797: dread dd/de/fcf [0,4194304] 0 2024-02-16T19:24:26.598 INFO:tasks.workunit.client.0.smithi040.stdout:5/999: mkdir d1/d45/d145 0 2024-02-16T19:24:26.598 INFO:tasks.workunit.client.0.smithi040.stdout:4/798: stat dd/de/f12 0 2024-02-16T19:24:26.599 INFO:tasks.workunit.client.0.smithi040.stdout:4/799: truncate dd/de/d10/d36/fda 298 0 2024-02-16T19:24:26.599 INFO:tasks.workunit.client.1.smithi163.stdout:6/380: write d0/f5 [7279450,87847] 0 2024-02-16T19:24:26.599 INFO:tasks.workunit.client.1.smithi163.stdout:6/381: dread - d0/dd/f6d zero size 2024-02-16T19:24:26.599 INFO:tasks.workunit.client.1.smithi163.stdout:6/382: creat d0/d6/d9/f73 x:0 0 0 2024-02-16T19:24:26.602 INFO:tasks.workunit.client.0.smithi040.stdout:7/884: write d21/d7e/fe9 [523839,96755] 0 2024-02-16T19:24:26.608 INFO:tasks.workunit.client.0.smithi040.stdout:7/885: dread d21/d29/f2b [0,4194304] 0 2024-02-16T19:24:26.608 INFO:tasks.workunit.client.0.smithi040.stdout:1/991: rmdir d17/d29/d8b/d10e/d112 39 2024-02-16T19:24:26.609 INFO:tasks.workunit.client.0.smithi040.stdout:1/992: getdents d17/d29/d8b/d10e/d121/d122/dab 0 2024-02-16T19:24:26.610 INFO:tasks.workunit.client.1.smithi163.stdout:6/383: mkdir d0/d6/df/d20/d40/d53/d74 0 2024-02-16T19:24:26.611 INFO:tasks.workunit.client.1.smithi163.stdout:7/414: dwrite d11/d24/d25/f4f [0,4194304] 0 2024-02-16T19:24:26.613 INFO:tasks.workunit.client.0.smithi040.stdout:7/886: creat d21/d4d/d77/d87/d9b/df2/f12a x:0 0 0 2024-02-16T19:24:26.614 INFO:tasks.workunit.client.1.smithi163.stdout:0/638: dwrite d11/d12/d67/d6a/d6d/fcb [0,4194304] 0 2024-02-16T19:24:26.614 INFO:tasks.workunit.client.1.smithi163.stdout:2/512: dwrite d2/f15 [0,4194304] 0 2024-02-16T19:24:26.616 INFO:tasks.workunit.client.1.smithi163.stdout:6/384: rename d0/d6/d9/d15/l28 to d0/d6/d9/d2e/d33/l75 0 2024-02-16T19:24:26.617 INFO:tasks.workunit.client.1.smithi163.stdout:4/319: rmdir d11/d2d/d4f 39 2024-02-16T19:24:26.623 INFO:tasks.workunit.client.0.smithi040.stdout:7/887: mkdir d21/d4d/d77/d7a/d12b 0 2024-02-16T19:24:26.623 INFO:tasks.workunit.client.0.smithi040.stdout:7/888: write d21/d4d/d7d/fb4 [726968,70174] 0 2024-02-16T19:24:26.629 INFO:tasks.workunit.client.1.smithi163.stdout:7/415: dwrite d11/d24/d25/f2d [0,4194304] 0 2024-02-16T19:24:26.632 INFO:tasks.workunit.client.1.smithi163.stdout:0/639: creat d11/d12/d20/fe4 x:0 0 0 2024-02-16T19:24:26.633 INFO:tasks.workunit.client.1.smithi163.stdout:4/320: symlink d11/d18/d1f/d29/d52/d57/l61 0 2024-02-16T19:24:26.634 INFO:tasks.workunit.client.1.smithi163.stdout:4/321: chown d11/d18/l25 59441 1 2024-02-16T19:24:26.641 INFO:tasks.workunit.client.0.smithi040.stdout:4/800: rmdir dd/de/d71 39 2024-02-16T19:24:26.647 INFO:tasks.workunit.client.1.smithi163.stdout:7/416: symlink d11/d24/d34/l8f 0 2024-02-16T19:24:26.648 INFO:tasks.workunit.client.1.smithi163.stdout:5/393: sync 2024-02-16T19:24:26.648 INFO:tasks.workunit.client.1.smithi163.stdout:3/415: sync 2024-02-16T19:24:26.648 INFO:tasks.workunit.client.1.smithi163.stdout:3/416: chown d0/dc/d1c/f23 47444 1 2024-02-16T19:24:26.648 INFO:tasks.workunit.client.1.smithi163.stdout:5/394: dread - d0/d2/f67 zero size 2024-02-16T19:24:26.649 INFO:tasks.workunit.client.0.smithi040.stdout:1/993: rename d17/d2d/lb4 to d17/d29/d8b/dd2/l146 0 2024-02-16T19:24:26.649 INFO:tasks.workunit.client.0.smithi040.stdout:1/994: truncate d17/d26/d9a/dfb/d92/daa/f13b 66456 0 2024-02-16T19:24:26.651 INFO:tasks.workunit.client.1.smithi163.stdout:0/640: symlink d11/le5 0 2024-02-16T19:24:26.651 INFO:tasks.workunit.client.1.smithi163.stdout:0/641: creat d11/d21/d7c/fe6 x:0 0 0 2024-02-16T19:24:26.652 INFO:tasks.workunit.client.0.smithi040.stdout:7/889: creat d21/d8b/d9c/d10e/f12c x:0 0 0 2024-02-16T19:24:26.652 INFO:tasks.workunit.client.0.smithi040.stdout:7/890: read f11 [934777,108604] 0 2024-02-16T19:24:26.652 INFO:tasks.workunit.client.1.smithi163.stdout:5/395: dread d0/d22/d3b/f66 [0,4194304] 0 2024-02-16T19:24:26.653 INFO:tasks.workunit.client.1.smithi163.stdout:5/396: write d0/f39 [1980120,458] 0 2024-02-16T19:24:26.653 INFO:tasks.workunit.client.1.smithi163.stdout:4/322: truncate d11/d18/d1f/f33 266930 0 2024-02-16T19:24:26.653 INFO:tasks.workunit.client.1.smithi163.stdout:8/386: sync 2024-02-16T19:24:26.653 INFO:tasks.workunit.client.1.smithi163.stdout:8/387: chown d0/d6f 194 1 2024-02-16T19:24:26.653 INFO:tasks.workunit.client.1.smithi163.stdout:8/388: truncate d0/dc/f6c 1196788 0 2024-02-16T19:24:26.654 INFO:tasks.workunit.client.0.smithi040.stdout:4/801: symlink dd/de/d10/d36/lf4 0 2024-02-16T19:24:26.654 INFO:tasks.workunit.client.0.smithi040.stdout:4/802: creat dd/de/d10/d36/d9e/ff5 x:0 0 0 2024-02-16T19:24:26.656 INFO:tasks.workunit.client.1.smithi163.stdout:3/417: mkdir d0/dc/d1d/d29/d95 0 2024-02-16T19:24:26.659 INFO:tasks.workunit.client.0.smithi040.stdout:1/995: rename d17/d26/d54/c114 to d17/d3c/d74/d126/c147 0 2024-02-16T19:24:26.662 INFO:tasks.workunit.client.1.smithi163.stdout:0/642: write d11/d21/f47 [2877552,41263] 0 2024-02-16T19:24:26.663 INFO:tasks.workunit.client.1.smithi163.stdout:7/417: dwrite d11/d50/d5d/d8b/f4e [0,4194304] 0 2024-02-16T19:24:26.664 INFO:tasks.workunit.client.1.smithi163.stdout:3/418: dread d0/dc/d1d/f1f [0,4194304] 0 2024-02-16T19:24:26.664 INFO:tasks.workunit.client.1.smithi163.stdout:5/397: creat d0/d22/d28/d36/d3c/d4f/f7b x:0 0 0 2024-02-16T19:24:26.664 INFO:tasks.workunit.client.1.smithi163.stdout:3/419: chown d0/dc/d1d/d49 1 1 2024-02-16T19:24:26.664 INFO:tasks.workunit.client.1.smithi163.stdout:5/398: write d0/d1c/f74 [49211,43286] 0 2024-02-16T19:24:26.665 INFO:tasks.workunit.client.0.smithi040.stdout:4/803: mknod dd/dd2/cf6 0 2024-02-16T19:24:26.666 INFO:tasks.workunit.client.1.smithi163.stdout:7/418: dread fc [0,4194304] 0 2024-02-16T19:24:26.667 INFO:tasks.workunit.client.1.smithi163.stdout:7/419: readlink d11/d24/d34/l8f 0 2024-02-16T19:24:26.667 INFO:tasks.workunit.client.0.smithi040.stdout:1/996: rename d17/d26/d9a/dfb/d38/l89 to d17/d29/d8b/d10e/d121/l148 0 2024-02-16T19:24:26.667 INFO:tasks.workunit.client.0.smithi040.stdout:1/997: chown d17/d26/d3d/l42 26980 1 2024-02-16T19:24:26.670 INFO:tasks.workunit.client.1.smithi163.stdout:9/582: sync 2024-02-16T19:24:26.672 INFO:tasks.workunit.client.1.smithi163.stdout:0/643: link d11/f1f d11/d22/d7f/d8e/fe7 0 2024-02-16T19:24:26.676 INFO:tasks.workunit.client.1.smithi163.stdout:3/420: readlink d0/dc/le 0 2024-02-16T19:24:26.676 INFO:tasks.workunit.client.1.smithi163.stdout:3/421: chown d0/l3 11989 1 2024-02-16T19:24:26.680 INFO:tasks.workunit.client.1.smithi163.stdout:5/399: mknod d0/d22/d28/d36/d3c/c7c 0 2024-02-16T19:24:26.680 INFO:tasks.workunit.client.1.smithi163.stdout:5/400: readlink d0/d2/l48 0 2024-02-16T19:24:26.681 INFO:tasks.workunit.client.1.smithi163.stdout:7/420: rename d11/d24/d25/f2c to d11/d50/d5d/d8b/f90 0 2024-02-16T19:24:26.683 INFO:tasks.workunit.client.1.smithi163.stdout:9/583: mknod d5/de/cd5 0 2024-02-16T19:24:26.685 INFO:tasks.workunit.client.1.smithi163.stdout:0/644: mknod d11/d22/d7f/ce8 0 2024-02-16T19:24:26.685 INFO:tasks.workunit.client.0.smithi040.stdout:1/998: dwrite d17/d29/d8b/d10e/d121/d122/fcd [0,4194304] 0 2024-02-16T19:24:26.689 INFO:tasks.workunit.client.0.smithi040.stdout:7/891: dread d21/f4a [0,4194304] 0 2024-02-16T19:24:26.693 INFO:tasks.workunit.client.1.smithi163.stdout:9/584: mknod d5/de/d30/d3d/d93/cd6 0 2024-02-16T19:24:26.698 INFO:tasks.workunit.client.1.smithi163.stdout:0/645: rename d11/d12/d67/fd2 to d11/d21/d50/d58/d6b/fe9 0 2024-02-16T19:24:26.700 INFO:tasks.workunit.client.1.smithi163.stdout:3/422: mkdir d0/dc/d96 0 2024-02-16T19:24:26.701 INFO:tasks.workunit.client.1.smithi163.stdout:5/401: dwrite d0/f32 [0,4194304] 0 2024-02-16T19:24:26.702 INFO:tasks.workunit.client.0.smithi040.stdout:7/892: rename d21/df9/dfe to d21/d7e/df8/db7/de7/d118/d12d 0 2024-02-16T19:24:26.705 INFO:tasks.workunit.client.1.smithi163.stdout:3/423: mknod d0/dc/d1d/d29/d5d/c97 0 2024-02-16T19:24:26.707 INFO:tasks.workunit.client.1.smithi163.stdout:9/585: write d5/de/d30/d3d/d49/d6d/f96 [2035420,87641] 0 2024-02-16T19:24:26.707 INFO:tasks.workunit.client.0.smithi040.stdout:7/893: getdents d21/d7e 0 2024-02-16T19:24:26.710 INFO:tasks.workunit.client.1.smithi163.stdout:0/646: dread d11/d12/d20/fb9 [0,4194304] 0 2024-02-16T19:24:26.712 INFO:tasks.workunit.client.1.smithi163.stdout:7/421: dwrite fb [0,4194304] 0 2024-02-16T19:24:26.712 INFO:tasks.workunit.client.1.smithi163.stdout:7/422: chown d11/d24/d25/d4b/d66/c6b 3 1 2024-02-16T19:24:26.712 INFO:tasks.workunit.client.1.smithi163.stdout:7/423: chown d11/d24/d25/f48 977402 1 2024-02-16T19:24:26.713 INFO:tasks.workunit.client.0.smithi040.stdout:7/894: read d21/d29/f45 [1332958,52916] 0 2024-02-16T19:24:26.713 INFO:tasks.workunit.client.0.smithi040.stdout:7/895: write d21/d29/d79/f80 [1394124,66267] 0 2024-02-16T19:24:26.713 INFO:tasks.workunit.client.1.smithi163.stdout:5/402: creat d0/d2/d34/d44/d53/f7d x:0 0 0 2024-02-16T19:24:26.714 INFO:tasks.workunit.client.1.smithi163.stdout:5/403: fdatasync d0/d22/d3b/f66 0 2024-02-16T19:24:26.714 INFO:tasks.workunit.client.0.smithi040.stdout:7/896: creat d21/d7e/df8/db7/de7/d10f/f12e x:0 0 0 2024-02-16T19:24:26.714 INFO:tasks.workunit.client.0.smithi040.stdout:7/897: fdatasync d21/d4d/d77/d87/fa9 0 2024-02-16T19:24:26.716 INFO:tasks.workunit.client.0.smithi040.stdout:7/898: mknod d21/d4d/d77/d7a/df4/c12f 0 2024-02-16T19:24:26.716 INFO:tasks.workunit.client.0.smithi040.stdout:7/899: dread - d21/d29/d79/f122 zero size 2024-02-16T19:24:26.717 INFO:tasks.workunit.client.0.smithi040.stdout:7/900: creat d21/d29/dad/f130 x:0 0 0 2024-02-16T19:24:26.717 INFO:tasks.workunit.client.0.smithi040.stdout:7/901: dread - d21/d4d/f6e zero size 2024-02-16T19:24:26.717 INFO:tasks.workunit.client.0.smithi040.stdout:7/902: chown d21/d29/d79/f7c 1716874 1 2024-02-16T19:24:26.717 INFO:tasks.workunit.client.0.smithi040.stdout:7/903: chown d21/d8b/cc3 6 1 2024-02-16T19:24:26.718 INFO:tasks.workunit.client.1.smithi163.stdout:9/586: rename d5/de/d30/f89 to d5/de/d30/d8a/fd7 0 2024-02-16T19:24:26.719 INFO:tasks.workunit.client.0.smithi040.stdout:7/904: dread d21/d29/d79/fc4 [0,4194304] 0 2024-02-16T19:24:26.721 INFO:tasks.workunit.client.1.smithi163.stdout:1/390: dwrite d7/f4f [0,4194304] 0 2024-02-16T19:24:26.722 INFO:tasks.workunit.client.1.smithi163.stdout:3/424: dread d0/dc/f17 [0,4194304] 0 2024-02-16T19:24:26.723 INFO:tasks.workunit.client.1.smithi163.stdout:0/647: creat d11/d21/fea x:0 0 0 2024-02-16T19:24:26.723 INFO:tasks.workunit.client.1.smithi163.stdout:0/648: chown d11/d12/d67/d6a/d6d/l3d 328 1 2024-02-16T19:24:26.725 INFO:tasks.workunit.client.1.smithi163.stdout:5/404: mknod d0/d2/c7e 0 2024-02-16T19:24:26.725 INFO:tasks.workunit.client.1.smithi163.stdout:5/405: readlink d0/l5b 0 2024-02-16T19:24:26.725 INFO:tasks.workunit.client.1.smithi163.stdout:5/406: chown d0/d22/d28/d36/d3c/d4f/f61 5634892 1 2024-02-16T19:24:26.725 INFO:tasks.workunit.client.1.smithi163.stdout:5/407: chown d0/c59 1186 1 2024-02-16T19:24:26.727 INFO:tasks.workunit.client.1.smithi163.stdout:9/587: creat d5/de/d30/d3d/d49/d6d/d97/fd8 x:0 0 0 2024-02-16T19:24:26.730 INFO:tasks.workunit.client.0.smithi040.stdout:2/939: sync 2024-02-16T19:24:26.730 INFO:tasks.workunit.client.0.smithi040.stdout:9/947: sync 2024-02-16T19:24:26.730 INFO:tasks.workunit.client.0.smithi040.stdout:2/940: stat d5 0 2024-02-16T19:24:26.730 INFO:tasks.workunit.client.0.smithi040.stdout:9/948: chown d3/d4/d11/d29/d38/d46/d6b/d85/d112/d70 0 1 2024-02-16T19:24:26.730 INFO:tasks.workunit.client.1.smithi163.stdout:3/425: mkdir d0/dc/d1d/d49/d98 0 2024-02-16T19:24:26.731 INFO:tasks.workunit.client.1.smithi163.stdout:7/424: write d11/f44 [2421103,107826] 0 2024-02-16T19:24:26.731 INFO:tasks.workunit.client.1.smithi163.stdout:0/649: symlink d11/d12/d67/d6a/d6d/leb 0 2024-02-16T19:24:26.733 INFO:tasks.workunit.client.0.smithi040.stdout:9/949: rename d3/d4/d11/d71/l78 to d3/d4/d11/d71/de8/l12c 0 2024-02-16T19:24:26.733 INFO:tasks.workunit.client.0.smithi040.stdout:9/950: write d3/d4/d11/d29/d38/d46/d6b/d85/d112/fc7 [107178,56684] 0 2024-02-16T19:24:26.733 INFO:tasks.workunit.client.0.smithi040.stdout:9/951: truncate d3/d4/d11/d71/dc9/dca/f124 947009 0 2024-02-16T19:24:26.733 INFO:tasks.workunit.client.0.smithi040.stdout:9/952: readlink d3/d4/d11/d29/d38/d46/d6b/d85/d112/l41 0 2024-02-16T19:24:26.734 INFO:tasks.workunit.client.1.smithi163.stdout:5/408: rename d0/d22/f23 to d0/d2/d34/d44/f7f 0 2024-02-16T19:24:26.735 INFO:tasks.workunit.client.0.smithi040.stdout:2/941: mkdir d5/d7/d10/d40/d117/d146 0 2024-02-16T19:24:26.739 INFO:tasks.workunit.client.0.smithi040.stdout:9/953: link d3/d4/d11/d29/d38/d46/d6b/d85/d112/d63/c65 d3/d4/d4a/db4/df7/c12d 0 2024-02-16T19:24:26.742 INFO:tasks.workunit.client.1.smithi163.stdout:1/391: write d7/f1d [1328675,97187] 0 2024-02-16T19:24:26.742 INFO:tasks.workunit.client.1.smithi163.stdout:1/392: readlink d7/l8 0 2024-02-16T19:24:26.742 INFO:tasks.workunit.client.1.smithi163.stdout:1/393: truncate d7/d20/f38 1190653 0 2024-02-16T19:24:26.742 INFO:tasks.workunit.client.0.smithi040.stdout:7/905: dwrite d21/d29/d79/f122 [0,4194304] 0 2024-02-16T19:24:26.744 INFO:tasks.workunit.client.0.smithi040.stdout:2/942: symlink d5/d7/d10/d10e/l147 0 2024-02-16T19:24:26.746 INFO:tasks.workunit.client.1.smithi163.stdout:4/323: dread d11/d18/d1f/f33 [0,4194304] 0 2024-02-16T19:24:26.746 INFO:tasks.workunit.client.1.smithi163.stdout:4/324: getdents d11/d5b 0 2024-02-16T19:24:26.747 INFO:tasks.workunit.client.0.smithi040.stdout:9/954: rename d3/d4/d11/d71/da8/f42 to d3/d4/d11/d29/d38/d46/d6b/d85/d112/f12e 0 2024-02-16T19:24:26.747 INFO:tasks.workunit.client.1.smithi163.stdout:7/425: dread d11/d24/d25/f4f [0,4194304] 0 2024-02-16T19:24:26.751 INFO:tasks.workunit.client.1.smithi163.stdout:3/426: mkdir d0/dc/d1c/d32/d62/d99 0 2024-02-16T19:24:26.753 INFO:tasks.workunit.client.1.smithi163.stdout:0/650: symlink d11/d12/d67/d6a/dae/dcc/dd9/lec 0 2024-02-16T19:24:26.753 INFO:tasks.workunit.client.1.smithi163.stdout:1/394: dread d7/dc/f33 [0,4194304] 0 2024-02-16T19:24:26.753 INFO:tasks.workunit.client.1.smithi163.stdout:1/395: truncate d7/dc/f33 5095752 0 2024-02-16T19:24:26.754 INFO:tasks.workunit.client.1.smithi163.stdout:1/396: stat d7/dc/d2d/d3f/f77 0 2024-02-16T19:24:26.754 INFO:tasks.workunit.client.1.smithi163.stdout:1/397: dread - d7/d20/f6d zero size 2024-02-16T19:24:26.755 INFO:tasks.workunit.client.1.smithi163.stdout:5/409: creat d0/d2/d55/f80 x:0 0 0 2024-02-16T19:24:26.755 INFO:tasks.workunit.client.1.smithi163.stdout:5/410: readlink d0/d22/d28/d36/l52 0 2024-02-16T19:24:26.755 INFO:tasks.workunit.client.1.smithi163.stdout:5/411: dread - d0/d22/d28/d36/d3c/f54 zero size 2024-02-16T19:24:26.758 INFO:tasks.workunit.client.0.smithi040.stdout:9/955: mknod d3/d4/d11/d71/dc9/dca/c12f 0 2024-02-16T19:24:26.758 INFO:tasks.workunit.client.1.smithi163.stdout:9/588: rmdir d5/de/d30/d31 39 2024-02-16T19:24:26.758 INFO:tasks.workunit.client.1.smithi163.stdout:5/412: write d0/d2/d34/d44/d53/f68 [380157,64489] 0 2024-02-16T19:24:26.759 INFO:tasks.workunit.client.1.smithi163.stdout:4/325: unlink d11/f17 0 2024-02-16T19:24:26.759 INFO:tasks.workunit.client.1.smithi163.stdout:4/326: truncate d11/d18/d1f/d34/f39 4369886 0 2024-02-16T19:24:26.762 INFO:tasks.workunit.client.1.smithi163.stdout:7/426: symlink d11/d30/l91 0 2024-02-16T19:24:26.763 INFO:tasks.workunit.client.0.smithi040.stdout:9/956: mknod d3/d4/d4a/db4/df7/daf/c130 0 2024-02-16T19:24:26.767 INFO:tasks.workunit.client.0.smithi040.stdout:9/957: creat d3/d4/d11/d71/dc9/f131 x:0 0 0 2024-02-16T19:24:26.767 INFO:tasks.workunit.client.0.smithi040.stdout:9/958: chown d3/d4/d11/c33 119768 1 2024-02-16T19:24:26.767 INFO:tasks.workunit.client.0.smithi040.stdout:9/959: read - d3/d4/d11/d71/da2/fc4 zero size 2024-02-16T19:24:26.769 INFO:tasks.workunit.client.0.smithi040.stdout:7/906: dwrite d21/d8b/d9c/ff7 [0,4194304] 0 2024-02-16T19:24:26.771 INFO:tasks.workunit.client.0.smithi040.stdout:2/943: rename d5/d7/d12/dca/d13b/d65 to d5/d7/d1e/d31/d4f/d63/d148 0 2024-02-16T19:24:26.773 INFO:tasks.workunit.client.0.smithi040.stdout:9/960: unlink d3/d4/d11/d71/da8/f5c 0 2024-02-16T19:24:26.775 INFO:tasks.workunit.client.1.smithi163.stdout:3/427: mknod d0/dc/d1d/d29/d5d/d71/c9a 0 2024-02-16T19:24:26.775 INFO:tasks.workunit.client.1.smithi163.stdout:3/428: truncate d0/f8a 143194 0 2024-02-16T19:24:26.776 INFO:tasks.workunit.client.1.smithi163.stdout:3/429: readlink d0/dc/d1d/d49/l53 0 2024-02-16T19:24:26.780 INFO:tasks.workunit.client.1.smithi163.stdout:0/651: dwrite d11/d12/d20/f39 [0,4194304] 0 2024-02-16T19:24:26.781 INFO:tasks.workunit.client.0.smithi040.stdout:4/804: dwrite dd/de/d10/d36/d9e/d32/f41 [0,4194304] 0 2024-02-16T19:24:26.781 INFO:tasks.workunit.client.1.smithi163.stdout:1/398: creat d7/d5b/f91 x:0 0 0 2024-02-16T19:24:26.782 INFO:tasks.workunit.client.0.smithi040.stdout:2/944: rmdir d5/d7/d1e/d4c/d52/daa 39 2024-02-16T19:24:26.785 INFO:tasks.workunit.client.1.smithi163.stdout:9/589: link d5/de/d30/d31/lbe d5/de/ld9 0 2024-02-16T19:24:26.785 INFO:tasks.workunit.client.0.smithi040.stdout:7/907: getdents d21/d8b/d9c/d10e 0 2024-02-16T19:24:26.785 INFO:tasks.workunit.client.0.smithi040.stdout:7/908: fsync d21/d4d/d77/d87/dc9/fef 0 2024-02-16T19:24:26.786 INFO:tasks.workunit.client.0.smithi040.stdout:7/909: truncate d21/faf 478304 0 2024-02-16T19:24:26.786 INFO:tasks.workunit.client.0.smithi040.stdout:7/910: write d21/d8b/f11e [4529612,77936] 0 2024-02-16T19:24:26.786 INFO:tasks.workunit.client.0.smithi040.stdout:7/911: truncate d21/d29/fc2 1168217 0 2024-02-16T19:24:26.786 INFO:tasks.workunit.client.0.smithi040.stdout:7/912: creat d21/d4d/d77/d7a/dc6/f131 x:0 0 0 2024-02-16T19:24:26.788 INFO:tasks.workunit.client.1.smithi163.stdout:5/413: link d0/d2/f21 d0/f81 0 2024-02-16T19:24:26.788 INFO:tasks.workunit.client.1.smithi163.stdout:5/414: truncate d0/d1c/d60/f6f 241847 0 2024-02-16T19:24:26.788 INFO:tasks.workunit.client.1.smithi163.stdout:5/415: read - d0/d22/d28/d36/d3c/f54 zero size 2024-02-16T19:24:26.789 INFO:tasks.workunit.client.1.smithi163.stdout:5/416: write d0/d22/d28/d36/d3c/d4f/f62 [4289970,70103] 0 2024-02-16T19:24:26.792 INFO:tasks.workunit.client.1.smithi163.stdout:4/327: creat d11/d3d/f62 x:0 0 0 2024-02-16T19:24:26.795 INFO:tasks.workunit.client.0.smithi040.stdout:9/961: dwrite d3/d4/d11/d29/d38/d46/d6b/d85/d112/f39 [4194304,4194304] 0 2024-02-16T19:24:26.798 INFO:tasks.workunit.client.1.smithi163.stdout:7/427: creat d11/d24/d34/d8d/f92 x:0 0 0 2024-02-16T19:24:26.801 INFO:tasks.workunit.client.0.smithi040.stdout:2/945: rename d5/d7/d10/df3/d42/f12d to d5/d7/d1e/d55/d133/f149 0 2024-02-16T19:24:26.803 INFO:tasks.workunit.client.1.smithi163.stdout:0/652: symlink d11/d12/dc6/led 0 2024-02-16T19:24:26.807 INFO:tasks.workunit.client.1.smithi163.stdout:5/417: mkdir d0/d2/d82 0 2024-02-16T19:24:26.807 INFO:tasks.workunit.client.1.smithi163.stdout:5/418: getdents d0/d2/d82 0 2024-02-16T19:24:26.807 INFO:tasks.workunit.client.1.smithi163.stdout:3/430: dwrite d0/dc/f17 [0,4194304] 0 2024-02-16T19:24:26.808 INFO:tasks.workunit.client.1.smithi163.stdout:3/431: getdents d0/dc/d94 0 2024-02-16T19:24:26.809 INFO:tasks.workunit.client.1.smithi163.stdout:4/328: creat d11/d2d/d4f/f63 x:0 0 0 2024-02-16T19:24:26.810 INFO:tasks.workunit.client.0.smithi040.stdout:9/962: mkdir d3/d4/d11/d29/d38/d46/d132 0 2024-02-16T19:24:26.810 INFO:tasks.workunit.client.0.smithi040.stdout:9/963: fdatasync d3/d4/d11/d71/da2/fce 0 2024-02-16T19:24:26.811 INFO:tasks.workunit.client.1.smithi163.stdout:7/428: mknod d11/d26/c93 0 2024-02-16T19:24:26.814 INFO:tasks.workunit.client.1.smithi163.stdout:0/653: mkdir d11/d22/d3e/dee 0 2024-02-16T19:24:26.815 INFO:tasks.workunit.client.0.smithi040.stdout:4/805: dwrite dd/fed [0,4194304] 0 2024-02-16T19:24:26.815 INFO:tasks.workunit.client.0.smithi040.stdout:4/806: fdatasync dd/de/d10/fa9 0 2024-02-16T19:24:26.815 INFO:tasks.workunit.client.0.smithi040.stdout:4/807: write dd/de/f12 [1909180,76954] 0 2024-02-16T19:24:26.816 INFO:tasks.workunit.client.1.smithi163.stdout:9/590: dwrite d5/de/d30/d8a/da8/fad [0,4194304] 0 2024-02-16T19:24:26.817 INFO:tasks.workunit.client.1.smithi163.stdout:1/399: dwrite d7/dc/d2d/d3f/f67 [0,4194304] 0 2024-02-16T19:24:26.819 INFO:tasks.workunit.client.1.smithi163.stdout:3/432: dread d0/dc/d1d/d29/f33 [0,4194304] 0 2024-02-16T19:24:26.819 INFO:tasks.workunit.client.1.smithi163.stdout:3/433: write d0/dc/d1d/d49/f59 [4067528,31749] 0 2024-02-16T19:24:26.819 INFO:tasks.workunit.client.1.smithi163.stdout:3/434: readlink d0/dc/d1d/d29/d38/d46/l6b 0 2024-02-16T19:24:26.819 INFO:tasks.workunit.client.1.smithi163.stdout:3/435: readlink d0/dc/d1d/d49/l6a 0 2024-02-16T19:24:26.820 INFO:tasks.workunit.client.1.smithi163.stdout:3/436: dread d0/f8a [0,4194304] 0 2024-02-16T19:24:26.820 INFO:tasks.workunit.client.1.smithi163.stdout:3/437: dread - d0/dc/d1d/d29/d5d/f81 zero size 2024-02-16T19:24:26.820 INFO:tasks.workunit.client.1.smithi163.stdout:3/438: truncate d0/f13 5059607 0 2024-02-16T19:24:26.821 INFO:tasks.workunit.client.0.smithi040.stdout:2/946: mknod d5/d7/d12/da6/c14a 0 2024-02-16T19:24:26.821 INFO:tasks.workunit.client.0.smithi040.stdout:2/947: fdatasync d5/d7/d1e/d4c/d52/f101 0 2024-02-16T19:24:26.822 INFO:tasks.workunit.client.1.smithi163.stdout:7/429: mknod d11/d50/d5d/d8b/d67/c94 0 2024-02-16T19:24:26.825 INFO:tasks.workunit.client.1.smithi163.stdout:0/654: mknod d11/d21/d7c/cef 0 2024-02-16T19:24:26.825 INFO:tasks.workunit.client.1.smithi163.stdout:4/329: dwrite d11/d3d/f62 [0,4194304] 0 2024-02-16T19:24:26.830 INFO:tasks.workunit.client.1.smithi163.stdout:9/591: creat d5/de/d30/d43/fda x:0 0 0 2024-02-16T19:24:26.830 INFO:tasks.workunit.client.1.smithi163.stdout:9/592: stat d5/de/c38 0 2024-02-16T19:24:26.834 INFO:tasks.workunit.client.0.smithi040.stdout:9/964: symlink d3/d4/dbe/l133 0 2024-02-16T19:24:26.834 INFO:tasks.workunit.client.0.smithi040.stdout:9/965: write d3/d4/d11/d29/d38/d46/d6b/d85/d112/d70/dcc/ffa [1004353,84653] 0 2024-02-16T19:24:26.835 INFO:tasks.workunit.client.1.smithi163.stdout:1/400: rmdir d7/dc/d2d/d32/d3b 39 2024-02-16T19:24:26.835 INFO:tasks.workunit.client.1.smithi163.stdout:1/401: readlink d7/dc/l3c 0 2024-02-16T19:24:26.839 INFO:tasks.workunit.client.1.smithi163.stdout:1/402: dread d7/dc/d2d/d32/d3b/f50 [0,4194304] 0 2024-02-16T19:24:26.840 INFO:tasks.workunit.client.1.smithi163.stdout:3/439: symlink d0/dc/d1c/d32/d62/d99/l9b 0 2024-02-16T19:24:26.841 INFO:tasks.workunit.client.0.smithi040.stdout:4/808: readlink dd/de/daa/le8 0 2024-02-16T19:24:26.842 INFO:tasks.workunit.client.1.smithi163.stdout:7/430: creat d11/d30/f95 x:0 0 0 2024-02-16T19:24:26.844 INFO:tasks.workunit.client.1.smithi163.stdout:5/419: getdents d0 0 2024-02-16T19:24:26.845 INFO:tasks.workunit.client.1.smithi163.stdout:6/385: sync 2024-02-16T19:24:26.845 INFO:tasks.workunit.client.1.smithi163.stdout:2/513: sync 2024-02-16T19:24:26.845 INFO:tasks.workunit.client.1.smithi163.stdout:8/389: sync 2024-02-16T19:24:26.846 INFO:tasks.workunit.client.0.smithi040.stdout:2/948: mknod d5/d7/d10/df3/dbe/d145/c14b 0 2024-02-16T19:24:26.848 INFO:tasks.workunit.client.1.smithi163.stdout:0/655: creat d11/d12/d67/d6a/dae/dcc/dd9/ff0 x:0 0 0 2024-02-16T19:24:26.849 INFO:tasks.workunit.client.1.smithi163.stdout:9/593: dwrite d5/f6 [0,4194304] 0 2024-02-16T19:24:26.853 INFO:tasks.workunit.client.0.smithi040.stdout:9/966: symlink d3/d4/d11/d29/d38/d46/d6b/d85/d112/dc3/l134 0 2024-02-16T19:24:26.853 INFO:tasks.workunit.client.0.smithi040.stdout:9/967: read d3/d4/d11/d29/d38/d46/d6b/f6f [2044348,66804] 0 2024-02-16T19:24:26.853 INFO:tasks.workunit.client.0.smithi040.stdout:9/968: fdatasync d3/d4/d11/d29/d38/d46/d6b/d85/d112/f12e 0 2024-02-16T19:24:26.857 INFO:tasks.workunit.client.0.smithi040.stdout:4/809: mknod dd/cf7 0 2024-02-16T19:24:26.858 INFO:tasks.workunit.client.1.smithi163.stdout:4/330: fsync d11/d18/f1d 0 2024-02-16T19:24:26.860 INFO:tasks.workunit.client.0.smithi040.stdout:2/949: link d5/d7/c1d d5/d7/d12/d1b/d136/c14c 0 2024-02-16T19:24:26.860 INFO:tasks.workunit.client.0.smithi040.stdout:2/950: stat d5/d7/d12/d1b 0 2024-02-16T19:24:26.862 INFO:tasks.workunit.client.1.smithi163.stdout:7/431: creat d11/d24/d25/d4b/d8c/f96 x:0 0 0 2024-02-16T19:24:26.862 INFO:tasks.workunit.client.1.smithi163.stdout:7/432: chown d11/d24/d25/c76 9 1 2024-02-16T19:24:26.863 INFO:tasks.workunit.client.1.smithi163.stdout:5/420: creat d0/d2/d5c/f83 x:0 0 0 2024-02-16T19:24:26.864 INFO:tasks.workunit.client.1.smithi163.stdout:1/403: dwrite d7/dc/d2d/d3f/f67 [0,4194304] 0 2024-02-16T19:24:26.865 INFO:tasks.workunit.client.0.smithi040.stdout:9/969: mkdir d3/d4/d11/d29/d135 0 2024-02-16T19:24:26.865 INFO:tasks.workunit.client.0.smithi040.stdout:9/970: write d3/d4/d11/d71/dc9/dca/f104 [94915,87098] 0 2024-02-16T19:24:26.865 INFO:tasks.workunit.client.0.smithi040.stdout:9/971: write d3/d4/f8 [775585,51802] 0 2024-02-16T19:24:26.865 INFO:tasks.workunit.client.0.smithi040.stdout:9/972: chown d3/d4/d11/d29/d38/d46/d6b/d85/d112 0 1 2024-02-16T19:24:26.865 INFO:tasks.workunit.client.1.smithi163.stdout:5/421: dread d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:26.866 INFO:tasks.workunit.client.1.smithi163.stdout:2/514: unlink d2/c1b 0 2024-02-16T19:24:26.867 INFO:tasks.workunit.client.0.smithi040.stdout:4/810: symlink dd/d58/d5e/dd1/dc6/df0/lf8 0 2024-02-16T19:24:26.868 INFO:tasks.workunit.client.1.smithi163.stdout:8/390: mknod d0/d7/c82 0 2024-02-16T19:24:26.871 INFO:tasks.workunit.client.1.smithi163.stdout:6/386: rename d0/d6/df/d20/l3b to d0/d6/d9/d15/d1c/d65/d63/l76 0 2024-02-16T19:24:26.871 INFO:tasks.workunit.client.1.smithi163.stdout:3/440: dwrite d0/f43 [0,4194304] 0 2024-02-16T19:24:26.871 INFO:tasks.workunit.client.1.smithi163.stdout:3/441: chown d0/dc/d1d/d49 437493 1 2024-02-16T19:24:26.872 INFO:tasks.workunit.client.1.smithi163.stdout:5/422: fdatasync d0/d22/d28/d36/d3c/d4f/f62 0 2024-02-16T19:24:26.872 INFO:tasks.workunit.client.1.smithi163.stdout:0/656: dwrite d11/d12/fdd [0,4194304] 0 2024-02-16T19:24:26.872 INFO:tasks.workunit.client.0.smithi040.stdout:2/951: creat d5/d7/d1e/d55/dd4/f14d x:0 0 0 2024-02-16T19:24:26.875 INFO:tasks.workunit.client.0.smithi040.stdout:4/811: mkdir dd/de/d10/d36/d9e/d32/df9 0 2024-02-16T19:24:26.875 INFO:tasks.workunit.client.1.smithi163.stdout:2/515: write d2/d2a/d39/d66/d6c/fa0 [2188693,84174] 0 2024-02-16T19:24:26.876 INFO:tasks.workunit.client.1.smithi163.stdout:9/594: creat d5/de/fdb x:0 0 0 2024-02-16T19:24:26.876 INFO:tasks.workunit.client.0.smithi040.stdout:9/973: read d3/d4/d11/d29/d36/f75 [1799935,76862] 0 2024-02-16T19:24:26.877 INFO:tasks.workunit.client.0.smithi040.stdout:2/952: symlink d5/d7/d1e/d55/d8d/l14e 0 2024-02-16T19:24:26.877 INFO:tasks.workunit.client.1.smithi163.stdout:5/423: dread d0/d1c/f40 [0,4194304] 0 2024-02-16T19:24:26.878 INFO:tasks.workunit.client.1.smithi163.stdout:4/331: link d11/d18/d1f/d29/f3e d11/d2d/d4f/f64 0 2024-02-16T19:24:26.879 INFO:tasks.workunit.client.0.smithi040.stdout:4/812: mkdir dd/d58/d5e/dd1/dc6/dfa 0 2024-02-16T19:24:26.880 INFO:tasks.workunit.client.1.smithi163.stdout:7/433: creat d11/d50/d5d/f97 x:0 0 0 2024-02-16T19:24:26.881 INFO:tasks.workunit.client.0.smithi040.stdout:2/953: mkdir d5/d7/d1e/d31/d4f/d63/d148/d14f 0 2024-02-16T19:24:26.884 INFO:tasks.workunit.client.1.smithi163.stdout:1/404: unlink d7/dc/d2d/c61 0 2024-02-16T19:24:26.886 INFO:tasks.workunit.client.0.smithi040.stdout:2/954: symlink d5/d7/d12/d1b/d136/l150 0 2024-02-16T19:24:26.890 INFO:tasks.workunit.client.0.smithi040.stdout:4/813: getdents dd/d58/d5e 0 2024-02-16T19:24:26.897 INFO:tasks.workunit.client.1.smithi163.stdout:3/442: symlink d0/dc/d1c/d32/d7f/l9c 0 2024-02-16T19:24:26.897 INFO:tasks.workunit.client.0.smithi040.stdout:9/974: dread d3/d4/d11/fdd [0,4194304] 0 2024-02-16T19:24:26.900 INFO:tasks.workunit.client.0.smithi040.stdout:4/814: write dd/de/d10/d36/d9e/d32/f41 [3982389,90615] 0 2024-02-16T19:24:26.900 INFO:tasks.workunit.client.0.smithi040.stdout:4/815: chown dd/de/db0/fb2 1961368 1 2024-02-16T19:24:26.902 INFO:tasks.workunit.client.1.smithi163.stdout:0/657: creat d11/d12/dc6/ff1 x:0 0 0 2024-02-16T19:24:26.902 INFO:tasks.workunit.client.1.smithi163.stdout:0/658: dread - d11/d12/d67/d6a/d6d/fe3 zero size 2024-02-16T19:24:26.902 INFO:tasks.workunit.client.1.smithi163.stdout:0/659: readlink d11/d22/d7f/d8e/lc0 0 2024-02-16T19:24:26.902 INFO:tasks.workunit.client.1.smithi163.stdout:0/660: getdents d11/d22/d7f/dcf 0 2024-02-16T19:24:26.906 INFO:tasks.workunit.client.0.smithi040.stdout:4/816: creat dd/dd2/ffb x:0 0 0 2024-02-16T19:24:26.909 INFO:tasks.workunit.client.1.smithi163.stdout:4/332: dwrite d11/d18/d1f/f38 [0,4194304] 0 2024-02-16T19:24:26.910 INFO:tasks.workunit.client.0.smithi040.stdout:4/817: unlink dd/de/d10/d36/d9e/d77/l76 0 2024-02-16T19:24:26.911 INFO:tasks.workunit.client.0.smithi040.stdout:4/818: write dd/de/d10/d36/d9e/fea [978691,57284] 0 2024-02-16T19:24:26.911 INFO:tasks.workunit.client.1.smithi163.stdout:1/405: write d7/dc/d2d/d32/f46 [337017,14293] 0 2024-02-16T19:24:26.911 INFO:tasks.workunit.client.1.smithi163.stdout:1/406: creat d7/dc/d2d/d32/d87/f92 x:0 0 0 2024-02-16T19:24:26.915 INFO:tasks.workunit.client.1.smithi163.stdout:3/443: write d0/dc/f28 [5007254,50221] 0 2024-02-16T19:24:26.916 INFO:tasks.workunit.client.1.smithi163.stdout:3/444: readlink d0/dc/d1c/d21/l2f 0 2024-02-16T19:24:26.916 INFO:tasks.workunit.client.1.smithi163.stdout:0/661: mknod d11/d21/d50/d58/d72/d9e/dd8/cf2 0 2024-02-16T19:24:26.916 INFO:tasks.workunit.client.1.smithi163.stdout:0/662: write d11/d12/d67/f91 [1284468,115936] 0 2024-02-16T19:24:26.917 INFO:tasks.workunit.client.1.smithi163.stdout:9/595: unlink d5/de/d16/d47/cbd 0 2024-02-16T19:24:26.918 INFO:tasks.workunit.client.1.smithi163.stdout:0/663: read d11/d21/f49 [385089,127339] 0 2024-02-16T19:24:26.918 INFO:tasks.workunit.client.1.smithi163.stdout:0/664: creat d11/d12/dc6/ff3 x:0 0 0 2024-02-16T19:24:26.919 INFO:tasks.workunit.client.1.smithi163.stdout:9/596: read d5/de/d30/d3d/d49/d6e/f92 [143744,24677] 0 2024-02-16T19:24:26.920 INFO:tasks.workunit.client.1.smithi163.stdout:7/434: dwrite ff [4194304,4194304] 0 2024-02-16T19:24:26.920 INFO:tasks.workunit.client.1.smithi163.stdout:3/445: dread d0/dc/d1c/d32/f57 [0,4194304] 0 2024-02-16T19:24:26.921 INFO:tasks.workunit.client.1.smithi163.stdout:3/446: fsync d0/f8a 0 2024-02-16T19:24:26.922 INFO:tasks.workunit.client.1.smithi163.stdout:6/387: rename d0/d6/df/d20/d40/d17/d47 to d0/d18/d77 0 2024-02-16T19:24:26.924 INFO:tasks.workunit.client.1.smithi163.stdout:7/435: dread d11/d24/d49/f51 [0,4194304] 0 2024-02-16T19:24:26.928 INFO:tasks.workunit.client.1.smithi163.stdout:4/333: dwrite d11/d18/d1f/d23/d3a/f3f [0,4194304] 0 2024-02-16T19:24:26.930 INFO:tasks.workunit.client.1.smithi163.stdout:3/447: mknod d0/dc/d1c/d32/d62/c9d 0 2024-02-16T19:24:26.932 INFO:tasks.workunit.client.1.smithi163.stdout:6/388: creat d0/d6/df/d20/d40/d53/d74/f78 x:0 0 0 2024-02-16T19:24:26.932 INFO:tasks.workunit.client.1.smithi163.stdout:6/389: creat d0/d6/d9/d15/d1c/d65/d63/f79 x:0 0 0 2024-02-16T19:24:26.934 INFO:tasks.workunit.client.1.smithi163.stdout:2/516: rename d2/c91 to d2/d2a/cb3 0 2024-02-16T19:24:26.935 INFO:tasks.workunit.client.1.smithi163.stdout:1/407: dwrite d7/f4f [0,4194304] 0 2024-02-16T19:24:26.936 INFO:tasks.workunit.client.1.smithi163.stdout:7/436: creat d11/d26/f98 x:0 0 0 2024-02-16T19:24:26.936 INFO:tasks.workunit.client.1.smithi163.stdout:7/437: fdatasync d11/d24/d25/f5a 0 2024-02-16T19:24:26.937 INFO:tasks.workunit.client.1.smithi163.stdout:9/597: mknod d5/de/d16/d46/d9b/cdc 0 2024-02-16T19:24:26.940 INFO:tasks.workunit.client.1.smithi163.stdout:1/408: write d7/dc/f30 [673582,42310] 0 2024-02-16T19:24:26.944 INFO:tasks.workunit.client.1.smithi163.stdout:4/334: dwrite d11/d18/d1f/d34/f51 [4194304,4194304] 0 2024-02-16T19:24:26.950 INFO:tasks.workunit.client.1.smithi163.stdout:3/448: mkdir d0/dc/d94/d9e 0 2024-02-16T19:24:26.952 INFO:tasks.workunit.client.1.smithi163.stdout:6/390: rmdir d0/d6/df/d20/d40/d17 39 2024-02-16T19:24:26.952 INFO:tasks.workunit.client.1.smithi163.stdout:6/391: write d0/d6/df/d20/d40/f59 [461605,65193] 0 2024-02-16T19:24:26.952 INFO:tasks.workunit.client.1.smithi163.stdout:6/392: chown d0/d6/d9/d2e/d33/f3d 6 1 2024-02-16T19:24:26.957 INFO:tasks.workunit.client.1.smithi163.stdout:7/438: dwrite d11/d30/f32 [4194304,4194304] 0 2024-02-16T19:24:26.957 INFO:tasks.workunit.client.1.smithi163.stdout:2/517: dwrite d2/d2a/d2d/d73/d84/f8e [0,4194304] 0 2024-02-16T19:24:26.968 INFO:tasks.workunit.client.1.smithi163.stdout:7/439: write d11/d26/f2b [3136630,28628] 0 2024-02-16T19:24:26.968 INFO:tasks.workunit.client.1.smithi163.stdout:7/440: read - d11/d24/d34/f8e zero size 2024-02-16T19:24:26.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:26 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:26.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:26 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:26.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:26 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:26.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:26 smithi163 ceph-mon[36452]: pgmap v252: 65 pgs: 65 active+clean; 2.7 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 190 MiB/s rd, 242 MiB/s wr, 296 op/s 2024-02-16T19:24:27.017 INFO:tasks.workunit.client.1.smithi163.stdout:9/598: creat d5/de/d72/da9/fdd x:0 0 0 2024-02-16T19:24:27.018 INFO:tasks.workunit.client.1.smithi163.stdout:1/409: fsync d7/d20/d2a/f40 0 2024-02-16T19:24:27.019 INFO:tasks.workunit.client.1.smithi163.stdout:9/599: write d5/de/d30/d31/f69 [531473,23390] 0 2024-02-16T19:24:27.019 INFO:tasks.workunit.client.1.smithi163.stdout:9/600: chown d5/de/d16/d47/cc2 140 1 2024-02-16T19:24:27.021 INFO:tasks.workunit.client.1.smithi163.stdout:4/335: unlink d11/d18/d1f/d29/d52/d57/l61 0 2024-02-16T19:24:27.023 INFO:tasks.workunit.client.1.smithi163.stdout:3/449: creat d0/d3e/f9f x:0 0 0 2024-02-16T19:24:27.024 INFO:tasks.workunit.client.1.smithi163.stdout:6/393: mknod d0/dd/c7a 0 2024-02-16T19:24:27.029 INFO:tasks.workunit.client.1.smithi163.stdout:2/518: link d2/d2a/d2d/d3c/la9 d2/d2a/d39/d66/d6c/lb4 0 2024-02-16T19:24:27.030 INFO:tasks.workunit.client.1.smithi163.stdout:4/336: dread d11/d18/d1f/f38 [0,4194304] 0 2024-02-16T19:24:27.030 INFO:tasks.workunit.client.1.smithi163.stdout:7/441: getdents d11/d24/d25/d4b 0 2024-02-16T19:24:27.031 INFO:tasks.workunit.client.1.smithi163.stdout:7/442: write d11/d30/f5b [694521,91280] 0 2024-02-16T19:24:27.037 INFO:tasks.workunit.client.1.smithi163.stdout:0/665: rename d11/d22/d61/lbb to d11/d4f/lf4 0 2024-02-16T19:24:27.041 INFO:tasks.workunit.client.1.smithi163.stdout:6/394: mkdir d0/d6/df/d7b 0 2024-02-16T19:24:27.041 INFO:tasks.workunit.client.1.smithi163.stdout:6/395: chown d0/d6/d9/d15/d1c/d65/d63/f6b 1558 1 2024-02-16T19:24:27.043 INFO:tasks.workunit.client.1.smithi163.stdout:2/519: mkdir d2/d2a/d2d/d73/db5 0 2024-02-16T19:24:27.043 INFO:tasks.workunit.client.1.smithi163.stdout:2/520: write d2/f23 [3639330,21894] 0 2024-02-16T19:24:27.043 INFO:tasks.workunit.client.1.smithi163.stdout:2/521: chown d2/d2a/l5c 630 1 2024-02-16T19:24:27.043 INFO:tasks.workunit.client.1.smithi163.stdout:3/450: dwrite d0/f79 [0,4194304] 0 2024-02-16T19:24:27.043 INFO:tasks.workunit.client.1.smithi163.stdout:3/451: fdatasync d0/dc/d1c/d32/f36 0 2024-02-16T19:24:27.045 INFO:tasks.workunit.client.1.smithi163.stdout:4/337: truncate f2 592649 0 2024-02-16T19:24:27.047 INFO:tasks.workunit.client.1.smithi163.stdout:7/443: mknod d11/d50/d5d/d8b/c99 0 2024-02-16T19:24:27.048 INFO:tasks.workunit.client.1.smithi163.stdout:0/666: creat d11/d21/d50/d58/d6b/ff5 x:0 0 0 2024-02-16T19:24:27.048 INFO:tasks.workunit.client.1.smithi163.stdout:0/667: write d11/d22/d3e/fc8 [478937,89704] 0 2024-02-16T19:24:27.051 INFO:tasks.workunit.client.1.smithi163.stdout:1/410: read d7/f14 [1116478,114521] 0 2024-02-16T19:24:27.051 INFO:tasks.workunit.client.1.smithi163.stdout:9/601: truncate d5/de/d16/f53 4970517 0 2024-02-16T19:24:27.054 INFO:tasks.workunit.client.1.smithi163.stdout:6/396: rmdir d0/d6/d9/d15 39 2024-02-16T19:24:27.056 INFO:tasks.workunit.client.1.smithi163.stdout:3/452: creat d0/dc/d50/d66/d80/fa0 x:0 0 0 2024-02-16T19:24:27.059 INFO:tasks.workunit.client.1.smithi163.stdout:0/668: mknod d11/d21/d50/d58/d6b/cf6 0 2024-02-16T19:24:27.062 INFO:tasks.workunit.client.1.smithi163.stdout:6/397: getdents d0/d6/d9/d15/d1c 0 2024-02-16T19:24:27.064 INFO:tasks.workunit.client.1.smithi163.stdout:4/338: getdents d11/d18 0 2024-02-16T19:24:27.065 INFO:tasks.workunit.client.1.smithi163.stdout:5/424: sync 2024-02-16T19:24:27.065 INFO:tasks.workunit.client.1.smithi163.stdout:8/391: sync 2024-02-16T19:24:27.065 INFO:tasks.workunit.client.1.smithi163.stdout:8/392: creat d0/d24/d2c/f83 x:0 0 0 2024-02-16T19:24:27.065 INFO:tasks.workunit.client.1.smithi163.stdout:8/393: truncate d0/d7/d19/d1f/d30/f52 156897 0 2024-02-16T19:24:27.066 INFO:tasks.workunit.client.1.smithi163.stdout:0/669: mknod d11/d21/d50/cf7 0 2024-02-16T19:24:27.066 INFO:tasks.workunit.client.1.smithi163.stdout:0/670: dread - d11/d21/d50/d58/d6b/ff5 zero size 2024-02-16T19:24:27.069 INFO:tasks.workunit.client.1.smithi163.stdout:9/602: mkdir d5/de/d30/d3d/dde 0 2024-02-16T19:24:27.071 INFO:tasks.workunit.client.1.smithi163.stdout:6/398: rename d0/d6/f1a to d0/d6/d9/d15/d1c/d65/d63/f7c 0 2024-02-16T19:24:27.072 INFO:tasks.workunit.client.1.smithi163.stdout:4/339: creat d11/f65 x:0 0 0 2024-02-16T19:24:27.072 INFO:tasks.workunit.client.1.smithi163.stdout:5/425: mknod d0/d1c/c84 0 2024-02-16T19:24:27.074 INFO:tasks.workunit.client.1.smithi163.stdout:0/671: truncate d11/d21/f46 1924831 0 2024-02-16T19:24:27.075 INFO:tasks.workunit.client.1.smithi163.stdout:2/522: dwrite d2/d26/d54/d58/d98/fa4 [0,4194304] 0 2024-02-16T19:24:27.075 INFO:tasks.workunit.client.1.smithi163.stdout:9/603: truncate d5/de/d30/d3d/f66 322671 0 2024-02-16T19:24:27.075 INFO:tasks.workunit.client.1.smithi163.stdout:9/604: creat d5/de/d30/d3d/d49/fdf x:0 0 0 2024-02-16T19:24:27.076 INFO:tasks.workunit.client.1.smithi163.stdout:6/399: creat d0/d6/d9/d15/d1c/f7d x:0 0 0 2024-02-16T19:24:27.076 INFO:tasks.workunit.client.1.smithi163.stdout:6/400: truncate d0/dd/f6d 924582 0 2024-02-16T19:24:27.082 INFO:tasks.workunit.client.0.smithi040.stdout:7/913: sync 2024-02-16T19:24:27.082 INFO:tasks.workunit.client.0.smithi040.stdout:1/999: sync 2024-02-16T19:24:27.082 INFO:tasks.workunit.client.0.smithi040.stdout:2/955: sync 2024-02-16T19:24:27.083 INFO:tasks.workunit.client.1.smithi163.stdout:5/426: symlink d0/d2/d82/l85 0 2024-02-16T19:24:27.083 INFO:tasks.workunit.client.1.smithi163.stdout:5/427: mknod d0/d1c/d5a/c86 0 2024-02-16T19:24:27.083 INFO:tasks.workunit.client.1.smithi163.stdout:2/523: symlink d2/d2a/d9f/lb6 0 2024-02-16T19:24:27.083 INFO:tasks.workunit.client.1.smithi163.stdout:2/524: fdatasync d2/d2a/fb2 0 2024-02-16T19:24:27.083 INFO:tasks.workunit.client.1.smithi163.stdout:0/672: mkdir d11/d21/d50/d58/d74/df8 0 2024-02-16T19:24:27.084 INFO:tasks.workunit.client.1.smithi163.stdout:7/444: dwrite f3 [0,4194304] 0 2024-02-16T19:24:27.084 INFO:tasks.workunit.client.1.smithi163.stdout:7/445: chown d11/d24/c35 3475612 1 2024-02-16T19:24:27.084 INFO:tasks.workunit.client.1.smithi163.stdout:5/428: truncate d0/f81 709927 0 2024-02-16T19:24:27.084 INFO:tasks.workunit.client.1.smithi163.stdout:5/429: truncate d0/d22/d28/d36/d3c/d4f/f62 5158629 0 2024-02-16T19:24:27.085 INFO:tasks.workunit.client.0.smithi040.stdout:7/914: rename d21/d29/f38 to d21/d4d/d77/d7a/f132 0 2024-02-16T19:24:27.087 INFO:tasks.workunit.client.1.smithi163.stdout:1/411: dwrite f6 [0,4194304] 0 2024-02-16T19:24:27.088 INFO:tasks.workunit.client.1.smithi163.stdout:3/453: dwrite d0/dc/f17 [0,4194304] 0 2024-02-16T19:24:27.088 INFO:tasks.workunit.client.1.smithi163.stdout:3/454: chown d0/d83/c77 23564 1 2024-02-16T19:24:27.088 INFO:tasks.workunit.client.1.smithi163.stdout:3/455: write d0/f6d [378744,36588] 0 2024-02-16T19:24:27.088 INFO:tasks.workunit.client.0.smithi040.stdout:7/915: rename d21/f3c to d21/d29/dad/f133 0 2024-02-16T19:24:27.090 INFO:tasks.workunit.client.1.smithi163.stdout:2/525: mkdir d2/d26/d97/db7 0 2024-02-16T19:24:27.090 INFO:tasks.workunit.client.1.smithi163.stdout:2/526: dread - d2/d2a/d39/d66/f9a zero size 2024-02-16T19:24:27.094 INFO:tasks.workunit.client.1.smithi163.stdout:0/673: link d11/l37 d11/d12/d67/d6a/lf9 0 2024-02-16T19:24:27.097 INFO:tasks.workunit.client.0.smithi040.stdout:2/956: dread d5/d7/d1e/d4c/d52/daa/dcd/ffc [0,4194304] 0 2024-02-16T19:24:27.097 INFO:tasks.workunit.client.0.smithi040.stdout:2/957: readlink d5/d7/d1e/d55/d8d/la8 0 2024-02-16T19:24:27.104 INFO:tasks.workunit.client.1.smithi163.stdout:7/446: dread ff [4194304,4194304] 0 2024-02-16T19:24:27.104 INFO:tasks.workunit.client.1.smithi163.stdout:7/447: chown d11/d50/f86 587475342 1 2024-02-16T19:24:27.104 INFO:tasks.workunit.client.1.smithi163.stdout:7/448: fdatasync f3 0 2024-02-16T19:24:27.104 INFO:tasks.workunit.client.1.smithi163.stdout:7/449: creat d11/d24/d25/f9a x:0 0 0 2024-02-16T19:24:27.109 INFO:tasks.workunit.client.1.smithi163.stdout:2/527: mknod d2/d2a/d2d/d73/db5/cb8 0 2024-02-16T19:24:27.111 INFO:tasks.workunit.client.0.smithi040.stdout:2/958: write d5/d7/d1e/d4c/d52/daa/fd2 [3845492,114632] 0 2024-02-16T19:24:27.111 INFO:tasks.workunit.client.0.smithi040.stdout:2/959: dread - d5/d7/d12/da6/d114/f119 zero size 2024-02-16T19:24:27.112 INFO:tasks.workunit.client.1.smithi163.stdout:7/450: creat d11/d50/d5d/d8b/d67/f9b x:0 0 0 2024-02-16T19:24:27.114 INFO:tasks.workunit.client.1.smithi163.stdout:9/605: dwrite d5/de/d16/d47/f86 [0,4194304] 0 2024-02-16T19:24:27.115 INFO:tasks.workunit.client.1.smithi163.stdout:2/528: rename d2/d26/d54/d58/fac to d2/d2a/d9f/fb9 0 2024-02-16T19:24:27.122 INFO:tasks.workunit.client.1.smithi163.stdout:9/606: creat d5/de/d84/fe0 x:0 0 0 2024-02-16T19:24:27.123 INFO:tasks.workunit.client.1.smithi163.stdout:2/529: rmdir d2/d2a/d2d/d73/db5 39 2024-02-16T19:24:27.123 INFO:tasks.workunit.client.1.smithi163.stdout:2/530: chown d2/d26/d54/d58/d7d 7933 1 2024-02-16T19:24:27.124 INFO:tasks.workunit.client.1.smithi163.stdout:3/456: dread d0/dc/f24 [0,4194304] 0 2024-02-16T19:24:27.125 INFO:tasks.workunit.client.0.smithi040.stdout:2/960: dread d5/d7/d10/df3/d42/fc8 [0,4194304] 0 2024-02-16T19:24:27.125 INFO:tasks.workunit.client.1.smithi163.stdout:9/607: link d5/de/d16/d46/d9b/fca d5/de/d30/d3d/d49/d6d/d97/fe1 0 2024-02-16T19:24:27.125 INFO:tasks.workunit.client.1.smithi163.stdout:3/457: dread d0/f13 [4194304,4194304] 0 2024-02-16T19:24:27.126 INFO:tasks.workunit.client.0.smithi040.stdout:2/961: write d5/d7/d1e/d31/d4f/ff2 [1297699,57494] 0 2024-02-16T19:24:27.126 INFO:tasks.workunit.client.0.smithi040.stdout:2/962: chown d5/d7/d1e/d4c/d52/f9f 4081712 1 2024-02-16T19:24:27.126 INFO:tasks.workunit.client.0.smithi040.stdout:2/963: creat d5/d7/d1e/d55/da3/f151 x:0 0 0 2024-02-16T19:24:27.126 INFO:tasks.workunit.client.0.smithi040.stdout:2/964: stat d5/d7/d12/d1b 0 2024-02-16T19:24:27.128 INFO:tasks.workunit.client.1.smithi163.stdout:9/608: creat d5/de/d16/d46/fe2 x:0 0 0 2024-02-16T19:24:27.130 INFO:tasks.workunit.client.1.smithi163.stdout:3/458: rename d0/l12 to d0/dc/d1d/d29/d5d/d71/la1 0 2024-02-16T19:24:27.133 INFO:tasks.workunit.client.0.smithi040.stdout:2/965: mkdir d5/d7/d12/dca/dea/d152 0 2024-02-16T19:24:27.134 INFO:tasks.workunit.client.1.smithi163.stdout:7/451: write d11/f15 [1733582,22873] 0 2024-02-16T19:24:27.134 INFO:tasks.workunit.client.1.smithi163.stdout:4/340: dwrite f9 [0,4194304] 0 2024-02-16T19:24:27.134 INFO:tasks.workunit.client.1.smithi163.stdout:4/341: getdents d11/d18/d1f/d23/d3a/d4a 0 2024-02-16T19:24:27.135 INFO:tasks.workunit.client.1.smithi163.stdout:3/459: symlink d0/dc/d50/d66/d80/la2 0 2024-02-16T19:24:27.136 INFO:tasks.workunit.client.0.smithi040.stdout:2/966: symlink d5/d7/d10/l153 0 2024-02-16T19:24:27.136 INFO:tasks.workunit.client.0.smithi040.stdout:2/967: dread - d5/d7/d1e/d55/f82 zero size 2024-02-16T19:24:27.143 INFO:tasks.workunit.client.1.smithi163.stdout:9/609: write d5/de/f5b [2827162,100468] 0 2024-02-16T19:24:27.143 INFO:tasks.workunit.client.1.smithi163.stdout:9/610: write d5/de/d16/d46/fe2 [698564,32750] 0 2024-02-16T19:24:27.145 INFO:tasks.workunit.client.1.smithi163.stdout:9/611: mknod d5/de/d84/ce3 0 2024-02-16T19:24:27.147 INFO:tasks.workunit.client.0.smithi040.stdout:2/968: read d5/d7/d1e/f11a [1624148,15914] 0 2024-02-16T19:24:27.149 INFO:tasks.workunit.client.0.smithi040.stdout:2/969: mkdir d5/d7/d12/da6/d114/d154 0 2024-02-16T19:24:27.149 INFO:tasks.workunit.client.0.smithi040.stdout:2/970: creat d5/f155 x:0 0 0 2024-02-16T19:24:27.149 INFO:tasks.workunit.client.0.smithi040.stdout:2/971: fdatasync d5/d7/d1e/d31/d4f/d63/f8e 0 2024-02-16T19:24:27.149 INFO:tasks.workunit.client.0.smithi040.stdout:2/972: creat d5/d7/d10/d10e/f156 x:0 0 0 2024-02-16T19:24:27.151 INFO:tasks.workunit.client.1.smithi163.stdout:2/531: read d2/d2a/d2d/f5e [1112842,71418] 0 2024-02-16T19:24:27.153 INFO:tasks.workunit.client.0.smithi040.stdout:2/973: truncate d5/d7/d1e/f6d 2306906 0 2024-02-16T19:24:27.154 INFO:tasks.workunit.client.1.smithi163.stdout:2/532: mkdir d2/d2a/d2d/d73/d84/dba 0 2024-02-16T19:24:27.155 INFO:tasks.workunit.client.0.smithi040.stdout:2/974: creat d5/d7/d12/dca/dea/f157 x:0 0 0 2024-02-16T19:24:27.156 INFO:tasks.workunit.client.1.smithi163.stdout:2/533: truncate d2/f16 3080777 0 2024-02-16T19:24:27.157 INFO:tasks.workunit.client.1.smithi163.stdout:2/534: symlink d2/d2a/d2d/lbb 0 2024-02-16T19:24:27.158 INFO:tasks.workunit.client.1.smithi163.stdout:2/535: unlink d2/f18 0 2024-02-16T19:24:27.158 INFO:tasks.workunit.client.0.smithi040.stdout:7/916: dwrite d21/d8b/fda [0,4194304] 0 2024-02-16T19:24:27.162 INFO:tasks.workunit.client.1.smithi163.stdout:0/674: dwrite d11/d21/d50/d58/d74/f7e [4194304,4194304] 0 2024-02-16T19:24:27.167 INFO:tasks.workunit.client.0.smithi040.stdout:2/975: dread d5/d7/d10/df3/d42/f7a [0,4194304] 0 2024-02-16T19:24:27.169 INFO:tasks.workunit.client.1.smithi163.stdout:6/401: dwrite d0/d6/df/f2a [0,4194304] 0 2024-02-16T19:24:27.169 INFO:tasks.workunit.client.1.smithi163.stdout:5/430: dwrite d0/d22/d28/d36/f69 [0,4194304] 0 2024-02-16T19:24:27.170 INFO:tasks.workunit.client.1.smithi163.stdout:6/402: creat d0/d6/df/d20/d40/f7e x:0 0 0 2024-02-16T19:24:27.172 INFO:tasks.workunit.client.1.smithi163.stdout:2/536: dread d2/fb [0,4194304] 0 2024-02-16T19:24:27.172 INFO:tasks.workunit.client.1.smithi163.stdout:1/412: dwrite d7/dc/f30 [0,4194304] 0 2024-02-16T19:24:27.173 INFO:tasks.workunit.client.1.smithi163.stdout:7/452: dwrite d11/f7e [0,4194304] 0 2024-02-16T19:24:27.173 INFO:tasks.workunit.client.1.smithi163.stdout:4/342: dwrite d11/d18/d1f/f38 [0,4194304] 0 2024-02-16T19:24:27.174 INFO:tasks.workunit.client.1.smithi163.stdout:0/675: dread d11/d21/f46 [0,4194304] 0 2024-02-16T19:24:27.174 INFO:tasks.workunit.client.1.smithi163.stdout:0/676: chown d11/d21/l36 46690554 1 2024-02-16T19:24:27.174 INFO:tasks.workunit.client.1.smithi163.stdout:0/677: fdatasync d11/d12/d67/d6a/d6d/fcb 0 2024-02-16T19:24:27.174 INFO:tasks.workunit.client.1.smithi163.stdout:9/612: dwrite d5/de/d30/d43/d8f/f90 [0,4194304] 0 2024-02-16T19:24:27.174 INFO:tasks.workunit.client.1.smithi163.stdout:5/431: link d0/d2/f57 d0/d2/d34/d44/d53/f87 0 2024-02-16T19:24:27.175 INFO:tasks.workunit.client.0.smithi040.stdout:7/917: read d21/d4d/d7b/fd6 [1930094,75958] 0 2024-02-16T19:24:27.175 INFO:tasks.workunit.client.1.smithi163.stdout:2/537: truncate d2/d2a/d2d/f5e 2462286 0 2024-02-16T19:24:27.177 INFO:tasks.workunit.client.1.smithi163.stdout:1/413: write d7/fa [1143210,20557] 0 2024-02-16T19:24:27.179 INFO:tasks.workunit.client.0.smithi040.stdout:7/918: unlink d21/d4d/l65 0 2024-02-16T19:24:27.179 INFO:tasks.workunit.client.0.smithi040.stdout:7/919: write d21/d29/f2b [5853107,76157] 0 2024-02-16T19:24:27.179 INFO:tasks.workunit.client.0.smithi040.stdout:2/976: write d5/d7/d1e/f25 [3913420,50970] 0 2024-02-16T19:24:27.179 INFO:tasks.workunit.client.0.smithi040.stdout:2/977: fsync d5/d7/d10/dbc/fe3 0 2024-02-16T19:24:27.179 INFO:tasks.workunit.client.0.smithi040.stdout:2/978: stat d5/d7/d1e/d31/d4f/d63/d8b/f108 0 2024-02-16T19:24:27.183 INFO:tasks.workunit.client.0.smithi040.stdout:7/920: creat d21/d4d/d77/d87/d9b/d112/f134 x:0 0 0 2024-02-16T19:24:27.186 INFO:tasks.workunit.client.1.smithi163.stdout:1/414: write d7/f4f [801984,98843] 0 2024-02-16T19:24:27.186 INFO:tasks.workunit.client.0.smithi040.stdout:2/979: truncate d5/d7/d12/d1b/f121 2025173 0 2024-02-16T19:24:27.186 INFO:tasks.workunit.client.0.smithi040.stdout:2/980: truncate d5/d7/d10/d40/f120 526800 0 2024-02-16T19:24:27.186 INFO:tasks.workunit.client.0.smithi040.stdout:2/981: read - d5/d7/d12/da6/f115 zero size 2024-02-16T19:24:27.187 INFO:tasks.workunit.client.1.smithi163.stdout:5/432: dread d0/d22/f33 [0,4194304] 0 2024-02-16T19:24:27.187 INFO:tasks.workunit.client.1.smithi163.stdout:5/433: write d0/d22/d28/d36/d3c/f54 [666777,99057] 0 2024-02-16T19:24:27.188 INFO:tasks.workunit.client.0.smithi040.stdout:7/921: dread d21/d4d/d77/fa3 [0,4194304] 0 2024-02-16T19:24:27.189 INFO:tasks.workunit.client.1.smithi163.stdout:0/678: symlink d11/d12/d67/d6a/d6d/lfa 0 2024-02-16T19:24:27.189 INFO:tasks.workunit.client.1.smithi163.stdout:4/343: dread d11/f30 [0,4194304] 0 2024-02-16T19:24:27.190 INFO:tasks.workunit.client.0.smithi040.stdout:2/982: creat d5/d7/d10/d40/d117/f158 x:0 0 0 2024-02-16T19:24:27.192 INFO:tasks.workunit.client.1.smithi163.stdout:3/460: rename d0/dc/d1d/d29/d5d to d0/dc/d50/d66/da3 0 2024-02-16T19:24:27.193 INFO:tasks.workunit.client.0.smithi040.stdout:7/922: mknod d21/d29/dad/c135 0 2024-02-16T19:24:27.195 INFO:tasks.workunit.client.0.smithi040.stdout:2/983: write d5/d7/d1e/d31/d4f/d139/f143 [783665,101438] 0 2024-02-16T19:24:27.195 INFO:tasks.workunit.client.0.smithi040.stdout:2/984: chown d5/d7/d12/d1b/lf6 7467 1 2024-02-16T19:24:27.196 INFO:tasks.workunit.client.1.smithi163.stdout:9/613: truncate d5/de/d30/d31/f54 592112 0 2024-02-16T19:24:27.197 INFO:tasks.workunit.client.0.smithi040.stdout:7/923: rename d21/d4d/d77/d87/dc9/dcb/f106 to d21/d4d/d77/d7a/df4/f136 0 2024-02-16T19:24:27.198 INFO:tasks.workunit.client.1.smithi163.stdout:2/538: mkdir d2/d2a/d39/d66/dbc 0 2024-02-16T19:24:27.199 INFO:tasks.workunit.client.0.smithi040.stdout:2/985: mknod d5/d7/d12/dca/dea/c159 0 2024-02-16T19:24:27.199 INFO:tasks.workunit.client.0.smithi040.stdout:2/986: creat d5/d7/d10/df3/f15a x:0 0 0 2024-02-16T19:24:27.201 INFO:tasks.workunit.client.0.smithi040.stdout:7/924: rename d21/d29/f128 to d21/f137 0 2024-02-16T19:24:27.201 INFO:tasks.workunit.client.0.smithi040.stdout:7/925: creat d21/d29/d79/f138 x:0 0 0 2024-02-16T19:24:27.204 INFO:tasks.workunit.client.1.smithi163.stdout:7/453: rmdir d11/d24/d34 39 2024-02-16T19:24:27.207 INFO:tasks.workunit.client.0.smithi040.stdout:2/987: dread - d5/d7/d1e/d31/d4f/f12c zero size 2024-02-16T19:24:27.207 INFO:tasks.workunit.client.0.smithi040.stdout:2/988: dread - d5/d7/d12/dca/f127 zero size 2024-02-16T19:24:27.209 INFO:tasks.workunit.client.1.smithi163.stdout:1/415: rmdir d7/d20/d2a/d8d 0 2024-02-16T19:24:27.211 INFO:tasks.workunit.client.1.smithi163.stdout:5/434: symlink d0/d22/d28/l88 0 2024-02-16T19:24:27.211 INFO:tasks.workunit.client.1.smithi163.stdout:5/435: chown d0/d22/d28/d36/d3c/d4f/f7b 834040056 1 2024-02-16T19:24:27.212 INFO:tasks.workunit.client.0.smithi040.stdout:2/989: rmdir d5/d7/d12/d1b/d136 39 2024-02-16T19:24:27.212 INFO:tasks.workunit.client.0.smithi040.stdout:2/990: stat d5/d7/d10/d40/d117 0 2024-02-16T19:24:27.213 INFO:tasks.workunit.client.1.smithi163.stdout:4/344: mkdir d11/d18/d66 0 2024-02-16T19:24:27.214 INFO:tasks.workunit.client.0.smithi040.stdout:2/991: rmdir d5/d7/d12/dca/d13b 39 2024-02-16T19:24:27.214 INFO:tasks.workunit.client.0.smithi040.stdout:2/992: truncate d5/d7/d1e/d31/d4f/d63/d148/f140 244438 0 2024-02-16T19:24:27.214 INFO:tasks.workunit.client.0.smithi040.stdout:2/993: write d5/d7/d12/fe6 [279270,78955] 0 2024-02-16T19:24:27.215 INFO:tasks.workunit.client.0.smithi040.stdout:2/994: creat d5/f15b x:0 0 0 2024-02-16T19:24:27.215 INFO:tasks.workunit.client.0.smithi040.stdout:2/995: fdatasync d5/d7/d10/f8f 0 2024-02-16T19:24:27.215 INFO:tasks.workunit.client.0.smithi040.stdout:4/819: sync 2024-02-16T19:24:27.215 INFO:tasks.workunit.client.0.smithi040.stdout:9/975: sync 2024-02-16T19:24:27.215 INFO:tasks.workunit.client.0.smithi040.stdout:9/976: stat d3/d4/d11/d29/d38/d46/d6b/f6f 0 2024-02-16T19:24:27.216 INFO:tasks.workunit.client.0.smithi040.stdout:7/926: dwrite d21/d29/dad/ffa [0,4194304] 0 2024-02-16T19:24:27.217 INFO:tasks.workunit.client.1.smithi163.stdout:3/461: creat d0/dc/d1c/d32/d62/fa4 x:0 0 0 2024-02-16T19:24:27.217 INFO:tasks.workunit.client.1.smithi163.stdout:3/462: fdatasync d0/dc/d1c/f20 0 2024-02-16T19:24:27.220 INFO:tasks.workunit.client.0.smithi040.stdout:2/996: mkdir d5/d7/d10/df3/d15c 0 2024-02-16T19:24:27.223 INFO:tasks.workunit.client.0.smithi040.stdout:4/820: mkdir dd/d58/d5e/dd1/d9d/dd6/dfc 0 2024-02-16T19:24:27.223 INFO:tasks.workunit.client.0.smithi040.stdout:4/821: chown dd/d58/ce1 39 1 2024-02-16T19:24:27.225 INFO:tasks.workunit.client.0.smithi040.stdout:9/977: mkdir d3/d4/d11/d29/d38/d46/d6b/d85/dfc/d136 0 2024-02-16T19:24:27.231 INFO:tasks.workunit.client.0.smithi040.stdout:7/927: mknod d21/d29/c139 0 2024-02-16T19:24:27.231 INFO:tasks.workunit.client.0.smithi040.stdout:7/928: read - d21/d4d/d7d/fce zero size 2024-02-16T19:24:27.232 INFO:tasks.workunit.client.0.smithi040.stdout:4/822: dread dd/de/f6b [0,4194304] 0 2024-02-16T19:24:27.232 INFO:tasks.workunit.client.0.smithi040.stdout:4/823: readlink dd/de/db0/ldb 0 2024-02-16T19:24:27.232 INFO:tasks.workunit.client.1.smithi163.stdout:4/345: dwrite d11/d18/f43 [0,4194304] 0 2024-02-16T19:24:27.233 INFO:tasks.workunit.client.1.smithi163.stdout:4/346: fdatasync d11/d2d/d4f/f64 0 2024-02-16T19:24:27.233 INFO:tasks.workunit.client.0.smithi040.stdout:2/997: mknod d5/d7/d1e/d55/c15d 0 2024-02-16T19:24:27.235 INFO:tasks.workunit.client.0.smithi040.stdout:9/978: link d3/d1c/f9e d3/d4/d11/d71/da2/f137 0 2024-02-16T19:24:27.239 INFO:tasks.workunit.client.0.smithi040.stdout:7/929: mknod d21/d4d/d77/d87/d9b/df2/d126/c13a 0 2024-02-16T19:24:27.239 INFO:tasks.workunit.client.0.smithi040.stdout:7/930: read d21/d4d/d7d/fb4 [735621,109100] 0 2024-02-16T19:24:27.241 INFO:tasks.workunit.client.0.smithi040.stdout:4/824: truncate dd/d58/d8c/fbe 3633037 0 2024-02-16T19:24:27.242 INFO:tasks.workunit.client.0.smithi040.stdout:2/998: dread d5/d7/d1e/f33 [0,4194304] 0 2024-02-16T19:24:27.242 INFO:tasks.workunit.client.0.smithi040.stdout:2/999: stat d5/d7/lb7 0 2024-02-16T19:24:27.244 INFO:tasks.workunit.client.0.smithi040.stdout:7/931: link d21/d29/d79/fc4 d21/d4d/d77/d87/d9b/df2/d126/f13b 0 2024-02-16T19:24:27.246 INFO:tasks.workunit.client.1.smithi163.stdout:9/614: truncate d5/de/f5f 3811866 0 2024-02-16T19:24:27.246 INFO:tasks.workunit.client.1.smithi163.stdout:9/615: read d5/f2b [4020211,50656] 0 2024-02-16T19:24:27.246 INFO:tasks.workunit.client.1.smithi163.stdout:9/616: write d5/fc8 [675329,68386] 0 2024-02-16T19:24:27.247 INFO:tasks.workunit.client.0.smithi040.stdout:4/825: truncate dd/de/d10/d36/d9e/d77/f96 1146100 0 2024-02-16T19:24:27.248 INFO:tasks.workunit.client.1.smithi163.stdout:2/539: creat d2/d2a/d2d/d73/d94/fbd x:0 0 0 2024-02-16T19:24:27.250 INFO:tasks.workunit.client.0.smithi040.stdout:4/826: mknod dd/de/d71/cfd 0 2024-02-16T19:24:27.250 INFO:tasks.workunit.client.0.smithi040.stdout:4/827: getdents dd/d58/d5e/dd1/dc6/dfa 0 2024-02-16T19:24:27.250 INFO:tasks.workunit.client.0.smithi040.stdout:4/828: chown dd/de/d10/d36/l70 2470791 1 2024-02-16T19:24:27.250 INFO:tasks.workunit.client.0.smithi040.stdout:4/829: creat dd/de/d10/d36/ffe x:0 0 0 2024-02-16T19:24:27.251 INFO:tasks.workunit.client.1.smithi163.stdout:1/416: stat d7/d20/d21/f45 0 2024-02-16T19:24:27.252 INFO:tasks.workunit.client.1.smithi163.stdout:4/347: dwrite d11/f14 [0,4194304] 0 2024-02-16T19:24:27.253 INFO:tasks.workunit.client.1.smithi163.stdout:5/436: mknod d0/d22/d28/d36/d3c/c89 0 2024-02-16T19:24:27.253 INFO:tasks.workunit.client.1.smithi163.stdout:5/437: chown d0/d2/d55/f80 2176 1 2024-02-16T19:24:27.253 INFO:tasks.workunit.client.0.smithi040.stdout:7/932: write d21/d29/dad/f133 [3081013,130588] 0 2024-02-16T19:24:27.260 INFO:tasks.workunit.client.0.smithi040.stdout:7/933: creat d21/d7e/df8/db7/de7/d118/d11b/f13c x:0 0 0 2024-02-16T19:24:27.263 INFO:tasks.workunit.client.0.smithi040.stdout:9/979: dwrite d3/d4/d11/d29/d38/d46/d6b/d85/dfc/f10c [0,4194304] 0 2024-02-16T19:24:27.265 INFO:tasks.workunit.client.1.smithi163.stdout:3/463: mkdir d0/d83/da5 0 2024-02-16T19:24:27.265 INFO:tasks.workunit.client.1.smithi163.stdout:3/464: fsync d0/dc/d1c/f23 0 2024-02-16T19:24:27.267 INFO:tasks.workunit.client.0.smithi040.stdout:4/830: dwrite dd/de/d10/f8a [0,4194304] 0 2024-02-16T19:24:27.269 INFO:tasks.workunit.client.0.smithi040.stdout:9/980: mknod d3/c138 0 2024-02-16T19:24:27.270 INFO:tasks.workunit.client.1.smithi163.stdout:9/617: mkdir d5/de/d16/d46/d9b/de4 0 2024-02-16T19:24:27.271 INFO:tasks.workunit.client.0.smithi040.stdout:4/831: mkdir dd/dd2/dff 0 2024-02-16T19:24:27.273 INFO:tasks.workunit.client.1.smithi163.stdout:2/540: symlink d2/d2a/d39/d66/dbc/lbe 0 2024-02-16T19:24:27.277 INFO:tasks.workunit.client.1.smithi163.stdout:4/348: mknod d11/d18/d1f/d34/c67 0 2024-02-16T19:24:27.280 INFO:tasks.workunit.client.1.smithi163.stdout:5/438: symlink d0/d2/d34/d44/d64/l8a 0 2024-02-16T19:24:27.280 INFO:tasks.workunit.client.1.smithi163.stdout:6/403: rename d0/d6/d9 to d0/dd/d7f 0 2024-02-16T19:24:27.280 INFO:tasks.workunit.client.0.smithi040.stdout:4/832: write dd/de/d10/fa9 [433274,73655] 0 2024-02-16T19:24:27.280 INFO:tasks.workunit.client.0.smithi040.stdout:4/833: rename dd/d58/d5e/dd1/d9d to dd/d58/d5e/dd1/d9d/d100 22 2024-02-16T19:24:27.280 INFO:tasks.workunit.client.0.smithi040.stdout:4/834: fsync dd/de/d10/f1b 0 2024-02-16T19:24:27.283 INFO:tasks.workunit.client.1.smithi163.stdout:3/465: link d0/dc/d1d/d29/f45 d0/dc/d1d/d29/d38/d46/fa6 0 2024-02-16T19:24:27.284 INFO:tasks.workunit.client.1.smithi163.stdout:1/417: mkdir d7/d20/d93 0 2024-02-16T19:24:27.287 INFO:tasks.workunit.client.1.smithi163.stdout:4/349: truncate d11/d18/d1f/d29/f49 1580627 0 2024-02-16T19:24:27.288 INFO:tasks.workunit.client.1.smithi163.stdout:5/439: mknod d0/d1c/c8b 0 2024-02-16T19:24:27.291 INFO:tasks.workunit.client.1.smithi163.stdout:0/679: rename d11/d12/c2b to d11/d21/d50/d58/d6b/cfb 0 2024-02-16T19:24:27.292 INFO:tasks.workunit.client.1.smithi163.stdout:3/466: mkdir d0/dc/d96/da7 0 2024-02-16T19:24:27.293 INFO:tasks.workunit.client.0.smithi040.stdout:4/835: dread dd/d58/d5e/f61 [0,4194304] 0 2024-02-16T19:24:27.293 INFO:tasks.workunit.client.1.smithi163.stdout:1/418: stat d7/d20/c2b 0 2024-02-16T19:24:27.293 INFO:tasks.workunit.client.1.smithi163.stdout:5/440: dread d0/d2/d34/d44/d53/f68 [0,4194304] 0 2024-02-16T19:24:27.293 INFO:tasks.workunit.client.1.smithi163.stdout:5/441: dread - d0/d2/d5c/f5d zero size 2024-02-16T19:24:27.296 INFO:tasks.workunit.client.1.smithi163.stdout:0/680: dread d11/d21/d50/d58/d6b/fe9 [0,4194304] 0 2024-02-16T19:24:27.297 INFO:tasks.workunit.client.1.smithi163.stdout:0/681: fsync d11/d21/d50/d58/d6b/ff5 0 2024-02-16T19:24:27.297 INFO:tasks.workunit.client.1.smithi163.stdout:0/682: fdatasync d11/d21/d50/d58/d6b/fe9 0 2024-02-16T19:24:27.297 INFO:tasks.workunit.client.1.smithi163.stdout:0/683: chown d11/d21/d7c/fc5 338 1 2024-02-16T19:24:27.297 INFO:tasks.workunit.client.1.smithi163.stdout:7/454: rename d11/d30/l91 to d11/d26/l9c 0 2024-02-16T19:24:27.297 INFO:tasks.workunit.client.1.smithi163.stdout:6/404: write d0/dd/d7f/d15/d1c/f2b [266972,67551] 0 2024-02-16T19:24:27.297 INFO:tasks.workunit.client.1.smithi163.stdout:3/467: mkdir d0/dc/d94/d9e/da8 0 2024-02-16T19:24:27.299 INFO:tasks.workunit.client.0.smithi040.stdout:4/836: creat dd/de/daa/f101 x:0 0 0 2024-02-16T19:24:27.302 INFO:tasks.workunit.client.1.smithi163.stdout:7/455: truncate fa 4686031 0 2024-02-16T19:24:27.302 INFO:tasks.workunit.client.1.smithi163.stdout:7/456: truncate d11/d50/f86 393434 0 2024-02-16T19:24:27.303 INFO:tasks.workunit.client.0.smithi040.stdout:7/934: dwrite d21/d4d/d77/d87/d9b/f98 [4194304,4194304] 0 2024-02-16T19:24:27.306 INFO:tasks.workunit.client.1.smithi163.stdout:3/468: link d0/d3e/l54 d0/dc/d50/d66/la9 0 2024-02-16T19:24:27.309 INFO:tasks.workunit.client.1.smithi163.stdout:7/457: mknod d11/d24/d25/c9d 0 2024-02-16T19:24:27.310 INFO:tasks.workunit.client.0.smithi040.stdout:4/837: mkdir dd/de/d71/d102 0 2024-02-16T19:24:27.312 INFO:tasks.workunit.client.0.smithi040.stdout:9/981: dwrite d3/d4/d4a/f62 [0,4194304] 0 2024-02-16T19:24:27.312 INFO:tasks.workunit.client.1.smithi163.stdout:9/618: dwrite d5/de/d16/d46/d9b/f9d [0,4194304] 0 2024-02-16T19:24:27.313 INFO:tasks.workunit.client.0.smithi040.stdout:7/935: dread d21/f24 [0,4194304] 0 2024-02-16T19:24:27.313 INFO:tasks.workunit.client.0.smithi040.stdout:4/838: write dd/de/d10/d36/f63 [2517370,79444] 0 2024-02-16T19:24:27.314 INFO:tasks.workunit.client.1.smithi163.stdout:3/469: mknod d0/d83/da5/caa 0 2024-02-16T19:24:27.315 INFO:tasks.workunit.client.1.smithi163.stdout:7/458: link d11/d50/d5d/d8b/l41 d11/d24/l9e 0 2024-02-16T19:24:27.316 INFO:tasks.workunit.client.1.smithi163.stdout:2/541: dwrite d2/d2a/d2d/d73/d94/fbd [0,4194304] 0 2024-02-16T19:24:27.321 INFO:tasks.workunit.client.0.smithi040.stdout:9/982: truncate d3/d4/d11/d29/d38/d46/d6b/d85/d112/d63/fa5 427479 0 2024-02-16T19:24:27.321 INFO:tasks.workunit.client.0.smithi040.stdout:9/983: creat d3/d4/d11/d71/dc9/f139 x:0 0 0 2024-02-16T19:24:27.322 INFO:tasks.workunit.client.1.smithi163.stdout:3/470: mknod d0/dc/d1d/d29/d38/cab 0 2024-02-16T19:24:27.323 INFO:tasks.workunit.client.0.smithi040.stdout:7/936: mknod d21/c13d 0 2024-02-16T19:24:27.323 INFO:tasks.workunit.client.0.smithi040.stdout:7/937: dread - d21/d29/d79/f138 zero size 2024-02-16T19:24:27.329 INFO:tasks.workunit.client.0.smithi040.stdout:9/984: mkdir d3/d4/d11/d29/d38/d46/d6b/d85/d112/d13a 0 2024-02-16T19:24:27.330 INFO:tasks.workunit.client.1.smithi163.stdout:0/684: dwrite d11/d4f/fc4 [0,4194304] 0 2024-02-16T19:24:27.330 INFO:tasks.workunit.client.1.smithi163.stdout:0/685: chown d11/d12/d20 7032172 1 2024-02-16T19:24:27.330 INFO:tasks.workunit.client.1.smithi163.stdout:0/686: chown d11/d21/d50/cf7 977628 1 2024-02-16T19:24:27.331 INFO:tasks.workunit.client.0.smithi040.stdout:4/839: write dd/f6e [2794182,38996] 0 2024-02-16T19:24:27.331 INFO:tasks.workunit.client.1.smithi163.stdout:2/542: dread - d2/d26/d54/f9b zero size 2024-02-16T19:24:27.331 INFO:tasks.workunit.client.1.smithi163.stdout:9/619: write d5/de/d72/fa7 [1435629,85490] 0 2024-02-16T19:24:27.331 INFO:tasks.workunit.client.1.smithi163.stdout:4/350: dwrite d11/f4b [0,4194304] 0 2024-02-16T19:24:27.332 INFO:tasks.workunit.client.1.smithi163.stdout:6/405: dwrite d0/dd/d7f/f6f [0,4194304] 0 2024-02-16T19:24:27.333 INFO:tasks.workunit.client.1.smithi163.stdout:8/394: sync 2024-02-16T19:24:27.334 INFO:tasks.workunit.client.1.smithi163.stdout:7/459: getdents d11/d24/d49 0 2024-02-16T19:24:27.334 INFO:tasks.workunit.client.1.smithi163.stdout:7/460: chown d11/d24/d34/l45 23 1 2024-02-16T19:24:27.335 INFO:tasks.workunit.client.1.smithi163.stdout:5/442: dwrite d0/d22/d28/d36/d3c/f43 [0,4194304] 0 2024-02-16T19:24:27.338 INFO:tasks.workunit.client.0.smithi040.stdout:4/840: mkdir dd/de/d10/d36/d9e/d32/d103 0 2024-02-16T19:24:27.339 INFO:tasks.workunit.client.0.smithi040.stdout:7/938: getdents d21/d4d/d77/d87/d9b/df2 0 2024-02-16T19:24:27.340 INFO:tasks.workunit.client.0.smithi040.stdout:7/939: creat d21/d29/f13e x:0 0 0 2024-02-16T19:24:27.341 INFO:tasks.workunit.client.1.smithi163.stdout:0/687: truncate d11/d22/f42 2069038 0 2024-02-16T19:24:27.342 INFO:tasks.workunit.client.0.smithi040.stdout:9/985: mknod d3/d4/c13b 0 2024-02-16T19:24:27.343 INFO:tasks.workunit.client.1.smithi163.stdout:5/443: dread d0/d1c/f74 [0,4194304] 0 2024-02-16T19:24:27.344 INFO:tasks.workunit.client.1.smithi163.stdout:2/543: symlink d2/d2a/d2d/d73/lbf 0 2024-02-16T19:24:27.349 INFO:tasks.workunit.client.0.smithi040.stdout:7/940: read d21/d7e/f84 [2650744,105846] 0 2024-02-16T19:24:27.350 INFO:tasks.workunit.client.0.smithi040.stdout:9/986: mknod d3/d1c/c13c 0 2024-02-16T19:24:27.351 INFO:tasks.workunit.client.0.smithi040.stdout:4/841: mknod dd/de/d45/c104 0 2024-02-16T19:24:27.352 INFO:tasks.workunit.client.1.smithi163.stdout:5/444: write d0/f2c [2565983,47355] 0 2024-02-16T19:24:27.352 INFO:tasks.workunit.client.1.smithi163.stdout:5/445: chown d0/d2/d34/d44/d63/d79 378 1 2024-02-16T19:24:27.354 INFO:tasks.workunit.client.0.smithi040.stdout:7/941: creat d21/d29/d79/d10c/f13f x:0 0 0 2024-02-16T19:24:27.354 INFO:tasks.workunit.client.1.smithi163.stdout:3/471: dwrite d0/dc/d1d/d49/f56 [4194304,4194304] 0 2024-02-16T19:24:27.355 INFO:tasks.workunit.client.1.smithi163.stdout:4/351: creat d11/d2d/d4f/d60/f68 x:0 0 0 2024-02-16T19:24:27.355 INFO:tasks.workunit.client.1.smithi163.stdout:8/395: dwrite d0/dc/f6d [0,4194304] 0 2024-02-16T19:24:27.355 INFO:tasks.workunit.client.1.smithi163.stdout:8/396: dread - d0/d7/f68 zero size 2024-02-16T19:24:27.356 INFO:tasks.workunit.client.0.smithi040.stdout:9/987: creat d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/d101/f13d x:0 0 0 2024-02-16T19:24:27.356 INFO:tasks.workunit.client.0.smithi040.stdout:9/988: write d3/d4/d11/d29/d38/d46/d6b/d85/d112/d63/fb6 [716150,61439] 0 2024-02-16T19:24:27.356 INFO:tasks.workunit.client.0.smithi040.stdout:9/989: chown d3/d4/d11/d29 7575140 1 2024-02-16T19:24:27.356 INFO:tasks.workunit.client.0.smithi040.stdout:9/990: write d3/d4/f2c [8901662,81214] 0 2024-02-16T19:24:27.360 INFO:tasks.workunit.client.0.smithi040.stdout:7/942: rename d21/d29/d79/f8c to d21/df9/f140 0 2024-02-16T19:24:27.360 INFO:tasks.workunit.client.0.smithi040.stdout:7/943: dread - d21/d4d/d7b/f119 zero size 2024-02-16T19:24:27.361 INFO:tasks.workunit.client.1.smithi163.stdout:7/461: truncate d11/f1c 1851761 0 2024-02-16T19:24:27.363 INFO:tasks.workunit.client.0.smithi040.stdout:4/842: rmdir dd/d58/d5e/dd1/d9d 39 2024-02-16T19:24:27.363 INFO:tasks.workunit.client.0.smithi040.stdout:4/843: read - dd/de/d45/fc4 zero size 2024-02-16T19:24:27.366 INFO:tasks.workunit.client.0.smithi040.stdout:9/991: dread d3/d4/d4a/f62 [0,4194304] 0 2024-02-16T19:24:27.367 INFO:tasks.workunit.client.0.smithi040.stdout:7/944: rename fa to d21/d7e/df8/f141 0 2024-02-16T19:24:27.370 INFO:tasks.workunit.client.1.smithi163.stdout:0/688: symlink d11/d12/d67/d6a/d6d/lfc 0 2024-02-16T19:24:27.370 INFO:tasks.workunit.client.1.smithi163.stdout:0/689: fdatasync d11/d4f/fb0 0 2024-02-16T19:24:27.371 INFO:tasks.workunit.client.0.smithi040.stdout:4/844: unlink dd/d58/d5e/ld3 0 2024-02-16T19:24:27.371 INFO:tasks.workunit.client.0.smithi040.stdout:4/845: dread - dd/de/d10/fb9 zero size 2024-02-16T19:24:27.372 INFO:tasks.workunit.client.0.smithi040.stdout:4/846: write dd/f43 [4481283,28801] 0 2024-02-16T19:24:27.372 INFO:tasks.workunit.client.0.smithi040.stdout:4/847: chown dd/d58/d8c/fbe 686739897 1 2024-02-16T19:24:27.373 INFO:tasks.workunit.client.0.smithi040.stdout:4/848: truncate dd/de/d45/fc4 495606 0 2024-02-16T19:24:27.373 INFO:tasks.workunit.client.0.smithi040.stdout:4/849: creat dd/de/d10/d36/dad/f105 x:0 0 0 2024-02-16T19:24:27.373 INFO:tasks.workunit.client.1.smithi163.stdout:2/544: mknod d2/d2a/cc0 0 2024-02-16T19:24:27.378 INFO:tasks.workunit.client.0.smithi040.stdout:9/992: dread d3/d4/d4a/db4/df7/f7d [0,4194304] 0 2024-02-16T19:24:27.378 INFO:tasks.workunit.client.0.smithi040.stdout:9/993: dread - d3/d4/dbe/de9/f11b zero size 2024-02-16T19:24:27.378 INFO:tasks.workunit.client.0.smithi040.stdout:9/994: fdatasync d3/d4/d11/d29/d38/d46/f5b 0 2024-02-16T19:24:27.380 INFO:tasks.workunit.client.0.smithi040.stdout:7/945: mknod d21/d4d/d77/d87/dc9/c142 0 2024-02-16T19:24:27.383 INFO:tasks.workunit.client.1.smithi163.stdout:2/545: write d2/d2a/d39/f42 [2491494,126423] 0 2024-02-16T19:24:27.383 INFO:tasks.workunit.client.1.smithi163.stdout:3/472: mkdir d0/dc/d94/d9e/dac 0 2024-02-16T19:24:27.385 INFO:tasks.workunit.client.1.smithi163.stdout:0/690: dwrite d11/d12/d20/fe4 [0,4194304] 0 2024-02-16T19:24:27.386 INFO:tasks.workunit.client.0.smithi040.stdout:4/850: mkdir dd/dd2/d106 0 2024-02-16T19:24:27.388 INFO:tasks.workunit.client.0.smithi040.stdout:9/995: unlink d3/d4/d4a/db4/df7/fb7 0 2024-02-16T19:24:27.388 INFO:tasks.workunit.client.0.smithi040.stdout:9/996: write d3/d1c/f64 [945765,94126] 0 2024-02-16T19:24:27.389 INFO:tasks.workunit.client.1.smithi163.stdout:8/397: rename d0/d7/d19/d1f/d30/f47 to d0/d24/d3c/f84 0 2024-02-16T19:24:27.390 INFO:tasks.workunit.client.1.smithi163.stdout:8/398: chown d0/d24/d3c/f4c 98617329 1 2024-02-16T19:24:27.391 INFO:tasks.workunit.client.0.smithi040.stdout:7/946: rename d21/d4d/d77/d7a/f132 to d21/d7e/df8/db7/de7/d118/d12d/f143 0 2024-02-16T19:24:27.392 INFO:tasks.workunit.client.0.smithi040.stdout:4/851: mkdir dd/de/d71/d102/d107 0 2024-02-16T19:24:27.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:27.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:27.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:26 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:27.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:26 smithi040 ceph-mon[29201]: pgmap v252: 65 pgs: 65 active+clean; 2.7 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 190 MiB/s rd, 242 MiB/s wr, 296 op/s 2024-02-16T19:24:27.394 INFO:tasks.workunit.client.1.smithi163.stdout:7/462: getdents d11/d50/d5d/d8b 0 2024-02-16T19:24:27.396 INFO:tasks.workunit.client.0.smithi040.stdout:9/997: creat d3/d4/d4a/db4/df7/f13e x:0 0 0 2024-02-16T19:24:27.396 INFO:tasks.workunit.client.0.smithi040.stdout:9/998: chown d3/d4/d11/d29/d38/d46/d6b/d85/d112/d31/d101 1909197 1 2024-02-16T19:24:27.397 INFO:tasks.workunit.client.1.smithi163.stdout:5/446: getdents d0/d2/d5c 0 2024-02-16T19:24:27.397 INFO:tasks.workunit.client.1.smithi163.stdout:5/447: chown d0/d2/d34/d44/d53/f87 325 1 2024-02-16T19:24:27.399 INFO:tasks.workunit.client.0.smithi040.stdout:7/947: rename d21/d29/dad/ffa to d21/d7e/df8/db7/de7/d10f/f144 0 2024-02-16T19:24:27.400 INFO:tasks.workunit.client.1.smithi163.stdout:9/620: getdents d5 0 2024-02-16T19:24:27.401 INFO:tasks.workunit.client.0.smithi040.stdout:4/852: mknod dd/d58/d8c/c108 0 2024-02-16T19:24:27.401 INFO:tasks.workunit.client.0.smithi040.stdout:7/948: write d21/d4d/d77/fa3 [592732,106327] 0 2024-02-16T19:24:27.403 INFO:tasks.workunit.client.0.smithi040.stdout:9/999: creat d3/d4/f13f x:0 0 0 2024-02-16T19:24:27.406 INFO:tasks.workunit.client.1.smithi163.stdout:2/546: chown d2/d26/d97/fb1 402439337 1 2024-02-16T19:24:27.406 INFO:tasks.workunit.client.1.smithi163.stdout:2/547: truncate d2/d2a/d2d/f6a 4786451 0 2024-02-16T19:24:27.406 INFO:tasks.workunit.client.1.smithi163.stdout:2/548: chown d2/d2a/d2d/c4b 70242 1 2024-02-16T19:24:27.406 INFO:tasks.workunit.client.1.smithi163.stdout:2/549: dread - d2/d26/d54/d58/f69 zero size 2024-02-16T19:24:27.407 INFO:tasks.workunit.client.0.smithi040.stdout:4/853: rename dd/de/d71/la6 to dd/de/d71/d102/l109 0 2024-02-16T19:24:27.409 INFO:tasks.workunit.client.1.smithi163.stdout:3/473: creat d0/dc/d94/d9e/fad x:0 0 0 2024-02-16T19:24:27.409 INFO:tasks.workunit.client.1.smithi163.stdout:3/474: read d0/dc/d1c/d21/f7a [3561499,112093] 0 2024-02-16T19:24:27.409 INFO:tasks.workunit.client.1.smithi163.stdout:3/475: chown d0/dc/d1c/d32/c6f 43802474 1 2024-02-16T19:24:27.409 INFO:tasks.workunit.client.0.smithi040.stdout:7/949: truncate d21/d4d/f6f 4070698 0 2024-02-16T19:24:27.410 INFO:tasks.workunit.client.1.smithi163.stdout:5/448: write d0/d22/d28/d36/f69 [1939710,101112] 0 2024-02-16T19:24:27.410 INFO:tasks.workunit.client.1.smithi163.stdout:5/449: dread - d0/d2/d34/d44/d63/f7a zero size 2024-02-16T19:24:27.413 INFO:tasks.workunit.client.0.smithi040.stdout:4/854: link dd/de/d10/d36/d9e/d32/c3f dd/de/c10a 0 2024-02-16T19:24:27.413 INFO:tasks.workunit.client.0.smithi040.stdout:4/855: chown dd/de/db0/ldb 8784087 1 2024-02-16T19:24:27.413 INFO:tasks.workunit.client.0.smithi040.stdout:4/856: readlink dd/de/l5a 0 2024-02-16T19:24:27.413 INFO:tasks.workunit.client.0.smithi040.stdout:4/857: creat dd/de/d10/d36/dad/f10b x:0 0 0 2024-02-16T19:24:27.415 INFO:tasks.workunit.client.0.smithi040.stdout:7/950: rename d21/d4d/f66 to d21/d4d/d77/d87/d9b/d112/f145 0 2024-02-16T19:24:27.417 INFO:tasks.workunit.client.1.smithi163.stdout:0/691: creat d11/d21/d50/d58/d72/d9e/ffd x:0 0 0 2024-02-16T19:24:27.418 INFO:tasks.workunit.client.1.smithi163.stdout:8/399: mkdir d0/d24/d78/d85 0 2024-02-16T19:24:27.418 INFO:tasks.workunit.client.1.smithi163.stdout:8/400: chown d0/f44 3 1 2024-02-16T19:24:27.419 INFO:tasks.workunit.client.0.smithi040.stdout:4/858: symlink dd/de/d71/l10c 0 2024-02-16T19:24:27.419 INFO:tasks.workunit.client.0.smithi040.stdout:4/859: chown dd/f6e 0 1 2024-02-16T19:24:27.420 INFO:tasks.workunit.client.1.smithi163.stdout:2/550: truncate d2/d2a/d39/d66/f77 129402 0 2024-02-16T19:24:27.422 INFO:tasks.workunit.client.1.smithi163.stdout:3/476: creat d0/dc/d94/fae x:0 0 0 2024-02-16T19:24:27.423 INFO:tasks.workunit.client.1.smithi163.stdout:9/621: dwrite d5/f77 [0,4194304] 0 2024-02-16T19:24:27.424 INFO:tasks.workunit.client.0.smithi040.stdout:7/951: creat d21/d4d/d77/d87/d9b/df2/f146 x:0 0 0 2024-02-16T19:24:27.424 INFO:tasks.workunit.client.0.smithi040.stdout:7/952: write d21/d4d/f61 [1722761,92779] 0 2024-02-16T19:24:27.424 INFO:tasks.workunit.client.0.smithi040.stdout:7/953: creat d21/d4d/d77/d87/d9b/df2/f147 x:0 0 0 2024-02-16T19:24:27.425 INFO:tasks.workunit.client.1.smithi163.stdout:7/463: dwrite d11/f19 [0,4194304] 0 2024-02-16T19:24:27.426 INFO:tasks.workunit.client.0.smithi040.stdout:4/860: creat dd/d58/d5e/f10d x:0 0 0 2024-02-16T19:24:27.428 INFO:tasks.workunit.client.1.smithi163.stdout:0/692: truncate d11/d12/d67/d6a/d6d/fcb 3060895 0 2024-02-16T19:24:27.428 INFO:tasks.workunit.client.1.smithi163.stdout:0/693: creat d11/d21/d50/d58/d72/d9e/ffe x:0 0 0 2024-02-16T19:24:27.428 INFO:tasks.workunit.client.1.smithi163.stdout:8/401: write d0/d24/d2c/f4d [539438,53210] 0 2024-02-16T19:24:27.430 INFO:tasks.workunit.client.1.smithi163.stdout:2/551: link d2/d26/d54/d58/d98/lb0 d2/d2a/d2d/d73/d84/dba/lc1 0 2024-02-16T19:24:27.430 INFO:tasks.workunit.client.0.smithi040.stdout:7/954: mknod d21/c148 0 2024-02-16T19:24:27.433 INFO:tasks.workunit.client.0.smithi040.stdout:4/861: rename dd/de/d71/l10c to dd/d58/l10e 0 2024-02-16T19:24:27.433 INFO:tasks.workunit.client.0.smithi040.stdout:4/862: stat dd/de/d45/f73 0 2024-02-16T19:24:27.435 INFO:tasks.workunit.client.0.smithi040.stdout:7/955: unlink d21/d29/l6d 0 2024-02-16T19:24:27.437 INFO:tasks.workunit.client.1.smithi163.stdout:9/622: unlink d5/ld 0 2024-02-16T19:24:27.437 INFO:tasks.workunit.client.1.smithi163.stdout:9/623: truncate d5/de/d30/d8a/da8/fcb 4811349 0 2024-02-16T19:24:27.437 INFO:tasks.workunit.client.1.smithi163.stdout:9/624: fsync d5/f2f 0 2024-02-16T19:24:27.437 INFO:tasks.workunit.client.1.smithi163.stdout:9/625: chown d5/de/d16/d46/d9b/cdc 32970664 1 2024-02-16T19:24:27.438 INFO:tasks.workunit.client.0.smithi040.stdout:4/863: unlink dd/de/d10/d36/d9e/ff5 0 2024-02-16T19:24:27.441 INFO:tasks.workunit.client.0.smithi040.stdout:7/956: creat d21/d29/d79/dbc/f149 x:0 0 0 2024-02-16T19:24:27.441 INFO:tasks.workunit.client.0.smithi040.stdout:7/957: chown d21/d7e/f105 2 1 2024-02-16T19:24:27.441 INFO:tasks.workunit.client.0.smithi040.stdout:7/958: creat d21/d29/d79/f14a x:0 0 0 2024-02-16T19:24:27.441 INFO:tasks.workunit.client.1.smithi163.stdout:7/464: rmdir d11/d24/d49 39 2024-02-16T19:24:27.442 INFO:tasks.workunit.client.0.smithi040.stdout:4/864: symlink dd/de/d10/d36/d9e/d32/df9/l10f 0 2024-02-16T19:24:27.446 INFO:tasks.workunit.client.1.smithi163.stdout:0/694: rename d11/d12/d67/d6a/d6d/lfa to d11/d21/d50/d58/d72/d9e/dbf/lff 0 2024-02-16T19:24:27.447 INFO:tasks.workunit.client.0.smithi040.stdout:7/959: creat d21/d4d/d77/d7a/f14b x:0 0 0 2024-02-16T19:24:27.448 INFO:tasks.workunit.client.0.smithi040.stdout:7/960: fsync d21/d4d/d77/d7a/dc6/f101 0 2024-02-16T19:24:27.448 INFO:tasks.workunit.client.1.smithi163.stdout:8/402: creat d0/d24/d78/d85/f86 x:0 0 0 2024-02-16T19:24:27.448 INFO:tasks.workunit.client.1.smithi163.stdout:8/403: write d0/d24/d3c/f84 [5165474,32952] 0 2024-02-16T19:24:27.449 INFO:tasks.workunit.client.1.smithi163.stdout:8/404: write d0/d7/d19/f26 [3040300,123444] 0 2024-02-16T19:24:27.450 INFO:tasks.workunit.client.0.smithi040.stdout:7/961: rename l1a to d21/d7e/df8/db7/l14c 0 2024-02-16T19:24:27.450 INFO:tasks.workunit.client.0.smithi040.stdout:7/962: chown d21/d4d/d7d/lfd 131831408 1 2024-02-16T19:24:27.451 INFO:tasks.workunit.client.1.smithi163.stdout:7/465: read d11/d24/d25/f39 [2044634,18724] 0 2024-02-16T19:24:27.452 INFO:tasks.workunit.client.0.smithi040.stdout:7/963: mkdir d21/d4d/d14d 0 2024-02-16T19:24:27.454 INFO:tasks.workunit.client.1.smithi163.stdout:3/477: rmdir d0/dc/d50/d66/d80 39 2024-02-16T19:24:27.457 INFO:tasks.workunit.client.0.smithi040.stdout:7/964: truncate d21/d4d/d77/d7a/dc6/fe6 2129404 0 2024-02-16T19:24:27.460 INFO:tasks.workunit.client.0.smithi040.stdout:7/965: creat d21/d7e/df8/db7/f14e x:0 0 0 2024-02-16T19:24:27.461 INFO:tasks.workunit.client.0.smithi040.stdout:7/966: creat d21/d7e/df8/db7/de7/d10f/f14f x:0 0 0 2024-02-16T19:24:27.461 INFO:tasks.workunit.client.0.smithi040.stdout:4/865: dwrite dd/de/f5d [0,4194304] 0 2024-02-16T19:24:27.462 INFO:tasks.workunit.client.1.smithi163.stdout:8/405: symlink d0/d7/d19/d1f/d30/d48/l87 0 2024-02-16T19:24:27.462 INFO:tasks.workunit.client.1.smithi163.stdout:8/406: dread - d0/d24/d3c/f7e zero size 2024-02-16T19:24:27.462 INFO:tasks.workunit.client.1.smithi163.stdout:8/407: dread - d0/d24/d3c/f7e zero size 2024-02-16T19:24:27.463 INFO:tasks.workunit.client.1.smithi163.stdout:7/466: link d11/d50/d5d/d8b/f3c d11/d24/d25/d4b/d66/f9f 0 2024-02-16T19:24:27.466 INFO:tasks.workunit.client.1.smithi163.stdout:7/467: dread d11/d24/d49/f51 [0,4194304] 0 2024-02-16T19:24:27.466 INFO:tasks.workunit.client.1.smithi163.stdout:3/478: rmdir d0/dc/d94/d9e/da8 0 2024-02-16T19:24:27.469 INFO:tasks.workunit.client.0.smithi040.stdout:7/967: rmdir d21/d4d/d77/d7a/d12b 0 2024-02-16T19:24:27.469 INFO:tasks.workunit.client.0.smithi040.stdout:7/968: readlink l5 0 2024-02-16T19:24:27.470 INFO:tasks.workunit.client.0.smithi040.stdout:7/969: truncate d21/d7e/f105 766448 0 2024-02-16T19:24:27.471 INFO:tasks.workunit.client.0.smithi040.stdout:4/866: write dd/f52 [1820384,4138] 0 2024-02-16T19:24:27.476 INFO:tasks.workunit.client.1.smithi163.stdout:9/626: rmdir d5/de/d30/d3d/d49 39 2024-02-16T19:24:27.477 INFO:tasks.workunit.client.0.smithi040.stdout:7/970: dread d21/d8b/fe0 [0,4194304] 0 2024-02-16T19:24:27.477 INFO:tasks.workunit.client.0.smithi040.stdout:7/971: dread - d21/d7e/fe4 zero size 2024-02-16T19:24:27.477 INFO:tasks.workunit.client.0.smithi040.stdout:7/972: chown d21/d29/d79/fdf 4242 1 2024-02-16T19:24:27.478 INFO:tasks.workunit.client.1.smithi163.stdout:0/695: symlink d11/d4f/l100 0 2024-02-16T19:24:27.478 INFO:tasks.workunit.client.0.smithi040.stdout:4/867: mknod dd/de/d45/c110 0 2024-02-16T19:24:27.480 INFO:tasks.workunit.client.1.smithi163.stdout:8/408: mknod d0/d24/d27/d45/c88 0 2024-02-16T19:24:27.481 INFO:tasks.workunit.client.0.smithi040.stdout:7/973: mknod d21/d7e/df8/db7/de7/d118/d12d/c150 0 2024-02-16T19:24:27.481 INFO:tasks.workunit.client.0.smithi040.stdout:7/974: write d21/d7e/fe1 [422152,116385] 0 2024-02-16T19:24:27.482 INFO:tasks.workunit.client.1.smithi163.stdout:2/552: write d2/f16 [1551589,52225] 0 2024-02-16T19:24:27.483 INFO:tasks.workunit.client.1.smithi163.stdout:7/468: symlink d11/d50/d5d/la0 0 2024-02-16T19:24:27.484 INFO:tasks.workunit.client.1.smithi163.stdout:3/479: mknod d0/dc/d94/d9e/dac/caf 0 2024-02-16T19:24:27.488 INFO:tasks.workunit.client.1.smithi163.stdout:0/696: dread d11/d21/d50/f52 [0,4194304] 0 2024-02-16T19:24:27.488 INFO:tasks.workunit.client.1.smithi163.stdout:0/697: fsync d11/d22/f34 0 2024-02-16T19:24:27.488 INFO:tasks.workunit.client.1.smithi163.stdout:0/698: write d11/d12/d67/f91 [2418055,65148] 0 2024-02-16T19:24:27.504 INFO:tasks.workunit.client.1.smithi163.stdout:0/699: mknod d11/c101 0 2024-02-16T19:24:27.505 INFO:tasks.workunit.client.1.smithi163.stdout:0/700: write d11/d21/d50/d58/d6b/fcd [4535153,8743] 0 2024-02-16T19:24:27.505 INFO:tasks.workunit.client.1.smithi163.stdout:0/701: chown d11/d12/f15 4673818 1 2024-02-16T19:24:27.506 INFO:tasks.workunit.client.1.smithi163.stdout:0/702: readlink d11/d22/d3e/l5d 0 2024-02-16T19:24:27.506 INFO:tasks.workunit.client.1.smithi163.stdout:7/469: rmdir d11/d26 39 2024-02-16T19:24:27.507 INFO:tasks.workunit.client.1.smithi163.stdout:7/470: symlink d11/d26/la1 0 2024-02-16T19:24:27.508 INFO:tasks.workunit.client.1.smithi163.stdout:7/471: mkdir d11/d50/da2 0 2024-02-16T19:24:27.510 INFO:tasks.workunit.client.1.smithi163.stdout:7/472: rename d11/d50/f86 to d11/d50/fa3 0 2024-02-16T19:24:27.520 INFO:tasks.workunit.client.0.smithi040.stdout:4/868: dwrite dd/d58/f8d [0,4194304] 0 2024-02-16T19:24:27.524 INFO:tasks.workunit.client.1.smithi163.stdout:7/473: dread d11/d50/d5d/d8b/f3c [0,4194304] 0 2024-02-16T19:24:27.525 INFO:tasks.workunit.client.1.smithi163.stdout:8/409: dwrite d0/d7/d18/f1e [0,4194304] 0 2024-02-16T19:24:27.526 INFO:tasks.workunit.client.0.smithi040.stdout:7/975: dwrite d21/d29/f50 [0,4194304] 0 2024-02-16T19:24:27.527 INFO:tasks.workunit.client.1.smithi163.stdout:8/410: getdents d0/d7/d18 0 2024-02-16T19:24:27.528 INFO:tasks.workunit.client.1.smithi163.stdout:7/474: rmdir d11/d24/d25 39 2024-02-16T19:24:27.528 INFO:tasks.workunit.client.1.smithi163.stdout:7/475: fdatasync d11/d26/f46 0 2024-02-16T19:24:27.529 INFO:tasks.workunit.client.1.smithi163.stdout:7/476: rmdir d11/d24 39 2024-02-16T19:24:27.530 INFO:tasks.workunit.client.1.smithi163.stdout:7/477: truncate d11/f1c 1147759 0 2024-02-16T19:24:27.530 INFO:tasks.workunit.client.1.smithi163.stdout:7/478: write d11/d24/d34/f8e [264132,111268] 0 2024-02-16T19:24:27.530 INFO:tasks.workunit.client.1.smithi163.stdout:7/479: chown d11/d24/d25/l87 147605774 1 2024-02-16T19:24:27.530 INFO:tasks.workunit.client.1.smithi163.stdout:7/480: creat d11/d50/d5d/d8b/d67/fa4 x:0 0 0 2024-02-16T19:24:27.530 INFO:tasks.workunit.client.1.smithi163.stdout:7/481: chown d11/d26/la1 5671 1 2024-02-16T19:24:27.531 INFO:tasks.workunit.client.1.smithi163.stdout:7/482: creat d11/d24/d49/fa5 x:0 0 0 2024-02-16T19:24:27.532 INFO:tasks.workunit.client.1.smithi163.stdout:7/483: write d11/d24/d25/f5a [1229511,38001] 0 2024-02-16T19:24:27.532 INFO:tasks.workunit.client.1.smithi163.stdout:7/484: mkdir d11/d50/da2/da6 0 2024-02-16T19:24:27.532 INFO:tasks.workunit.client.1.smithi163.stdout:7/485: stat d11/d30/f3e 0 2024-02-16T19:24:27.533 INFO:tasks.workunit.client.1.smithi163.stdout:7/486: creat d11/d24/d49/fa7 x:0 0 0 2024-02-16T19:24:27.533 INFO:tasks.workunit.client.1.smithi163.stdout:7/487: creat d11/fa8 x:0 0 0 2024-02-16T19:24:27.534 INFO:tasks.workunit.client.1.smithi163.stdout:7/488: symlink d11/d50/da2/da6/la9 0 2024-02-16T19:24:27.535 INFO:tasks.workunit.client.1.smithi163.stdout:7/489: unlink d11/l13 0 2024-02-16T19:24:27.535 INFO:tasks.workunit.client.1.smithi163.stdout:7/490: dread - d11/d24/d25/d4b/d8c/f96 zero size 2024-02-16T19:24:27.537 INFO:tasks.workunit.client.1.smithi163.stdout:7/491: symlink d11/d24/d34/d8d/laa 0 2024-02-16T19:24:27.537 INFO:tasks.workunit.client.1.smithi163.stdout:7/492: write d11/d24/d25/f7f [324481,99541] 0 2024-02-16T19:24:27.537 INFO:tasks.workunit.client.1.smithi163.stdout:7/493: write d11/d24/d49/fa5 [908936,126814] 0 2024-02-16T19:24:27.537 INFO:tasks.workunit.client.0.smithi040.stdout:7/976: dread d21/d4d/f6c [0,4194304] 0 2024-02-16T19:24:27.537 INFO:tasks.workunit.client.0.smithi040.stdout:7/977: mkdir d21/d4d/d77/d151 0 2024-02-16T19:24:27.538 INFO:tasks.workunit.client.1.smithi163.stdout:2/553: dwrite d2/d26/d54/d58/d98/fa4 [0,4194304] 0 2024-02-16T19:24:27.538 INFO:tasks.workunit.client.1.smithi163.stdout:2/554: creat d2/d2a/d39/d3a/fc2 x:0 0 0 2024-02-16T19:24:27.539 INFO:tasks.workunit.client.1.smithi163.stdout:7/494: creat d11/d24/d34/fab x:0 0 0 2024-02-16T19:24:27.541 INFO:tasks.workunit.client.1.smithi163.stdout:8/411: dread d0/d7/d19/d1f/d30/d48/f65 [0,4194304] 0 2024-02-16T19:24:27.541 INFO:tasks.workunit.client.0.smithi040.stdout:4/869: dwrite dd/d58/d68/fcb [0,4194304] 0 2024-02-16T19:24:27.542 INFO:tasks.workunit.client.1.smithi163.stdout:2/555: symlink d2/d26/d54/d58/da6/lc3 0 2024-02-16T19:24:27.542 INFO:tasks.workunit.client.1.smithi163.stdout:2/556: chown d2/d2a/d2d/d73/d84/f8e 76639635 1 2024-02-16T19:24:27.542 INFO:tasks.workunit.client.1.smithi163.stdout:2/557: stat d2/d2a/d2d/d73/d84/f8e 0 2024-02-16T19:24:27.543 INFO:tasks.workunit.client.1.smithi163.stdout:9/627: dwrite d5/de/d30/d3d/d49/d6d/fc3 [0,4194304] 0 2024-02-16T19:24:27.543 INFO:tasks.workunit.client.1.smithi163.stdout:9/628: creat d5/de/d72/da9/fe5 x:0 0 0 2024-02-16T19:24:27.543 INFO:tasks.workunit.client.1.smithi163.stdout:0/703: dwrite d11/d22/fd7 [0,4194304] 0 2024-02-16T19:24:27.545 INFO:tasks.workunit.client.1.smithi163.stdout:2/558: mknod d2/d2a/d39/d3a/cc4 0 2024-02-16T19:24:27.545 INFO:tasks.workunit.client.0.smithi040.stdout:4/870: creat dd/de/d10/d36/d9e/d32/f111 x:0 0 0 2024-02-16T19:24:27.549 INFO:tasks.workunit.client.0.smithi040.stdout:4/871: symlink dd/de/db0/l112 0 2024-02-16T19:24:27.551 INFO:tasks.workunit.client.1.smithi163.stdout:9/629: symlink d5/de/d30/le6 0 2024-02-16T19:24:27.553 INFO:tasks.workunit.client.0.smithi040.stdout:4/872: symlink dd/de/l113 0 2024-02-16T19:24:27.555 INFO:tasks.workunit.client.0.smithi040.stdout:4/873: dread dd/de/d10/fa2 [0,4194304] 0 2024-02-16T19:24:27.556 INFO:tasks.workunit.client.0.smithi040.stdout:7/978: dwrite d21/d4d/d77/d7a/dc6/f101 [0,4194304] 0 2024-02-16T19:24:27.556 INFO:tasks.workunit.client.0.smithi040.stdout:7/979: chown d21/d29/ff1 370639230 1 2024-02-16T19:24:27.557 INFO:tasks.workunit.client.0.smithi040.stdout:4/874: truncate dd/de/d10/d36/d9e/d77/f96 492766 0 2024-02-16T19:24:27.557 INFO:tasks.workunit.client.0.smithi040.stdout:4/875: creat dd/d58/d5e/dd1/f114 x:0 0 0 2024-02-16T19:24:27.559 INFO:tasks.workunit.client.1.smithi163.stdout:0/704: mkdir d11/d12/dc6/d102 0 2024-02-16T19:24:27.559 INFO:tasks.workunit.client.1.smithi163.stdout:0/705: dread - d11/d21/d50/d58/d72/d9e/ffd zero size 2024-02-16T19:24:27.559 INFO:tasks.workunit.client.1.smithi163.stdout:0/706: chown d11/d22/f2d 9135 1 2024-02-16T19:24:27.565 INFO:tasks.workunit.client.0.smithi040.stdout:7/980: link d21/d29/ceb d21/d4d/d77/d87/dc9/ddd/dbe/c152 0 2024-02-16T19:24:27.566 INFO:tasks.workunit.client.0.smithi040.stdout:7/981: chown d21/d4d/d77/d87/dc9/ddd/dbe/lcf 1797517157 1 2024-02-16T19:24:27.566 INFO:tasks.workunit.client.0.smithi040.stdout:7/982: fsync d21/d29/d79/dbc/f149 0 2024-02-16T19:24:27.566 INFO:tasks.workunit.client.0.smithi040.stdout:7/983: write d21/f24 [3423544,128852] 0 2024-02-16T19:24:27.566 INFO:tasks.workunit.client.0.smithi040.stdout:7/984: chown d21/d4d/d77/d87/d9b/df2 1930012 1 2024-02-16T19:24:27.567 INFO:tasks.workunit.client.0.smithi040.stdout:4/876: read dd/de/d10/d36/d9e/d32/f64 [3462465,29677] 0 2024-02-16T19:24:27.569 INFO:tasks.workunit.client.1.smithi163.stdout:2/559: mkdir d2/d26/d54/d58/d8c/dc5 0 2024-02-16T19:24:27.569 INFO:tasks.workunit.client.1.smithi163.stdout:2/560: creat d2/d2a/d2d/fc6 x:0 0 0 2024-02-16T19:24:27.569 INFO:tasks.workunit.client.1.smithi163.stdout:2/561: readlink d2/d2a/d39/d3a/l4e 0 2024-02-16T19:24:27.570 INFO:tasks.workunit.client.1.smithi163.stdout:9/630: write d5/f9 [1315345,83648] 0 2024-02-16T19:24:27.571 INFO:tasks.workunit.client.0.smithi040.stdout:7/985: truncate d21/df9/f140 744085 0 2024-02-16T19:24:27.573 INFO:tasks.workunit.client.0.smithi040.stdout:4/877: mknod dd/d58/d5e/dd1/dc6/dfa/c115 0 2024-02-16T19:24:27.573 INFO:tasks.workunit.client.0.smithi040.stdout:4/878: fdatasync dd/de/d10/d36/d9e/f46 0 2024-02-16T19:24:27.575 INFO:tasks.workunit.client.0.smithi040.stdout:7/986: rmdir d21/d4d/d77/d87/d9b 39 2024-02-16T19:24:27.575 INFO:tasks.workunit.client.0.smithi040.stdout:7/987: write d21/d29/d79/f7c [3675302,124495] 0 2024-02-16T19:24:27.578 INFO:tasks.workunit.client.1.smithi163.stdout:0/707: dwrite d11/d4f/fb0 [0,4194304] 0 2024-02-16T19:24:27.578 INFO:tasks.workunit.client.1.smithi163.stdout:0/708: write d11/d21/fea [860454,119980] 0 2024-02-16T19:24:27.578 INFO:tasks.workunit.client.1.smithi163.stdout:0/709: write d11/d22/d7f/f95 [917152,37992] 0 2024-02-16T19:24:27.579 INFO:tasks.workunit.client.0.smithi040.stdout:4/879: fsync dd/f52 0 2024-02-16T19:24:27.579 INFO:tasks.workunit.client.0.smithi040.stdout:4/880: dread - dd/de/fee zero size 2024-02-16T19:24:27.579 INFO:tasks.workunit.client.1.smithi163.stdout:2/562: rmdir d2/d26/d97/db7 0 2024-02-16T19:24:27.581 INFO:tasks.workunit.client.1.smithi163.stdout:9/631: chown d5/l12 306614377 1 2024-02-16T19:24:27.581 INFO:tasks.workunit.client.1.smithi163.stdout:2/563: dread d2/d2a/f5f [0,4194304] 0 2024-02-16T19:24:27.584 INFO:tasks.workunit.client.0.smithi040.stdout:4/881: symlink dd/d58/d5e/dd1/d9d/dd6/l116 0 2024-02-16T19:24:27.656 INFO:tasks.workunit.client.0.smithi040.stdout:4/882: creat dd/d58/d5e/f117 x:0 0 0 2024-02-16T19:24:27.657 INFO:tasks.workunit.client.0.smithi040.stdout:4/883: write dd/f6e [4710912,119570] 0 2024-02-16T19:24:27.661 INFO:tasks.workunit.client.0.smithi040.stdout:7/988: rmdir d21/d7e/df8/db7/de7/d10f 39 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/989: write d21/d8b/f11e [5181614,129288] 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/990: readlink d21/d29/l5a 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/884: unlink dd/d58/l85 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/991: creat d21/d7e/df8/f153 x:0 0 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/992: dread - d21/d29/dad/ff5 zero size 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/993: dread - d21/d4d/d77/d87/d9b/df2/f12a zero size 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/885: rename dd/f6e to dd/dd2/dff/f118 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/994: getdents d21/d4d/d77/d87/d9b/df2/d126 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/886: rename dd/d58/d5e/f7f to dd/de/d10/d36/d9e/f119 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/995: creat d21/d4d/d77/d87/d9b/f154 x:0 0 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/996: symlink d21/d8b/d9c/l155 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:7/997: stat d21/d4d/c6a 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/887: dread dd/de/d10/d36/d9e/f15 [0,4194304] 0 2024-02-16T19:24:27.662 INFO:tasks.workunit.client.0.smithi040.stdout:4/888: write dd/de/d71/ff2 [3070894,76554] 0 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/889: unlink dd/de/d10/d36/d9e/d77/c3a 0 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/890: fdatasync dd/de/d10/d36/d9e/d77/f59 0 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/891: dread - dd/d58/d5e/f10d zero size 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/892: mkdir dd/d58/d11a 0 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:7/998: dwrite d21/d7e/faa [0,4194304] 0 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:7/999: rename d21/d4d/d7b/f94 to d21/d4d/d77/d87/dc9/dcb/f156 0 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/893: symlink dd/de/daa/l11b 0 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/894: rename dd/de/d10/f8a to dd/d58/d5e/dd1/f11c 0 2024-02-16T19:24:27.663 INFO:tasks.workunit.client.0.smithi040.stdout:4/895: getdents dd/de/d10/d36/d9e/d32/df9 0 2024-02-16T19:24:27.664 INFO:tasks.workunit.client.1.smithi163.stdout:9/632: rmdir d5/de/d30/d3d/d93 39 2024-02-16T19:24:27.664 INFO:tasks.workunit.client.1.smithi163.stdout:2/564: unlink d2/d2a/f46 0 2024-02-16T19:24:27.664 INFO:tasks.workunit.client.1.smithi163.stdout:2/565: read d2/fb [7492660,121901] 0 2024-02-16T19:24:27.664 INFO:tasks.workunit.client.1.smithi163.stdout:0/710: dwrite d11/d22/d61/f9b [4194304,4194304] 0 2024-02-16T19:24:27.664 INFO:tasks.workunit.client.1.smithi163.stdout:0/711: dread d11/d21/fdb [0,4194304] 0 2024-02-16T19:24:27.664 INFO:tasks.workunit.client.1.smithi163.stdout:1/419: sync 2024-02-16T19:24:27.664 INFO:tasks.workunit.client.1.smithi163.stdout:0/712: link d11/d12/la9 d11/d22/l103 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:0/713: truncate d11/d21/d7c/fc5 1722120 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:2/566: mknod d2/d26/cc7 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:1/420: mknod d7/dc/d2d/c94 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:2/567: mkdir d2/d2a/d2d/d3c/dc8 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:1/421: mknod d7/dc/d2d/d32/d87/c95 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:2/568: mkdir d2/d2a/d2d/d73/d94/dc9 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:2/569: fsync d2/f1f 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:1/422: getdents d7/d20 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:1/423: write d7/d20/f38 [1689844,34601] 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:2/570: truncate d2/ff 3037619 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:1/424: mkdir d7/d20/d96 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:2/571: unlink d2/d2a/d39/f86 0 2024-02-16T19:24:27.665 INFO:tasks.workunit.client.1.smithi163.stdout:2/572: truncate d2/d2a/d39/d66/f9a 413352 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:0/714: dwrite d11/d12/d20/f5c [0,4194304] 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/425: mkdir d7/dc/d2d/d32/d87/d97 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:2/573: mkdir d2/d26/d54/dca 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:0/715: creat d11/d21/d50/d58/d72/f104 x:0 0 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:2/574: read d2/d2a/d2d/d73/d84/f89 [3911187,123476] 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/426: symlink d7/dc/l98 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/427: chown d7/dc/l4b 3530 1 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/428: chown d7/d5b/l5e 6358 1 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/429: write d7/d20/f35 [671151,82120] 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/430: creat d7/d20/f99 x:0 0 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/431: rename d7/c4a to d7/d20/d21/d4e/c9a 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/432: stat d7/d5b/f6a 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/433: chown d7/dc/f33 1 1 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:1/434: symlink d7/l9b 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:0/716: dwrite d11/d12/d67/d6a/d6d/fe3 [0,4194304] 0 2024-02-16T19:24:27.666 INFO:tasks.workunit.client.1.smithi163.stdout:0/717: readlink d11/d22/d3e/l40 0 2024-02-16T19:24:27.667 INFO:tasks.workunit.client.1.smithi163.stdout:2/575: dwrite d2/f1c [4194304,4194304] 0 2024-02-16T19:24:27.667 INFO:tasks.workunit.client.1.smithi163.stdout:2/576: write d2/d2a/d39/d3a/fc2 [360020,93759] 0 2024-02-16T19:24:27.667 INFO:tasks.workunit.client.1.smithi163.stdout:0/718: truncate d11/d21/d50/f79 2095826 0 2024-02-16T19:24:27.669 INFO:tasks.workunit.client.0.smithi040.stdout:4/896: dread dd/de/d10/fa9 [0,4194304] 0 2024-02-16T19:24:27.669 INFO:tasks.workunit.client.0.smithi040.stdout:4/897: chown dd/l78 397864 1 2024-02-16T19:24:27.669 INFO:tasks.workunit.client.0.smithi040.stdout:4/898: write dd/d58/d5e/dd1/fec [128130,23199] 0 2024-02-16T19:24:27.670 INFO:tasks.workunit.client.1.smithi163.stdout:2/577: unlink d2/fc 0 2024-02-16T19:24:27.671 INFO:tasks.workunit.client.0.smithi040.stdout:4/899: mknod dd/d58/d5e/dd1/c11d 0 2024-02-16T19:24:27.672 INFO:tasks.workunit.client.1.smithi163.stdout:0/719: creat d11/d12/d67/d6a/dae/f105 x:0 0 0 2024-02-16T19:24:27.672 INFO:tasks.workunit.client.1.smithi163.stdout:0/720: fdatasync d11/f62 0 2024-02-16T19:24:27.676 INFO:tasks.workunit.client.1.smithi163.stdout:2/578: mkdir d2/d2a/d39/d66/d6c/dcb 0 2024-02-16T19:24:27.676 INFO:tasks.workunit.client.1.smithi163.stdout:2/579: creat d2/d26/d54/d58/fcc x:0 0 0 2024-02-16T19:24:27.677 INFO:tasks.workunit.client.1.smithi163.stdout:0/721: creat d11/d21/d50/d58/d74/df8/f106 x:0 0 0 2024-02-16T19:24:27.678 INFO:tasks.workunit.client.1.smithi163.stdout:0/722: read d11/d22/f24 [545720,59503] 0 2024-02-16T19:24:27.680 INFO:tasks.workunit.client.0.smithi040.stdout:4/900: write dd/de/d10/d36/d9e/d32/f95 [2648356,57594] 0 2024-02-16T19:24:27.680 INFO:tasks.workunit.client.1.smithi163.stdout:2/580: creat d2/d26/d54/d58/da6/fcd x:0 0 0 2024-02-16T19:24:27.706 INFO:tasks.workunit.client.0.smithi040.stdout:4/901: dwrite dd/f1d [4194304,4194304] 0 2024-02-16T19:24:27.706 INFO:tasks.workunit.client.0.smithi040.stdout:4/902: creat dd/de/db0/f11e x:0 0 0 2024-02-16T19:24:27.706 INFO:tasks.workunit.client.0.smithi040.stdout:4/903: fdatasync dd/d58/fbb 0 2024-02-16T19:24:27.706 INFO:tasks.workunit.client.1.smithi163.stdout:0/723: dwrite d11/d22/f24 [0,4194304] 0 2024-02-16T19:24:27.709 INFO:tasks.workunit.client.0.smithi040.stdout:4/904: getdents dd/de/d10 0 2024-02-16T19:24:27.711 INFO:tasks.workunit.client.1.smithi163.stdout:0/724: mknod d11/d21/d50/d58/d74/df8/c107 0 2024-02-16T19:24:27.712 INFO:tasks.workunit.client.1.smithi163.stdout:0/725: mknod d11/d12/d67/d6a/c108 0 2024-02-16T19:24:27.712 INFO:tasks.workunit.client.1.smithi163.stdout:0/726: write d11/d21/f46 [1714831,38622] 0 2024-02-16T19:24:27.713 INFO:tasks.workunit.client.1.smithi163.stdout:0/727: truncate d11/d12/d67/d6a/dae/dcc/dd9/ff0 742666 0 2024-02-16T19:24:27.713 INFO:tasks.workunit.client.1.smithi163.stdout:0/728: fdatasync d11/d22/d3e/f6f 0 2024-02-16T19:24:27.713 INFO:tasks.workunit.client.1.smithi163.stdout:0/729: write d11/d21/d7c/fe6 [703908,62290] 0 2024-02-16T19:24:27.724 INFO:tasks.workunit.client.1.smithi163.stdout:3/480: sync 2024-02-16T19:24:27.724 INFO:tasks.workunit.client.1.smithi163.stdout:4/352: sync 2024-02-16T19:24:27.724 INFO:tasks.workunit.client.1.smithi163.stdout:6/406: sync 2024-02-16T19:24:27.724 INFO:tasks.workunit.client.1.smithi163.stdout:5/450: sync 2024-02-16T19:24:27.724 INFO:tasks.workunit.client.1.smithi163.stdout:6/407: creat d0/d6/df/d20/d40/d53/f80 x:0 0 0 2024-02-16T19:24:27.724 INFO:tasks.workunit.client.1.smithi163.stdout:8/412: sync 2024-02-16T19:24:27.724 INFO:tasks.workunit.client.1.smithi163.stdout:7/495: sync 2024-02-16T19:24:27.725 INFO:tasks.workunit.client.1.smithi163.stdout:7/496: readlink d11/d24/d34/l45 0 2024-02-16T19:24:27.726 INFO:tasks.workunit.client.1.smithi163.stdout:3/481: symlink d0/dc/d1c/d32/d62/d99/lb0 0 2024-02-16T19:24:27.727 INFO:tasks.workunit.client.1.smithi163.stdout:5/451: creat d0/d2/d5c/f8c x:0 0 0 2024-02-16T19:24:27.727 INFO:tasks.workunit.client.1.smithi163.stdout:5/452: write d0/d22/d28/d36/d3c/d4f/f7b [998204,58728] 0 2024-02-16T19:24:27.728 INFO:tasks.workunit.client.1.smithi163.stdout:5/453: dread d0/d2/f57 [4194304,4194304] 0 2024-02-16T19:24:27.728 INFO:tasks.workunit.client.1.smithi163.stdout:8/413: creat d0/d7/d19/d1f/d30/d6a/d7b/f89 x:0 0 0 2024-02-16T19:24:27.729 INFO:tasks.workunit.client.1.smithi163.stdout:7/497: mkdir d11/d24/d49/dac 0 2024-02-16T19:24:27.730 INFO:tasks.workunit.client.1.smithi163.stdout:4/353: rmdir d11/d2d/d4f 39 2024-02-16T19:24:27.730 INFO:tasks.workunit.client.1.smithi163.stdout:4/354: fdatasync d11/d18/d1f/d29/d52/f5a 0 2024-02-16T19:24:27.731 INFO:tasks.workunit.client.1.smithi163.stdout:3/482: creat d0/dc/d96/da7/fb1 x:0 0 0 2024-02-16T19:24:27.733 INFO:tasks.workunit.client.1.smithi163.stdout:5/454: mkdir d0/d22/d3b/d8d 0 2024-02-16T19:24:27.733 INFO:tasks.workunit.client.0.smithi040.stdout:4/905: dwrite dd/de/d10/d36/d9e/fea [0,4194304] 0 2024-02-16T19:24:27.734 INFO:tasks.workunit.client.1.smithi163.stdout:8/414: creat d0/d24/d3c/f8a x:0 0 0 2024-02-16T19:24:27.734 INFO:tasks.workunit.client.1.smithi163.stdout:7/498: symlink d11/d24/d34/d8d/lad 0 2024-02-16T19:24:27.736 INFO:tasks.workunit.client.1.smithi163.stdout:0/730: dwrite d11/d22/fa3 [0,4194304] 0 2024-02-16T19:24:27.737 INFO:tasks.workunit.client.1.smithi163.stdout:4/355: rename d11/c13 to d11/d2d/d4f/c69 0 2024-02-16T19:24:27.739 INFO:tasks.workunit.client.1.smithi163.stdout:8/415: link d0/d7/d19/d1f/d30/d48/f65 d0/d7/d19/d1f/d32/d70/f8b 0 2024-02-16T19:24:27.739 INFO:tasks.workunit.client.1.smithi163.stdout:8/416: readlink d0/d7/d18/l3e 0 2024-02-16T19:24:27.739 INFO:tasks.workunit.client.1.smithi163.stdout:8/417: fdatasync d0/d7/d18/f3f 0 2024-02-16T19:24:27.744 INFO:tasks.workunit.client.1.smithi163.stdout:6/408: dwrite d0/dd/f6d [0,4194304] 0 2024-02-16T19:24:27.744 INFO:tasks.workunit.client.1.smithi163.stdout:6/409: fdatasync d0/d6/f31 0 2024-02-16T19:24:27.746 INFO:tasks.workunit.client.0.smithi040.stdout:4/906: write dd/de/d71/ff2 [511296,100122] 0 2024-02-16T19:24:27.747 INFO:tasks.workunit.client.1.smithi163.stdout:5/455: rename d0/d1c/d60/f6f to d0/f8e 0 2024-02-16T19:24:27.749 INFO:tasks.workunit.client.1.smithi163.stdout:4/356: dread f9 [0,4194304] 0 2024-02-16T19:24:27.752 INFO:tasks.workunit.client.1.smithi163.stdout:8/418: unlink d0/d7/d55/c63 0 2024-02-16T19:24:27.758 INFO:tasks.workunit.client.1.smithi163.stdout:6/410: mkdir d0/dd/d7f/d81 0 2024-02-16T19:24:27.760 INFO:tasks.workunit.client.1.smithi163.stdout:8/419: mknod d0/d24/c8c 0 2024-02-16T19:24:27.760 INFO:tasks.workunit.client.1.smithi163.stdout:8/420: dread - d0/d24/d3c/f8a zero size 2024-02-16T19:24:27.761 INFO:tasks.workunit.client.1.smithi163.stdout:4/357: dread d11/d18/f24 [0,4194304] 0 2024-02-16T19:24:27.761 INFO:tasks.workunit.client.1.smithi163.stdout:6/411: symlink d0/dd/l82 0 2024-02-16T19:24:27.762 INFO:tasks.workunit.client.1.smithi163.stdout:6/412: readlink d0/l3f 0 2024-02-16T19:24:27.762 INFO:tasks.workunit.client.1.smithi163.stdout:8/421: mknod d0/d24/d2c/d74/c8d 0 2024-02-16T19:24:27.764 INFO:tasks.workunit.client.1.smithi163.stdout:6/413: creat d0/d6/df/f83 x:0 0 0 2024-02-16T19:24:27.764 INFO:tasks.workunit.client.1.smithi163.stdout:6/414: fsync d0/d6/df/d20/d40/d17/f4b 0 2024-02-16T19:24:27.766 INFO:tasks.workunit.client.1.smithi163.stdout:8/422: link d0/d7/d19/d1f/d30/d48/l87 d0/d7/d19/d1f/l8e 0 2024-02-16T19:24:27.766 INFO:tasks.workunit.client.1.smithi163.stdout:8/423: write d0/d7/d18/d2e/f5f [610095,50113] 0 2024-02-16T19:24:27.767 INFO:tasks.workunit.client.1.smithi163.stdout:4/358: getdents d11/d2d 0 2024-02-16T19:24:27.768 INFO:tasks.workunit.client.1.smithi163.stdout:6/415: unlink d0/d6/df/l25 0 2024-02-16T19:24:27.769 INFO:tasks.workunit.client.1.smithi163.stdout:3/483: dwrite d0/f1b [0,4194304] 0 2024-02-16T19:24:27.771 INFO:tasks.workunit.client.1.smithi163.stdout:8/424: truncate d0/d7/d18/d2e/f7f 1405246 0 2024-02-16T19:24:27.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/416: mkdir d0/d6/df/d20/d40/d53/d74/d84 0 2024-02-16T19:24:27.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/417: write d0/d6/df/d20/d40/d17/f5e [393140,20734] 0 2024-02-16T19:24:27.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/418: readlink d0/dd/d7f/d15/d1c/d65/d63/l76 0 2024-02-16T19:24:27.772 INFO:tasks.workunit.client.1.smithi163.stdout:3/484: mknod d0/d83/cb2 0 2024-02-16T19:24:27.774 INFO:tasks.workunit.client.1.smithi163.stdout:6/419: mkdir d0/dd/d7f/d81/d85 0 2024-02-16T19:24:27.774 INFO:tasks.workunit.client.1.smithi163.stdout:3/485: stat d0/dc/d1d/d49/l67 0 2024-02-16T19:24:27.775 INFO:tasks.workunit.client.1.smithi163.stdout:6/420: mkdir d0/dd/d7f/d15/d1c/d65/d63/d86 0 2024-02-16T19:24:27.777 INFO:tasks.workunit.client.1.smithi163.stdout:3/486: rename d0/l16 to d0/dc/d1d/d29/d95/lb3 0 2024-02-16T19:24:27.778 INFO:tasks.workunit.client.1.smithi163.stdout:7/499: dwrite d11/d24/d49/f81 [0,4194304] 0 2024-02-16T19:24:27.778 INFO:tasks.workunit.client.0.smithi040.stdout:4/907: dwrite dd/de/d45/fd4 [0,4194304] 0 2024-02-16T19:24:27.779 INFO:tasks.workunit.client.1.smithi163.stdout:6/421: mkdir d0/d6/df/d7b/d87 0 2024-02-16T19:24:27.779 INFO:tasks.workunit.client.1.smithi163.stdout:5/456: dwrite d0/f2c [4194304,4194304] 0 2024-02-16T19:24:27.779 INFO:tasks.workunit.client.1.smithi163.stdout:6/422: write d0/d6/df/d20/d40/d53/f72 [234187,106624] 0 2024-02-16T19:24:27.779 INFO:tasks.workunit.client.1.smithi163.stdout:6/423: truncate d0/d6/df/d20/d40/d53/f80 919793 0 2024-02-16T19:24:27.779 INFO:tasks.workunit.client.1.smithi163.stdout:6/424: fsync d0/dd/d7f/d2e/d33/f3d 0 2024-02-16T19:24:27.781 INFO:tasks.workunit.client.1.smithi163.stdout:3/487: rename d0/dc/d1d/d49/l65 to d0/dc/d50/d66/lb4 0 2024-02-16T19:24:27.781 INFO:tasks.workunit.client.1.smithi163.stdout:3/488: write d0/f8a [463391,7592] 0 2024-02-16T19:24:27.783 INFO:tasks.workunit.client.1.smithi163.stdout:0/731: dwrite d11/fb4 [0,4194304] 0 2024-02-16T19:24:27.784 INFO:tasks.workunit.client.1.smithi163.stdout:6/425: dread d0/dd/d7f/d15/d1c/f56 [0,4194304] 0 2024-02-16T19:24:27.784 INFO:tasks.workunit.client.1.smithi163.stdout:6/426: chown d0/dd/d7f/d15/d1c/d65/d63/f6b 72 1 2024-02-16T19:24:27.784 INFO:tasks.workunit.client.0.smithi040.stdout:4/908: dread dd/d58/fc5 [0,4194304] 0 2024-02-16T19:24:27.785 INFO:tasks.workunit.client.0.smithi040.stdout:4/909: write dd/f43 [1469501,53341] 0 2024-02-16T19:24:27.785 INFO:tasks.workunit.client.0.smithi040.stdout:4/910: creat dd/de/d10/d36/dc3/f11f x:0 0 0 2024-02-16T19:24:27.785 INFO:tasks.workunit.client.0.smithi040.stdout:4/911: dread - dd/d58/d5e/f10d zero size 2024-02-16T19:24:27.785 INFO:tasks.workunit.client.0.smithi040.stdout:4/912: chown dd/dd2/d106 1374 1 2024-02-16T19:24:27.792 INFO:tasks.workunit.client.1.smithi163.stdout:7/500: mknod d11/d24/d25/cae 0 2024-02-16T19:24:27.794 INFO:tasks.workunit.client.1.smithi163.stdout:3/489: dread d0/dc/d1c/f2c [0,4194304] 0 2024-02-16T19:24:27.798 INFO:tasks.workunit.client.1.smithi163.stdout:6/427: creat d0/d6/df/d20/d40/d17/f88 x:0 0 0 2024-02-16T19:24:27.799 INFO:tasks.workunit.client.1.smithi163.stdout:4/359: dwrite d11/d18/d1f/f20 [4194304,4194304] 0 2024-02-16T19:24:27.799 INFO:tasks.workunit.client.1.smithi163.stdout:7/501: unlink l0 0 2024-02-16T19:24:27.799 INFO:tasks.workunit.client.1.smithi163.stdout:7/502: chown d11/d26/f33 63360 1 2024-02-16T19:24:27.800 INFO:tasks.workunit.client.1.smithi163.stdout:3/490: symlink d0/dc/d1c/lb5 0 2024-02-16T19:24:27.806 INFO:tasks.workunit.client.1.smithi163.stdout:7/503: dread d11/d24/d25/f5a [0,4194304] 0 2024-02-16T19:24:27.809 INFO:tasks.workunit.client.1.smithi163.stdout:4/360: link d11/d2d/d4f/d60/f68 d11/d18/d1f/d29/d52/f6a 0 2024-02-16T19:24:27.810 INFO:tasks.workunit.client.1.smithi163.stdout:7/504: mknod d11/d24/d34/caf 0 2024-02-16T19:24:27.811 INFO:tasks.workunit.client.1.smithi163.stdout:4/361: mknod d11/d18/d66/c6b 0 2024-02-16T19:24:27.811 INFO:tasks.workunit.client.1.smithi163.stdout:4/362: dread - d11/d2d/d4f/f55 zero size 2024-02-16T19:24:27.812 INFO:tasks.workunit.client.1.smithi163.stdout:6/428: dread d0/d18/f48 [0,4194304] 0 2024-02-16T19:24:27.813 INFO:tasks.workunit.client.1.smithi163.stdout:6/429: unlink d0/d6/c37 0 2024-02-16T19:24:27.816 INFO:tasks.workunit.client.1.smithi163.stdout:6/430: creat d0/d6/df/d20/d40/f89 x:0 0 0 2024-02-16T19:24:27.819 INFO:tasks.workunit.client.1.smithi163.stdout:6/431: unlink d0/dd/f13 0 2024-02-16T19:24:27.819 INFO:tasks.workunit.client.1.smithi163.stdout:6/432: fdatasync d0/d6/df/d30/f5a 0 2024-02-16T19:24:27.819 INFO:tasks.workunit.client.1.smithi163.stdout:6/433: write d0/d6/df/d20/d40/d17/f88 [152353,29187] 0 2024-02-16T19:24:27.821 INFO:tasks.workunit.client.1.smithi163.stdout:8/425: dwrite d0/d7/d18/d53/f7a [0,4194304] 0 2024-02-16T19:24:27.821 INFO:tasks.workunit.client.1.smithi163.stdout:8/426: readlink d0/d24/d3c/d3d/l2b 0 2024-02-16T19:24:27.821 INFO:tasks.workunit.client.1.smithi163.stdout:8/427: creat d0/d7/d18/d2e/f8f x:0 0 0 2024-02-16T19:24:27.821 INFO:tasks.workunit.client.1.smithi163.stdout:5/457: dwrite d0/d22/d28/d36/d3c/d4f/f61 [0,4194304] 0 2024-02-16T19:24:27.822 INFO:tasks.workunit.client.1.smithi163.stdout:6/434: mknod d0/d18/c8a 0 2024-02-16T19:24:27.822 INFO:tasks.workunit.client.1.smithi163.stdout:0/732: dwrite d11/d4f/fb0 [0,4194304] 0 2024-02-16T19:24:27.822 INFO:tasks.workunit.client.1.smithi163.stdout:0/733: fdatasync d11/d21/d7c/fb7 0 2024-02-16T19:24:27.823 INFO:tasks.workunit.client.0.smithi040.stdout:4/913: dwrite dd/d58/d5e/dd1/d9d/fce [0,4194304] 0 2024-02-16T19:24:27.831 INFO:tasks.workunit.client.0.smithi040.stdout:4/914: link dd/d58/d5e/dd1/dc6/dfa/c115 dd/d58/c120 0 2024-02-16T19:24:27.832 INFO:tasks.workunit.client.1.smithi163.stdout:5/458: mknod d0/d22/d28/d36/c8f 0 2024-02-16T19:24:27.834 INFO:tasks.workunit.client.1.smithi163.stdout:6/435: mknod d0/c8b 0 2024-02-16T19:24:27.835 INFO:tasks.workunit.client.1.smithi163.stdout:0/734: write d11/d12/d67/d6a/d6d/f93 [6314874,44722] 0 2024-02-16T19:24:27.837 INFO:tasks.workunit.client.0.smithi040.stdout:4/915: symlink dd/de/d10/d36/d9e/d32/d103/l121 0 2024-02-16T19:24:27.840 INFO:tasks.workunit.client.1.smithi163.stdout:5/459: unlink d0/d22/f3e 0 2024-02-16T19:24:27.841 INFO:tasks.workunit.client.1.smithi163.stdout:5/460: symlink d0/d22/d3b/l90 0 2024-02-16T19:24:27.841 INFO:tasks.workunit.client.1.smithi163.stdout:5/461: fdatasync d0/d22/d28/f38 0 2024-02-16T19:24:27.845 INFO:tasks.workunit.client.1.smithi163.stdout:7/505: dwrite d11/f1c [0,4194304] 0 2024-02-16T19:24:27.845 INFO:tasks.workunit.client.1.smithi163.stdout:7/506: chown d11/d24/d25/d4b/d66/f70 95 1 2024-02-16T19:24:27.845 INFO:tasks.workunit.client.1.smithi163.stdout:6/436: dread d0/d6/df/d20/d40/d1e/f52 [0,4194304] 0 2024-02-16T19:24:27.847 INFO:tasks.workunit.client.1.smithi163.stdout:7/507: symlink d11/d24/d25/lb0 0 2024-02-16T19:24:27.847 INFO:tasks.workunit.client.1.smithi163.stdout:4/363: dwrite d11/d18/d1f/d29/d52/f6a [0,4194304] 0 2024-02-16T19:24:27.847 INFO:tasks.workunit.client.1.smithi163.stdout:4/364: fsync d11/d18/d1f/d23/d3a/f3f 0 2024-02-16T19:24:27.850 INFO:tasks.workunit.client.1.smithi163.stdout:0/735: dread d11/d21/f47 [0,4194304] 0 2024-02-16T19:24:27.852 INFO:tasks.workunit.client.1.smithi163.stdout:8/428: dwrite d0/d24/d2c/f4d [0,4194304] 0 2024-02-16T19:24:27.852 INFO:tasks.workunit.client.1.smithi163.stdout:6/437: creat d0/d18/d77/d70/f8c x:0 0 0 2024-02-16T19:24:27.852 INFO:tasks.workunit.client.1.smithi163.stdout:4/365: mkdir d11/d18/d1f/d29/d52/d57/d6c 0 2024-02-16T19:24:27.852 INFO:tasks.workunit.client.1.smithi163.stdout:4/366: chown d11/f46 32 1 2024-02-16T19:24:27.852 INFO:tasks.workunit.client.1.smithi163.stdout:9/633: sync 2024-02-16T19:24:27.855 INFO:tasks.workunit.client.1.smithi163.stdout:9/634: write d5/fc8 [1391594,19841] 0 2024-02-16T19:24:27.858 INFO:tasks.workunit.client.1.smithi163.stdout:4/367: rename d11/l5e to d11/d18/d66/l6d 0 2024-02-16T19:24:27.859 INFO:tasks.workunit.client.1.smithi163.stdout:4/368: write d11/d18/d1f/d29/f36 [1013655,54878] 0 2024-02-16T19:24:27.865 INFO:tasks.workunit.client.0.smithi040.stdout:4/916: dwrite dd/de/d10/d36/d9e/d32/f111 [0,4194304] 0 2024-02-16T19:24:27.865 INFO:tasks.workunit.client.0.smithi040.stdout:4/917: creat dd/dd2/f122 x:0 0 0 2024-02-16T19:24:27.865 INFO:tasks.workunit.client.1.smithi163.stdout:5/462: dwrite d0/d2/f16 [4194304,4194304] 0 2024-02-16T19:24:27.866 INFO:tasks.workunit.client.1.smithi163.stdout:5/463: stat d0/d2/d34/f6d 0 2024-02-16T19:24:27.866 INFO:tasks.workunit.client.1.smithi163.stdout:8/429: rmdir d0/d24 39 2024-02-16T19:24:27.866 INFO:tasks.workunit.client.1.smithi163.stdout:8/430: stat d0/d24/d78 0 2024-02-16T19:24:27.867 INFO:tasks.workunit.client.1.smithi163.stdout:9/635: creat d5/de/d30/d31/d52/fe7 x:0 0 0 2024-02-16T19:24:27.870 INFO:tasks.workunit.client.1.smithi163.stdout:4/369: creat d11/d18/d1f/d23/d3a/d4a/f6e x:0 0 0 2024-02-16T19:24:27.870 INFO:tasks.workunit.client.1.smithi163.stdout:4/370: truncate d11/f65 165580 0 2024-02-16T19:24:27.870 INFO:tasks.workunit.client.1.smithi163.stdout:4/371: read - d11/d18/d1f/d23/d3a/d4a/f6e zero size 2024-02-16T19:24:27.870 INFO:tasks.workunit.client.1.smithi163.stdout:4/372: read f10 [3708677,103036] 0 2024-02-16T19:24:27.870 INFO:tasks.workunit.client.1.smithi163.stdout:4/373: rename d11/d18/d1f/d23 to d11/d18/d1f/d23/d3a/d4a/d6f 22 2024-02-16T19:24:27.870 INFO:tasks.workunit.client.1.smithi163.stdout:4/374: chown d11/d18/f24 145 1 2024-02-16T19:24:27.873 INFO:tasks.workunit.client.1.smithi163.stdout:0/736: dread d11/d22/f24 [0,4194304] 0 2024-02-16T19:24:27.874 INFO:tasks.workunit.client.1.smithi163.stdout:8/431: creat d0/d24/d2c/f90 x:0 0 0 2024-02-16T19:24:27.876 INFO:tasks.workunit.client.0.smithi040.stdout:4/918: mkdir dd/d58/d5e/dd1/d9d/d123 0 2024-02-16T19:24:27.877 INFO:tasks.workunit.client.1.smithi163.stdout:2/581: sync 2024-02-16T19:24:27.877 INFO:tasks.workunit.client.1.smithi163.stdout:1/435: sync 2024-02-16T19:24:27.877 INFO:tasks.workunit.client.1.smithi163.stdout:1/436: fsync d7/dc/d2d/d32/d87/f92 0 2024-02-16T19:24:27.880 INFO:tasks.workunit.client.1.smithi163.stdout:7/508: dwrite f3 [4194304,4194304] 0 2024-02-16T19:24:27.880 INFO:tasks.workunit.client.1.smithi163.stdout:6/438: dwrite d0/dd/d7f/d15/d1c/d65/d63/f6b [0,4194304] 0 2024-02-16T19:24:27.880 INFO:tasks.workunit.client.1.smithi163.stdout:0/737: symlink d11/d12/dc6/l109 0 2024-02-16T19:24:27.882 INFO:tasks.workunit.client.1.smithi163.stdout:8/432: mknod d0/d24/d2c/c91 0 2024-02-16T19:24:27.883 INFO:tasks.workunit.client.1.smithi163.stdout:1/437: symlink d7/d20/d21/d4e/l9c 0 2024-02-16T19:24:27.886 INFO:tasks.workunit.client.1.smithi163.stdout:9/636: write d5/de/d72/fb0 [1608589,53273] 0 2024-02-16T19:24:27.892 INFO:tasks.workunit.client.1.smithi163.stdout:7/509: dread fa [0,4194304] 0 2024-02-16T19:24:27.892 INFO:tasks.workunit.client.1.smithi163.stdout:7/510: readlink d11/d24/d34/l45 0 2024-02-16T19:24:27.893 INFO:tasks.workunit.client.1.smithi163.stdout:6/439: creat d0/d6/df/d20/d69/f8d x:0 0 0 2024-02-16T19:24:27.893 INFO:tasks.workunit.client.1.smithi163.stdout:4/375: rmdir d11/d18/d1f/d23/d3a 39 2024-02-16T19:24:27.895 INFO:tasks.workunit.client.1.smithi163.stdout:8/433: truncate d0/d7/d19/d1f/d32/d70/f8b 1612289 0 2024-02-16T19:24:27.904 INFO:tasks.workunit.client.1.smithi163.stdout:4/376: truncate d11/f16 3803591 0 2024-02-16T19:24:27.904 INFO:tasks.workunit.client.1.smithi163.stdout:4/377: chown d11/d2d 7 1 2024-02-16T19:24:27.904 INFO:tasks.workunit.client.1.smithi163.stdout:4/378: readlink d11/d18/d1f/l42 0 2024-02-16T19:24:27.904 INFO:tasks.workunit.client.1.smithi163.stdout:0/738: dwrite d11/d12/d67/d6a/d6d/fcb [0,4194304] 0 2024-02-16T19:24:27.907 INFO:tasks.workunit.client.1.smithi163.stdout:0/739: symlink d11/d21/d50/d58/d72/l10a 0 2024-02-16T19:24:27.907 INFO:tasks.workunit.client.1.smithi163.stdout:0/740: dread - d11/d21/fad zero size 2024-02-16T19:24:27.910 INFO:tasks.workunit.client.1.smithi163.stdout:1/438: write d7/dc/d2d/d3f/f67 [3774455,93633] 0 2024-02-16T19:24:27.910 INFO:tasks.workunit.client.1.smithi163.stdout:8/434: dread d0/d7/d19/f26 [0,4194304] 0 2024-02-16T19:24:27.911 INFO:tasks.workunit.client.1.smithi163.stdout:1/439: mknod d7/dc/d2d/d32/d87/c9d 0 2024-02-16T19:24:27.922 INFO:tasks.workunit.client.1.smithi163.stdout:8/435: dread d0/dc/f16 [0,4194304] 0 2024-02-16T19:24:27.925 INFO:tasks.workunit.client.1.smithi163.stdout:1/440: read d7/d5b/d63/f7b [4172620,50162] 0 2024-02-16T19:24:27.926 INFO:tasks.workunit.client.1.smithi163.stdout:1/441: chown d7/c18 100197951 1 2024-02-16T19:24:27.926 INFO:tasks.workunit.client.1.smithi163.stdout:1/442: read d7/d5b/d63/f7b [3083768,17052] 0 2024-02-16T19:24:27.933 INFO:tasks.workunit.client.1.smithi163.stdout:2/582: dwrite d2/d26/d54/d58/f69 [0,4194304] 0 2024-02-16T19:24:27.933 INFO:tasks.workunit.client.1.smithi163.stdout:2/583: write d2/d26/d54/d58/fcc [398306,1962] 0 2024-02-16T19:24:27.938 INFO:tasks.workunit.client.1.smithi163.stdout:1/443: dread d7/f6b [0,4194304] 0 2024-02-16T19:24:27.938 INFO:tasks.workunit.client.1.smithi163.stdout:1/444: read d7/f49 [498267,1582] 0 2024-02-16T19:24:27.939 INFO:tasks.workunit.client.1.smithi163.stdout:1/445: truncate d7/dc/f33 5112066 0 2024-02-16T19:24:27.943 INFO:tasks.workunit.client.1.smithi163.stdout:2/584: dread d2/d2a/d39/d66/d6c/f6d [0,4194304] 0 2024-02-16T19:24:27.943 INFO:tasks.workunit.client.1.smithi163.stdout:2/585: write d2/d2a/d39/f75 [342014,105589] 0 2024-02-16T19:24:27.943 INFO:tasks.workunit.client.1.smithi163.stdout:2/586: fdatasync d2/d26/d54/d58/f87 0 2024-02-16T19:24:27.946 INFO:tasks.workunit.client.1.smithi163.stdout:0/741: dwrite d11/f62 [0,4194304] 0 2024-02-16T19:24:27.946 INFO:tasks.workunit.client.1.smithi163.stdout:0/742: creat d11/d12/f10b x:0 0 0 2024-02-16T19:24:27.946 INFO:tasks.workunit.client.1.smithi163.stdout:9/637: dwrite d5/de/d30/d3d/d49/d6d/f96 [0,4194304] 0 2024-02-16T19:24:27.946 INFO:tasks.workunit.client.1.smithi163.stdout:0/743: creat d11/d12/d20/f10c x:0 0 0 2024-02-16T19:24:27.947 INFO:tasks.workunit.client.0.smithi040.stdout:4/919: dwrite dd/de/d45/f74 [4194304,4194304] 0 2024-02-16T19:24:27.947 INFO:tasks.workunit.client.0.smithi040.stdout:4/920: creat dd/f124 x:0 0 0 2024-02-16T19:24:27.947 INFO:tasks.workunit.client.0.smithi040.stdout:4/921: truncate dd/d58/d5e/f117 982737 0 2024-02-16T19:24:27.947 INFO:tasks.workunit.client.1.smithi163.stdout:6/440: dwrite d0/dd/d7f/d2e/d33/f5f [0,4194304] 0 2024-02-16T19:24:27.947 INFO:tasks.workunit.client.1.smithi163.stdout:8/436: dwrite d0/d24/d27/d45/f75 [0,4194304] 0 2024-02-16T19:24:27.949 INFO:tasks.workunit.client.0.smithi040.stdout:4/922: symlink dd/de/d10/d36/d9e/d32/d103/l125 0 2024-02-16T19:24:27.954 INFO:tasks.workunit.client.1.smithi163.stdout:2/587: dread d2/d26/d54/d58/f69 [0,4194304] 0 2024-02-16T19:24:27.954 INFO:tasks.workunit.client.0.smithi040.stdout:4/923: symlink dd/de/d10/d36/d9e/l126 0 2024-02-16T19:24:27.954 INFO:tasks.workunit.client.0.smithi040.stdout:4/924: getdents dd/d58/d11a 0 2024-02-16T19:24:27.957 INFO:tasks.workunit.client.1.smithi163.stdout:9/638: creat d5/de/d30/d3d/d49/d6d/d97/fe8 x:0 0 0 2024-02-16T19:24:27.959 INFO:tasks.workunit.client.1.smithi163.stdout:0/744: rename d11/d21/d50/d58/c77 to d11/d12/d67/d6a/dae/dcc/dd9/c10d 0 2024-02-16T19:24:27.967 INFO:tasks.workunit.client.0.smithi040.stdout:4/925: write dd/d58/d68/fcb [1666815,11969] 0 2024-02-16T19:24:27.967 INFO:tasks.workunit.client.0.smithi040.stdout:4/926: write dd/d58/fbc [601486,58377] 0 2024-02-16T19:24:27.967 INFO:tasks.workunit.client.1.smithi163.stdout:1/446: dwrite d7/d20/f99 [0,4194304] 0 2024-02-16T19:24:27.967 INFO:tasks.workunit.client.1.smithi163.stdout:9/639: truncate d5/de/d30/d43/d8f/f90 1815418 0 2024-02-16T19:24:27.969 INFO:tasks.workunit.client.1.smithi163.stdout:8/437: rename d0/d24/d3c/f7e to d0/d7/d19/d1f/d30/f92 0 2024-02-16T19:24:27.970 INFO:tasks.workunit.client.1.smithi163.stdout:0/745: symlink d11/d12/dc6/l10e 0 2024-02-16T19:24:27.971 INFO:tasks.workunit.client.1.smithi163.stdout:0/746: readlink d11/d22/lb8 0 2024-02-16T19:24:27.971 INFO:tasks.workunit.client.1.smithi163.stdout:0/747: fsync d11/d4f/fb0 0 2024-02-16T19:24:27.971 INFO:tasks.workunit.client.1.smithi163.stdout:0/748: fdatasync d11/d21/fea 0 2024-02-16T19:24:27.971 INFO:tasks.workunit.client.1.smithi163.stdout:0/749: fsync d11/f98 0 2024-02-16T19:24:27.974 INFO:tasks.workunit.client.1.smithi163.stdout:6/441: dwrite d0/d6/df/d20/d40/f89 [0,4194304] 0 2024-02-16T19:24:27.975 INFO:tasks.workunit.client.1.smithi163.stdout:1/447: rename f6 to d7/d20/d21/d4e/f9e 0 2024-02-16T19:24:27.976 INFO:tasks.workunit.client.1.smithi163.stdout:8/438: creat d0/d24/d2c/d74/f93 x:0 0 0 2024-02-16T19:24:27.976 INFO:tasks.workunit.client.1.smithi163.stdout:0/750: mkdir d11/d22/d61/d10f 0 2024-02-16T19:24:27.977 INFO:tasks.workunit.client.1.smithi163.stdout:0/751: write d11/d12/d67/fa6 [1489898,75735] 0 2024-02-16T19:24:27.979 INFO:tasks.workunit.client.1.smithi163.stdout:6/442: unlink d0/dd/d7f/d2e/f61 0 2024-02-16T19:24:27.979 INFO:tasks.workunit.client.1.smithi163.stdout:6/443: stat d0/dd/d7f/d81 0 2024-02-16T19:24:27.979 INFO:tasks.workunit.client.1.smithi163.stdout:1/448: stat d7/dc/d2d/d32/f75 0 2024-02-16T19:24:27.980 INFO:tasks.workunit.client.1.smithi163.stdout:8/439: mknod d0/d24/c94 0 2024-02-16T19:24:27.981 INFO:tasks.workunit.client.1.smithi163.stdout:2/588: dwrite d2/d26/d97/fb1 [0,4194304] 0 2024-02-16T19:24:27.982 INFO:tasks.workunit.client.1.smithi163.stdout:0/752: getdents d11/d22/d7f 0 2024-02-16T19:24:27.984 INFO:tasks.workunit.client.1.smithi163.stdout:1/449: creat d7/d20/d2a/f9f x:0 0 0 2024-02-16T19:24:27.986 INFO:tasks.workunit.client.0.smithi040.stdout:4/927: dwrite dd/de/d10/d36/ffe [0,4194304] 0 2024-02-16T19:24:27.986 INFO:tasks.workunit.client.1.smithi163.stdout:8/440: mkdir d0/d95 0 2024-02-16T19:24:27.987 INFO:tasks.workunit.client.1.smithi163.stdout:9/640: write d5/f18 [1626068,56005] 0 2024-02-16T19:24:27.990 INFO:tasks.workunit.client.1.smithi163.stdout:2/589: mkdir d2/dce 0 2024-02-16T19:24:27.990 INFO:tasks.workunit.client.1.smithi163.stdout:6/444: dread d0/dd/d7f/f2d [0,4194304] 0 2024-02-16T19:24:27.992 INFO:tasks.workunit.client.1.smithi163.stdout:0/753: truncate d11/f98 419374 0 2024-02-16T19:24:27.993 INFO:tasks.workunit.client.0.smithi040.stdout:4/928: creat dd/d58/d5e/dd1/d9d/d123/f127 x:0 0 0 2024-02-16T19:24:27.993 INFO:tasks.workunit.client.0.smithi040.stdout:4/929: readlink dd/de/d10/l79 0 2024-02-16T19:24:27.993 INFO:tasks.workunit.client.0.smithi040.stdout:4/930: creat dd/de/d10/d36/dad/f128 x:0 0 0 2024-02-16T19:24:27.995 INFO:tasks.workunit.client.1.smithi163.stdout:8/441: symlink d0/d24/d2c/l96 0 2024-02-16T19:24:27.996 INFO:tasks.workunit.client.1.smithi163.stdout:9/641: getdents d5/de/d30/d43/d8f 0 2024-02-16T19:24:27.997 INFO:tasks.workunit.client.1.smithi163.stdout:1/450: dread d7/f1d [0,4194304] 0 2024-02-16T19:24:27.997 INFO:tasks.workunit.client.1.smithi163.stdout:2/590: read d2/d2a/d39/f75 [251947,80243] 0 2024-02-16T19:24:27.997 INFO:tasks.workunit.client.1.smithi163.stdout:2/591: readlink d2/d2a/d39/l85 0 2024-02-16T19:24:27.999 INFO:tasks.workunit.client.1.smithi163.stdout:9/642: truncate d5/de/d16/fb2 116287 0 2024-02-16T19:24:28.000 INFO:tasks.workunit.client.1.smithi163.stdout:0/754: dread d11/d22/d7f/f95 [0,4194304] 0 2024-02-16T19:24:28.000 INFO:tasks.workunit.client.1.smithi163.stdout:1/451: truncate d7/f49 1219280 0 2024-02-16T19:24:28.002 INFO:tasks.workunit.client.1.smithi163.stdout:2/592: link d2/f1a d2/d2a/d2d/d73/d94/dc9/fcf 0 2024-02-16T19:24:28.003 INFO:tasks.workunit.client.1.smithi163.stdout:2/593: read - d2/d26/d54/f9b zero size 2024-02-16T19:24:28.007 INFO:tasks.workunit.client.1.smithi163.stdout:1/452: mknod d7/d5b/ca0 0 2024-02-16T19:24:28.009 INFO:tasks.workunit.client.1.smithi163.stdout:2/594: symlink d2/d2a/d39/d66/dbc/ld0 0 2024-02-16T19:24:28.011 INFO:tasks.workunit.client.1.smithi163.stdout:6/445: dwrite d0/dd/f50 [0,4194304] 0 2024-02-16T19:24:28.016 INFO:tasks.workunit.client.1.smithi163.stdout:6/446: rename d0/d6/df/d20/d40/d53/d74/f78 to d0/dd/d7f/f8e 0 2024-02-16T19:24:28.016 INFO:tasks.workunit.client.1.smithi163.stdout:6/447: stat d0/d18/d77/d70/f8c 0 2024-02-16T19:24:28.016 INFO:tasks.workunit.client.1.smithi163.stdout:6/448: chown d0/l42 414635 1 2024-02-16T19:24:28.029 INFO:tasks.workunit.client.0.smithi040.stdout:4/931: dwrite dd/de/d45/fd4 [4194304,4194304] 0 2024-02-16T19:24:28.029 INFO:tasks.workunit.client.0.smithi040.stdout:4/932: chown dd/d58/c120 52 1 2024-02-16T19:24:28.031 INFO:tasks.workunit.client.0.smithi040.stdout:4/933: unlink dd/de/c10a 0 2024-02-16T19:24:28.038 INFO:tasks.workunit.client.1.smithi163.stdout:8/442: dwrite d0/d24/d2c/d74/f93 [0,4194304] 0 2024-02-16T19:24:28.038 INFO:tasks.workunit.client.1.smithi163.stdout:8/443: truncate d0/d24/d27/f64 1965399 0 2024-02-16T19:24:28.038 INFO:tasks.workunit.client.1.smithi163.stdout:0/755: dwrite d11/d12/d20/f5c [0,4194304] 0 2024-02-16T19:24:28.038 INFO:tasks.workunit.client.1.smithi163.stdout:0/756: dread - d11/d12/d67/d6a/f75 zero size 2024-02-16T19:24:28.041 INFO:tasks.workunit.client.1.smithi163.stdout:8/444: link d0/d24/f60 d0/f97 0 2024-02-16T19:24:28.042 INFO:tasks.workunit.client.1.smithi163.stdout:8/445: creat d0/d6f/f98 x:0 0 0 2024-02-16T19:24:28.049 INFO:tasks.workunit.client.1.smithi163.stdout:6/449: dwrite d0/dd/d7f/d15/d1c/f56 [0,4194304] 0 2024-02-16T19:24:28.049 INFO:tasks.workunit.client.1.smithi163.stdout:6/450: write d0/d6/df/d20/d40/d17/f4b [270669,37852] 0 2024-02-16T19:24:28.049 INFO:tasks.workunit.client.1.smithi163.stdout:0/757: write d11/d22/f2d [1373909,36151] 0 2024-02-16T19:24:28.049 INFO:tasks.workunit.client.1.smithi163.stdout:0/758: truncate d11/d21/d50/d58/d72/fa8 4708173 0 2024-02-16T19:24:28.052 INFO:tasks.workunit.client.1.smithi163.stdout:0/759: truncate d11/d21/f47 3799086 0 2024-02-16T19:24:28.052 INFO:tasks.workunit.client.0.smithi040.stdout:4/934: dwrite dd/de/d10/d36/d9e/d77/f96 [0,4194304] 0 2024-02-16T19:24:28.053 INFO:tasks.workunit.client.1.smithi163.stdout:9/643: dwrite d5/de/d30/d3d/f66 [0,4194304] 0 2024-02-16T19:24:28.055 INFO:tasks.workunit.client.1.smithi163.stdout:9/644: readlink d5/l27 0 2024-02-16T19:24:28.056 INFO:tasks.workunit.client.0.smithi040.stdout:4/935: rmdir dd/d58/d5e/dd1 39 2024-02-16T19:24:28.056 INFO:tasks.workunit.client.0.smithi040.stdout:4/936: rmdir dd/d58/d5e/dd1/d9d/dd6 39 2024-02-16T19:24:28.059 INFO:tasks.workunit.client.1.smithi163.stdout:6/451: read d0/dd/f50 [3010690,53724] 0 2024-02-16T19:24:28.060 INFO:tasks.workunit.client.1.smithi163.stdout:6/452: write d0/d6/df/f83 [238158,55863] 0 2024-02-16T19:24:28.061 INFO:tasks.workunit.client.1.smithi163.stdout:6/453: dread d0/dd/d7f/f4c [0,4194304] 0 2024-02-16T19:24:28.064 INFO:tasks.workunit.client.1.smithi163.stdout:8/446: dwrite d0/d7/d18/d2e/f5f [0,4194304] 0 2024-02-16T19:24:28.064 INFO:tasks.workunit.client.1.smithi163.stdout:8/447: readlink d0/d6f/l67 0 2024-02-16T19:24:28.070 INFO:tasks.workunit.client.1.smithi163.stdout:8/448: symlink d0/d7/d19/d1f/d30/l99 0 2024-02-16T19:24:28.070 INFO:tasks.workunit.client.1.smithi163.stdout:8/449: chown d0/d24/d27/d45/f75 1421 1 2024-02-16T19:24:28.098 INFO:tasks.workunit.client.1.smithi163.stdout:0/760: dwrite d11/d21/fdb [0,4194304] 0 2024-02-16T19:24:28.100 INFO:tasks.workunit.client.1.smithi163.stdout:0/761: mknod d11/d21/d50/d58/d72/d9e/dd8/c110 0 2024-02-16T19:24:28.101 INFO:tasks.workunit.client.1.smithi163.stdout:0/762: link d11/d21/l29 d11/d21/d50/d58/d6b/l111 0 2024-02-16T19:24:28.102 INFO:tasks.workunit.client.1.smithi163.stdout:6/454: dwrite d0/dd/d7f/d2e/d33/f3d [0,4194304] 0 2024-02-16T19:24:28.102 INFO:tasks.workunit.client.1.smithi163.stdout:6/455: stat d0/d6/df/d20/d40/d53/d74/d84 0 2024-02-16T19:24:28.102 INFO:tasks.workunit.client.1.smithi163.stdout:9/645: dwrite d5/de/d30/f36 [0,4194304] 0 2024-02-16T19:24:28.103 INFO:tasks.workunit.client.0.smithi040.stdout:4/937: dwrite dd/de/d10/d36/d9e/d77/f7b [0,4194304] 0 2024-02-16T19:24:28.113 INFO:tasks.workunit.client.1.smithi163.stdout:0/763: dread d11/d21/d50/d58/d72/fa8 [0,4194304] 0 2024-02-16T19:24:28.133 INFO:tasks.workunit.client.1.smithi163.stdout:6/456: truncate d0/d6/df/f32 1454442 0 2024-02-16T19:24:28.135 INFO:tasks.workunit.client.0.smithi040.stdout:4/938: mknod dd/dd2/d106/c129 0 2024-02-16T19:24:28.135 INFO:tasks.workunit.client.0.smithi040.stdout:4/939: chown dd/de/d10/d36/d9e/ca4 60168 1 2024-02-16T19:24:28.142 INFO:tasks.workunit.client.1.smithi163.stdout:6/457: dread d0/dd/d7f/d2e/d33/f5f [0,4194304] 0 2024-02-16T19:24:28.143 INFO:tasks.workunit.client.1.smithi163.stdout:6/458: fsync d0/dd/d7f/d15/d1c/f7d 0 2024-02-16T19:24:28.148 INFO:tasks.workunit.client.0.smithi040.stdout:4/940: dread dd/de/d10/d36/d9e/f46 [4194304,4194304] 0 2024-02-16T19:24:28.164 INFO:tasks.workunit.client.1.smithi163.stdout:6/459: dwrite d0/dd/d7f/f8e [0,4194304] 0 2024-02-16T19:24:28.168 INFO:tasks.workunit.client.1.smithi163.stdout:6/460: write d0/d6/df/d20/d40/d1e/f52 [1792058,64896] 0 2024-02-16T19:24:28.195 INFO:tasks.workunit.client.1.smithi163.stdout:0/764: rename d11/d12/d20/f10c to d11/d22/d7f/f112 0 2024-02-16T19:24:28.209 INFO:tasks.workunit.client.1.smithi163.stdout:3/491: sync 2024-02-16T19:24:28.210 INFO:tasks.workunit.client.1.smithi163.stdout:5/464: sync 2024-02-16T19:24:28.215 INFO:tasks.workunit.client.1.smithi163.stdout:0/765: dwrite d11/d22/fa7 [0,4194304] 0 2024-02-16T19:24:28.216 INFO:tasks.workunit.client.1.smithi163.stdout:0/766: chown d11/d22/d7f/ce8 558 1 2024-02-16T19:24:28.223 INFO:tasks.workunit.client.1.smithi163.stdout:3/492: dwrite d0/dc/d1d/d29/d38/f40 [0,4194304] 0 2024-02-16T19:24:28.252 INFO:tasks.workunit.client.1.smithi163.stdout:1/453: sync 2024-02-16T19:24:28.252 INFO:tasks.workunit.client.1.smithi163.stdout:4/379: sync 2024-02-16T19:24:28.252 INFO:tasks.workunit.client.1.smithi163.stdout:7/511: sync 2024-02-16T19:24:28.252 INFO:tasks.workunit.client.1.smithi163.stdout:2/595: sync 2024-02-16T19:24:28.252 INFO:tasks.workunit.client.1.smithi163.stdout:9/646: rmdir d5/de/d30/d3d 39 2024-02-16T19:24:28.252 INFO:tasks.workunit.client.1.smithi163.stdout:8/450: sync 2024-02-16T19:24:28.261 INFO:tasks.workunit.client.1.smithi163.stdout:5/465: getdents d0/d22/d3b 0 2024-02-16T19:24:28.261 INFO:tasks.workunit.client.1.smithi163.stdout:5/466: creat d0/d2/d55/f91 x:0 0 0 2024-02-16T19:24:28.261 INFO:tasks.workunit.client.1.smithi163.stdout:5/467: chown d0/d22/d3b/f42 1249 1 2024-02-16T19:24:28.266 INFO:tasks.workunit.client.1.smithi163.stdout:8/451: dwrite d0/d7/d19/d1f/d32/f39 [0,4194304] 0 2024-02-16T19:24:28.268 INFO:tasks.workunit.client.1.smithi163.stdout:8/452: dread d0/f97 [4194304,4194304] 0 2024-02-16T19:24:28.271 INFO:tasks.workunit.client.1.smithi163.stdout:4/380: rmdir d11/d2d/d4f 39 2024-02-16T19:24:28.272 INFO:tasks.workunit.client.1.smithi163.stdout:1/454: creat d7/d20/d96/fa1 x:0 0 0 2024-02-16T19:24:28.274 INFO:tasks.workunit.client.1.smithi163.stdout:7/512: creat d11/d50/d5d/d8b/fb1 x:0 0 0 2024-02-16T19:24:28.276 INFO:tasks.workunit.client.1.smithi163.stdout:2/596: symlink d2/d2a/d2d/d73/ld1 0 2024-02-16T19:24:28.276 INFO:tasks.workunit.client.1.smithi163.stdout:2/597: creat d2/d26/d54/d58/d7d/fd2 x:0 0 0 2024-02-16T19:24:28.281 INFO:tasks.workunit.client.1.smithi163.stdout:3/493: dread d0/dc/f3c [0,4194304] 0 2024-02-16T19:24:28.281 INFO:tasks.workunit.client.1.smithi163.stdout:3/494: truncate d0/dc/d1d/d49/f7d 478235 0 2024-02-16T19:24:28.281 INFO:tasks.workunit.client.1.smithi163.stdout:0/767: mkdir d11/d12/d113 0 2024-02-16T19:24:28.282 INFO:tasks.workunit.client.1.smithi163.stdout:5/468: rmdir d0 39 2024-02-16T19:24:28.282 INFO:tasks.workunit.client.1.smithi163.stdout:3/495: write d0/f92 [63135,130719] 0 2024-02-16T19:24:28.288 INFO:tasks.workunit.client.1.smithi163.stdout:8/453: rename d0/d24/d3c/d3d/d41/c73 to d0/d24/d3c/d3d/c9a 0 2024-02-16T19:24:28.290 INFO:tasks.workunit.client.1.smithi163.stdout:4/381: mkdir d11/d2d/d4f/d5f/d70 0 2024-02-16T19:24:28.291 INFO:tasks.workunit.client.1.smithi163.stdout:6/461: mkdir d0/d6/df/d8f 0 2024-02-16T19:24:28.291 INFO:tasks.workunit.client.1.smithi163.stdout:6/462: readlink d0/d6/l1b 0 2024-02-16T19:24:28.291 INFO:tasks.workunit.client.1.smithi163.stdout:6/463: write d0/f2 [2457977,96665] 0 2024-02-16T19:24:28.292 INFO:tasks.workunit.client.1.smithi163.stdout:2/598: write d2/d2a/d2d/f5e [3343192,102482] 0 2024-02-16T19:24:28.293 INFO:tasks.workunit.client.1.smithi163.stdout:1/455: dread d7/dc/f30 [0,4194304] 0 2024-02-16T19:24:28.295 INFO:tasks.workunit.client.1.smithi163.stdout:5/469: symlink d0/d22/d28/l92 0 2024-02-16T19:24:28.295 INFO:tasks.workunit.client.1.smithi163.stdout:5/470: chown d0/d2/d34/d44/d64/l8a 1687633 1 2024-02-16T19:24:28.297 INFO:tasks.workunit.client.1.smithi163.stdout:0/768: rename d11/d22/l44 to d11/d22/d3e/l114 0 2024-02-16T19:24:28.297 INFO:tasks.workunit.client.1.smithi163.stdout:3/496: dread d0/dc/f3c [0,4194304] 0 2024-02-16T19:24:28.298 INFO:tasks.workunit.client.1.smithi163.stdout:8/454: symlink d0/d24/d2c/d74/l9b 0 2024-02-16T19:24:28.299 INFO:tasks.workunit.client.1.smithi163.stdout:0/769: dread d11/d22/d3e/fc8 [0,4194304] 0 2024-02-16T19:24:28.301 INFO:tasks.workunit.client.1.smithi163.stdout:6/464: creat d0/d6/df/d20/f90 x:0 0 0 2024-02-16T19:24:28.302 INFO:tasks.workunit.client.1.smithi163.stdout:5/471: rmdir d0/d1c/d5a 39 2024-02-16T19:24:28.302 INFO:tasks.workunit.client.1.smithi163.stdout:5/472: truncate d0/f4d 1420165 0 2024-02-16T19:24:28.304 INFO:tasks.workunit.client.1.smithi163.stdout:4/382: rename d11/d2d/d4f/d60/f68 to d11/d2d/f71 0 2024-02-16T19:24:28.305 INFO:tasks.workunit.client.1.smithi163.stdout:8/455: mknod d0/d7/d19/d1f/c9c 0 2024-02-16T19:24:28.306 INFO:tasks.workunit.client.1.smithi163.stdout:6/465: creat d0/d6/df/d8f/f91 x:0 0 0 2024-02-16T19:24:28.308 INFO:tasks.workunit.client.1.smithi163.stdout:2/599: rename d2/d26/cc7 to d2/d2a/d39/d66/dbc/cd3 0 2024-02-16T19:24:28.309 INFO:tasks.workunit.client.1.smithi163.stdout:8/456: unlink d0/d7/d19/l49 0 2024-02-16T19:24:28.309 INFO:tasks.workunit.client.1.smithi163.stdout:8/457: fsync d0/d7/d19/d1f/d30/f92 0 2024-02-16T19:24:28.310 INFO:tasks.workunit.client.1.smithi163.stdout:8/458: creat d0/d7/d18/d53/f9d x:0 0 0 2024-02-16T19:24:28.311 INFO:tasks.workunit.client.1.smithi163.stdout:3/497: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:28.311 INFO:tasks.workunit.client.1.smithi163.stdout:3/498: creat d0/dc/d94/d9e/fb6 x:0 0 0 2024-02-16T19:24:28.313 INFO:tasks.workunit.client.1.smithi163.stdout:9/647: dwrite d5/f2f [0,4194304] 0 2024-02-16T19:24:28.313 INFO:tasks.workunit.client.1.smithi163.stdout:9/648: getdents d5/de/d30/d3d/d49/d6e/da1 0 2024-02-16T19:24:28.313 INFO:tasks.workunit.client.1.smithi163.stdout:0/770: write d11/d12/d20/f39 [4122895,59215] 0 2024-02-16T19:24:28.314 INFO:tasks.workunit.client.1.smithi163.stdout:0/771: read - d11/d22/d3e/fdf zero size 2024-02-16T19:24:28.314 INFO:tasks.workunit.client.1.smithi163.stdout:0/772: chown d11/d22/d61/f78 3887561 1 2024-02-16T19:24:28.320 INFO:tasks.workunit.client.1.smithi163.stdout:1/456: rename d7/d20/l47 to d7/dc/d2d/d32/la2 0 2024-02-16T19:24:28.320 INFO:tasks.workunit.client.1.smithi163.stdout:8/459: write d0/d7/d19/d1f/d30/d48/f65 [219599,87291] 0 2024-02-16T19:24:28.321 INFO:tasks.workunit.client.1.smithi163.stdout:3/499: creat d0/dc/d50/d66/d80/fb7 x:0 0 0 2024-02-16T19:24:28.321 INFO:tasks.workunit.client.1.smithi163.stdout:3/500: fsync d0/dc/d96/da7/fb1 0 2024-02-16T19:24:28.321 INFO:tasks.workunit.client.1.smithi163.stdout:3/501: write d0/dc/d1c/d32/f3f [4599688,113540] 0 2024-02-16T19:24:28.321 INFO:tasks.workunit.client.1.smithi163.stdout:3/502: chown d0/dc/d1c/d32/d62 7 1 2024-02-16T19:24:28.321 INFO:tasks.workunit.client.1.smithi163.stdout:9/649: symlink d5/de/d72/le9 0 2024-02-16T19:24:28.322 INFO:tasks.workunit.client.1.smithi163.stdout:1/457: symlink d7/dc/d2d/la3 0 2024-02-16T19:24:28.323 INFO:tasks.workunit.client.1.smithi163.stdout:2/600: dread d2/f37 [0,4194304] 0 2024-02-16T19:24:28.323 INFO:tasks.workunit.client.1.smithi163.stdout:2/601: fdatasync d2/d26/d54/d58/f69 0 2024-02-16T19:24:28.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/650: rmdir d5/de/d16/d46/d9b/dc7 0 2024-02-16T19:24:28.327 INFO:tasks.workunit.client.1.smithi163.stdout:2/602: creat d2/d2a/d2d/d3c/dc8/fd4 x:0 0 0 2024-02-16T19:24:28.329 INFO:tasks.workunit.client.1.smithi163.stdout:3/503: rename d0/dc/d1c/d32/d62 to d0/dc/d1c/d32/db8 0 2024-02-16T19:24:28.331 INFO:tasks.workunit.client.1.smithi163.stdout:3/504: dread d0/f39 [0,4194304] 0 2024-02-16T19:24:28.331 INFO:tasks.workunit.client.1.smithi163.stdout:3/505: chown d0/dc/d1c/d32/f36 504 1 2024-02-16T19:24:28.333 INFO:tasks.workunit.client.1.smithi163.stdout:3/506: mkdir d0/d83/da5/db9 0 2024-02-16T19:24:28.333 INFO:tasks.workunit.client.1.smithi163.stdout:3/507: readlink d0/dc/d1c/d32/l61 0 2024-02-16T19:24:28.334 INFO:tasks.workunit.client.1.smithi163.stdout:3/508: rmdir d0 39 2024-02-16T19:24:28.339 INFO:tasks.workunit.client.1.smithi163.stdout:9/651: write d5/f2f [881886,21464] 0 2024-02-16T19:24:28.346 INFO:tasks.workunit.client.1.smithi163.stdout:1/458: write d7/dc/d2d/d32/f64 [3896835,60056] 0 2024-02-16T19:24:28.346 INFO:tasks.workunit.client.1.smithi163.stdout:1/459: creat d7/d5b/fa4 x:0 0 0 2024-02-16T19:24:28.346 INFO:tasks.workunit.client.1.smithi163.stdout:1/460: chown d7/d20/d2a/c7a 0 1 2024-02-16T19:24:28.347 INFO:tasks.workunit.client.1.smithi163.stdout:1/461: write d7/d20/d2a/f40 [378619,37701] 0 2024-02-16T19:24:28.347 INFO:tasks.workunit.client.1.smithi163.stdout:1/462: write d7/d20/f38 [2234696,86897] 0 2024-02-16T19:24:28.351 INFO:tasks.workunit.client.1.smithi163.stdout:9/652: write d5/de/d16/d46/f8c [4058719,96963] 0 2024-02-16T19:24:28.354 INFO:tasks.workunit.client.1.smithi163.stdout:1/463: unlink d7/dc/d2d/d32/d87/c95 0 2024-02-16T19:24:28.356 INFO:tasks.workunit.client.1.smithi163.stdout:9/653: mkdir d5/de/d30/d3d/d49/dea 0 2024-02-16T19:24:28.358 INFO:tasks.workunit.client.1.smithi163.stdout:9/654: dread - d5/fbb zero size 2024-02-16T19:24:28.361 INFO:tasks.workunit.client.1.smithi163.stdout:9/655: rename d5/de/d30/l75 to d5/de/d84/leb 0 2024-02-16T19:24:28.362 INFO:tasks.workunit.client.1.smithi163.stdout:9/656: chown d5/de/d72/c79 476 1 2024-02-16T19:24:28.362 INFO:tasks.workunit.client.1.smithi163.stdout:9/657: chown d5/de/d16/d47/fb4 53462112 1 2024-02-16T19:24:28.362 INFO:tasks.workunit.client.1.smithi163.stdout:5/473: dwrite d0/d2/d34/f4b [0,4194304] 0 2024-02-16T19:24:28.364 INFO:tasks.workunit.client.1.smithi163.stdout:0/773: dwrite d11/d12/f6c [0,4194304] 0 2024-02-16T19:24:28.364 INFO:tasks.workunit.client.1.smithi163.stdout:9/658: symlink d5/de/d16/d46/d9b/lec 0 2024-02-16T19:24:28.364 INFO:tasks.workunit.client.1.smithi163.stdout:4/383: dwrite d11/d18/d1f/d29/f3e [0,4194304] 0 2024-02-16T19:24:28.364 INFO:tasks.workunit.client.1.smithi163.stdout:8/460: dwrite d0/dc/f22 [0,4194304] 0 2024-02-16T19:24:28.364 INFO:tasks.workunit.client.1.smithi163.stdout:8/461: chown d0/d7/d18/d2e/f5d 1119332061 1 2024-02-16T19:24:28.365 INFO:tasks.workunit.client.1.smithi163.stdout:2/603: dwrite d2/f9 [4194304,4194304] 0 2024-02-16T19:24:28.375 INFO:tasks.workunit.client.1.smithi163.stdout:8/462: symlink d0/d7/d19/d1f/d30/l9e 0 2024-02-16T19:24:28.375 INFO:tasks.workunit.client.1.smithi163.stdout:8/463: fdatasync d0/d7/d19/d1f/d32/f59 0 2024-02-16T19:24:28.376 INFO:tasks.workunit.client.1.smithi163.stdout:0/774: dread d11/d12/d20/fe4 [0,4194304] 0 2024-02-16T19:24:28.376 INFO:tasks.workunit.client.1.smithi163.stdout:0/775: read - d11/d21/d50/d58/d74/df8/f106 zero size 2024-02-16T19:24:28.376 INFO:tasks.workunit.client.1.smithi163.stdout:0/776: fdatasync d11/d22/f34 0 2024-02-16T19:24:28.378 INFO:tasks.workunit.client.1.smithi163.stdout:1/464: dwrite d7/d20/d2a/f9f [0,4194304] 0 2024-02-16T19:24:28.378 INFO:tasks.workunit.client.1.smithi163.stdout:4/384: dread d11/d2d/f71 [0,4194304] 0 2024-02-16T19:24:28.384 INFO:tasks.workunit.client.1.smithi163.stdout:8/464: unlink d0/d7/c82 0 2024-02-16T19:24:28.384 INFO:tasks.workunit.client.1.smithi163.stdout:8/465: readlink d0/d24/d3c/d3d/l2b 0 2024-02-16T19:24:28.392 INFO:tasks.workunit.client.1.smithi163.stdout:1/465: mknod d7/d20/ca5 0 2024-02-16T19:24:28.392 INFO:tasks.workunit.client.1.smithi163.stdout:1/466: fsync d7/f4f 0 2024-02-16T19:24:28.395 INFO:tasks.workunit.client.1.smithi163.stdout:5/474: rename d0/d2/d34/d44/d63 to d0/d2/d34/d44/d53/d93 0 2024-02-16T19:24:28.396 INFO:tasks.workunit.client.1.smithi163.stdout:8/466: symlink d0/d24/d27/l9f 0 2024-02-16T19:24:28.398 INFO:tasks.workunit.client.1.smithi163.stdout:1/467: mkdir d7/dc/d2d/d32/d87/da6 0 2024-02-16T19:24:28.400 INFO:tasks.workunit.client.1.smithi163.stdout:8/467: creat d0/d7/d18/fa0 x:0 0 0 2024-02-16T19:24:28.400 INFO:tasks.workunit.client.1.smithi163.stdout:8/468: dread - d0/d7/d18/d53/f9d zero size 2024-02-16T19:24:28.400 INFO:tasks.workunit.client.1.smithi163.stdout:9/659: dwrite d5/de/d30/d3d/f66 [0,4194304] 0 2024-02-16T19:24:28.401 INFO:tasks.workunit.client.1.smithi163.stdout:9/660: fsync d5/de/d72/da9/fdd 0 2024-02-16T19:24:28.401 INFO:tasks.workunit.client.1.smithi163.stdout:9/661: truncate d5/de/d16/d46/d9b/fca 107494 0 2024-02-16T19:24:28.402 INFO:tasks.workunit.client.1.smithi163.stdout:8/469: symlink d0/d24/d2c/d74/la1 0 2024-02-16T19:24:28.402 INFO:tasks.workunit.client.1.smithi163.stdout:8/470: write d0/d24/d27/d45/f4a [5045374,111089] 0 2024-02-16T19:24:28.402 INFO:tasks.workunit.client.1.smithi163.stdout:8/471: write d0/d7/d19/f26 [2083609,15271] 0 2024-02-16T19:24:28.415 INFO:tasks.workunit.client.1.smithi163.stdout:8/472: dread d0/d24/d2c/d74/f79 [4194304,4194304] 0 2024-02-16T19:24:28.426 INFO:tasks.workunit.client.1.smithi163.stdout:4/385: dwrite d11/d18/d1f/d29/f36 [0,4194304] 0 2024-02-16T19:24:28.428 INFO:tasks.workunit.client.1.smithi163.stdout:8/473: dread d0/d24/d2c/f3b [0,4194304] 0 2024-02-16T19:24:28.428 INFO:tasks.workunit.client.1.smithi163.stdout:0/777: dwrite d11/d21/d50/d58/f92 [0,4194304] 0 2024-02-16T19:24:28.429 INFO:tasks.workunit.client.1.smithi163.stdout:8/474: write d0/f97 [6036357,34769] 0 2024-02-16T19:24:28.429 INFO:tasks.workunit.client.1.smithi163.stdout:0/778: fsync d11/d12/d20/f3a 0 2024-02-16T19:24:28.429 INFO:tasks.workunit.client.1.smithi163.stdout:0/779: fdatasync d11/d21/d7c/f81 0 2024-02-16T19:24:28.430 INFO:tasks.workunit.client.1.smithi163.stdout:1/468: dwrite d7/f22 [4194304,4194304] 0 2024-02-16T19:24:28.430 INFO:tasks.workunit.client.1.smithi163.stdout:2/604: dwrite d2/f23 [0,4194304] 0 2024-02-16T19:24:28.430 INFO:tasks.workunit.client.1.smithi163.stdout:2/605: write d2/d2a/d2d/f92 [876567,51212] 0 2024-02-16T19:24:28.430 INFO:tasks.workunit.client.1.smithi163.stdout:2/606: write d2/d2a/d39/d3a/f7b [630989,16280] 0 2024-02-16T19:24:28.433 INFO:tasks.workunit.client.1.smithi163.stdout:2/607: creat d2/d26/d54/fd5 x:0 0 0 2024-02-16T19:24:28.433 INFO:tasks.workunit.client.1.smithi163.stdout:8/475: rmdir d0/d7/d18/d2e 39 2024-02-16T19:24:28.433 INFO:tasks.workunit.client.1.smithi163.stdout:8/476: readlink d0/d7/d19/d1f/d30/l99 0 2024-02-16T19:24:28.434 INFO:tasks.workunit.client.1.smithi163.stdout:0/780: dread f10 [0,4194304] 0 2024-02-16T19:24:28.435 INFO:tasks.workunit.client.1.smithi163.stdout:8/477: truncate d0/dc/f16 2935985 0 2024-02-16T19:24:28.435 INFO:tasks.workunit.client.1.smithi163.stdout:8/478: creat d0/d7/d19/d1f/d32/fa2 x:0 0 0 2024-02-16T19:24:28.436 INFO:tasks.workunit.client.1.smithi163.stdout:0/781: symlink d11/d12/dc6/d102/l115 0 2024-02-16T19:24:28.437 INFO:tasks.workunit.client.1.smithi163.stdout:4/386: dread d11/f14 [0,4194304] 0 2024-02-16T19:24:28.438 INFO:tasks.workunit.client.1.smithi163.stdout:8/479: symlink d0/d6f/la3 0 2024-02-16T19:24:28.439 INFO:tasks.workunit.client.1.smithi163.stdout:4/387: truncate d11/d2d/d4f/f53 3468748 0 2024-02-16T19:24:28.440 INFO:tasks.workunit.client.1.smithi163.stdout:9/662: dwrite d5/f2f [0,4194304] 0 2024-02-16T19:24:28.440 INFO:tasks.workunit.client.1.smithi163.stdout:9/663: fsync d5/de/d16/fb2 0 2024-02-16T19:24:28.442 INFO:tasks.workunit.client.1.smithi163.stdout:8/480: rename d0/f97 to d0/d24/d27/fa4 0 2024-02-16T19:24:28.442 INFO:tasks.workunit.client.1.smithi163.stdout:8/481: write d0/d7/d19/d1f/d32/fa2 [37221,23002] 0 2024-02-16T19:24:28.442 INFO:tasks.workunit.client.1.smithi163.stdout:8/482: creat d0/d7/d19/d1f/d30/d48/fa5 x:0 0 0 2024-02-16T19:24:28.442 INFO:tasks.workunit.client.1.smithi163.stdout:8/483: write d0/d24/d27/fa4 [483510,14302] 0 2024-02-16T19:24:28.442 INFO:tasks.workunit.client.1.smithi163.stdout:9/664: dread d5/de/d30/d3d/d49/d6d/d97/fe1 [0,4194304] 0 2024-02-16T19:24:28.444 INFO:tasks.workunit.client.1.smithi163.stdout:4/388: link d11/d18/d1f/f20 d11/d2d/d4f/f72 0 2024-02-16T19:24:28.444 INFO:tasks.workunit.client.1.smithi163.stdout:2/608: dread d2/d2a/d39/d3a/f7b [0,4194304] 0 2024-02-16T19:24:28.444 INFO:tasks.workunit.client.1.smithi163.stdout:2/609: write d2/f1c [9414239,34805] 0 2024-02-16T19:24:28.451 INFO:tasks.workunit.client.1.smithi163.stdout:4/389: link f2 d11/d18/d1f/d29/d52/d57/f73 0 2024-02-16T19:24:28.457 INFO:tasks.workunit.client.1.smithi163.stdout:2/610: write d2/d26/d54/d58/d98/fa4 [2921496,7855] 0 2024-02-16T19:24:28.457 INFO:tasks.workunit.client.1.smithi163.stdout:2/611: write d2/d26/d54/d58/d7d/f83 [772476,118248] 0 2024-02-16T19:24:28.457 INFO:tasks.workunit.client.1.smithi163.stdout:2/612: chown d2/d2a/d39/d3a/f7b 3616255 1 2024-02-16T19:24:28.457 INFO:tasks.workunit.client.1.smithi163.stdout:2/613: chown d2/d2a/d39/d3a 0 1 2024-02-16T19:24:28.457 INFO:tasks.workunit.client.1.smithi163.stdout:2/614: creat d2/d26/d97/fd6 x:0 0 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/469: dwrite d7/d20/d21/f55 [0,4194304] 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/470: fdatasync d7/d20/d21/d4e/f8f 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/471: write d7/d20/d21/d4e/f79 [869916,69267] 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/472: fsync d7/dc/d78/d7c/f3d 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/473: creat d7/dc/d2d/d32/d87/fa7 x:0 0 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/474: chown d7/dc/d2d/d32 95589511 1 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:8/484: dwrite d0/d7/d18/d53/f9d [0,4194304] 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:8/485: write d0/d7/d18/d2e/f5f [4352578,82303] 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:8/486: write d0/d7/d19/d1f/d32/fa2 [868955,19567] 0 2024-02-16T19:24:28.465 INFO:tasks.workunit.client.1.smithi163.stdout:8/487: truncate d0/d7/d18/d2e/f8f 789180 0 2024-02-16T19:24:28.512 INFO:tasks.workunit.client.1.smithi163.stdout:9/665: rmdir d5/de 39 2024-02-16T19:24:28.513 INFO:tasks.workunit.client.1.smithi163.stdout:4/390: mkdir d11/d2d/d4f/d5f/d70/d74 0 2024-02-16T19:24:28.516 INFO:tasks.workunit.client.1.smithi163.stdout:2/615: mknod d2/d2a/d39/d66/cd7 0 2024-02-16T19:24:28.519 INFO:tasks.workunit.client.1.smithi163.stdout:1/475: symlink d7/d5b/d63/la8 0 2024-02-16T19:24:28.519 INFO:tasks.workunit.client.1.smithi163.stdout:2/616: dread d2/d2a/d2d/d73/d84/f89 [0,4194304] 0 2024-02-16T19:24:28.520 INFO:tasks.workunit.client.1.smithi163.stdout:1/476: chown d7/c18 85334 1 2024-02-16T19:24:28.520 INFO:tasks.workunit.client.1.smithi163.stdout:1/477: write d7/d5b/d63/f7b [2184446,8343] 0 2024-02-16T19:24:28.523 INFO:tasks.workunit.client.1.smithi163.stdout:8/488: symlink d0/d7/d19/d1f/d32/la6 0 2024-02-16T19:24:28.524 INFO:tasks.workunit.client.1.smithi163.stdout:8/489: dread d0/d24/d27/f64 [0,4194304] 0 2024-02-16T19:24:28.525 INFO:tasks.workunit.client.1.smithi163.stdout:9/666: dread d5/de/d16/fb2 [0,4194304] 0 2024-02-16T19:24:28.525 INFO:tasks.workunit.client.1.smithi163.stdout:9/667: write d5/de/d30/d31/d52/fe7 [86415,110825] 0 2024-02-16T19:24:28.525 INFO:tasks.workunit.client.1.smithi163.stdout:9/668: chown d5/de/d72/c79 11081520 1 2024-02-16T19:24:28.525 INFO:tasks.workunit.client.1.smithi163.stdout:9/669: write d5/de/d30/d31/f74 [1157099,116477] 0 2024-02-16T19:24:28.525 INFO:tasks.workunit.client.1.smithi163.stdout:4/391: unlink d11/d18/f24 0 2024-02-16T19:24:28.526 INFO:tasks.workunit.client.1.smithi163.stdout:2/617: unlink d2/d2a/c68 0 2024-02-16T19:24:28.529 INFO:tasks.workunit.client.1.smithi163.stdout:1/478: unlink d7/dc/d2d/f8e 0 2024-02-16T19:24:28.535 INFO:tasks.workunit.client.1.smithi163.stdout:4/392: mkdir d11/d2d/d4f/d5f/d70/d74/d75 0 2024-02-16T19:24:28.535 INFO:tasks.workunit.client.1.smithi163.stdout:8/490: dread d0/d7/d18/f37 [0,4194304] 0 2024-02-16T19:24:28.535 INFO:tasks.workunit.client.1.smithi163.stdout:8/491: stat d0/dc/f2d 0 2024-02-16T19:24:28.535 INFO:tasks.workunit.client.1.smithi163.stdout:8/492: creat d0/d7/d19/d1f/d30/d48/fa7 x:0 0 0 2024-02-16T19:24:28.537 INFO:tasks.workunit.client.1.smithi163.stdout:1/479: unlink d7/dc/d2d/d32/d87/f92 0 2024-02-16T19:24:28.537 INFO:tasks.workunit.client.1.smithi163.stdout:1/480: write d7/d20/f99 [4741669,75376] 0 2024-02-16T19:24:28.538 INFO:tasks.workunit.client.1.smithi163.stdout:4/393: rmdir d11/d18/d1f/d29/d52 39 2024-02-16T19:24:28.541 INFO:tasks.workunit.client.1.smithi163.stdout:1/481: creat d7/d20/d93/fa9 x:0 0 0 2024-02-16T19:24:28.541 INFO:tasks.workunit.client.1.smithi163.stdout:1/482: fsync d7/d20/d21/d4e/f9e 0 2024-02-16T19:24:28.542 INFO:tasks.workunit.client.1.smithi163.stdout:4/394: mkdir d11/d18/d1f/d29/d52/d76 0 2024-02-16T19:24:28.545 INFO:tasks.workunit.client.1.smithi163.stdout:1/483: unlink d7/dc/d2d/d3f/f77 0 2024-02-16T19:24:28.547 INFO:tasks.workunit.client.1.smithi163.stdout:4/395: symlink d11/d18/d66/l77 0 2024-02-16T19:24:28.548 INFO:tasks.workunit.client.1.smithi163.stdout:4/396: creat d11/d5b/f78 x:0 0 0 2024-02-16T19:24:28.549 INFO:tasks.workunit.client.1.smithi163.stdout:8/493: dread d0/d7/d19/d1f/d32/f39 [0,4194304] 0 2024-02-16T19:24:28.550 INFO:tasks.workunit.client.1.smithi163.stdout:4/397: mkdir d11/d2d/d4f/d60/d79 0 2024-02-16T19:24:28.551 INFO:tasks.workunit.client.1.smithi163.stdout:4/398: dread - d11/d2d/d4f/f55 zero size 2024-02-16T19:24:28.553 INFO:tasks.workunit.client.1.smithi163.stdout:8/494: creat d0/d7/d19/fa8 x:0 0 0 2024-02-16T19:24:28.554 INFO:tasks.workunit.client.1.smithi163.stdout:2/618: dwrite d2/f16 [0,4194304] 0 2024-02-16T19:24:28.555 INFO:tasks.workunit.client.1.smithi163.stdout:9/670: dwrite d5/de/d30/d8a/da8/fcb [0,4194304] 0 2024-02-16T19:24:28.560 INFO:tasks.workunit.client.1.smithi163.stdout:8/495: creat d0/fa9 x:0 0 0 2024-02-16T19:24:28.564 INFO:tasks.workunit.client.1.smithi163.stdout:9/671: creat d5/fed x:0 0 0 2024-02-16T19:24:28.564 INFO:tasks.workunit.client.1.smithi163.stdout:4/399: dread d11/d18/d1f/d34/f51 [4194304,4194304] 0 2024-02-16T19:24:28.565 INFO:tasks.workunit.client.1.smithi163.stdout:4/400: read d11/f26 [556882,91446] 0 2024-02-16T19:24:28.566 INFO:tasks.workunit.client.1.smithi163.stdout:2/619: mknod d2/d26/cd8 0 2024-02-16T19:24:28.569 INFO:tasks.workunit.client.1.smithi163.stdout:1/484: dwrite d7/f14 [0,4194304] 0 2024-02-16T19:24:28.570 INFO:tasks.workunit.client.1.smithi163.stdout:4/401: rename d11/d18/d1f/c56 to d11/d18/d1f/d29/d52/d57/d6c/c7a 0 2024-02-16T19:24:28.571 INFO:tasks.workunit.client.1.smithi163.stdout:2/620: creat d2/d26/fd9 x:0 0 0 2024-02-16T19:24:28.573 INFO:tasks.workunit.client.1.smithi163.stdout:1/485: unlink d7/dc/l56 0 2024-02-16T19:24:28.576 INFO:tasks.workunit.client.1.smithi163.stdout:4/402: link d11/d18/d1f/d23/l2c d11/d2d/l7b 0 2024-02-16T19:24:28.576 INFO:tasks.workunit.client.1.smithi163.stdout:2/621: truncate d2/d2a/d2d/d73/d94/dc9/fcf 2355106 0 2024-02-16T19:24:28.576 INFO:tasks.workunit.client.1.smithi163.stdout:2/622: rename d2 to d2/d2a/d39/d66/dbc/dda 22 2024-02-16T19:24:28.576 INFO:tasks.workunit.client.1.smithi163.stdout:8/496: dread d0/dc/f6c [0,4194304] 0 2024-02-16T19:24:28.576 INFO:tasks.workunit.client.1.smithi163.stdout:8/497: creat d0/faa x:0 0 0 2024-02-16T19:24:28.581 INFO:tasks.workunit.client.1.smithi163.stdout:8/498: read d0/d7/d19/d1f/d32/f39 [1404759,74690] 0 2024-02-16T19:24:28.581 INFO:tasks.workunit.client.1.smithi163.stdout:8/499: write d0/fa9 [202454,31010] 0 2024-02-16T19:24:28.582 INFO:tasks.workunit.client.1.smithi163.stdout:2/623: getdents d2/d2a/d39/d3a 0 2024-02-16T19:24:28.584 INFO:tasks.workunit.client.1.smithi163.stdout:8/500: mknod d0/d24/d78/d85/cab 0 2024-02-16T19:24:28.585 INFO:tasks.workunit.client.1.smithi163.stdout:4/403: write d11/d2d/d4f/f64 [1302685,110796] 0 2024-02-16T19:24:28.587 INFO:tasks.workunit.client.1.smithi163.stdout:9/672: write d5/de/d30/d31/f54 [1439865,60993] 0 2024-02-16T19:24:28.588 INFO:tasks.workunit.client.1.smithi163.stdout:2/624: fsync d2/d26/fd9 0 2024-02-16T19:24:28.589 INFO:tasks.workunit.client.1.smithi163.stdout:2/625: dread d2/d2a/d39/f57 [0,4194304] 0 2024-02-16T19:24:28.589 INFO:tasks.workunit.client.1.smithi163.stdout:2/626: chown d2/d2a/d39/f3b 61603154 1 2024-02-16T19:24:28.589 INFO:tasks.workunit.client.1.smithi163.stdout:4/404: mkdir d11/d18/d1f/d29/d52/d57/d6c/d7c 0 2024-02-16T19:24:28.589 INFO:tasks.workunit.client.1.smithi163.stdout:4/405: read d11/d18/f1d [55425,2949] 0 2024-02-16T19:24:28.589 INFO:tasks.workunit.client.1.smithi163.stdout:9/673: rmdir d5/de/d30/d3d/d93 39 2024-02-16T19:24:28.592 INFO:tasks.workunit.client.1.smithi163.stdout:4/406: mknod d11/d18/d1f/d34/c7d 0 2024-02-16T19:24:28.593 INFO:tasks.workunit.client.1.smithi163.stdout:9/674: rmdir d5/de/d16 39 2024-02-16T19:24:28.593 INFO:tasks.workunit.client.1.smithi163.stdout:9/675: chown d5/f18 1 1 2024-02-16T19:24:28.593 INFO:tasks.workunit.client.1.smithi163.stdout:9/676: fdatasync d5/de/d30/f7b 0 2024-02-16T19:24:28.596 INFO:tasks.workunit.client.1.smithi163.stdout:4/407: creat d11/d18/d1f/d29/d52/d57/f7e x:0 0 0 2024-02-16T19:24:28.596 INFO:tasks.workunit.client.1.smithi163.stdout:9/677: creat d5/de/d16/d46/d9b/fee x:0 0 0 2024-02-16T19:24:28.597 INFO:tasks.workunit.client.1.smithi163.stdout:2/627: rename d2/d2a/d2d/d3c to d2/ddb 0 2024-02-16T19:24:28.599 INFO:tasks.workunit.client.1.smithi163.stdout:2/628: rmdir d2/d2a/d39/d3a 39 2024-02-16T19:24:28.601 INFO:tasks.workunit.client.1.smithi163.stdout:1/486: dwrite d7/dc/f33 [4194304,4194304] 0 2024-02-16T19:24:28.601 INFO:tasks.workunit.client.1.smithi163.stdout:1/487: creat d7/d5b/faa x:0 0 0 2024-02-16T19:24:28.602 INFO:tasks.workunit.client.1.smithi163.stdout:1/488: chown d7/d20/c23 94 1 2024-02-16T19:24:28.602 INFO:tasks.workunit.client.1.smithi163.stdout:1/489: write d7/d20/d2a/f40 [723933,72113] 0 2024-02-16T19:24:28.602 INFO:tasks.workunit.client.1.smithi163.stdout:9/678: rename d5/de/d72/fa7 to d5/de/d30/d3d/d49/fef 0 2024-02-16T19:24:28.602 INFO:tasks.workunit.client.1.smithi163.stdout:9/679: read d5/de/d30/d31/f69 [885098,21008] 0 2024-02-16T19:24:28.605 INFO:tasks.workunit.client.1.smithi163.stdout:8/501: dread d0/d24/f60 [0,4194304] 0 2024-02-16T19:24:28.612 INFO:tasks.workunit.client.1.smithi163.stdout:2/629: dread d2/d2a/d2d/f6e [0,4194304] 0 2024-02-16T19:24:28.613 INFO:tasks.workunit.client.1.smithi163.stdout:2/630: stat d2/d2a/d2d/l4c 0 2024-02-16T19:24:28.625 INFO:tasks.workunit.client.1.smithi163.stdout:9/680: write d5/f6 [1291626,53219] 0 2024-02-16T19:24:28.626 INFO:tasks.workunit.client.1.smithi163.stdout:7/513: sync 2024-02-16T19:24:28.627 INFO:tasks.workunit.client.1.smithi163.stdout:6/466: sync 2024-02-16T19:24:28.629 INFO:tasks.workunit.client.1.smithi163.stdout:2/631: dread d2/d2a/d39/f42 [0,4194304] 0 2024-02-16T19:24:28.630 INFO:tasks.workunit.client.1.smithi163.stdout:7/514: mkdir d11/d50/d5d/d8b/d67/db2 0 2024-02-16T19:24:28.630 INFO:tasks.workunit.client.1.smithi163.stdout:6/467: creat d0/d6/f92 x:0 0 0 2024-02-16T19:24:28.630 INFO:tasks.workunit.client.1.smithi163.stdout:7/515: creat d11/d24/d49/fb3 x:0 0 0 2024-02-16T19:24:28.631 INFO:tasks.workunit.client.1.smithi163.stdout:2/632: readlink d2/d26/l61 0 2024-02-16T19:24:28.632 INFO:tasks.workunit.client.1.smithi163.stdout:8/502: dwrite d0/d7/d19/d1f/d30/f52 [0,4194304] 0 2024-02-16T19:24:28.632 INFO:tasks.workunit.client.1.smithi163.stdout:8/503: write d0/d7/d18/d2e/f5c [724534,67803] 0 2024-02-16T19:24:28.632 INFO:tasks.workunit.client.1.smithi163.stdout:4/408: dwrite d11/d2d/d4f/f53 [0,4194304] 0 2024-02-16T19:24:28.635 INFO:tasks.workunit.client.1.smithi163.stdout:6/468: symlink d0/d6/df/d7b/l93 0 2024-02-16T19:24:28.635 INFO:tasks.workunit.client.1.smithi163.stdout:6/469: write d0/d18/d77/f6a [667692,51190] 0 2024-02-16T19:24:28.635 INFO:tasks.workunit.client.1.smithi163.stdout:7/516: rmdir d11/d24/d34/d8d 39 2024-02-16T19:24:28.635 INFO:tasks.workunit.client.1.smithi163.stdout:1/490: dwrite d7/d20/d21/d74/f76 [0,4194304] 0 2024-02-16T19:24:28.636 INFO:tasks.workunit.client.1.smithi163.stdout:1/491: chown d7/dc/c10 0 1 2024-02-16T19:24:28.636 INFO:tasks.workunit.client.1.smithi163.stdout:1/492: fsync d7/d20/d21/f55 0 2024-02-16T19:24:28.639 INFO:tasks.workunit.client.1.smithi163.stdout:8/504: rename d0/d7/d19/d1f/d32/f81 to d0/d7/d19/d1f/d30/d6a/d7b/fac 0 2024-02-16T19:24:28.639 INFO:tasks.workunit.client.1.smithi163.stdout:8/505: creat d0/d7/d18/fad x:0 0 0 2024-02-16T19:24:28.639 INFO:tasks.workunit.client.1.smithi163.stdout:8/506: write d0/d6f/f98 [981320,44663] 0 2024-02-16T19:24:28.639 INFO:tasks.workunit.client.1.smithi163.stdout:8/507: write d0/d24/d3c/f84 [537619,123768] 0 2024-02-16T19:24:28.640 INFO:tasks.workunit.client.1.smithi163.stdout:6/470: mkdir d0/d18/d94 0 2024-02-16T19:24:28.640 INFO:tasks.workunit.client.1.smithi163.stdout:7/517: symlink d11/d50/d5d/d8b/d67/db2/lb4 0 2024-02-16T19:24:28.643 INFO:tasks.workunit.client.1.smithi163.stdout:1/493: mkdir d7/d20/d21/d74/dab 0 2024-02-16T19:24:28.646 INFO:tasks.workunit.client.1.smithi163.stdout:4/409: rename d11/d3d/l5d to d11/d18/l7f 0 2024-02-16T19:24:28.647 INFO:tasks.workunit.client.1.smithi163.stdout:4/410: chown d11/d18/l7f 80167 1 2024-02-16T19:24:28.649 INFO:tasks.workunit.client.1.smithi163.stdout:6/471: creat d0/d18/d94/f95 x:0 0 0 2024-02-16T19:24:28.649 INFO:tasks.workunit.client.1.smithi163.stdout:6/472: write d0/dd/d7f/d2e/d33/f60 [1655635,8869] 0 2024-02-16T19:24:28.649 INFO:tasks.workunit.client.1.smithi163.stdout:6/473: write d0/d6/f92 [207010,127277] 0 2024-02-16T19:24:28.649 INFO:tasks.workunit.client.1.smithi163.stdout:1/494: mknod d7/dc/d78/cac 0 2024-02-16T19:24:28.651 INFO:tasks.workunit.client.1.smithi163.stdout:4/411: creat d11/d18/d1f/d29/d52/f80 x:0 0 0 2024-02-16T19:24:28.653 INFO:tasks.workunit.client.1.smithi163.stdout:1/495: write d7/dc/d2d/d32/f51 [4045974,11164] 0 2024-02-16T19:24:28.655 INFO:tasks.workunit.client.1.smithi163.stdout:4/412: mkdir d11/d2d/d81 0 2024-02-16T19:24:28.656 INFO:tasks.workunit.client.1.smithi163.stdout:1/496: truncate d7/d20/d21/d4e/f9e 833149 0 2024-02-16T19:24:28.657 INFO:tasks.workunit.client.1.smithi163.stdout:1/497: stat d7/d20/d21/d4e 0 2024-02-16T19:24:28.657 INFO:tasks.workunit.client.1.smithi163.stdout:4/413: getdents d11/d18/d1f/d29/d52/d57/d6c 0 2024-02-16T19:24:28.659 INFO:tasks.workunit.client.1.smithi163.stdout:4/414: symlink d11/d18/d1f/d23/l82 0 2024-02-16T19:24:28.659 INFO:tasks.workunit.client.1.smithi163.stdout:4/415: fdatasync d11/d18/d1f/d23/d3a/f3f 0 2024-02-16T19:24:28.661 INFO:tasks.workunit.client.1.smithi163.stdout:4/416: mknod d11/d18/d1f/d23/d3a/d4a/c83 0 2024-02-16T19:24:28.661 INFO:tasks.workunit.client.1.smithi163.stdout:4/417: creat d11/d18/d1f/d29/d52/d57/f84 x:0 0 0 2024-02-16T19:24:28.661 INFO:tasks.workunit.client.1.smithi163.stdout:4/418: getdents d11/d18/d1f/d29/d52/d57/d6c/d7c 0 2024-02-16T19:24:28.661 INFO:tasks.workunit.client.1.smithi163.stdout:4/419: creat d11/d18/d1f/d29/f85 x:0 0 0 2024-02-16T19:24:28.663 INFO:tasks.workunit.client.1.smithi163.stdout:4/420: rmdir d11/d18/d1f/d29/d52/d57/d6c 39 2024-02-16T19:24:28.668 INFO:tasks.workunit.client.1.smithi163.stdout:9/681: dwrite d5/f2c [0,4194304] 0 2024-02-16T19:24:28.672 INFO:tasks.workunit.client.1.smithi163.stdout:9/682: dread d5/de/d16/f98 [0,4194304] 0 2024-02-16T19:24:28.673 INFO:tasks.workunit.client.1.smithi163.stdout:9/683: creat d5/de/d16/d46/d9b/de4/ff0 x:0 0 0 2024-02-16T19:24:28.677 INFO:tasks.workunit.client.1.smithi163.stdout:9/684: dread d5/de/d16/d47/f87 [0,4194304] 0 2024-02-16T19:24:28.677 INFO:tasks.workunit.client.1.smithi163.stdout:9/685: readlink d5/de/d30/d3d/d49/d6d/l70 0 2024-02-16T19:24:28.677 INFO:tasks.workunit.client.1.smithi163.stdout:2/633: dwrite d2/d2a/d39/f75 [0,4194304] 0 2024-02-16T19:24:28.677 INFO:tasks.workunit.client.1.smithi163.stdout:8/508: dwrite d0/d7/d55/f80 [4194304,4194304] 0 2024-02-16T19:24:28.677 INFO:tasks.workunit.client.1.smithi163.stdout:2/634: write d2/d2a/d2d/f2e [969929,62570] 0 2024-02-16T19:24:28.678 INFO:tasks.workunit.client.1.smithi163.stdout:0/782: sync 2024-02-16T19:24:28.678 INFO:tasks.workunit.client.1.smithi163.stdout:3/509: sync 2024-02-16T19:24:28.678 INFO:tasks.workunit.client.1.smithi163.stdout:5/475: sync 2024-02-16T19:24:28.683 INFO:tasks.workunit.client.1.smithi163.stdout:7/518: dwrite d11/f1b [0,4194304] 0 2024-02-16T19:24:28.683 INFO:tasks.workunit.client.1.smithi163.stdout:8/509: getdents d0/d7/d19/d1f/d30/d6a 0 2024-02-16T19:24:28.683 INFO:tasks.workunit.client.1.smithi163.stdout:8/510: chown d0/l1c 2593 1 2024-02-16T19:24:28.689 INFO:tasks.workunit.client.1.smithi163.stdout:2/635: link d2/d2a/d2d/d73/d84/la7 d2/ddb/d7c/d9c/ldc 0 2024-02-16T19:24:28.691 INFO:tasks.workunit.client.1.smithi163.stdout:5/476: rename d0/d2/l1e to d0/d2/l94 0 2024-02-16T19:24:28.695 INFO:tasks.workunit.client.1.smithi163.stdout:9/686: rename d5/c29 to d5/de/cf1 0 2024-02-16T19:24:28.695 INFO:tasks.workunit.client.1.smithi163.stdout:9/687: creat d5/de/d16/d46/d9b/de4/ff2 x:0 0 0 2024-02-16T19:24:28.697 INFO:tasks.workunit.client.1.smithi163.stdout:8/511: unlink d0/d6f/l67 0 2024-02-16T19:24:28.697 INFO:tasks.workunit.client.1.smithi163.stdout:8/512: fdatasync d0/d7/d19/f2f 0 2024-02-16T19:24:28.698 INFO:tasks.workunit.client.1.smithi163.stdout:4/421: getdents d11/d18/d1f/d29/d52 0 2024-02-16T19:24:28.698 INFO:tasks.workunit.client.1.smithi163.stdout:4/422: truncate d11/d18/f43 4428283 0 2024-02-16T19:24:28.700 INFO:tasks.workunit.client.1.smithi163.stdout:5/477: creat d0/d22/d28/f95 x:0 0 0 2024-02-16T19:24:28.704 INFO:tasks.workunit.client.1.smithi163.stdout:2/636: read d2/f1f [2391261,119876] 0 2024-02-16T19:24:28.704 INFO:tasks.workunit.client.1.smithi163.stdout:4/423: mknod d11/d3d/c86 0 2024-02-16T19:24:28.706 INFO:tasks.workunit.client.1.smithi163.stdout:2/637: mknod d2/ddb/dc8/cdd 0 2024-02-16T19:24:28.706 INFO:tasks.workunit.client.1.smithi163.stdout:4/424: symlink d11/d18/d1f/d29/d52/d57/d6c/l87 0 2024-02-16T19:24:28.711 INFO:tasks.workunit.client.1.smithi163.stdout:9/688: read d5/de/d30/d8a/da8/fad [1756833,80842] 0 2024-02-16T19:24:28.711 INFO:tasks.workunit.client.1.smithi163.stdout:9/689: write d5/de/d16/d46/d9b/fc6 [279256,119081] 0 2024-02-16T19:24:28.714 INFO:tasks.workunit.client.1.smithi163.stdout:9/690: getdents d5/de/d30/d3d/d93 0 2024-02-16T19:24:28.714 INFO:tasks.workunit.client.1.smithi163.stdout:9/691: truncate d5/fed 324876 0 2024-02-16T19:24:28.714 INFO:tasks.workunit.client.1.smithi163.stdout:9/692: stat d5/de/d30/d8a 0 2024-02-16T19:24:28.714 INFO:tasks.workunit.client.1.smithi163.stdout:3/510: dwrite d0/f39 [0,4194304] 0 2024-02-16T19:24:28.718 INFO:tasks.workunit.client.1.smithi163.stdout:3/511: link d0/dc/c10 d0/d83/da5/cba 0 2024-02-16T19:24:28.721 INFO:tasks.workunit.client.1.smithi163.stdout:2/638: dread d2/d26/f28 [0,4194304] 0 2024-02-16T19:24:28.739 INFO:tasks.workunit.client.1.smithi163.stdout:3/512: write d0/f43 [1717579,85774] 0 2024-02-16T19:24:28.739 INFO:tasks.workunit.client.1.smithi163.stdout:7/519: dwrite ff [4194304,4194304] 0 2024-02-16T19:24:28.739 INFO:tasks.workunit.client.1.smithi163.stdout:3/513: dread d0/dc/d1d/d49/f7d [0,4194304] 0 2024-02-16T19:24:28.741 INFO:tasks.workunit.client.1.smithi163.stdout:7/520: link d11/c22 d11/d50/d5d/d8b/cb5 0 2024-02-16T19:24:28.741 INFO:tasks.workunit.client.1.smithi163.stdout:7/521: stat d11/d24/d34/d8d/f92 0 2024-02-16T19:24:28.741 INFO:tasks.workunit.client.1.smithi163.stdout:7/522: read - d11/d26/f4c zero size 2024-02-16T19:24:28.741 INFO:tasks.workunit.client.1.smithi163.stdout:7/523: chown d11/d24/d25/f9a 1 1 2024-02-16T19:24:28.742 INFO:tasks.workunit.client.1.smithi163.stdout:3/514: creat d0/dc/d96/fbb x:0 0 0 2024-02-16T19:24:28.742 INFO:tasks.workunit.client.1.smithi163.stdout:2/639: write d2/d2a/d2d/d73/d84/f8e [1661618,16082] 0 2024-02-16T19:24:28.742 INFO:tasks.workunit.client.1.smithi163.stdout:3/515: truncate d0/dc/d1d/d49/f7d 371112 0 2024-02-16T19:24:28.744 INFO:tasks.workunit.client.1.smithi163.stdout:3/516: mknod d0/dc/d50/cbc 0 2024-02-16T19:24:28.745 INFO:tasks.workunit.client.1.smithi163.stdout:0/783: dwrite fd [8388608,4194304] 0 2024-02-16T19:24:28.746 INFO:tasks.workunit.client.1.smithi163.stdout:3/517: rmdir d0/dc/d50/d66/d80 39 2024-02-16T19:24:28.746 INFO:tasks.workunit.client.1.smithi163.stdout:3/518: write d0/dc/d94/d9e/fb6 [41170,90454] 0 2024-02-16T19:24:28.746 INFO:tasks.workunit.client.1.smithi163.stdout:3/519: dread - d0/dc/d94/d9e/fad zero size 2024-02-16T19:24:28.746 INFO:tasks.workunit.client.1.smithi163.stdout:3/520: dread - d0/dc/d50/d66/d80/fa0 zero size 2024-02-16T19:24:28.746 INFO:tasks.workunit.client.1.smithi163.stdout:4/425: dwrite d11/d18/d1f/f38 [0,4194304] 0 2024-02-16T19:24:28.746 INFO:tasks.workunit.client.1.smithi163.stdout:5/478: dwrite d0/d1c/f74 [0,4194304] 0 2024-02-16T19:24:28.747 INFO:tasks.workunit.client.1.smithi163.stdout:0/784: link d11/d22/d61/f94 d11/d21/d50/f116 0 2024-02-16T19:24:28.747 INFO:tasks.workunit.client.1.smithi163.stdout:0/785: chown d11/d21/d7c/fb7 2077 1 2024-02-16T19:24:28.747 INFO:tasks.workunit.client.1.smithi163.stdout:3/521: creat d0/dc/d1c/d32/d7f/fbd x:0 0 0 2024-02-16T19:24:28.750 INFO:tasks.workunit.client.1.smithi163.stdout:9/693: dwrite d5/de/fdb [0,4194304] 0 2024-02-16T19:24:28.750 INFO:tasks.workunit.client.1.smithi163.stdout:9/694: dread - d5/de/d30/d3d/d49/fdf zero size 2024-02-16T19:24:28.752 INFO:tasks.workunit.client.1.smithi163.stdout:5/479: creat d0/d1c/d5a/f96 x:0 0 0 2024-02-16T19:24:28.756 INFO:tasks.workunit.client.1.smithi163.stdout:0/786: symlink d11/d12/d67/d6a/dae/l117 0 2024-02-16T19:24:28.757 INFO:tasks.workunit.client.1.smithi163.stdout:2/640: write d2/d26/f2f [3215008,114402] 0 2024-02-16T19:24:28.760 INFO:tasks.workunit.client.1.smithi163.stdout:9/695: mkdir d5/de/d30/d3d/d49/df3 0 2024-02-16T19:24:28.761 INFO:tasks.workunit.client.1.smithi163.stdout:5/480: rename d0/l25 to d0/d1c/d5a/l97 0 2024-02-16T19:24:28.761 INFO:tasks.workunit.client.1.smithi163.stdout:5/481: chown d0/d2/d34/d44/d53/f87 1 1 2024-02-16T19:24:28.762 INFO:tasks.workunit.client.1.smithi163.stdout:2/641: symlink d2/ddb/d7c/lde 0 2024-02-16T19:24:28.762 INFO:tasks.workunit.client.1.smithi163.stdout:2/642: creat d2/d26/d97/fdf x:0 0 0 2024-02-16T19:24:28.762 INFO:tasks.workunit.client.1.smithi163.stdout:2/643: chown d2/d2a/d2d/d73/d84/f8e 5872 1 2024-02-16T19:24:28.764 INFO:tasks.workunit.client.1.smithi163.stdout:9/696: mkdir d5/de/d30/d3d/d49/d6e/df4 0 2024-02-16T19:24:28.767 INFO:tasks.workunit.client.1.smithi163.stdout:3/522: dwrite d0/dc/d1d/d29/d38/f40 [0,4194304] 0 2024-02-16T19:24:28.768 INFO:tasks.workunit.client.1.smithi163.stdout:2/644: rename d2/d26/d54/d58/f87 to d2/d2a/d39/d3a/fe0 0 2024-02-16T19:24:28.769 INFO:tasks.workunit.client.1.smithi163.stdout:1/498: truncate d7/d20/d21/d4e/f9e 831517 0 2024-02-16T19:24:28.772 INFO:tasks.workunit.client.1.smithi163.stdout:9/697: rename d5/de/d16/d46/l85 to d5/de/d30/d3d/d49/d6d/d97/lf5 0 2024-02-16T19:24:28.774 INFO:tasks.workunit.client.1.smithi163.stdout:1/499: rename d7/dc/d2d/d32/f75 to d7/dc/d2d/d3f/fad 0 2024-02-16T19:24:28.774 INFO:tasks.workunit.client.1.smithi163.stdout:1/500: creat d7/dc/d78/d7c/fae x:0 0 0 2024-02-16T19:24:28.774 INFO:tasks.workunit.client.1.smithi163.stdout:1/501: chown d7/d20/f24 749 1 2024-02-16T19:24:28.774 INFO:tasks.workunit.client.1.smithi163.stdout:1/502: chown d7/dc/d2d/d3f/l86 1630583 1 2024-02-16T19:24:28.774 INFO:tasks.workunit.client.1.smithi163.stdout:4/426: dwrite d11/f2f [0,4194304] 0 2024-02-16T19:24:28.776 INFO:tasks.workunit.client.1.smithi163.stdout:9/698: mkdir d5/de/df6 0 2024-02-16T19:24:28.776 INFO:tasks.workunit.client.1.smithi163.stdout:9/699: write d5/de/d30/d3d/d49/d6d/d97/fe1 [206350,77542] 0 2024-02-16T19:24:28.776 INFO:tasks.workunit.client.1.smithi163.stdout:9/700: chown d5/de/d30/d3d/d49/d6e/df4 301646 1 2024-02-16T19:24:28.782 INFO:tasks.workunit.client.1.smithi163.stdout:0/787: dwrite d11/d21/fdb [0,4194304] 0 2024-02-16T19:24:28.794 INFO:tasks.workunit.client.1.smithi163.stdout:0/788: write d11/d21/d50/f79 [907771,43031] 0 2024-02-16T19:24:28.807 INFO:tasks.workunit.client.1.smithi163.stdout:3/523: dwrite d0/dc/d1d/d29/d38/d46/f90 [0,4194304] 0 2024-02-16T19:24:28.807 INFO:tasks.workunit.client.1.smithi163.stdout:5/482: dwrite d0/d2/d5c/f8c [0,4194304] 0 2024-02-16T19:24:28.807 INFO:tasks.workunit.client.1.smithi163.stdout:2/645: dwrite d2/d2a/d2d/d73/d94/dc9/fcf [0,4194304] 0 2024-02-16T19:24:28.807 INFO:tasks.workunit.client.1.smithi163.stdout:2/646: creat d2/d26/d54/fe1 x:0 0 0 2024-02-16T19:24:28.807 INFO:tasks.workunit.client.1.smithi163.stdout:4/427: dwrite d11/f30 [0,4194304] 0 2024-02-16T19:24:28.815 INFO:tasks.workunit.client.1.smithi163.stdout:5/483: dread d0/d2/d34/d44/f7f [0,4194304] 0 2024-02-16T19:24:28.977 INFO:tasks.workunit.client.1.smithi163.stdout:1/503: link d7/dc/d78/d7c/c7f d7/dc/d78/d7c/caf 0 2024-02-16T19:24:28.982 INFO:tasks.workunit.client.1.smithi163.stdout:9/701: stat d5/de/d16/lb1 0 2024-02-16T19:24:28.983 INFO:tasks.workunit.client.1.smithi163.stdout:9/702: dread d5/de/d30/d43/f4b [0,4194304] 0 2024-02-16T19:24:28.984 INFO:tasks.workunit.client.1.smithi163.stdout:0/789: unlink d11/d22/d3e/l38 0 2024-02-16T19:24:28.985 INFO:tasks.workunit.client.1.smithi163.stdout:3/524: symlink d0/dc/d1c/d32/d7f/lbe 0 2024-02-16T19:24:28.992 INFO:tasks.workunit.client.1.smithi163.stdout:0/790: dread d11/f1f [0,4194304] 0 2024-02-16T19:24:28.993 INFO:tasks.workunit.client.1.smithi163.stdout:4/428: stat d11/d18/l7f 0 2024-02-16T19:24:28.996 INFO:tasks.workunit.client.1.smithi163.stdout:0/791: write d11/d12/d20/fe4 [1635684,115072] 0 2024-02-16T19:24:29.001 INFO:tasks.workunit.client.1.smithi163.stdout:5/484: rename d0/d22/d28/f70 to d0/d22/d28/d36/f98 0 2024-02-16T19:24:29.010 INFO:tasks.workunit.client.1.smithi163.stdout:1/504: creat d7/dc/d2d/d32/d87/d97/fb0 x:0 0 0 2024-02-16T19:24:29.010 INFO:tasks.workunit.client.1.smithi163.stdout:9/703: rmdir d5/de/d30/d3d/d49 39 2024-02-16T19:24:29.010 INFO:tasks.workunit.client.1.smithi163.stdout:4/429: symlink d11/d18/d1f/d29/d52/l88 0 2024-02-16T19:24:29.013 INFO:tasks.workunit.client.1.smithi163.stdout:5/485: rename d0/d2/d82 to d0/d2/d5c/d99 0 2024-02-16T19:24:29.014 INFO:tasks.workunit.client.1.smithi163.stdout:1/505: symlink d7/dc/d2d/d32/d87/da6/lb1 0 2024-02-16T19:24:29.014 INFO:tasks.workunit.client.1.smithi163.stdout:1/506: read d7/f49 [31016,42242] 0 2024-02-16T19:24:29.015 INFO:tasks.workunit.client.1.smithi163.stdout:9/704: creat d5/de/d30/d3d/d49/d6d/d97/ff7 x:0 0 0 2024-02-16T19:24:29.016 INFO:tasks.workunit.client.1.smithi163.stdout:9/705: mknod d5/de/d30/d3d/d49/dea/cf8 0 2024-02-16T19:24:29.016 INFO:tasks.workunit.client.1.smithi163.stdout:9/706: truncate d5/de/d30/d43/fda 599930 0 2024-02-16T19:24:29.017 INFO:tasks.workunit.client.1.smithi163.stdout:9/707: mkdir d5/de/d30/d3d/d49/d50/df9 0 2024-02-16T19:24:29.017 INFO:tasks.workunit.client.1.smithi163.stdout:9/708: chown d5/de/d30/d3d/d49/d6e/laf 107 1 2024-02-16T19:24:29.017 INFO:tasks.workunit.client.1.smithi163.stdout:9/709: dread - d5/de/d30/d8a/fd2 zero size 2024-02-16T19:24:29.017 INFO:tasks.workunit.client.1.smithi163.stdout:0/792: dwrite d11/d12/d20/fb9 [0,4194304] 0 2024-02-16T19:24:29.018 INFO:tasks.workunit.client.1.smithi163.stdout:9/710: creat d5/dc0/ffa x:0 0 0 2024-02-16T19:24:29.026 INFO:tasks.workunit.client.1.smithi163.stdout:6/474: sync 2024-02-16T19:24:29.026 INFO:tasks.workunit.client.1.smithi163.stdout:4/430: write d11/d2d/d4f/f64 [45123,49158] 0 2024-02-16T19:24:29.026 INFO:tasks.workunit.client.1.smithi163.stdout:5/486: write d0/d1c/f74 [1272117,31225] 0 2024-02-16T19:24:29.026 INFO:tasks.workunit.client.1.smithi163.stdout:8/513: sync 2024-02-16T19:24:29.027 INFO:tasks.workunit.client.1.smithi163.stdout:7/524: sync 2024-02-16T19:24:29.027 INFO:tasks.workunit.client.1.smithi163.stdout:2/647: sync 2024-02-16T19:24:29.033 INFO:tasks.workunit.client.1.smithi163.stdout:4/431: mkdir d11/d18/d1f/d29/d52/d76/d89 0 2024-02-16T19:24:29.034 INFO:tasks.workunit.client.1.smithi163.stdout:5/487: mknod d0/d2/c9a 0 2024-02-16T19:24:29.034 INFO:tasks.workunit.client.1.smithi163.stdout:5/488: chown d0/d2/d34 1806 1 2024-02-16T19:24:29.035 INFO:tasks.workunit.client.1.smithi163.stdout:2/648: creat d2/d2a/d2d/d73/db5/fe2 x:0 0 0 2024-02-16T19:24:29.036 INFO:tasks.workunit.client.1.smithi163.stdout:6/475: mknod d0/dd/d7f/d15/d1c/d65/d63/d86/c96 0 2024-02-16T19:24:29.036 INFO:tasks.workunit.client.1.smithi163.stdout:6/476: stat d0/dd/d7f/f73 0 2024-02-16T19:24:29.037 INFO:tasks.workunit.client.1.smithi163.stdout:6/477: stat d0/d6/df/d20/d40/d17/c39 0 2024-02-16T19:24:29.037 INFO:tasks.workunit.client.1.smithi163.stdout:6/478: fsync d0/dd/d7f/d15/d1c/f7d 0 2024-02-16T19:24:29.037 INFO:tasks.workunit.client.1.smithi163.stdout:4/432: creat d11/f8a x:0 0 0 2024-02-16T19:24:29.037 INFO:tasks.workunit.client.1.smithi163.stdout:4/433: fdatasync d11/d18/d1f/d29/f36 0 2024-02-16T19:24:29.038 INFO:tasks.workunit.client.1.smithi163.stdout:6/479: truncate d0/dd/d7f/d15/d1c/d65/d63/f79 124614 0 2024-02-16T19:24:29.047 INFO:tasks.workunit.client.1.smithi163.stdout:2/649: dread d2/d26/f2f [0,4194304] 0 2024-02-16T19:24:29.050 INFO:tasks.workunit.client.1.smithi163.stdout:2/650: creat d2/ddb/dc8/fe3 x:0 0 0 2024-02-16T19:24:29.050 INFO:tasks.workunit.client.1.smithi163.stdout:5/489: dread d0/d2/f30 [0,4194304] 0 2024-02-16T19:24:29.050 INFO:tasks.workunit.client.1.smithi163.stdout:1/507: dwrite d7/d20/d2a/f9f [0,4194304] 0 2024-02-16T19:24:29.051 INFO:tasks.workunit.client.1.smithi163.stdout:5/490: read d0/d22/d28/d36/d3c/d4f/f62 [4771420,374] 0 2024-02-16T19:24:29.051 INFO:tasks.workunit.client.1.smithi163.stdout:2/651: mkdir d2/d44/de4 0 2024-02-16T19:24:29.051 INFO:tasks.workunit.client.1.smithi163.stdout:2/652: truncate d2/f1a 4822025 0 2024-02-16T19:24:29.051 INFO:tasks.workunit.client.1.smithi163.stdout:0/793: dwrite d11/d12/f6c [0,4194304] 0 2024-02-16T19:24:29.053 INFO:tasks.workunit.client.1.smithi163.stdout:5/491: creat d0/d22/d3b/f9b x:0 0 0 2024-02-16T19:24:29.053 INFO:tasks.workunit.client.1.smithi163.stdout:2/653: write d2/d2a/d2d/f2e [430066,85120] 0 2024-02-16T19:24:29.053 INFO:tasks.workunit.client.1.smithi163.stdout:1/508: symlink d7/dc/d2d/d3f/lb2 0 2024-02-16T19:24:29.055 INFO:tasks.workunit.client.1.smithi163.stdout:5/492: symlink d0/d2/d5c/l9c 0 2024-02-16T19:24:29.055 INFO:tasks.workunit.client.1.smithi163.stdout:5/493: write d0/d2/f16 [3638831,114241] 0 2024-02-16T19:24:29.057 INFO:tasks.workunit.client.1.smithi163.stdout:2/654: rename d2/f23 to d2/fe5 0 2024-02-16T19:24:29.058 INFO:tasks.workunit.client.1.smithi163.stdout:1/509: rename d7/f49 to d7/d20/d93/fb3 0 2024-02-16T19:24:29.059 INFO:tasks.workunit.client.1.smithi163.stdout:0/794: rmdir d11/d21/d7c 39 2024-02-16T19:24:29.061 INFO:tasks.workunit.client.1.smithi163.stdout:5/494: creat d0/d2/d31/f9d x:0 0 0 2024-02-16T19:24:29.061 INFO:tasks.workunit.client.1.smithi163.stdout:5/495: chown d0/d2/l37 19 1 2024-02-16T19:24:29.061 INFO:tasks.workunit.client.1.smithi163.stdout:5/496: write d0/d2/d34/d44/d53/f7d [647584,84903] 0 2024-02-16T19:24:29.062 INFO:tasks.workunit.client.1.smithi163.stdout:2/655: mkdir d2/d2a/d2d/d73/db5/de6 0 2024-02-16T19:24:29.062 INFO:tasks.workunit.client.1.smithi163.stdout:2/656: readlink d2/d2a/l5c 0 2024-02-16T19:24:29.062 INFO:tasks.workunit.client.1.smithi163.stdout:2/657: write d2/ddb/d7c/f99 [1481005,80813] 0 2024-02-16T19:24:29.063 INFO:tasks.workunit.client.0.smithi040.stdout:4/941: symlink dd/de/d71/d102/d107/l12a 0 2024-02-16T19:24:29.063 INFO:tasks.workunit.client.0.smithi040.stdout:4/942: write dd/d58/d5e/f10d [736010,30927] 0 2024-02-16T19:24:29.065 INFO:tasks.workunit.client.1.smithi163.stdout:2/658: symlink d2/d2a/d2d/d73/d84/le7 0 2024-02-16T19:24:29.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/943: unlink dd/de/c51 0 2024-02-16T19:24:29.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/944: fdatasync dd/d58/d8c/fae 0 2024-02-16T19:24:29.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/945: dread - dd/de/d10/d36/dc3/fdf zero size 2024-02-16T19:24:29.069 INFO:tasks.workunit.client.0.smithi040.stdout:4/946: fdatasync dd/de/d10/d36/d9e/fea 0 2024-02-16T19:24:29.074 INFO:tasks.workunit.client.1.smithi163.stdout:9/711: dwrite d5/de/d30/d43/f5c [0,4194304] 0 2024-02-16T19:24:29.074 INFO:tasks.workunit.client.1.smithi163.stdout:9/712: truncate d5/de/d30/d8a/fd7 264266 0 2024-02-16T19:24:29.074 INFO:tasks.workunit.client.1.smithi163.stdout:4/434: dwrite d11/d5b/f78 [0,4194304] 0 2024-02-16T19:24:29.074 INFO:tasks.workunit.client.1.smithi163.stdout:4/435: readlink d11/d18/d1f/d23/l32 0 2024-02-16T19:24:29.074 INFO:tasks.workunit.client.1.smithi163.stdout:6/480: dwrite d0/f5 [8388608,4194304] 0 2024-02-16T19:24:29.074 INFO:tasks.workunit.client.1.smithi163.stdout:7/525: dwrite d11/f1b [0,4194304] 0 2024-02-16T19:24:29.079 INFO:tasks.workunit.client.1.smithi163.stdout:4/436: mknod d11/d2d/d4f/d60/c8b 0 2024-02-16T19:24:29.080 INFO:tasks.workunit.client.1.smithi163.stdout:6/481: mkdir d0/d18/d77/d97 0 2024-02-16T19:24:29.083 INFO:tasks.workunit.client.1.smithi163.stdout:7/526: symlink d11/d50/d5d/d8b/lb6 0 2024-02-16T19:24:29.083 INFO:tasks.workunit.client.1.smithi163.stdout:7/527: chown d11/d24/d25/f5a 105270 1 2024-02-16T19:24:29.086 INFO:tasks.workunit.client.1.smithi163.stdout:4/437: creat d11/d2d/d4f/f8c x:0 0 0 2024-02-16T19:24:29.086 INFO:tasks.workunit.client.1.smithi163.stdout:4/438: chown d11/f30 104322377 1 2024-02-16T19:24:29.086 INFO:tasks.workunit.client.1.smithi163.stdout:4/439: dread - d11/f46 zero size 2024-02-16T19:24:29.086 INFO:tasks.workunit.client.1.smithi163.stdout:4/440: read - d11/d18/d1f/d29/d52/d57/f84 zero size 2024-02-16T19:24:29.086 INFO:tasks.workunit.client.1.smithi163.stdout:6/482: dread d0/dd/d7f/d2e/d33/f38 [0,4194304] 0 2024-02-16T19:24:29.087 INFO:tasks.workunit.client.1.smithi163.stdout:7/528: link d11/d24/c35 d11/d50/cb7 0 2024-02-16T19:24:29.087 INFO:tasks.workunit.client.1.smithi163.stdout:7/529: write d11/d50/f88 [576041,27416] 0 2024-02-16T19:24:29.087 INFO:tasks.workunit.client.1.smithi163.stdout:7/530: chown d11/d24/d34/caf 2998 1 2024-02-16T19:24:29.088 INFO:tasks.workunit.client.1.smithi163.stdout:9/713: read d5/de/d30/d3d/d49/f62 [2842459,3802] 0 2024-02-16T19:24:29.088 INFO:tasks.workunit.client.1.smithi163.stdout:4/441: link d11/d18/f43 d11/d2d/d4f/d60/f8d 0 2024-02-16T19:24:29.088 INFO:tasks.workunit.client.1.smithi163.stdout:4/442: dread - d11/d18/d1f/d29/d52/f5a zero size 2024-02-16T19:24:29.088 INFO:tasks.workunit.client.1.smithi163.stdout:4/443: chown d11/d3d/l44 104 1 2024-02-16T19:24:29.089 INFO:tasks.workunit.client.1.smithi163.stdout:6/483: rmdir d0/dd/d7f/d2e/d33 39 2024-02-16T19:24:29.089 INFO:tasks.workunit.client.1.smithi163.stdout:6/484: fdatasync d0/dd/d7f/d2e/d33/f5f 0 2024-02-16T19:24:29.091 INFO:tasks.workunit.client.1.smithi163.stdout:4/444: link d11/d18/l25 d11/d18/d1f/d34/l8e 0 2024-02-16T19:24:29.091 INFO:tasks.workunit.client.1.smithi163.stdout:4/445: truncate d11/f41 745226 0 2024-02-16T19:24:29.093 INFO:tasks.workunit.client.1.smithi163.stdout:6/485: mkdir d0/dd/d7f/d81/d98 0 2024-02-16T19:24:29.096 INFO:tasks.workunit.client.1.smithi163.stdout:9/714: rename d5/de/d16/c3f to d5/dc0/cfb 0 2024-02-16T19:24:29.096 INFO:tasks.workunit.client.1.smithi163.stdout:9/715: dread - d5/de/d16/d46/d9b/fee zero size 2024-02-16T19:24:29.096 INFO:tasks.workunit.client.1.smithi163.stdout:0/795: dwrite d11/d21/fea [0,4194304] 0 2024-02-16T19:24:29.096 INFO:tasks.workunit.client.1.smithi163.stdout:0/796: chown d11/d12/fdd 22473 1 2024-02-16T19:24:29.096 INFO:tasks.workunit.client.1.smithi163.stdout:1/510: dwrite d7/f1d [4194304,4194304] 0 2024-02-16T19:24:29.097 INFO:tasks.workunit.client.1.smithi163.stdout:2/659: dwrite d2/d26/d97/fb1 [0,4194304] 0 2024-02-16T19:24:29.097 INFO:tasks.workunit.client.1.smithi163.stdout:2/660: dread - d2/d26/d54/fd5 zero size 2024-02-16T19:24:29.097 INFO:tasks.workunit.client.1.smithi163.stdout:4/446: mkdir d11/d5b/d8f 0 2024-02-16T19:24:29.101 INFO:tasks.workunit.client.1.smithi163.stdout:9/716: creat d5/de/d30/d3d/d49/df3/ffc x:0 0 0 2024-02-16T19:24:29.101 INFO:tasks.workunit.client.1.smithi163.stdout:9/717: write d5/de/d30/d3d/d49/fdf [957787,70528] 0 2024-02-16T19:24:29.105 INFO:tasks.workunit.client.1.smithi163.stdout:2/661: unlink d2/ddb/c63 0 2024-02-16T19:24:29.107 INFO:tasks.workunit.client.1.smithi163.stdout:4/447: creat d11/d2d/d4f/d60/d79/f90 x:0 0 0 2024-02-16T19:24:29.108 INFO:tasks.workunit.client.1.smithi163.stdout:4/448: creat d11/f91 x:0 0 0 2024-02-16T19:24:29.112 INFO:tasks.workunit.client.1.smithi163.stdout:9/718: write d5/de/d30/d8a/da8/fcb [1801371,113706] 0 2024-02-16T19:24:29.112 INFO:tasks.workunit.client.1.smithi163.stdout:9/719: chown d5/fc8 1408337 1 2024-02-16T19:24:29.117 INFO:tasks.workunit.client.1.smithi163.stdout:7/531: dwrite d11/d24/d25/f4f [0,4194304] 0 2024-02-16T19:24:29.117 INFO:tasks.workunit.client.1.smithi163.stdout:9/720: dread d5/de/d16/d46/fe2 [0,4194304] 0 2024-02-16T19:24:29.119 INFO:tasks.workunit.client.1.smithi163.stdout:7/532: mknod d11/d50/d5d/cb8 0 2024-02-16T19:24:29.120 INFO:tasks.workunit.client.1.smithi163.stdout:9/721: truncate d5/de/f5f 2732454 0 2024-02-16T19:24:29.122 INFO:tasks.workunit.client.1.smithi163.stdout:7/533: creat d11/d24/d25/d4b/d66/fb9 x:0 0 0 2024-02-16T19:24:29.123 INFO:tasks.workunit.client.1.smithi163.stdout:1/511: dwrite d7/f6b [4194304,4194304] 0 2024-02-16T19:24:29.123 INFO:tasks.workunit.client.1.smithi163.stdout:7/534: unlink d11/d24/d34/f8e 0 2024-02-16T19:24:29.125 INFO:tasks.workunit.client.1.smithi163.stdout:9/722: link d5/l7 d5/de/d30/d3d/d49/dea/lfd 0 2024-02-16T19:24:29.125 INFO:tasks.workunit.client.1.smithi163.stdout:9/723: chown d5/de/d16/d46/d9b/fca 2 1 2024-02-16T19:24:29.125 INFO:tasks.workunit.client.0.smithi040.stdout:4/947: dwrite dd/d58/d5e/dd1/fec [0,4194304] 0 2024-02-16T19:24:29.126 INFO:tasks.workunit.client.1.smithi163.stdout:7/535: link d11/d26/f98 d11/d50/d5d/d8b/d67/db2/fba 0 2024-02-16T19:24:29.127 INFO:tasks.workunit.client.1.smithi163.stdout:7/536: readlink d11/l4d 0 2024-02-16T19:24:29.127 INFO:tasks.workunit.client.0.smithi040.stdout:4/948: truncate dd/de/f6b 1047731 0 2024-02-16T19:24:29.128 INFO:tasks.workunit.client.1.smithi163.stdout:9/724: unlink d5/de/d84/leb 0 2024-02-16T19:24:29.129 INFO:tasks.workunit.client.1.smithi163.stdout:9/725: write d5/de/d72/fb0 [3475286,115859] 0 2024-02-16T19:24:29.129 INFO:tasks.workunit.client.1.smithi163.stdout:9/726: chown d5/de/d30 132064615 1 2024-02-16T19:24:29.130 INFO:tasks.workunit.client.0.smithi040.stdout:4/949: rename dd/de/c16 to dd/d58/d5e/dd1/d9d/dd6/c12b 0 2024-02-16T19:24:29.131 INFO:tasks.workunit.client.0.smithi040.stdout:4/950: getdents dd/d58/d5e/dd1/dc6 0 2024-02-16T19:24:29.131 INFO:tasks.workunit.client.0.smithi040.stdout:4/951: chown dd/de/d10/d36/dad 1945575287 1 2024-02-16T19:24:29.131 INFO:tasks.workunit.client.1.smithi163.stdout:9/727: rename d5/f2f to d5/de/d30/d3d/d49/d6e/ffe 0 2024-02-16T19:24:29.133 INFO:tasks.workunit.client.1.smithi163.stdout:9/728: mknod d5/de/d30/d3d/d49/dea/cff 0 2024-02-16T19:24:29.135 INFO:tasks.workunit.client.1.smithi163.stdout:2/662: dwrite d2/ddb/dc8/fd4 [0,4194304] 0 2024-02-16T19:24:29.135 INFO:tasks.workunit.client.1.smithi163.stdout:4/449: dwrite d11/d18/d1f/d29/f49 [0,4194304] 0 2024-02-16T19:24:29.135 INFO:tasks.workunit.client.1.smithi163.stdout:4/450: readlink d11/d18/d66/l6d 0 2024-02-16T19:24:29.135 INFO:tasks.workunit.client.1.smithi163.stdout:4/451: write d11/d18/d1f/d29/d52/d57/f7e [298446,55661] 0 2024-02-16T19:24:29.136 INFO:tasks.workunit.client.0.smithi040.stdout:4/952: truncate dd/de/d45/fc4 388774 0 2024-02-16T19:24:29.136 INFO:tasks.workunit.client.0.smithi040.stdout:4/953: chown dd/de/daa/dd8/cd5 125956 1 2024-02-16T19:24:29.139 INFO:tasks.workunit.client.1.smithi163.stdout:2/663: mknod d2/d2a/d2d/ce8 0 2024-02-16T19:24:29.140 INFO:tasks.workunit.client.0.smithi040.stdout:4/954: link dd/d58/l87 dd/dd2/d106/l12c 0 2024-02-16T19:24:29.140 INFO:tasks.workunit.client.1.smithi163.stdout:4/452: creat d11/d18/d1f/d29/d52/d57/d6c/d7c/f92 x:0 0 0 2024-02-16T19:24:29.140 INFO:tasks.workunit.client.1.smithi163.stdout:1/512: dread d7/d20/f99 [0,4194304] 0 2024-02-16T19:24:29.140 INFO:tasks.workunit.client.0.smithi040.stdout:4/955: mknod dd/d58/d5e/dd1/c12d 0 2024-02-16T19:24:29.141 INFO:tasks.workunit.client.1.smithi163.stdout:9/729: truncate d5/f40 8878950 0 2024-02-16T19:24:29.143 INFO:tasks.workunit.client.1.smithi163.stdout:2/664: symlink d2/d2a/d2d/d73/d94/dc9/le9 0 2024-02-16T19:24:29.143 INFO:tasks.workunit.client.1.smithi163.stdout:2/665: write d2/d2a/d2d/f2e [341829,103940] 0 2024-02-16T19:24:29.143 INFO:tasks.workunit.client.1.smithi163.stdout:1/513: symlink d7/d20/d21/d74/dab/lb4 0 2024-02-16T19:24:29.143 INFO:tasks.workunit.client.1.smithi163.stdout:1/514: write d7/d20/d21/d4e/f8f [886481,29468] 0 2024-02-16T19:24:29.145 INFO:tasks.workunit.client.1.smithi163.stdout:7/537: dwrite d11/d50/fa3 [0,4194304] 0 2024-02-16T19:24:29.147 INFO:tasks.workunit.client.1.smithi163.stdout:2/666: rename d2/d26/d54/d58/d98/fa3 to d2/ddb/d7c/d9c/fea 0 2024-02-16T19:24:29.148 INFO:tasks.workunit.client.1.smithi163.stdout:2/667: chown d2/d26/d54/fe1 1 1 2024-02-16T19:24:29.148 INFO:tasks.workunit.client.1.smithi163.stdout:2/668: truncate d2/d26/d54/d58/fcc 727151 0 2024-02-16T19:24:29.148 INFO:tasks.workunit.client.1.smithi163.stdout:2/669: creat d2/d26/d54/d58/d98/feb x:0 0 0 2024-02-16T19:24:29.148 INFO:tasks.workunit.client.1.smithi163.stdout:1/515: stat d7/c65 0 2024-02-16T19:24:29.152 INFO:tasks.workunit.client.1.smithi163.stdout:7/538: creat d11/d26/fbb x:0 0 0 2024-02-16T19:24:29.160 INFO:tasks.workunit.client.1.smithi163.stdout:2/670: mknod d2/d26/d54/d58/da6/cec 0 2024-02-16T19:24:29.161 INFO:tasks.workunit.client.1.smithi163.stdout:7/539: symlink d11/d50/d5d/lbc 0 2024-02-16T19:24:29.163 INFO:tasks.workunit.client.1.smithi163.stdout:2/671: rename d2/d2a/c41 to d2/d26/d97/ced 0 2024-02-16T19:24:29.164 INFO:tasks.workunit.client.1.smithi163.stdout:1/516: rmdir d7/dc/d78/d7c 39 2024-02-16T19:24:29.166 INFO:tasks.workunit.client.1.smithi163.stdout:7/540: truncate d11/d24/d25/f5a 1112003 0 2024-02-16T19:24:29.167 INFO:tasks.workunit.client.1.smithi163.stdout:2/672: mkdir d2/dce/dee 0 2024-02-16T19:24:29.168 INFO:tasks.workunit.client.1.smithi163.stdout:7/541: symlink d11/d24/lbd 0 2024-02-16T19:24:29.168 INFO:tasks.workunit.client.1.smithi163.stdout:7/542: dread - d11/d24/f79 zero size 2024-02-16T19:24:29.168 INFO:tasks.workunit.client.1.smithi163.stdout:2/673: truncate d2/d26/d54/d58/f69 2632540 0 2024-02-16T19:24:29.170 INFO:tasks.workunit.client.1.smithi163.stdout:2/674: mkdir d2/d26/d54/d58/d7d/daf/def 0 2024-02-16T19:24:29.170 INFO:tasks.workunit.client.1.smithi163.stdout:2/675: chown d2/d2a/d39/l59 3648 1 2024-02-16T19:24:29.170 INFO:tasks.workunit.client.1.smithi163.stdout:2/676: fsync d2/f1c 0 2024-02-16T19:24:29.171 INFO:tasks.workunit.client.1.smithi163.stdout:9/730: dwrite d5/de/d30/d3d/d49/d6d/fc3 [0,4194304] 0 2024-02-16T19:24:29.171 INFO:tasks.workunit.client.1.smithi163.stdout:2/677: symlink d2/d26/d54/d58/da6/lf0 0 2024-02-16T19:24:29.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/956: dwrite dd/de/d45/fc4 [0,4194304] 0 2024-02-16T19:24:29.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/957: chown dd/d58/d5e/dd1/d9d 61419330 1 2024-02-16T19:24:29.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/958: chown dd/d58/d8c 1935718 1 2024-02-16T19:24:29.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/959: write dd/f124 [850615,24001] 0 2024-02-16T19:24:29.172 INFO:tasks.workunit.client.0.smithi040.stdout:4/960: dread - dd/de/d10/d36/dad/f128 zero size 2024-02-16T19:24:29.174 INFO:tasks.workunit.client.1.smithi163.stdout:9/731: rmdir d5/de/d72 39 2024-02-16T19:24:29.178 INFO:tasks.workunit.client.1.smithi163.stdout:9/732: symlink d5/de/d30/d3d/l100 0 2024-02-16T19:24:29.178 INFO:tasks.workunit.client.1.smithi163.stdout:9/733: readlink d5/de/d30/d3d/d49/d50/lb3 0 2024-02-16T19:24:29.179 INFO:tasks.workunit.client.1.smithi163.stdout:7/543: write d11/f1b [1122114,64081] 0 2024-02-16T19:24:29.179 INFO:tasks.workunit.client.1.smithi163.stdout:7/544: fdatasync d11/d24/d25/f5a 0 2024-02-16T19:24:29.181 INFO:tasks.workunit.client.1.smithi163.stdout:1/517: dwrite d7/d5b/f6a [0,4194304] 0 2024-02-16T19:24:29.181 INFO:tasks.workunit.client.1.smithi163.stdout:7/545: link fc d11/d50/da2/fbe 0 2024-02-16T19:24:29.183 INFO:tasks.workunit.client.1.smithi163.stdout:9/734: getdents d5 0 2024-02-16T19:24:29.187 INFO:tasks.workunit.client.1.smithi163.stdout:9/735: mknod d5/de/d30/d3d/d49/d6d/d97/c101 0 2024-02-16T19:24:29.187 INFO:tasks.workunit.client.1.smithi163.stdout:9/736: readlink d5/l10 0 2024-02-16T19:24:29.189 INFO:tasks.workunit.client.1.smithi163.stdout:9/737: mkdir d5/dc0/d102 0 2024-02-16T19:24:29.190 INFO:tasks.workunit.client.1.smithi163.stdout:9/738: rename d5/de/d16/d46/d9b/cdc to d5/de/d30/c103 0 2024-02-16T19:24:29.190 INFO:tasks.workunit.client.1.smithi163.stdout:9/739: chown d5/de/d30/f7b 127542947 1 2024-02-16T19:24:29.192 INFO:tasks.workunit.client.1.smithi163.stdout:2/678: dwrite d2/d26/f28 [0,4194304] 0 2024-02-16T19:24:29.192 INFO:tasks.workunit.client.1.smithi163.stdout:9/740: rename d5/de/d30/d3d/dde to d5/de/d72/da9/d104 0 2024-02-16T19:24:29.200 INFO:tasks.workunit.client.1.smithi163.stdout:2/679: link d2/d26/f31 d2/d26/d54/d58/d8c/ff1 0 2024-02-16T19:24:29.203 INFO:tasks.workunit.client.0.smithi040.stdout:4/961: rename dd/de/d71/d102 to dd/d58/d5e/dd1/d9d/d123/d12e 0 2024-02-16T19:24:29.206 INFO:tasks.workunit.client.0.smithi040.stdout:4/962: truncate dd/de/d10/f30 3725456 0 2024-02-16T19:24:29.206 INFO:tasks.workunit.client.0.smithi040.stdout:4/963: fsync dd/de/d10/f4a 0 2024-02-16T19:24:29.216 INFO:tasks.workunit.client.1.smithi163.stdout:1/518: dwrite d7/fb [0,4194304] 0 2024-02-16T19:24:29.216 INFO:tasks.workunit.client.1.smithi163.stdout:1/519: truncate d7/d20/d21/f45 1165805 0 2024-02-16T19:24:29.216 INFO:tasks.workunit.client.1.smithi163.stdout:1/520: write d7/d20/d2a/f40 [1302699,8160] 0 2024-02-16T19:24:29.217 INFO:tasks.workunit.client.1.smithi163.stdout:9/741: dwrite d5/de/d30/d8a/fd2 [0,4194304] 0 2024-02-16T19:24:29.217 INFO:tasks.workunit.client.1.smithi163.stdout:9/742: chown d5/de/d30/d43/f60 966 1 2024-02-16T19:24:29.217 INFO:tasks.workunit.client.1.smithi163.stdout:9/743: chown d5/de/d72 3 1 2024-02-16T19:24:29.228 INFO:tasks.workunit.client.1.smithi163.stdout:1/521: unlink d7/d20/d21/c60 0 2024-02-16T19:24:29.230 INFO:tasks.workunit.client.1.smithi163.stdout:1/522: symlink d7/d5b/d63/lb5 0 2024-02-16T19:24:29.230 INFO:tasks.workunit.client.1.smithi163.stdout:9/744: rmdir d5/dc0 39 2024-02-16T19:24:29.232 INFO:tasks.workunit.client.1.smithi163.stdout:1/523: getdents d7/dc/d2d/d32/d87 0 2024-02-16T19:24:29.233 INFO:tasks.workunit.client.0.smithi040.stdout:4/964: dwrite dd/de/d45/f74 [0,4194304] 0 2024-02-16T19:24:29.233 INFO:tasks.workunit.client.1.smithi163.stdout:2/680: dwrite d2/d2a/d39/f42 [0,4194304] 0 2024-02-16T19:24:29.235 INFO:tasks.workunit.client.1.smithi163.stdout:1/524: rename d7/dc/d2d/d32/c7e to d7/d20/d93/cb6 0 2024-02-16T19:24:29.235 INFO:tasks.workunit.client.1.smithi163.stdout:1/525: fdatasync d7/d20/d21/d74/f76 0 2024-02-16T19:24:29.241 INFO:tasks.workunit.client.0.smithi040.stdout:4/965: write dd/de/d45/f73 [2280321,86308] 0 2024-02-16T19:24:29.241 INFO:tasks.workunit.client.0.smithi040.stdout:4/966: chown dd/d58/d5e/dd1/d9d/dd6/ce6 27 1 2024-02-16T19:24:29.244 INFO:tasks.workunit.client.0.smithi040.stdout:4/967: mkdir dd/d58/d5e/dd1/d9d/d12f 0 2024-02-16T19:24:29.245 INFO:tasks.workunit.client.1.smithi163.stdout:2/681: dread d2/d2a/d2d/d73/d94/fbd [0,4194304] 0 2024-02-16T19:24:29.247 INFO:tasks.workunit.client.1.smithi163.stdout:2/682: rename d2/d26/d54/d58/d98/lb0 to d2/ddb/dc8/lf2 0 2024-02-16T19:24:29.248 INFO:tasks.workunit.client.1.smithi163.stdout:2/683: symlink d2/d2a/d2d/d73/d94/dc9/lf3 0 2024-02-16T19:24:29.248 INFO:tasks.workunit.client.1.smithi163.stdout:2/684: chown d2/d2a/d2d/d73/ld1 455074 1 2024-02-16T19:24:29.249 INFO:tasks.workunit.client.1.smithi163.stdout:2/685: rename d2/d26/d97/fb1 to d2/d44/ff4 0 2024-02-16T19:24:29.250 INFO:tasks.workunit.client.1.smithi163.stdout:2/686: rmdir d2/d26/d54/d58/d7d/daf/def 0 2024-02-16T19:24:29.250 INFO:tasks.workunit.client.1.smithi163.stdout:2/687: creat d2/d2a/d2d/d73/d84/ff5 x:0 0 0 2024-02-16T19:24:29.251 INFO:tasks.workunit.client.1.smithi163.stdout:2/688: symlink d2/dce/lf6 0 2024-02-16T19:24:29.253 INFO:tasks.workunit.client.1.smithi163.stdout:9/745: dwrite d5/fbb [0,4194304] 0 2024-02-16T19:24:29.253 INFO:tasks.workunit.client.1.smithi163.stdout:9/746: creat d5/de/d30/d3d/d49/d6d/d97/f105 x:0 0 0 2024-02-16T19:24:29.254 INFO:tasks.workunit.client.1.smithi163.stdout:1/526: dwrite d7/d20/d21/d4e/f8f [0,4194304] 0 2024-02-16T19:24:29.254 INFO:tasks.workunit.client.1.smithi163.stdout:2/689: dread d2/d2a/d39/f3b [0,4194304] 0 2024-02-16T19:24:29.256 INFO:tasks.workunit.client.1.smithi163.stdout:9/747: mkdir d5/de/d16/d46/d9b/d106 0 2024-02-16T19:24:29.256 INFO:tasks.workunit.client.0.smithi040.stdout:4/968: dread dd/de/d10/d36/d9e/d77/f96 [0,4194304] 0 2024-02-16T19:24:29.259 INFO:tasks.workunit.client.1.smithi163.stdout:1/527: mkdir d7/dc/d2d/d32/d87/d97/db7 0 2024-02-16T19:24:29.259 INFO:tasks.workunit.client.1.smithi163.stdout:1/528: chown d7/dc/d2d/d32/d3b/f58 0 1 2024-02-16T19:24:29.261 INFO:tasks.workunit.client.1.smithi163.stdout:2/690: mknod d2/d2a/d9f/cf7 0 2024-02-16T19:24:29.263 INFO:tasks.workunit.client.1.smithi163.stdout:9/748: mkdir d5/de/d107 0 2024-02-16T19:24:29.263 INFO:tasks.workunit.client.1.smithi163.stdout:9/749: write d5/f2b [654956,38341] 0 2024-02-16T19:24:29.264 INFO:tasks.workunit.client.1.smithi163.stdout:1/529: creat d7/d5b/d63/fb8 x:0 0 0 2024-02-16T19:24:29.266 INFO:tasks.workunit.client.1.smithi163.stdout:2/691: mkdir d2/d2a/d9f/df8 0 2024-02-16T19:24:29.266 INFO:tasks.workunit.client.1.smithi163.stdout:2/692: dread - d2/d2a/d2d/d73/d84/ff5 zero size 2024-02-16T19:24:29.267 INFO:tasks.workunit.client.1.smithi163.stdout:1/530: truncate d7/dc/d78/d7c/f3d 1688617 0 2024-02-16T19:24:29.268 INFO:tasks.workunit.client.1.smithi163.stdout:2/693: link d2/d26/d54/d58/fcc d2/d26/d54/d58/d8c/ff9 0 2024-02-16T19:24:29.270 INFO:tasks.workunit.client.1.smithi163.stdout:1/531: rename d7/f57 to d7/dc/d2d/d32/d3b/fb9 0 2024-02-16T19:24:29.270 INFO:tasks.workunit.client.1.smithi163.stdout:1/532: chown d7/dc/d78/d7c/c29 78 1 2024-02-16T19:24:29.270 INFO:tasks.workunit.client.1.smithi163.stdout:1/533: creat d7/dc/d2d/fba x:0 0 0 2024-02-16T19:24:29.272 INFO:tasks.workunit.client.1.smithi163.stdout:1/534: creat d7/d20/d21/fbb x:0 0 0 2024-02-16T19:24:29.276 INFO:tasks.workunit.client.1.smithi163.stdout:9/750: dread d5/f82 [0,4194304] 0 2024-02-16T19:24:29.277 INFO:tasks.workunit.client.1.smithi163.stdout:9/751: mknod d5/de/d30/d8a/da8/c108 0 2024-02-16T19:24:29.277 INFO:tasks.workunit.client.1.smithi163.stdout:9/752: write d5/de/d30/d3d/d49/d6d/fac [258516,61955] 0 2024-02-16T19:24:29.279 INFO:tasks.workunit.client.0.smithi040.stdout:4/969: dwrite dd/dd2/ffb [0,4194304] 0 2024-02-16T19:24:29.282 INFO:tasks.workunit.client.1.smithi163.stdout:2/694: dread d2/f15 [0,4194304] 0 2024-02-16T19:24:29.283 INFO:tasks.workunit.client.1.smithi163.stdout:9/753: creat d5/de/d72/f109 x:0 0 0 2024-02-16T19:24:29.287 INFO:tasks.workunit.client.1.smithi163.stdout:9/754: mkdir d5/de/d30/d8a/d10a 0 2024-02-16T19:24:29.289 INFO:tasks.workunit.client.1.smithi163.stdout:1/535: write d7/f14 [3407068,3951] 0 2024-02-16T19:24:29.290 INFO:tasks.workunit.client.1.smithi163.stdout:9/755: mkdir d5/de/d84/d10b 0 2024-02-16T19:24:29.297 INFO:tasks.workunit.client.1.smithi163.stdout:9/756: dread d5/de/d16/f2a [0,4194304] 0 2024-02-16T19:24:29.297 INFO:tasks.workunit.client.1.smithi163.stdout:9/757: readlink d5/de/d72/le9 0 2024-02-16T19:24:29.297 INFO:tasks.workunit.client.1.smithi163.stdout:9/758: fsync d5/de/d30/d3d/f6f 0 2024-02-16T19:24:29.298 INFO:tasks.workunit.client.1.smithi163.stdout:1/536: dread d7/d5b/d63/f7b [0,4194304] 0 2024-02-16T19:24:29.298 INFO:tasks.workunit.client.1.smithi163.stdout:9/759: unlink d5/de/d30/d3d/d49/d6d/d97/f105 0 2024-02-16T19:24:29.298 INFO:tasks.workunit.client.1.smithi163.stdout:9/760: write d5/de/d16/d46/d9b/fca [772780,66349] 0 2024-02-16T19:24:29.299 INFO:tasks.workunit.client.1.smithi163.stdout:1/537: symlink d7/dc/d2d/d3f/lbc 0 2024-02-16T19:24:29.299 INFO:tasks.workunit.client.1.smithi163.stdout:1/538: fdatasync d7/dc/d2d/d32/f51 0 2024-02-16T19:24:29.313 INFO:tasks.workunit.client.0.smithi040.stdout:4/970: dwrite dd/de/d10/d36/d9e/d77/f7b [0,4194304] 0 2024-02-16T19:24:29.313 INFO:tasks.workunit.client.1.smithi163.stdout:1/539: dread d7/d5b/f6a [0,4194304] 0 2024-02-16T19:24:29.315 INFO:tasks.workunit.client.0.smithi040.stdout:4/971: mkdir dd/de/d10/d36/d9e/d32/d130 0 2024-02-16T19:24:29.315 INFO:tasks.workunit.client.1.smithi163.stdout:1/540: write d7/d20/d21/d4e/f79 [480288,9420] 0 2024-02-16T19:24:29.316 INFO:tasks.workunit.client.1.smithi163.stdout:1/541: chown d7/dc/d2d/d32/f43 1042045 1 2024-02-16T19:24:29.317 INFO:tasks.workunit.client.1.smithi163.stdout:9/761: dwrite d5/de/d30/d31/f54 [0,4194304] 0 2024-02-16T19:24:29.322 INFO:tasks.workunit.client.0.smithi040.stdout:4/972: dread dd/de/d10/d36/f63 [0,4194304] 0 2024-02-16T19:24:29.322 INFO:tasks.workunit.client.0.smithi040.stdout:4/973: fdatasync dd/d58/d8c/fbe 0 2024-02-16T19:24:29.323 INFO:tasks.workunit.client.0.smithi040.stdout:4/974: mkdir dd/d58/d5e/dd1/d9d/d12f/d131 0 2024-02-16T19:24:29.323 INFO:tasks.workunit.client.0.smithi040.stdout:4/975: dread - dd/de/d10/d36/dc3/fdf zero size 2024-02-16T19:24:29.324 INFO:tasks.workunit.client.0.smithi040.stdout:4/976: creat dd/de/db0/f132 x:0 0 0 2024-02-16T19:24:29.325 INFO:tasks.workunit.client.0.smithi040.stdout:4/977: write dd/de/d10/d36/dad/f10b [531712,124696] 0 2024-02-16T19:24:29.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/762: dread d5/de/d30/d3d/f88 [0,4194304] 0 2024-02-16T19:24:29.326 INFO:tasks.workunit.client.1.smithi163.stdout:9/763: truncate d5/fc8 1569536 0 2024-02-16T19:24:29.326 INFO:tasks.workunit.client.0.smithi040.stdout:4/978: creat dd/d58/d5e/dd1/dc6/f133 x:0 0 0 2024-02-16T19:24:29.327 INFO:tasks.workunit.client.1.smithi163.stdout:9/764: symlink d5/de/d16/d46/l10c 0 2024-02-16T19:24:29.327 INFO:tasks.workunit.client.1.smithi163.stdout:9/765: read d5/de/d16/f98 [496833,41477] 0 2024-02-16T19:24:29.328 INFO:tasks.workunit.client.0.smithi040.stdout:4/979: creat dd/de/d45/f134 x:0 0 0 2024-02-16T19:24:29.328 INFO:tasks.workunit.client.0.smithi040.stdout:4/980: read - dd/de/d10/fb9 zero size 2024-02-16T19:24:29.340 INFO:tasks.workunit.client.1.smithi163.stdout:9/766: write d5/de/d30/d3d/d49/d6e/ffe [1689478,14342] 0 2024-02-16T19:24:29.340 INFO:tasks.workunit.client.1.smithi163.stdout:9/767: fsync d5/de/d30/d43/f4b 0 2024-02-16T19:24:29.343 INFO:tasks.workunit.client.1.smithi163.stdout:9/768: unlink d5/de/d16/c57 0 2024-02-16T19:24:29.368 INFO:tasks.workunit.client.1.smithi163.stdout:9/769: dwrite d5/de/d30/d3d/d93/fab [0,4194304] 0 2024-02-16T19:24:29.368 INFO:tasks.workunit.client.1.smithi163.stdout:9/770: chown d5/de/d30/le6 317596 1 2024-02-16T19:24:29.381 INFO:tasks.workunit.client.1.smithi163.stdout:9/771: dwrite d5/de/d16/f98 [0,4194304] 0 2024-02-16T19:24:29.382 INFO:tasks.workunit.client.1.smithi163.stdout:1/542: dwrite d7/d20/d21/f6c [0,4194304] 0 2024-02-16T19:24:29.387 INFO:tasks.workunit.client.1.smithi163.stdout:9/772: getdents d5/de/d30/d3d/d49/df3 0 2024-02-16T19:24:29.388 INFO:tasks.workunit.client.1.smithi163.stdout:1/543: getdents d7/d20/d93 0 2024-02-16T19:24:29.388 INFO:tasks.workunit.client.1.smithi163.stdout:1/544: truncate d7/f7d 193336 0 2024-02-16T19:24:29.389 INFO:tasks.workunit.client.1.smithi163.stdout:9/773: mknod d5/de/d30/d3d/d49/d50/c10d 0 2024-02-16T19:24:29.389 INFO:tasks.workunit.client.1.smithi163.stdout:9/774: truncate d5/fa0 57009 0 2024-02-16T19:24:29.389 INFO:tasks.workunit.client.1.smithi163.stdout:1/545: rmdir d7/dc/d2d/d32 39 2024-02-16T19:24:29.390 INFO:tasks.workunit.client.1.smithi163.stdout:3/525: sync 2024-02-16T19:24:29.390 INFO:tasks.workunit.client.1.smithi163.stdout:8/514: sync 2024-02-16T19:24:29.390 INFO:tasks.workunit.client.1.smithi163.stdout:9/775: truncate d5/de/d30/d8a/da8/fad 18986 0 2024-02-16T19:24:29.391 INFO:tasks.workunit.client.1.smithi163.stdout:1/546: creat d7/dc/d78/fbd x:0 0 0 2024-02-16T19:24:29.392 INFO:tasks.workunit.client.1.smithi163.stdout:3/526: unlink d0/dc/d50/d66/da3/d71/la1 0 2024-02-16T19:24:29.392 INFO:tasks.workunit.client.1.smithi163.stdout:3/527: readlink d0/dc/le 0 2024-02-16T19:24:29.393 INFO:tasks.workunit.client.1.smithi163.stdout:8/515: creat d0/d24/d27/d45/fae x:0 0 0 2024-02-16T19:24:29.394 INFO:tasks.workunit.client.1.smithi163.stdout:8/516: chown d0/l1c 28830823 1 2024-02-16T19:24:29.395 INFO:tasks.workunit.client.1.smithi163.stdout:9/776: symlink d5/de/d30/d8a/d10a/l10e 0 2024-02-16T19:24:29.398 INFO:tasks.workunit.client.1.smithi163.stdout:9/777: rename d5/f19 to d5/de/d84/d10b/f10f 0 2024-02-16T19:24:29.398 INFO:tasks.workunit.client.1.smithi163.stdout:9/778: chown d5/de/d16/d47/f86 541462 1 2024-02-16T19:24:29.398 INFO:tasks.workunit.client.1.smithi163.stdout:9/779: fsync d5/de/d30/d3d/f66 0 2024-02-16T19:24:29.400 INFO:tasks.workunit.client.1.smithi163.stdout:9/780: rename d5/de/d30/d43/f60 to d5/de/d30/d3d/d49/dea/f110 0 2024-02-16T19:24:29.403 INFO:tasks.workunit.client.1.smithi163.stdout:9/781: creat d5/de/d30/d3d/d49/d50/df9/f111 x:0 0 0 2024-02-16T19:24:29.405 INFO:tasks.workunit.client.1.smithi163.stdout:9/782: symlink d5/de/d30/d3d/d49/d6d/d97/l112 0 2024-02-16T19:24:29.406 INFO:tasks.workunit.client.1.smithi163.stdout:9/783: read - d5/de/d16/fbf zero size 2024-02-16T19:24:29.406 INFO:tasks.workunit.client.1.smithi163.stdout:9/784: chown d5/de/d16/lb1 0 1 2024-02-16T19:24:29.413 INFO:tasks.workunit.client.1.smithi163.stdout:9/785: dread d5/fc8 [0,4194304] 0 2024-02-16T19:24:29.416 INFO:tasks.workunit.client.1.smithi163.stdout:1/547: dwrite d7/d5b/d63/f7b [4194304,4194304] 0 2024-02-16T19:24:29.417 INFO:tasks.workunit.client.1.smithi163.stdout:1/548: stat d7/dc/d2d/d32/d87/fa7 0 2024-02-16T19:24:29.417 INFO:tasks.workunit.client.1.smithi163.stdout:9/786: dread d5/de/d16/d46/d9b/fc6 [0,4194304] 0 2024-02-16T19:24:29.418 INFO:tasks.workunit.client.1.smithi163.stdout:1/549: creat d7/dc/d2d/d32/fbe x:0 0 0 2024-02-16T19:24:29.419 INFO:tasks.workunit.client.1.smithi163.stdout:9/787: symlink d5/de/d30/d8a/da8/l113 0 2024-02-16T19:24:29.420 INFO:tasks.workunit.client.1.smithi163.stdout:1/550: symlink d7/dc/d2d/d32/lbf 0 2024-02-16T19:24:29.420 INFO:tasks.workunit.client.1.smithi163.stdout:1/551: mknod d7/dc/cc0 0 2024-02-16T19:24:29.423 INFO:tasks.workunit.client.1.smithi163.stdout:1/552: rename d7/dc/d2d/d32/d87/d97/db7 to d7/dc/dc1 0 2024-02-16T19:24:29.426 INFO:tasks.workunit.client.1.smithi163.stdout:3/528: dwrite d0/dc/d1d/d29/f30 [4194304,4194304] 0 2024-02-16T19:24:29.426 INFO:tasks.workunit.client.1.smithi163.stdout:3/529: read d0/f6d [929712,38131] 0 2024-02-16T19:24:29.426 INFO:tasks.workunit.client.1.smithi163.stdout:3/530: chown d0/dc/d1d/d29/d95/lb3 905721 1 2024-02-16T19:24:29.426 INFO:tasks.workunit.client.1.smithi163.stdout:8/517: dwrite d0/d7/d19/d1f/d32/f59 [0,4194304] 0 2024-02-16T19:24:29.427 INFO:tasks.workunit.client.1.smithi163.stdout:3/531: creat d0/dc/d94/d9e/dac/fbf x:0 0 0 2024-02-16T19:24:29.427 INFO:tasks.workunit.client.1.smithi163.stdout:3/532: creat d0/dc/d94/d9e/dac/fc0 x:0 0 0 2024-02-16T19:24:29.431 INFO:tasks.workunit.client.1.smithi163.stdout:8/518: creat d0/d24/d3c/d3d/faf x:0 0 0 2024-02-16T19:24:29.431 INFO:tasks.workunit.client.1.smithi163.stdout:8/519: fdatasync d0/d7/d55/f80 0 2024-02-16T19:24:29.436 INFO:tasks.workunit.client.1.smithi163.stdout:9/788: dwrite d5/de/d30/d3d/d49/d6d/fac [0,4194304] 0 2024-02-16T19:24:29.438 INFO:tasks.workunit.client.1.smithi163.stdout:3/533: link d0/d83/cb2 d0/dc/d1d/d29/d38/cc1 0 2024-02-16T19:24:29.438 INFO:tasks.workunit.client.1.smithi163.stdout:8/520: rename d0/dc/f16 to d0/d24/d2c/d74/fb0 0 2024-02-16T19:24:29.444 INFO:tasks.workunit.client.1.smithi163.stdout:8/521: unlink d0/d7/c12 0 2024-02-16T19:24:29.445 INFO:tasks.workunit.client.1.smithi163.stdout:3/534: link d0/dc/d50/d66/la9 d0/dc/d1d/d49/d98/lc2 0 2024-02-16T19:24:29.445 INFO:tasks.workunit.client.1.smithi163.stdout:3/535: dread - d0/d3e/f60 zero size 2024-02-16T19:24:29.446 INFO:tasks.workunit.client.1.smithi163.stdout:8/522: mkdir d0/d24/d78/db1 0 2024-02-16T19:24:29.446 INFO:tasks.workunit.client.1.smithi163.stdout:1/553: dwrite d7/d20/d2a/f9f [0,4194304] 0 2024-02-16T19:24:29.448 INFO:tasks.workunit.client.1.smithi163.stdout:3/536: truncate d0/dc/d1d/d29/d38/d46/f90 4178670 0 2024-02-16T19:24:29.450 INFO:tasks.workunit.client.1.smithi163.stdout:8/523: unlink d0/d24/d3c/d3d/faf 0 2024-02-16T19:24:29.452 INFO:tasks.workunit.client.1.smithi163.stdout:1/554: mknod d7/d5b/cc2 0 2024-02-16T19:24:29.452 INFO:tasks.workunit.client.1.smithi163.stdout:1/555: dread - d7/dc/d2d/d3f/fad zero size 2024-02-16T19:24:29.453 INFO:tasks.workunit.client.1.smithi163.stdout:3/537: creat d0/dc/d1c/d91/fc3 x:0 0 0 2024-02-16T19:24:29.454 INFO:tasks.workunit.client.1.smithi163.stdout:1/556: symlink d7/dc/d2d/d90/lc3 0 2024-02-16T19:24:29.454 INFO:tasks.workunit.client.1.smithi163.stdout:1/557: read - d7/dc/d2d/d32/d87/f88 zero size 2024-02-16T19:24:29.455 INFO:tasks.workunit.client.1.smithi163.stdout:3/538: link d0/dc/d1d/d29/d38/c4b d0/d83/cc4 0 2024-02-16T19:24:29.455 INFO:tasks.workunit.client.1.smithi163.stdout:9/789: dwrite d5/de/d30/d3d/d49/d6e/f73 [0,4194304] 0 2024-02-16T19:24:29.456 INFO:tasks.workunit.client.1.smithi163.stdout:1/558: mknod d7/dc/d2d/d32/d87/da6/cc4 0 2024-02-16T19:24:29.458 INFO:tasks.workunit.client.1.smithi163.stdout:3/539: symlink d0/dc/d50/d66/d93/lc5 0 2024-02-16T19:24:29.467 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:29 smithi163 ceph-mon[36452]: pgmap v253: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 244 MiB/s rd, 299 MiB/s wr, 417 op/s 2024-02-16T19:24:29.468 INFO:tasks.workunit.client.1.smithi163.stdout:3/540: rmdir d0/dc/d50/d66/d93 39 2024-02-16T19:24:29.469 INFO:tasks.workunit.client.1.smithi163.stdout:1/559: dread d7/dc/f30 [0,4194304] 0 2024-02-16T19:24:29.470 INFO:tasks.workunit.client.1.smithi163.stdout:3/541: getdents d0/dc/d96/da7 0 2024-02-16T19:24:29.471 INFO:tasks.workunit.client.1.smithi163.stdout:1/560: link d7/dc/d78/c84 d7/dc/d2d/d32/d87/d97/cc5 0 2024-02-16T19:24:29.471 INFO:tasks.workunit.client.1.smithi163.stdout:1/561: chown d7/d20/d2a/l3a 21517945 1 2024-02-16T19:24:29.472 INFO:tasks.workunit.client.1.smithi163.stdout:1/562: dread - d7/dc/d78/d7c/f81 zero size 2024-02-16T19:24:29.472 INFO:tasks.workunit.client.1.smithi163.stdout:1/563: write d7/dc/d2d/d32/f51 [965394,96809] 0 2024-02-16T19:24:29.474 INFO:tasks.workunit.client.1.smithi163.stdout:3/542: dread d0/dc/d94/d9e/fb6 [0,4194304] 0 2024-02-16T19:24:29.474 INFO:tasks.workunit.client.1.smithi163.stdout:3/543: dread - d0/f8e zero size 2024-02-16T19:24:29.474 INFO:tasks.workunit.client.1.smithi163.stdout:3/544: read d0/dc/d1c/d21/f2d [49203,102683] 0 2024-02-16T19:24:29.475 INFO:tasks.workunit.client.1.smithi163.stdout:3/545: dread - d0/dc/f8c zero size 2024-02-16T19:24:29.475 INFO:tasks.workunit.client.1.smithi163.stdout:3/546: write d0/dc/d1c/d32/f36 [683447,58822] 0 2024-02-16T19:24:29.477 INFO:tasks.workunit.client.1.smithi163.stdout:3/547: symlink d0/lc6 0 2024-02-16T19:24:29.480 INFO:tasks.workunit.client.1.smithi163.stdout:1/564: write d7/d20/f99 [2500244,84565] 0 2024-02-16T19:24:29.481 INFO:tasks.workunit.client.1.smithi163.stdout:9/790: dwrite d5/de/d16/d46/d9b/f9d [0,4194304] 0 2024-02-16T19:24:29.482 INFO:tasks.workunit.client.1.smithi163.stdout:3/548: unlink d0/dc/d1c/d32/db8/d99/lb0 0 2024-02-16T19:24:29.483 INFO:tasks.workunit.client.1.smithi163.stdout:8/524: dwrite d0/d7/d19/d1f/d32/f59 [0,4194304] 0 2024-02-16T19:24:29.483 INFO:tasks.workunit.client.1.smithi163.stdout:8/525: creat d0/d7/d19/d1f/d30/fb2 x:0 0 0 2024-02-16T19:24:29.484 INFO:tasks.workunit.client.1.smithi163.stdout:1/565: write d7/dc/d2d/d32/d3b/f58 [902411,50019] 0 2024-02-16T19:24:29.484 INFO:tasks.workunit.client.1.smithi163.stdout:1/566: chown d7/fa 1 1 2024-02-16T19:24:29.484 INFO:tasks.workunit.client.1.smithi163.stdout:1/567: chown d7/dc/c10 88688 1 2024-02-16T19:24:29.486 INFO:tasks.workunit.client.1.smithi163.stdout:3/549: unlink d0/dc/d1c/d32/l61 0 2024-02-16T19:24:29.488 INFO:tasks.workunit.client.1.smithi163.stdout:4/453: sync 2024-02-16T19:24:29.489 INFO:tasks.workunit.client.1.smithi163.stdout:7/546: sync 2024-02-16T19:24:29.489 INFO:tasks.workunit.client.1.smithi163.stdout:6/486: sync 2024-02-16T19:24:29.492 INFO:tasks.workunit.client.1.smithi163.stdout:4/454: symlink d11/d3d/l93 0 2024-02-16T19:24:29.492 INFO:tasks.workunit.client.1.smithi163.stdout:0/797: sync 2024-02-16T19:24:29.492 INFO:tasks.workunit.client.1.smithi163.stdout:5/497: sync 2024-02-16T19:24:29.492 INFO:tasks.workunit.client.1.smithi163.stdout:5/498: chown d0/d1c/f74 0 1 2024-02-16T19:24:29.492 INFO:tasks.workunit.client.1.smithi163.stdout:2/695: sync 2024-02-16T19:24:29.492 INFO:tasks.workunit.client.1.smithi163.stdout:2/696: fdatasync d2/f15 0 2024-02-16T19:24:29.493 INFO:tasks.workunit.client.1.smithi163.stdout:7/547: creat d11/d24/d25/fbf x:0 0 0 2024-02-16T19:24:29.494 INFO:tasks.workunit.client.1.smithi163.stdout:1/568: rmdir d7/d5b/d63 39 2024-02-16T19:24:29.496 INFO:tasks.workunit.client.1.smithi163.stdout:0/798: dread d11/d22/d7f/f83 [0,4194304] 0 2024-02-16T19:24:29.496 INFO:tasks.workunit.client.1.smithi163.stdout:3/550: write d0/f8 [3388116,85756] 0 2024-02-16T19:24:29.496 INFO:tasks.workunit.client.1.smithi163.stdout:4/455: mkdir d11/d18/d1f/d29/d52/d76/d89/d94 0 2024-02-16T19:24:29.497 INFO:tasks.workunit.client.1.smithi163.stdout:4/456: fdatasync d11/d18/d1f/d34/f51 0 2024-02-16T19:24:29.497 INFO:tasks.workunit.client.1.smithi163.stdout:4/457: read - d11/d18/d1f/d23/d3a/d4a/f4e zero size 2024-02-16T19:24:29.497 INFO:tasks.workunit.client.1.smithi163.stdout:4/458: chown d11/d18/d1f 81285 1 2024-02-16T19:24:29.497 INFO:tasks.workunit.client.1.smithi163.stdout:4/459: fsync d11/d18/d1f/f27 0 2024-02-16T19:24:29.498 INFO:tasks.workunit.client.1.smithi163.stdout:5/499: symlink d0/d2/d34/d44/d53/l9e 0 2024-02-16T19:24:29.500 INFO:tasks.workunit.client.1.smithi163.stdout:8/526: dread d0/d24/d2c/f4d [0,4194304] 0 2024-02-16T19:24:29.500 INFO:tasks.workunit.client.1.smithi163.stdout:8/527: fsync d0/d7/d19/d1f/d30/d6a/d7b/f89 0 2024-02-16T19:24:29.501 INFO:tasks.workunit.client.1.smithi163.stdout:2/697: unlink d2/d26/d97/fd6 0 2024-02-16T19:24:29.503 INFO:tasks.workunit.client.1.smithi163.stdout:0/799: getdents d11/d12 0 2024-02-16T19:24:29.504 INFO:tasks.workunit.client.1.smithi163.stdout:0/800: fdatasync d11/d22/d7f/f96 0 2024-02-16T19:24:29.506 INFO:tasks.workunit.client.1.smithi163.stdout:7/548: rmdir d11/d24/d49 39 2024-02-16T19:24:29.506 INFO:tasks.workunit.client.1.smithi163.stdout:4/460: creat d11/d18/d1f/d29/d52/d57/d6c/f95 x:0 0 0 2024-02-16T19:24:29.506 INFO:tasks.workunit.client.1.smithi163.stdout:7/549: write d11/d24/d34/d8d/f92 [861187,81805] 0 2024-02-16T19:24:29.506 INFO:tasks.workunit.client.1.smithi163.stdout:4/461: write d11/d2d/d4f/f8c [220633,120354] 0 2024-02-16T19:24:29.507 INFO:tasks.workunit.client.1.smithi163.stdout:7/550: readlink d11/d24/d34/d8d/lad 0 2024-02-16T19:24:29.507 INFO:tasks.workunit.client.1.smithi163.stdout:7/551: creat d11/d24/d34/fc0 x:0 0 0 2024-02-16T19:24:29.507 INFO:tasks.workunit.client.1.smithi163.stdout:0/801: mkdir d11/d12/d67/d6a/dae/d118 0 2024-02-16T19:24:29.507 INFO:tasks.workunit.client.1.smithi163.stdout:1/569: dread d7/dc/f33 [4194304,4194304] 0 2024-02-16T19:24:29.509 INFO:tasks.workunit.client.1.smithi163.stdout:5/500: getdents d0/d2/d55 0 2024-02-16T19:24:29.509 INFO:tasks.workunit.client.1.smithi163.stdout:4/462: rename d11/d2d/f71 to d11/d18/d1f/d29/d52/d57/d6c/d7c/f96 0 2024-02-16T19:24:29.510 INFO:tasks.workunit.client.1.smithi163.stdout:5/501: mknod d0/d2/d34/c9f 0 2024-02-16T19:24:29.511 INFO:tasks.workunit.client.1.smithi163.stdout:4/463: truncate d11/f2f 713289 0 2024-02-16T19:24:29.518 INFO:tasks.workunit.client.1.smithi163.stdout:5/502: getdents d0/d22/d28/d36/d3c/d4f 0 2024-02-16T19:24:29.522 INFO:tasks.workunit.client.1.smithi163.stdout:0/802: dread d11/d21/fdb [0,4194304] 0 2024-02-16T19:24:29.525 INFO:tasks.workunit.client.1.smithi163.stdout:0/803: rmdir d11/d21/d50/d58/d72/d9e/dbf 39 2024-02-16T19:24:29.525 INFO:tasks.workunit.client.1.smithi163.stdout:6/487: dwrite d0/dd/f50 [4194304,4194304] 0 2024-02-16T19:24:29.529 INFO:tasks.workunit.client.1.smithi163.stdout:0/804: creat d11/d12/d67/d6a/f119 x:0 0 0 2024-02-16T19:24:29.529 INFO:tasks.workunit.client.1.smithi163.stdout:0/805: creat d11/d21/d50/d58/d74/df8/f11a x:0 0 0 2024-02-16T19:24:29.533 INFO:tasks.workunit.client.1.smithi163.stdout:5/503: write d0/d1c/f40 [5867237,100020] 0 2024-02-16T19:24:29.533 INFO:tasks.workunit.client.1.smithi163.stdout:9/791: dwrite d5/de/d30/d3d/d49/d6e/ffe [0,4194304] 0 2024-02-16T19:24:29.533 INFO:tasks.workunit.client.1.smithi163.stdout:0/806: creat d11/d22/f11b x:0 0 0 2024-02-16T19:24:29.534 INFO:tasks.workunit.client.1.smithi163.stdout:6/488: dread d0/d6/df/d20/d40/d1e/f52 [0,4194304] 0 2024-02-16T19:24:29.534 INFO:tasks.workunit.client.1.smithi163.stdout:6/489: chown d0/d6/f31 1791245643 1 2024-02-16T19:24:29.536 INFO:tasks.workunit.client.1.smithi163.stdout:9/792: rename d5/de/d30/d3d/d49/d6d/d97/fe8 to d5/de/df6/f114 0 2024-02-16T19:24:29.537 INFO:tasks.workunit.client.1.smithi163.stdout:5/504: creat d0/d2/d34/d44/d53/d93/fa0 x:0 0 0 2024-02-16T19:24:29.538 INFO:tasks.workunit.client.1.smithi163.stdout:0/807: mkdir d11/d21/d7c/d11c 0 2024-02-16T19:24:29.544 INFO:tasks.workunit.client.1.smithi163.stdout:9/793: read d5/de/d30/d43/d8f/f90 [904259,86472] 0 2024-02-16T19:24:29.544 INFO:tasks.workunit.client.1.smithi163.stdout:9/794: write d5/de/d30/d3d/d49/d6e/f8d [447823,58608] 0 2024-02-16T19:24:29.545 INFO:tasks.workunit.client.1.smithi163.stdout:8/528: dwrite d0/d7/d18/d2e/f5f [0,4194304] 0 2024-02-16T19:24:29.546 INFO:tasks.workunit.client.1.smithi163.stdout:9/795: getdents d5/de 0 2024-02-16T19:24:29.546 INFO:tasks.workunit.client.1.smithi163.stdout:9/796: write d5/de/d30/f36 [4585872,97476] 0 2024-02-16T19:24:29.548 INFO:tasks.workunit.client.1.smithi163.stdout:9/797: write d5/de/d30/d31/d52/fe7 [1714,90339] 0 2024-02-16T19:24:29.548 INFO:tasks.workunit.client.1.smithi163.stdout:8/529: mknod d0/cb3 0 2024-02-16T19:24:29.549 INFO:tasks.workunit.client.1.smithi163.stdout:8/530: mknod d0/d7/d18/cb4 0 2024-02-16T19:24:29.550 INFO:tasks.workunit.client.1.smithi163.stdout:1/570: dwrite d7/d5b/d63/f7b [4194304,4194304] 0 2024-02-16T19:24:29.550 INFO:tasks.workunit.client.1.smithi163.stdout:1/571: chown d7/d20/d2a/f5c 87669 1 2024-02-16T19:24:29.551 INFO:tasks.workunit.client.1.smithi163.stdout:7/552: dwrite d11/d24/d34/fc0 [0,4194304] 0 2024-02-16T19:24:29.553 INFO:tasks.workunit.client.1.smithi163.stdout:1/572: mkdir d7/d20/d21/d4e/dc6 0 2024-02-16T19:24:29.553 INFO:tasks.workunit.client.1.smithi163.stdout:8/531: rename d0/d7/d19/d1f/d30/d48/fa5 to d0/d7/d19/d1f/d30/fb5 0 2024-02-16T19:24:29.554 INFO:tasks.workunit.client.1.smithi163.stdout:1/573: chown d7/d20/f35 148166561 1 2024-02-16T19:24:29.554 INFO:tasks.workunit.client.1.smithi163.stdout:8/532: dread - d0/d24/d3c/f8a zero size 2024-02-16T19:24:29.554 INFO:tasks.workunit.client.1.smithi163.stdout:1/574: write d7/d20/d93/fa9 [244740,22842] 0 2024-02-16T19:24:29.554 INFO:tasks.workunit.client.1.smithi163.stdout:1/575: chown d7/d20/d21/d4e 9 1 2024-02-16T19:24:29.560 INFO:tasks.workunit.client.1.smithi163.stdout:1/576: dread d7/d20/d2a/f40 [0,4194304] 0 2024-02-16T19:24:29.560 INFO:tasks.workunit.client.1.smithi163.stdout:4/464: dwrite d11/d18/d1f/d29/d52/d57/d6c/d7c/f92 [0,4194304] 0 2024-02-16T19:24:29.568 INFO:tasks.workunit.client.1.smithi163.stdout:4/465: rmdir d11/d2d/d4f/d5f/d70/d74 39 2024-02-16T19:24:29.568 INFO:tasks.workunit.client.1.smithi163.stdout:4/466: read - d11/f8a zero size 2024-02-16T19:24:29.571 INFO:tasks.workunit.client.1.smithi163.stdout:4/467: mknod d11/d18/d1f/d34/c97 0 2024-02-16T19:24:29.580 INFO:tasks.workunit.client.1.smithi163.stdout:5/505: dwrite d0/d22/d3b/f66 [0,4194304] 0 2024-02-16T19:24:29.581 INFO:tasks.workunit.client.1.smithi163.stdout:0/808: dwrite d11/d22/fa3 [0,4194304] 0 2024-02-16T19:24:29.584 INFO:tasks.workunit.client.1.smithi163.stdout:5/506: rename d0/f39 to d0/d1c/d60/fa1 0 2024-02-16T19:24:29.584 INFO:tasks.workunit.client.1.smithi163.stdout:4/468: write d11/d18/d1f/d29/f36 [1732035,40740] 0 2024-02-16T19:24:29.585 INFO:tasks.workunit.client.1.smithi163.stdout:0/809: mkdir d11/d11d 0 2024-02-16T19:24:29.586 INFO:tasks.workunit.client.1.smithi163.stdout:0/810: write d11/d22/d7f/d8e/fa4 [5201708,106573] 0 2024-02-16T19:24:29.586 INFO:tasks.workunit.client.1.smithi163.stdout:0/811: readlink d11/d22/l64 0 2024-02-16T19:24:29.586 INFO:tasks.workunit.client.1.smithi163.stdout:0/812: write d11/d21/d7c/f81 [229701,89844] 0 2024-02-16T19:24:29.586 INFO:tasks.workunit.client.1.smithi163.stdout:0/813: chown d11/d21/d7c/fc5 347828261 1 2024-02-16T19:24:29.586 INFO:tasks.workunit.client.1.smithi163.stdout:0/814: write d11/d21/d50/d58/d72/d9e/ffe [768004,80827] 0 2024-02-16T19:24:29.587 INFO:tasks.workunit.client.1.smithi163.stdout:7/553: dwrite d11/d24/d49/f69 [0,4194304] 0 2024-02-16T19:24:29.587 INFO:tasks.workunit.client.1.smithi163.stdout:7/554: fdatasync d11/d24/d49/f51 0 2024-02-16T19:24:29.587 INFO:tasks.workunit.client.1.smithi163.stdout:8/533: dwrite d0/d7/d18/fad [0,4194304] 0 2024-02-16T19:24:29.588 INFO:tasks.workunit.client.1.smithi163.stdout:8/534: fsync d0/d7/d18/d2e/f5d 0 2024-02-16T19:24:29.589 INFO:tasks.workunit.client.1.smithi163.stdout:5/507: rmdir d0/d2/d34/d44/d53/d93/d79 0 2024-02-16T19:24:29.589 INFO:tasks.workunit.client.1.smithi163.stdout:1/577: dwrite d7/dc/d2d/d32/f51 [0,4194304] 0 2024-02-16T19:24:29.597 INFO:tasks.workunit.client.1.smithi163.stdout:5/508: getdents d0/d22 0 2024-02-16T19:24:29.600 INFO:tasks.workunit.client.1.smithi163.stdout:1/578: rename d7/d20/d93/fa9 to d7/dc/d2d/fc7 0 2024-02-16T19:24:29.600 INFO:tasks.workunit.client.1.smithi163.stdout:1/579: dread - d7/d20/f6d zero size 2024-02-16T19:24:29.600 INFO:tasks.workunit.client.1.smithi163.stdout:1/580: write d7/dc/d2d/d32/f51 [4214606,107471] 0 2024-02-16T19:24:29.603 INFO:tasks.workunit.client.1.smithi163.stdout:8/535: getdents d0 0 2024-02-16T19:24:29.605 INFO:tasks.workunit.client.1.smithi163.stdout:5/509: rename d0/d1c/l26 to d0/d2/d55/la2 0 2024-02-16T19:24:29.605 INFO:tasks.workunit.client.1.smithi163.stdout:5/510: dread - d0/d2/d34/d44/f72 zero size 2024-02-16T19:24:29.605 INFO:tasks.workunit.client.1.smithi163.stdout:5/511: fsync d0/d2/d5c/f83 0 2024-02-16T19:24:29.609 INFO:tasks.workunit.client.1.smithi163.stdout:1/581: creat d7/dc/d2d/d3f/fc8 x:0 0 0 2024-02-16T19:24:29.609 INFO:tasks.workunit.client.1.smithi163.stdout:1/582: chown d7/d20/d21/fbb 7478 1 2024-02-16T19:24:29.609 INFO:tasks.workunit.client.1.smithi163.stdout:1/583: fsync d7/dc/d78/fbd 0 2024-02-16T19:24:29.610 INFO:tasks.workunit.client.1.smithi163.stdout:5/512: rmdir d0/d2/d34/d44/d64 39 2024-02-16T19:24:29.610 INFO:tasks.workunit.client.1.smithi163.stdout:5/513: truncate d0/d2/d34/d44/f72 499420 0 2024-02-16T19:24:29.612 INFO:tasks.workunit.client.1.smithi163.stdout:4/469: dwrite d11/d2d/d4f/f64 [0,4194304] 0 2024-02-16T19:24:29.613 INFO:tasks.workunit.client.1.smithi163.stdout:1/584: creat d7/dc/d2d/d32/d87/fc9 x:0 0 0 2024-02-16T19:24:29.613 INFO:tasks.workunit.client.1.smithi163.stdout:1/585: write d7/dc/d2d/d32/d3b/fb9 [60275,70555] 0 2024-02-16T19:24:29.614 INFO:tasks.workunit.client.1.smithi163.stdout:7/555: dwrite d11/d50/d5d/d8b/d67/f85 [0,4194304] 0 2024-02-16T19:24:29.615 INFO:tasks.workunit.client.1.smithi163.stdout:4/470: unlink d11/d18/d1f/d34/l8e 0 2024-02-16T19:24:29.615 INFO:tasks.workunit.client.1.smithi163.stdout:4/471: chown d11/d2d/d4f/f64 24 1 2024-02-16T19:24:29.617 INFO:tasks.workunit.client.1.smithi163.stdout:1/586: dread d7/d20/d21/d4e/f9e [0,4194304] 0 2024-02-16T19:24:29.617 INFO:tasks.workunit.client.1.smithi163.stdout:1/587: write d7/d5b/fa4 [996855,18203] 0 2024-02-16T19:24:29.618 INFO:tasks.workunit.client.1.smithi163.stdout:1/588: creat d7/dc/fca x:0 0 0 2024-02-16T19:24:29.618 INFO:tasks.workunit.client.1.smithi163.stdout:1/589: creat d7/d20/d96/fcb x:0 0 0 2024-02-16T19:24:29.619 INFO:tasks.workunit.client.1.smithi163.stdout:5/514: dread d0/d2/f21 [0,4194304] 0 2024-02-16T19:24:29.619 INFO:tasks.workunit.client.1.smithi163.stdout:5/515: chown d0/d2/l37 934142 1 2024-02-16T19:24:29.620 INFO:tasks.workunit.client.1.smithi163.stdout:1/590: symlink d7/d20/d21/lcc 0 2024-02-16T19:24:29.621 INFO:tasks.workunit.client.1.smithi163.stdout:5/516: mkdir d0/d1c/d60/da3 0 2024-02-16T19:24:29.624 INFO:tasks.workunit.client.1.smithi163.stdout:1/591: link d7/d20/d21/d74/dab/lb4 d7/dc/d2d/d32/d87/lcd 0 2024-02-16T19:24:29.625 INFO:tasks.workunit.client.1.smithi163.stdout:1/592: getdents d7/dc/dc1 0 2024-02-16T19:24:29.625 INFO:tasks.workunit.client.1.smithi163.stdout:1/593: fdatasync d7/d20/d96/fa1 0 2024-02-16T19:24:29.625 INFO:tasks.workunit.client.1.smithi163.stdout:1/594: chown d7/dc/d2d/d3f 339 1 2024-02-16T19:24:29.625 INFO:tasks.workunit.client.1.smithi163.stdout:1/595: read - d7/d5b/faa zero size 2024-02-16T19:24:29.625 INFO:tasks.workunit.client.1.smithi163.stdout:1/596: creat d7/d20/d96/fce x:0 0 0 2024-02-16T19:24:29.625 INFO:tasks.workunit.client.1.smithi163.stdout:1/597: truncate d7/dc/d2d/d32/d87/fc9 898648 0 2024-02-16T19:24:29.625 INFO:tasks.workunit.client.1.smithi163.stdout:5/517: mknod d0/d22/d28/d36/ca4 0 2024-02-16T19:24:29.627 INFO:tasks.workunit.client.1.smithi163.stdout:1/598: unlink d7/dc/cc0 0 2024-02-16T19:24:29.628 INFO:tasks.workunit.client.1.smithi163.stdout:1/599: fsync d7/d20/d21/d4e/f79 0 2024-02-16T19:24:29.628 INFO:tasks.workunit.client.1.smithi163.stdout:5/518: mkdir d0/d1c/d5a/da5 0 2024-02-16T19:24:29.630 INFO:tasks.workunit.client.1.smithi163.stdout:1/600: link d7/dc/d2d/la3 d7/d20/d96/lcf 0 2024-02-16T19:24:29.630 INFO:tasks.workunit.client.1.smithi163.stdout:5/519: creat d0/d22/d28/d36/fa6 x:0 0 0 2024-02-16T19:24:29.631 INFO:tasks.workunit.client.1.smithi163.stdout:1/601: mknod d7/dc/d2d/d3f/cd0 0 2024-02-16T19:24:29.633 INFO:tasks.workunit.client.1.smithi163.stdout:1/602: mknod d7/d20/d21/d74/dab/cd1 0 2024-02-16T19:24:29.640 INFO:tasks.workunit.client.1.smithi163.stdout:8/536: dwrite d0/d7/d18/f37 [0,4194304] 0 2024-02-16T19:24:29.641 INFO:tasks.workunit.client.1.smithi163.stdout:8/537: rename d0/d7/d19/c58 to d0/d7/d19/d1f/d30/d6a/cb6 0 2024-02-16T19:24:29.641 INFO:tasks.workunit.client.1.smithi163.stdout:8/538: write d0/d24/f60 [6180564,112495] 0 2024-02-16T19:24:29.642 INFO:tasks.workunit.client.1.smithi163.stdout:8/539: readlink d0/d7/d18/l77 0 2024-02-16T19:24:29.643 INFO:tasks.workunit.client.1.smithi163.stdout:7/556: dwrite d11/d24/d49/fa5 [0,4194304] 0 2024-02-16T19:24:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:29 smithi040 ceph-mon[29201]: pgmap v253: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 244 MiB/s rd, 299 MiB/s wr, 417 op/s 2024-02-16T19:24:29.648 INFO:tasks.workunit.client.1.smithi163.stdout:7/557: mkdir d11/d24/d25/d4b/d66/dc1 0 2024-02-16T19:24:29.650 INFO:tasks.workunit.client.1.smithi163.stdout:7/558: symlink d11/d24/d25/d4b/d66/lc2 0 2024-02-16T19:24:29.650 INFO:tasks.workunit.client.1.smithi163.stdout:7/559: creat d11/d26/fc3 x:0 0 0 2024-02-16T19:24:29.650 INFO:tasks.workunit.client.1.smithi163.stdout:7/560: chown d11/d50/d5d/d8b/d67/c8a 1 1 2024-02-16T19:24:29.653 INFO:tasks.workunit.client.1.smithi163.stdout:7/561: unlink d11/f44 0 2024-02-16T19:24:29.654 INFO:tasks.workunit.client.1.smithi163.stdout:1/603: dwrite d7/dc/d2d/d32/f43 [0,4194304] 0 2024-02-16T19:24:29.654 INFO:tasks.workunit.client.1.smithi163.stdout:1/604: write d7/d20/f35 [3819532,90843] 0 2024-02-16T19:24:29.654 INFO:tasks.workunit.client.1.smithi163.stdout:7/562: mkdir d11/d24/d49/dc4 0 2024-02-16T19:24:29.654 INFO:tasks.workunit.client.1.smithi163.stdout:7/563: read - d11/d24/d34/fab zero size 2024-02-16T19:24:29.655 INFO:tasks.workunit.client.1.smithi163.stdout:7/564: symlink d11/d50/d5d/d8b/lc5 0 2024-02-16T19:24:29.655 INFO:tasks.workunit.client.1.smithi163.stdout:7/565: chown d11/d50/d5d/d8b/c6c 859 1 2024-02-16T19:24:29.657 INFO:tasks.workunit.client.1.smithi163.stdout:7/566: symlink d11/d26/lc6 0 2024-02-16T19:24:29.658 INFO:tasks.workunit.client.1.smithi163.stdout:8/540: dwrite d0/d24/d27/d45/f4a [4194304,4194304] 0 2024-02-16T19:24:29.658 INFO:tasks.workunit.client.1.smithi163.stdout:8/541: chown d0/d7/d18/d53/f9d 374 1 2024-02-16T19:24:29.662 INFO:tasks.workunit.client.1.smithi163.stdout:8/542: dread d0/d7/d18/d2e/f7f [0,4194304] 0 2024-02-16T19:24:29.662 INFO:tasks.workunit.client.1.smithi163.stdout:8/543: write d0/d7/d18/d2e/f8f [1337465,62426] 0 2024-02-16T19:24:29.666 INFO:tasks.workunit.client.1.smithi163.stdout:5/520: rmdir d0/d22/d28/d36 39 2024-02-16T19:24:29.666 INFO:tasks.workunit.client.1.smithi163.stdout:5/521: chown d0/d22/d28/d36/d3c/d4f/f50 128947 1 2024-02-16T19:24:29.669 INFO:tasks.workunit.client.1.smithi163.stdout:5/522: mknod d0/d22/ca7 0 2024-02-16T19:24:29.669 INFO:tasks.workunit.client.1.smithi163.stdout:5/523: chown d0/d22/d28/d36/d3c/d4f/f62 0 1 2024-02-16T19:24:29.671 INFO:tasks.workunit.client.1.smithi163.stdout:5/524: mkdir d0/d2/d34/d44/d53/d93/da8 0 2024-02-16T19:24:29.672 INFO:tasks.workunit.client.1.smithi163.stdout:5/525: creat d0/d1c/fa9 x:0 0 0 2024-02-16T19:24:29.682 INFO:tasks.workunit.client.1.smithi163.stdout:5/526: dread d0/d2/d34/d44/f51 [0,4194304] 0 2024-02-16T19:24:29.683 INFO:tasks.workunit.client.1.smithi163.stdout:5/527: creat d0/d22/d28/d36/faa x:0 0 0 2024-02-16T19:24:29.685 INFO:tasks.workunit.client.1.smithi163.stdout:5/528: rename d0/d2/d31/f9d to d0/d22/d3b/d8d/fab 0 2024-02-16T19:24:29.685 INFO:tasks.workunit.client.1.smithi163.stdout:5/529: fsync d0/d22/d28/d36/d3c/d4f/f7b 0 2024-02-16T19:24:29.686 INFO:tasks.workunit.client.1.smithi163.stdout:5/530: symlink d0/d22/d28/d36/d3c/d4f/lac 0 2024-02-16T19:24:29.686 INFO:tasks.workunit.client.1.smithi163.stdout:5/531: creat d0/d22/d28/fad x:0 0 0 2024-02-16T19:24:29.686 INFO:tasks.workunit.client.1.smithi163.stdout:5/532: chown d0/d22/f33 0 1 2024-02-16T19:24:29.687 INFO:tasks.workunit.client.1.smithi163.stdout:5/533: creat d0/d2/d55/fae x:0 0 0 2024-02-16T19:24:29.705 INFO:tasks.workunit.client.1.smithi163.stdout:5/534: dwrite d0/d1c/fa9 [0,4194304] 0 2024-02-16T19:24:29.706 INFO:tasks.workunit.client.1.smithi163.stdout:5/535: readlink d0/d1c/l73 0 2024-02-16T19:24:29.707 INFO:tasks.workunit.client.1.smithi163.stdout:5/536: symlink d0/d22/d28/laf 0 2024-02-16T19:24:29.734 INFO:tasks.workunit.client.1.smithi163.stdout:8/544: dwrite d0/d7/d18/d2e/f5d [0,4194304] 0 2024-02-16T19:24:29.735 INFO:tasks.workunit.client.1.smithi163.stdout:5/537: dwrite d0/d1c/d5a/f96 [0,4194304] 0 2024-02-16T19:24:29.737 INFO:tasks.workunit.client.1.smithi163.stdout:8/545: mknod d0/d7/d55/cb7 0 2024-02-16T19:24:29.737 INFO:tasks.workunit.client.1.smithi163.stdout:8/546: read d0/d7/d19/d1f/d32/d70/f8b [365822,115215] 0 2024-02-16T19:24:29.737 INFO:tasks.workunit.client.1.smithi163.stdout:8/547: readlink d0/l28 0 2024-02-16T19:24:29.740 INFO:tasks.workunit.client.1.smithi163.stdout:5/538: creat d0/d2/d5c/d99/fb0 x:0 0 0 2024-02-16T19:24:29.741 INFO:tasks.workunit.client.1.smithi163.stdout:5/539: mkdir d0/d2/db1 0 2024-02-16T19:24:29.742 INFO:tasks.workunit.client.1.smithi163.stdout:5/540: getdents d0/d22/d28/d36/d3c 0 2024-02-16T19:24:29.742 INFO:tasks.workunit.client.1.smithi163.stdout:5/541: creat d0/d22/d3b/fb2 x:0 0 0 2024-02-16T19:24:29.742 INFO:tasks.workunit.client.1.smithi163.stdout:5/542: creat d0/d2/d31/fb3 x:0 0 0 2024-02-16T19:24:29.746 INFO:tasks.workunit.client.1.smithi163.stdout:5/543: mknod d0/d2/db1/cb4 0 2024-02-16T19:24:29.759 INFO:tasks.workunit.client.1.smithi163.stdout:8/548: dwrite d0/d7/d19/d1f/d30/f92 [0,4194304] 0 2024-02-16T19:24:29.759 INFO:tasks.workunit.client.1.smithi163.stdout:5/544: dread d0/d2/d34/d44/f7f [0,4194304] 0 2024-02-16T19:24:29.759 INFO:tasks.workunit.client.1.smithi163.stdout:5/545: creat d0/d1c/d5a/fb5 x:0 0 0 2024-02-16T19:24:29.761 INFO:tasks.workunit.client.1.smithi163.stdout:8/549: mkdir d0/d7/d19/d1f/d30/d6a/db8 0 2024-02-16T19:24:29.761 INFO:tasks.workunit.client.1.smithi163.stdout:8/550: dread - d0/d24/d2c/f90 zero size 2024-02-16T19:24:29.762 INFO:tasks.workunit.client.1.smithi163.stdout:5/546: mknod d0/d2/d31/cb6 0 2024-02-16T19:24:29.763 INFO:tasks.workunit.client.1.smithi163.stdout:8/551: creat d0/d24/d78/fb9 x:0 0 0 2024-02-16T19:24:29.763 INFO:tasks.workunit.client.1.smithi163.stdout:5/547: write d0/d22/d28/d36/d3c/f54 [72256,58258] 0 2024-02-16T19:24:29.763 INFO:tasks.workunit.client.1.smithi163.stdout:8/552: creat d0/d7/d18/d53/fba x:0 0 0 2024-02-16T19:24:29.764 INFO:tasks.workunit.client.1.smithi163.stdout:8/553: write d0/d7/d19/f2f [736277,72383] 0 2024-02-16T19:24:29.765 INFO:tasks.workunit.client.1.smithi163.stdout:8/554: unlink d0/d7/d19/l57 0 2024-02-16T19:24:29.765 INFO:tasks.workunit.client.1.smithi163.stdout:8/555: write d0/dc/f7d [739347,57366] 0 2024-02-16T19:24:29.772 INFO:tasks.workunit.client.1.smithi163.stdout:5/548: write d0/d1c/d5a/f96 [1944441,82683] 0 2024-02-16T19:24:29.774 INFO:tasks.workunit.client.1.smithi163.stdout:5/549: getdents d0/d22/d28 0 2024-02-16T19:24:29.777 INFO:tasks.workunit.client.1.smithi163.stdout:8/556: dread d0/d24/d2c/f4d [0,4194304] 0 2024-02-16T19:24:29.779 INFO:tasks.workunit.client.1.smithi163.stdout:8/557: creat d0/d24/fbb x:0 0 0 2024-02-16T19:24:29.787 INFO:tasks.workunit.client.1.smithi163.stdout:8/558: write d0/dc/f6c [1216853,26811] 0 2024-02-16T19:24:29.804 INFO:tasks.workunit.client.1.smithi163.stdout:5/550: dwrite d0/d2/d34/f4b [0,4194304] 0 2024-02-16T19:24:29.805 INFO:tasks.workunit.client.1.smithi163.stdout:5/551: mkdir d0/d22/d28/d36/db7 0 2024-02-16T19:24:29.824 INFO:tasks.workunit.client.1.smithi163.stdout:5/552: dwrite d0/d22/d3b/d8d/fab [0,4194304] 0 2024-02-16T19:24:29.827 INFO:tasks.workunit.client.1.smithi163.stdout:5/553: link d0/d22/f35 d0/d22/d28/d36/db7/fb8 0 2024-02-16T19:24:29.828 INFO:tasks.workunit.client.1.smithi163.stdout:5/554: read d0/d2/d34/d44/d53/f68 [432084,67322] 0 2024-02-16T19:24:29.829 INFO:tasks.workunit.client.1.smithi163.stdout:5/555: creat d0/d22/d3b/d8d/fb9 x:0 0 0 2024-02-16T19:24:29.830 INFO:tasks.workunit.client.1.smithi163.stdout:5/556: symlink d0/d22/d28/d36/db7/lba 0 2024-02-16T19:24:29.831 INFO:tasks.workunit.client.1.smithi163.stdout:5/557: truncate d0/d22/f3a 1319605 0 2024-02-16T19:24:29.831 INFO:tasks.workunit.client.1.smithi163.stdout:5/558: creat d0/d2/d5c/fbb x:0 0 0 2024-02-16T19:24:29.833 INFO:tasks.workunit.client.1.smithi163.stdout:5/559: dread d0/d22/d28/d36/d3c/d4f/f7b [0,4194304] 0 2024-02-16T19:24:29.833 INFO:tasks.workunit.client.1.smithi163.stdout:5/560: read d0/d2/f57 [3525030,1623] 0 2024-02-16T19:24:29.835 INFO:tasks.workunit.client.1.smithi163.stdout:5/561: symlink d0/d2/d5c/lbc 0 2024-02-16T19:24:29.835 INFO:tasks.workunit.client.1.smithi163.stdout:5/562: chown d0/d2/d34/f6e 18442 1 2024-02-16T19:24:29.835 INFO:tasks.workunit.client.1.smithi163.stdout:5/563: write d0/d2/f30 [3688304,1757] 0 2024-02-16T19:24:29.835 INFO:tasks.workunit.client.1.smithi163.stdout:5/564: fsync d0/f4d 0 2024-02-16T19:24:29.835 INFO:tasks.workunit.client.1.smithi163.stdout:5/565: readlink d0/d22/d28/d36/db7/lba 0 2024-02-16T19:24:29.835 INFO:tasks.workunit.client.1.smithi163.stdout:5/566: fdatasync d0/d2/d5c/f8c 0 2024-02-16T19:24:29.837 INFO:tasks.workunit.client.1.smithi163.stdout:5/567: unlink d0/d2/c27 0 2024-02-16T19:24:29.837 INFO:tasks.workunit.client.1.smithi163.stdout:5/568: readlink d0/d2/d34/d44/d64/l77 0 2024-02-16T19:24:29.837 INFO:tasks.workunit.client.1.smithi163.stdout:5/569: chown d0/d2/d5c/f8c 147918 1 2024-02-16T19:24:29.854 INFO:tasks.workunit.client.1.smithi163.stdout:5/570: dwrite d0/d1c/f40 [4194304,4194304] 0 2024-02-16T19:24:29.858 INFO:tasks.workunit.client.1.smithi163.stdout:5/571: dread d0/d22/d28/d36/f6b [0,4194304] 0 2024-02-16T19:24:29.858 INFO:tasks.workunit.client.1.smithi163.stdout:5/572: readlink d0/l19 0 2024-02-16T19:24:29.858 INFO:tasks.workunit.client.1.smithi163.stdout:5/573: dread - d0/d2/d5c/f83 zero size 2024-02-16T19:24:29.858 INFO:tasks.workunit.client.1.smithi163.stdout:5/574: chown d0/d22/d28/laf 7553632 1 2024-02-16T19:24:29.858 INFO:tasks.workunit.client.1.smithi163.stdout:5/575: symlink d0/d2/d34/lbd 0 2024-02-16T19:24:29.859 INFO:tasks.workunit.client.1.smithi163.stdout:5/576: unlink d0/d2/fb 0 2024-02-16T19:24:29.860 INFO:tasks.workunit.client.1.smithi163.stdout:5/577: chown d0/d2/d5c/f75 2077065 1 2024-02-16T19:24:29.860 INFO:tasks.workunit.client.1.smithi163.stdout:5/578: mkdir d0/d1c/dbe 0 2024-02-16T19:24:29.860 INFO:tasks.workunit.client.1.smithi163.stdout:5/579: chown d0/d2/c9a 65273 1 2024-02-16T19:24:29.861 INFO:tasks.workunit.client.1.smithi163.stdout:5/580: symlink d0/d22/d28/d36/lbf 0 2024-02-16T19:24:29.861 INFO:tasks.workunit.client.1.smithi163.stdout:5/581: stat d0/d22/f33 0 2024-02-16T19:24:29.861 INFO:tasks.workunit.client.1.smithi163.stdout:5/582: write d0/d22/d3b/f42 [1296307,18165] 0 2024-02-16T19:24:29.863 INFO:tasks.workunit.client.1.smithi163.stdout:5/583: rename d0/d2/d5c/d99 to d0/d22/d3b/dc0 0 2024-02-16T19:24:29.865 INFO:tasks.workunit.client.1.smithi163.stdout:5/584: rmdir d0/d2/d31 39 2024-02-16T19:24:29.901 INFO:tasks.workunit.client.1.smithi163.stdout:3/551: sync 2024-02-16T19:24:29.902 INFO:tasks.workunit.client.1.smithi163.stdout:3/552: creat d0/dc/d1c/d32/db8/fc7 x:0 0 0 2024-02-16T19:24:29.903 INFO:tasks.workunit.client.1.smithi163.stdout:3/553: symlink d0/d83/da5/lc8 0 2024-02-16T19:24:29.920 INFO:tasks.workunit.client.1.smithi163.stdout:3/554: dwrite d0/d3e/f78 [0,4194304] 0 2024-02-16T19:24:29.931 INFO:tasks.workunit.client.1.smithi163.stdout:3/555: write d0/dc/d1d/d49/f59 [6436273,46621] 0 2024-02-16T19:24:29.932 INFO:tasks.workunit.client.1.smithi163.stdout:3/556: creat d0/dc/d1d/d49/fc9 x:0 0 0 2024-02-16T19:24:29.932 INFO:tasks.workunit.client.1.smithi163.stdout:3/557: truncate d0/dc/d50/d66/da3/d71/f89 558022 0 2024-02-16T19:24:29.933 INFO:tasks.workunit.client.1.smithi163.stdout:3/558: mknod d0/dc/cca 0 2024-02-16T19:24:29.934 INFO:tasks.workunit.client.1.smithi163.stdout:3/559: symlink d0/dc/d1c/d32/lcb 0 2024-02-16T19:24:29.950 INFO:tasks.workunit.client.1.smithi163.stdout:3/560: dwrite d0/dc/d1d/d29/f72 [0,4194304] 0 2024-02-16T19:24:29.950 INFO:tasks.workunit.client.1.smithi163.stdout:3/561: write d0/dc/d1c/d32/d7f/fbd [435746,1343] 0 2024-02-16T19:24:29.954 INFO:tasks.workunit.client.1.smithi163.stdout:3/562: creat d0/dc/d1d/d49/d98/fcc x:0 0 0 2024-02-16T19:24:29.954 INFO:tasks.workunit.client.1.smithi163.stdout:3/563: truncate d0/f6d 1100567 0 2024-02-16T19:24:29.969 INFO:tasks.workunit.client.1.smithi163.stdout:3/564: dwrite d0/dc/d1d/d29/d38/d46/fa6 [0,4194304] 0 2024-02-16T19:24:29.970 INFO:tasks.workunit.client.1.smithi163.stdout:3/565: symlink d0/dc/d1c/lcd 0 2024-02-16T19:24:29.971 INFO:tasks.workunit.client.1.smithi163.stdout:3/566: rmdir d0/dc/d50/d66/da3 39 2024-02-16T19:24:29.971 INFO:tasks.workunit.client.1.smithi163.stdout:3/567: chown d0/dc/d1d/d29/d95/lb3 12171626 1 2024-02-16T19:24:29.971 INFO:tasks.workunit.client.1.smithi163.stdout:2/698: sync 2024-02-16T19:24:29.971 INFO:tasks.workunit.client.1.smithi163.stdout:9/798: sync 2024-02-16T19:24:29.971 INFO:tasks.workunit.client.1.smithi163.stdout:9/799: read d5/de/d30/d3d/d49/d6e/f92 [170067,68653] 0 2024-02-16T19:24:29.972 INFO:tasks.workunit.client.1.smithi163.stdout:7/567: sync 2024-02-16T19:24:29.972 INFO:tasks.workunit.client.1.smithi163.stdout:6/490: sync 2024-02-16T19:24:29.972 INFO:tasks.workunit.client.1.smithi163.stdout:1/605: sync 2024-02-16T19:24:29.973 INFO:tasks.workunit.client.1.smithi163.stdout:0/815: sync 2024-02-16T19:24:29.973 INFO:tasks.workunit.client.1.smithi163.stdout:4/472: sync 2024-02-16T19:24:29.973 INFO:tasks.workunit.client.1.smithi163.stdout:4/473: write d11/f26 [468394,93602] 0 2024-02-16T19:24:29.974 INFO:tasks.workunit.client.1.smithi163.stdout:4/474: stat d11/d18/d1f/d34/c67 0 2024-02-16T19:24:29.974 INFO:tasks.workunit.client.1.smithi163.stdout:2/699: symlink d2/d26/d54/d58/d98/lfa 0 2024-02-16T19:24:29.974 INFO:tasks.workunit.client.1.smithi163.stdout:9/800: truncate d5/dc0/fcc 91698 0 2024-02-16T19:24:29.974 INFO:tasks.workunit.client.1.smithi163.stdout:9/801: chown d5/de/d30/d3d/d93/la4 755426924 1 2024-02-16T19:24:29.974 INFO:tasks.workunit.client.1.smithi163.stdout:1/606: dread d7/d5b/fa4 [0,4194304] 0 2024-02-16T19:24:29.974 INFO:tasks.workunit.client.1.smithi163.stdout:7/568: unlink d11/d50/d5d/d8b/cb5 0 2024-02-16T19:24:29.975 INFO:tasks.workunit.client.1.smithi163.stdout:6/491: creat d0/dd/d7f/d15/d1c/f99 x:0 0 0 2024-02-16T19:24:29.975 INFO:tasks.workunit.client.1.smithi163.stdout:6/492: truncate d0/d6/df/d20/d40/d53/f72 1380941 0 2024-02-16T19:24:29.977 INFO:tasks.workunit.client.1.smithi163.stdout:4/475: creat d11/d18/d66/f98 x:0 0 0 2024-02-16T19:24:29.977 INFO:tasks.workunit.client.1.smithi163.stdout:0/816: rename d11/d12/f6c to d11/d22/d3e/f11e 0 2024-02-16T19:24:29.978 INFO:tasks.workunit.client.1.smithi163.stdout:2/700: mkdir d2/d26/d54/dfb 0 2024-02-16T19:24:29.978 INFO:tasks.workunit.client.1.smithi163.stdout:2/701: write d2/d2a/d39/d66/d6c/fa0 [4477631,39358] 0 2024-02-16T19:24:29.978 INFO:tasks.workunit.client.1.smithi163.stdout:2/702: dread - d2/f9d zero size 2024-02-16T19:24:29.978 INFO:tasks.workunit.client.1.smithi163.stdout:9/802: rmdir d5/de/d30/d43 39 2024-02-16T19:24:29.978 INFO:tasks.workunit.client.1.smithi163.stdout:9/803: chown d5/de/d30/f36 1054 1 2024-02-16T19:24:29.980 INFO:tasks.workunit.client.1.smithi163.stdout:4/476: creat d11/d2d/f99 x:0 0 0 2024-02-16T19:24:29.981 INFO:tasks.workunit.client.1.smithi163.stdout:2/703: creat d2/dce/ffc x:0 0 0 2024-02-16T19:24:29.982 INFO:tasks.workunit.client.1.smithi163.stdout:9/804: truncate d5/de/d30/f36 3264612 0 2024-02-16T19:24:29.985 INFO:tasks.workunit.client.1.smithi163.stdout:2/704: rename d2/d2a/d39/d3a/l7e to d2/lfd 0 2024-02-16T19:24:29.985 INFO:tasks.workunit.client.1.smithi163.stdout:2/705: readlink d2/d26/d54/l67 0 2024-02-16T19:24:29.987 INFO:tasks.workunit.client.1.smithi163.stdout:9/805: unlink d5/de/d30/d3d/d93/ld1 0 2024-02-16T19:24:29.988 INFO:tasks.workunit.client.1.smithi163.stdout:1/607: dread d7/d20/d21/f55 [0,4194304] 0 2024-02-16T19:24:29.988 INFO:tasks.workunit.client.1.smithi163.stdout:9/806: mknod d5/de/d72/da9/d104/c115 0 2024-02-16T19:24:29.991 INFO:tasks.workunit.client.1.smithi163.stdout:9/807: unlink d5/f2b 0 2024-02-16T19:24:29.991 INFO:tasks.workunit.client.1.smithi163.stdout:0/817: write d11/d21/d50/d58/f92 [3154791,65405] 0 2024-02-16T19:24:29.991 INFO:tasks.workunit.client.1.smithi163.stdout:9/808: getdents d5/de/df6 0 2024-02-16T19:24:29.992 INFO:tasks.workunit.client.1.smithi163.stdout:9/809: write d5/de/d30/d3d/d49/d50/f63 [3656786,99731] 0 2024-02-16T19:24:29.992 INFO:tasks.workunit.client.1.smithi163.stdout:6/493: dread d0/dd/d7f/d2e/f4f [0,4194304] 0 2024-02-16T19:24:29.993 INFO:tasks.workunit.client.1.smithi163.stdout:0/818: mkdir d11/d12/d11f 0 2024-02-16T19:24:29.993 INFO:tasks.workunit.client.1.smithi163.stdout:0/819: write d11/d21/f47 [230414,44998] 0 2024-02-16T19:24:29.993 INFO:tasks.workunit.client.1.smithi163.stdout:9/810: mknod d5/de/df6/c116 0 2024-02-16T19:24:29.994 INFO:tasks.workunit.client.1.smithi163.stdout:4/477: dread d11/d18/d1f/d29/d52/f6a [0,4194304] 0 2024-02-16T19:24:29.994 INFO:tasks.workunit.client.1.smithi163.stdout:4/478: truncate d11/d18/d1f/d29/d52/f80 716475 0 2024-02-16T19:24:29.994 INFO:tasks.workunit.client.1.smithi163.stdout:4/479: write d11/f91 [290369,93648] 0 2024-02-16T19:24:29.994 INFO:tasks.workunit.client.1.smithi163.stdout:6/494: mkdir d0/dd/d7f/d81/d9a 0 2024-02-16T19:24:29.994 INFO:tasks.workunit.client.1.smithi163.stdout:6/495: read d0/d6/f8 [669339,82225] 0 2024-02-16T19:24:29.996 INFO:tasks.workunit.client.1.smithi163.stdout:3/568: dwrite d0/dc/d1c/d32/f36 [0,4194304] 0 2024-02-16T19:24:29.996 INFO:tasks.workunit.client.1.smithi163.stdout:9/811: symlink d5/de/d16/d46/d9b/d106/l117 0 2024-02-16T19:24:29.996 INFO:tasks.workunit.client.1.smithi163.stdout:9/812: stat d5/de/d16/d47/c5a 0 2024-02-16T19:24:29.996 INFO:tasks.workunit.client.1.smithi163.stdout:3/569: stat d0/dc/d50/d66/d80/fa0 0 2024-02-16T19:24:29.996 INFO:tasks.workunit.client.1.smithi163.stdout:4/480: rename d11/l22 to d11/d18/d1f/d29/d52/d57/d6c/d7c/l9a 0 2024-02-16T19:24:30.000 INFO:tasks.workunit.client.1.smithi163.stdout:9/813: creat d5/de/d30/d3d/d49/d6d/f118 x:0 0 0 2024-02-16T19:24:30.001 INFO:tasks.workunit.client.1.smithi163.stdout:3/570: creat d0/dc/d1c/d21/fce x:0 0 0 2024-02-16T19:24:30.002 INFO:tasks.workunit.client.1.smithi163.stdout:3/571: readlink d0/dc/d1c/lcd 0 2024-02-16T19:24:30.003 INFO:tasks.workunit.client.1.smithi163.stdout:4/481: mknod d11/d18/d1f/d29/d52/d57/c9b 0 2024-02-16T19:24:30.004 INFO:tasks.workunit.client.1.smithi163.stdout:9/814: truncate d5/de/df6/f114 712042 0 2024-02-16T19:24:30.005 INFO:tasks.workunit.client.1.smithi163.stdout:3/572: mknod d0/dc/d1c/d32/d7f/ccf 0 2024-02-16T19:24:30.005 INFO:tasks.workunit.client.1.smithi163.stdout:3/573: creat d0/dc/d94/d9e/dac/fd0 x:0 0 0 2024-02-16T19:24:30.006 INFO:tasks.workunit.client.1.smithi163.stdout:0/820: read d11/d22/fd7 [2781,87021] 0 2024-02-16T19:24:30.006 INFO:tasks.workunit.client.1.smithi163.stdout:0/821: fsync d11/d21/d50/f116 0 2024-02-16T19:24:30.006 INFO:tasks.workunit.client.1.smithi163.stdout:4/482: truncate d11/d18/d1f/f20 127093 0 2024-02-16T19:24:30.007 INFO:tasks.workunit.client.1.smithi163.stdout:9/815: unlink d5/de/ld9 0 2024-02-16T19:24:30.011 INFO:tasks.workunit.client.1.smithi163.stdout:3/574: creat d0/d83/da5/db9/fd1 x:0 0 0 2024-02-16T19:24:30.013 INFO:tasks.workunit.client.1.smithi163.stdout:7/569: dwrite d11/d30/f5b [0,4194304] 0 2024-02-16T19:24:30.014 INFO:tasks.workunit.client.1.smithi163.stdout:9/816: rename d5/de/d30/d3d/l7e to d5/de/d30/d3d/d49/d6d/d97/l119 0 2024-02-16T19:24:30.014 INFO:tasks.workunit.client.1.smithi163.stdout:6/496: dwrite d0/d6/df/d20/d40/d17/f4b [0,4194304] 0 2024-02-16T19:24:30.014 INFO:tasks.workunit.client.1.smithi163.stdout:6/497: chown d0/d6/df/c4a 65628358 1 2024-02-16T19:24:30.015 INFO:tasks.workunit.client.1.smithi163.stdout:4/483: read d11/f2f [461214,50246] 0 2024-02-16T19:24:30.015 INFO:tasks.workunit.client.1.smithi163.stdout:3/575: truncate d0/dc/f24 3557031 0 2024-02-16T19:24:30.015 INFO:tasks.workunit.client.1.smithi163.stdout:4/484: dread - d11/d18/d1f/d23/d3a/d4a/f4e zero size 2024-02-16T19:24:30.018 INFO:tasks.workunit.client.1.smithi163.stdout:7/570: symlink d11/d24/d25/d4b/lc7 0 2024-02-16T19:24:30.018 INFO:tasks.workunit.client.1.smithi163.stdout:7/571: dread - d11/d26/f4c zero size 2024-02-16T19:24:30.019 INFO:tasks.workunit.client.1.smithi163.stdout:9/817: mknod d5/de/d30/d8a/d10a/c11a 0 2024-02-16T19:24:30.023 INFO:tasks.workunit.client.1.smithi163.stdout:6/498: rename d0/dd/d7f/f8e to d0/d18/d77/d70/f9b 0 2024-02-16T19:24:30.025 INFO:tasks.workunit.client.1.smithi163.stdout:9/818: dread d5/de/d30/d31/d52/fe7 [0,4194304] 0 2024-02-16T19:24:30.025 INFO:tasks.workunit.client.1.smithi163.stdout:9/819: chown d5/f2c 19291 1 2024-02-16T19:24:30.028 INFO:tasks.workunit.client.1.smithi163.stdout:3/576: creat d0/dc/fd2 x:0 0 0 2024-02-16T19:24:30.028 INFO:tasks.workunit.client.1.smithi163.stdout:9/820: symlink d5/dc0/l11b 0 2024-02-16T19:24:30.030 INFO:tasks.workunit.client.1.smithi163.stdout:0/822: dwrite d11/d21/d7c/f81 [0,4194304] 0 2024-02-16T19:24:30.031 INFO:tasks.workunit.client.1.smithi163.stdout:9/821: creat d5/de/d16/d46/f11c x:0 0 0 2024-02-16T19:24:30.032 INFO:tasks.workunit.client.1.smithi163.stdout:4/485: write d11/d3d/f62 [3337711,16921] 0 2024-02-16T19:24:30.032 INFO:tasks.workunit.client.1.smithi163.stdout:4/486: fdatasync d11/d2d/d4f/f63 0 2024-02-16T19:24:30.037 INFO:tasks.workunit.client.1.smithi163.stdout:4/487: dread d11/f41 [0,4194304] 0 2024-02-16T19:24:30.037 INFO:tasks.workunit.client.1.smithi163.stdout:4/488: dread - d11/d18/d66/f98 zero size 2024-02-16T19:24:30.042 INFO:tasks.workunit.client.1.smithi163.stdout:0/823: link d11/d22/f2d d11/d21/d50/d58/d72/f120 0 2024-02-16T19:24:30.043 INFO:tasks.workunit.client.1.smithi163.stdout:9/822: getdents d5/de/d30/d43 0 2024-02-16T19:24:30.046 INFO:tasks.workunit.client.1.smithi163.stdout:4/489: mkdir d11/d18/d1f/d29/d52/d57/d6c/d9c 0 2024-02-16T19:24:30.046 INFO:tasks.workunit.client.1.smithi163.stdout:4/490: truncate d11/d18/d1f/f33 502371 0 2024-02-16T19:24:30.047 INFO:tasks.workunit.client.1.smithi163.stdout:0/824: rename d11/d12/d67/d6a/dae/l117 to d11/d12/d67/d6a/d6d/l121 0 2024-02-16T19:24:30.048 INFO:tasks.workunit.client.1.smithi163.stdout:0/825: truncate d11/d21/d50/d58/d72/d9e/fab 369724 0 2024-02-16T19:24:30.048 INFO:tasks.workunit.client.1.smithi163.stdout:0/826: getdents d11/d22/d61/d10f 0 2024-02-16T19:24:30.048 INFO:tasks.workunit.client.1.smithi163.stdout:9/823: mkdir d5/de/d30/d3d/d49/dea/d11d 0 2024-02-16T19:24:30.048 INFO:tasks.workunit.client.1.smithi163.stdout:9/824: read d5/de/d30/d3d/f3b [2067229,3972] 0 2024-02-16T19:24:30.049 INFO:tasks.workunit.client.1.smithi163.stdout:9/825: mkdir d5/dc0/d102/d11e 0 2024-02-16T19:24:30.050 INFO:tasks.workunit.client.1.smithi163.stdout:9/826: chown d5/de/d16/d47 12127 1 2024-02-16T19:24:30.051 INFO:tasks.workunit.client.1.smithi163.stdout:9/827: getdents d5/de/d16 0 2024-02-16T19:24:30.051 INFO:tasks.workunit.client.1.smithi163.stdout:9/828: read - d5/de/d84/fe0 zero size 2024-02-16T19:24:30.052 INFO:tasks.workunit.client.1.smithi163.stdout:7/572: dwrite d11/d50/d5d/d8b/fb1 [0,4194304] 0 2024-02-16T19:24:30.053 INFO:tasks.workunit.client.1.smithi163.stdout:9/829: rename d5/de/d16/f3c to d5/dc0/d102/d11e/f11f 0 2024-02-16T19:24:30.053 INFO:tasks.workunit.client.1.smithi163.stdout:9/830: fsync d5/f1a 0 2024-02-16T19:24:30.055 INFO:tasks.workunit.client.1.smithi163.stdout:9/831: mknod d5/de/d107/c120 0 2024-02-16T19:24:30.057 INFO:tasks.workunit.client.1.smithi163.stdout:7/573: getdents d11/d24/d25 0 2024-02-16T19:24:30.057 INFO:tasks.workunit.client.1.smithi163.stdout:0/827: write d11/d22/f24 [3658856,47868] 0 2024-02-16T19:24:30.058 INFO:tasks.workunit.client.1.smithi163.stdout:7/574: mkdir d11/d50/d5d/d8b/dc8 0 2024-02-16T19:24:30.058 INFO:tasks.workunit.client.1.smithi163.stdout:7/575: fsync d11/d24/d25/f9a 0 2024-02-16T19:24:30.059 INFO:tasks.workunit.client.1.smithi163.stdout:0/828: truncate d11/d12/d20/f3a 1942094 0 2024-02-16T19:24:30.059 INFO:tasks.workunit.client.1.smithi163.stdout:3/577: dwrite d0/f1b [4194304,4194304] 0 2024-02-16T19:24:30.060 INFO:tasks.workunit.client.1.smithi163.stdout:4/491: dread f9 [0,4194304] 0 2024-02-16T19:24:30.063 INFO:tasks.workunit.client.1.smithi163.stdout:6/499: dwrite d0/dd/d7f/d2e/f4f [0,4194304] 0 2024-02-16T19:24:30.064 INFO:tasks.workunit.client.1.smithi163.stdout:6/500: chown d0/d6/f45 3756 1 2024-02-16T19:24:30.066 INFO:tasks.workunit.client.1.smithi163.stdout:3/578: creat d0/dc/d50/fd3 x:0 0 0 2024-02-16T19:24:30.067 INFO:tasks.workunit.client.1.smithi163.stdout:9/832: write d5/de/d30/d3d/d49/dea/f110 [1250553,96261] 0 2024-02-16T19:24:30.067 INFO:tasks.workunit.client.1.smithi163.stdout:9/833: chown d5/de/d30/d31/c71 2444486 1 2024-02-16T19:24:30.067 INFO:tasks.workunit.client.1.smithi163.stdout:4/492: mkdir d11/d18/d1f/d29/d52/d57/d6c/d9d 0 2024-02-16T19:24:30.071 INFO:tasks.workunit.client.1.smithi163.stdout:6/501: mknod d0/d6/df/c9c 0 2024-02-16T19:24:30.072 INFO:tasks.workunit.client.1.smithi163.stdout:3/579: creat d0/dc/d1d/d49/fd4 x:0 0 0 2024-02-16T19:24:30.074 INFO:tasks.workunit.client.1.smithi163.stdout:3/580: mknod d0/dc/d50/d66/d93/cd5 0 2024-02-16T19:24:30.075 INFO:tasks.workunit.client.1.smithi163.stdout:3/581: write d0/dc/d94/d9e/dac/fd0 [432631,82631] 0 2024-02-16T19:24:30.076 INFO:tasks.workunit.client.1.smithi163.stdout:9/834: mkdir d5/de/d16/d46/d9b/d121 0 2024-02-16T19:24:30.078 INFO:tasks.workunit.client.1.smithi163.stdout:3/582: mkdir d0/dc/d1d/d29/d95/dd6 0 2024-02-16T19:24:30.079 INFO:tasks.workunit.client.1.smithi163.stdout:9/835: truncate d5/dc0/d102/d11e/f11f 474573 0 2024-02-16T19:24:30.085 INFO:tasks.workunit.client.1.smithi163.stdout:4/493: dread d11/d18/d1f/d29/f31 [0,4194304] 0 2024-02-16T19:24:30.085 INFO:tasks.workunit.client.1.smithi163.stdout:4/494: write d11/d18/d1f/f20 [369205,118386] 0 2024-02-16T19:24:30.088 INFO:tasks.workunit.client.1.smithi163.stdout:4/495: symlink d11/d18/d1f/d29/l9e 0 2024-02-16T19:24:30.089 INFO:tasks.workunit.client.1.smithi163.stdout:9/836: dread d5/de/d72/fb0 [0,4194304] 0 2024-02-16T19:24:30.090 INFO:tasks.workunit.client.1.smithi163.stdout:9/837: fdatasync d5/de/d16/d46/f8c 0 2024-02-16T19:24:30.090 INFO:tasks.workunit.client.1.smithi163.stdout:9/838: chown d5/de/d30/c39 1727881 1 2024-02-16T19:24:30.090 INFO:tasks.workunit.client.1.smithi163.stdout:7/576: dwrite d11/d50/d5d/d8b/d67/db2/fba [0,4194304] 0 2024-02-16T19:24:30.090 INFO:tasks.workunit.client.1.smithi163.stdout:4/496: creat d11/d18/f9f x:0 0 0 2024-02-16T19:24:30.090 INFO:tasks.workunit.client.1.smithi163.stdout:8/559: sync 2024-02-16T19:24:30.091 INFO:tasks.workunit.client.1.smithi163.stdout:5/585: sync 2024-02-16T19:24:30.091 INFO:tasks.workunit.client.1.smithi163.stdout:5/586: write d0/d22/d28/d36/fa6 [461292,125104] 0 2024-02-16T19:24:30.091 INFO:tasks.workunit.client.1.smithi163.stdout:9/839: creat d5/dc0/f122 x:0 0 0 2024-02-16T19:24:30.094 INFO:tasks.workunit.client.1.smithi163.stdout:7/577: mkdir d11/d50/d5d/dc9 0 2024-02-16T19:24:30.095 INFO:tasks.workunit.client.1.smithi163.stdout:4/497: creat d11/d18/d1f/d29/d52/d57/d6c/d9d/fa0 x:0 0 0 2024-02-16T19:24:30.095 INFO:tasks.workunit.client.1.smithi163.stdout:4/498: creat d11/d18/d1f/d29/fa1 x:0 0 0 2024-02-16T19:24:30.096 INFO:tasks.workunit.client.1.smithi163.stdout:5/587: mknod d0/d1c/d60/cc1 0 2024-02-16T19:24:30.098 INFO:tasks.workunit.client.1.smithi163.stdout:9/840: creat d5/de/d30/d3d/d49/df3/f123 x:0 0 0 2024-02-16T19:24:30.100 INFO:tasks.workunit.client.1.smithi163.stdout:4/499: mkdir d11/d18/d1f/d29/d52/d76/da2 0 2024-02-16T19:24:30.100 INFO:tasks.workunit.client.1.smithi163.stdout:4/500: stat d11/c1c 0 2024-02-16T19:24:30.100 INFO:tasks.workunit.client.1.smithi163.stdout:4/501: write d11/d2d/d4f/f63 [367074,39991] 0 2024-02-16T19:24:30.100 INFO:tasks.workunit.client.1.smithi163.stdout:4/502: dread - d11/d2d/f99 zero size 2024-02-16T19:24:30.100 INFO:tasks.workunit.client.1.smithi163.stdout:5/588: mknod d0/d22/d28/d36/d3c/cc2 0 2024-02-16T19:24:30.101 INFO:tasks.workunit.client.1.smithi163.stdout:9/841: rename d5/de/d30/d43/l67 to d5/de/d30/d8a/d10a/l124 0 2024-02-16T19:24:30.101 INFO:tasks.workunit.client.1.smithi163.stdout:9/842: write d5/de/d30/d3d/d49/fef [4585586,95861] 0 2024-02-16T19:24:30.102 INFO:tasks.workunit.client.1.smithi163.stdout:5/589: symlink d0/d2/d34/d44/d53/d93/lc3 0 2024-02-16T19:24:30.105 INFO:tasks.workunit.client.1.smithi163.stdout:4/503: rename d11/d18/d1f/d29/d52/d57/d6c/c7a to d11/d18/d1f/d29/d52/d57/d6c/d9d/ca3 0 2024-02-16T19:24:30.105 INFO:tasks.workunit.client.1.smithi163.stdout:5/590: mkdir d0/d22/d28/d36/dc4 0 2024-02-16T19:24:30.107 INFO:tasks.workunit.client.1.smithi163.stdout:5/591: unlink d0/d22/d3b/d8d/fb9 0 2024-02-16T19:24:30.107 INFO:tasks.workunit.client.1.smithi163.stdout:5/592: fdatasync d0/d22/d28/d36/d3c/f54 0 2024-02-16T19:24:30.107 INFO:tasks.workunit.client.1.smithi163.stdout:5/593: readlink d0/d22/d28/d36/d3c/d4f/lac 0 2024-02-16T19:24:30.107 INFO:tasks.workunit.client.1.smithi163.stdout:5/594: symlink d0/d22/d3b/dc0/lc5 0 2024-02-16T19:24:30.107 INFO:tasks.workunit.client.1.smithi163.stdout:5/595: read - d0/d2/d55/f91 zero size 2024-02-16T19:24:30.109 INFO:tasks.workunit.client.1.smithi163.stdout:5/596: dread d0/d22/d28/d36/f6b [0,4194304] 0 2024-02-16T19:24:30.115 INFO:tasks.workunit.client.1.smithi163.stdout:4/504: dread d11/d18/d1f/f27 [4194304,4194304] 0 2024-02-16T19:24:30.135 INFO:tasks.workunit.client.1.smithi163.stdout:0/829: dwrite d11/d21/d50/d58/d6b/fcd [0,4194304] 0 2024-02-16T19:24:30.135 INFO:tasks.workunit.client.1.smithi163.stdout:6/502: dwrite d0/dd/f6d [0,4194304] 0 2024-02-16T19:24:30.135 INFO:tasks.workunit.client.1.smithi163.stdout:6/503: write d0/d6/df/d20/d40/d53/f72 [921864,109148] 0 2024-02-16T19:24:30.139 INFO:tasks.workunit.client.1.smithi163.stdout:3/583: dwrite d0/dc/d1c/f20 [4194304,4194304] 0 2024-02-16T19:24:30.139 INFO:tasks.workunit.client.1.smithi163.stdout:3/584: dread - d0/dc/d50/d66/da3/f86 zero size 2024-02-16T19:24:30.145 INFO:tasks.workunit.client.1.smithi163.stdout:0/830: write d11/d12/d20/f5c [4177133,4133] 0 2024-02-16T19:24:30.148 INFO:tasks.workunit.client.1.smithi163.stdout:0/831: dread d11/d21/d50/d58/d72/d9e/ffe [0,4194304] 0 2024-02-16T19:24:30.148 INFO:tasks.workunit.client.1.smithi163.stdout:0/832: write d11/d22/fd7 [2983998,52983] 0 2024-02-16T19:24:30.149 INFO:tasks.workunit.client.1.smithi163.stdout:0/833: mkdir d11/d12/d67/d6a/d122 0 2024-02-16T19:24:30.149 INFO:tasks.workunit.client.1.smithi163.stdout:0/834: stat d11/d22/d3e/c8c 0 2024-02-16T19:24:30.158 INFO:tasks.workunit.client.1.smithi163.stdout:0/835: dread d11/d22/d7f/fb6 [0,4194304] 0 2024-02-16T19:24:30.173 INFO:tasks.workunit.client.1.smithi163.stdout:7/578: dwrite d11/d24/d49/fa5 [0,4194304] 0 2024-02-16T19:24:30.173 INFO:tasks.workunit.client.1.smithi163.stdout:7/579: truncate d11/d26/f46 226608 0 2024-02-16T19:24:30.173 INFO:tasks.workunit.client.1.smithi163.stdout:7/580: chown d11/d24/f79 183394658 1 2024-02-16T19:24:30.174 INFO:tasks.workunit.client.1.smithi163.stdout:8/560: dwrite d0/d7/f68 [0,4194304] 0 2024-02-16T19:24:30.176 INFO:tasks.workunit.client.1.smithi163.stdout:7/581: rename d11/d24/d25/f4f to d11/d24/d49/dc4/fca 0 2024-02-16T19:24:30.176 INFO:tasks.workunit.client.1.smithi163.stdout:7/582: dread - d11/d24/d34/fab zero size 2024-02-16T19:24:30.176 INFO:tasks.workunit.client.1.smithi163.stdout:7/583: write d11/d24/d25/f48 [211622,100053] 0 2024-02-16T19:24:30.176 INFO:tasks.workunit.client.1.smithi163.stdout:7/584: chown d11/d24/l2e 848 1 2024-02-16T19:24:30.179 INFO:tasks.workunit.client.1.smithi163.stdout:9/843: dwrite d5/de/d30/d8a/fd2 [0,4194304] 0 2024-02-16T19:24:30.179 INFO:tasks.workunit.client.1.smithi163.stdout:5/597: dwrite d0/d1c/d5a/fb5 [0,4194304] 0 2024-02-16T19:24:30.179 INFO:tasks.workunit.client.1.smithi163.stdout:4/505: dwrite d11/d18/d1f/d29/fa1 [0,4194304] 0 2024-02-16T19:24:30.179 INFO:tasks.workunit.client.1.smithi163.stdout:4/506: chown d11/d18/d1f/d29/fa1 1496 1 2024-02-16T19:24:30.180 INFO:tasks.workunit.client.1.smithi163.stdout:0/836: dwrite d11/d21/d50/d58/d74/df8/f11a [0,4194304] 0 2024-02-16T19:24:30.182 INFO:tasks.workunit.client.1.smithi163.stdout:7/585: link d11/d24/d25/f48 d11/d24/d25/d4b/d66/dc1/fcb 0 2024-02-16T19:24:30.186 INFO:tasks.workunit.client.1.smithi163.stdout:5/598: creat d0/d2/d34/fc6 x:0 0 0 2024-02-16T19:24:30.186 INFO:tasks.workunit.client.1.smithi163.stdout:5/599: fdatasync d0/d22/d28/d36/d3c/d4f/f61 0 2024-02-16T19:24:30.188 INFO:tasks.workunit.client.1.smithi163.stdout:8/561: symlink d0/d24/d3c/d3d/d41/lbc 0 2024-02-16T19:24:30.190 INFO:tasks.workunit.client.1.smithi163.stdout:0/837: dread d11/d21/fdb [0,4194304] 0 2024-02-16T19:24:30.191 INFO:tasks.workunit.client.1.smithi163.stdout:4/507: truncate d11/f41 364296 0 2024-02-16T19:24:30.199 INFO:tasks.workunit.client.1.smithi163.stdout:5/600: link d0/d22/f3a d0/d1c/dbe/fc7 0 2024-02-16T19:24:30.199 INFO:tasks.workunit.client.1.smithi163.stdout:8/562: write d0/d24/d2c/d74/f93 [2874056,96712] 0 2024-02-16T19:24:30.200 INFO:tasks.workunit.client.1.smithi163.stdout:4/508: symlink d11/d18/d1f/d29/la4 0 2024-02-16T19:24:30.200 INFO:tasks.workunit.client.1.smithi163.stdout:4/509: fsync d11/f46 0 2024-02-16T19:24:30.202 INFO:tasks.workunit.client.1.smithi163.stdout:0/838: write d11/d21/d7c/f81 [1420589,87526] 0 2024-02-16T19:24:30.203 INFO:tasks.workunit.client.1.smithi163.stdout:5/601: mknod d0/d22/d3b/cc8 0 2024-02-16T19:24:30.203 INFO:tasks.workunit.client.1.smithi163.stdout:5/602: chown d0/d1c/fa9 1601 1 2024-02-16T19:24:30.204 INFO:tasks.workunit.client.1.smithi163.stdout:9/844: dwrite d5/de/d30/d3d/d49/dea/f110 [0,4194304] 0 2024-02-16T19:24:30.204 INFO:tasks.workunit.client.1.smithi163.stdout:9/845: truncate d5/de/d16/f98 5098253 0 2024-02-16T19:24:30.204 INFO:tasks.workunit.client.1.smithi163.stdout:8/563: mkdir d0/d7/d19/d1f/d30/d48/dbd 0 2024-02-16T19:24:30.206 INFO:tasks.workunit.client.1.smithi163.stdout:0/839: getdents d11/d22/d7f/d8e 0 2024-02-16T19:24:30.207 INFO:tasks.workunit.client.1.smithi163.stdout:5/603: symlink d0/d22/d3b/dc0/lc9 0 2024-02-16T19:24:30.208 INFO:tasks.workunit.client.1.smithi163.stdout:0/840: dread d11/d22/d7f/f83 [0,4194304] 0 2024-02-16T19:24:30.209 INFO:tasks.workunit.client.1.smithi163.stdout:9/846: truncate d5/de/d30/d3d/d49/d6e/fb9 728154 0 2024-02-16T19:24:30.211 INFO:tasks.workunit.client.1.smithi163.stdout:8/564: mknod d0/d7/d19/d1f/cbe 0 2024-02-16T19:24:30.212 INFO:tasks.workunit.client.1.smithi163.stdout:5/604: mknod d0/d22/d28/d36/db7/cca 0 2024-02-16T19:24:30.212 INFO:tasks.workunit.client.1.smithi163.stdout:5/605: fsync d0/d22/d3b/f4c 0 2024-02-16T19:24:30.212 INFO:tasks.workunit.client.1.smithi163.stdout:5/606: stat d0/d22/d28/d36/d3c/d4f/f7b 0 2024-02-16T19:24:30.212 INFO:tasks.workunit.client.1.smithi163.stdout:5/607: chown d0/d2/l2a 2 1 2024-02-16T19:24:30.214 INFO:tasks.workunit.client.1.smithi163.stdout:4/510: dread f9 [4194304,4194304] 0 2024-02-16T19:24:30.216 INFO:tasks.workunit.client.1.smithi163.stdout:0/841: rename d11/f1f to d11/d21/d7c/d11c/f123 0 2024-02-16T19:24:30.217 INFO:tasks.workunit.client.1.smithi163.stdout:9/847: symlink d5/de/d30/d8a/da8/l125 0 2024-02-16T19:24:30.218 INFO:tasks.workunit.client.1.smithi163.stdout:8/565: symlink d0/d7/d18/d53/lbf 0 2024-02-16T19:24:30.221 INFO:tasks.workunit.client.1.smithi163.stdout:0/842: truncate d11/d4f/fc4 3544907 0 2024-02-16T19:24:30.223 INFO:tasks.workunit.client.1.smithi163.stdout:9/848: rename d5/de/d30/d3d/d49/dea/f110 to d5/de/d16/d46/d9b/d106/f126 0 2024-02-16T19:24:30.224 INFO:tasks.workunit.client.1.smithi163.stdout:8/566: mknod d0/d7/d19/d1f/d32/cc0 0 2024-02-16T19:24:30.226 INFO:tasks.workunit.client.1.smithi163.stdout:0/843: creat d11/d12/d67/dd4/f124 x:0 0 0 2024-02-16T19:24:30.227 INFO:tasks.workunit.client.1.smithi163.stdout:9/849: getdents d5/dc0/d102/d11e 0 2024-02-16T19:24:30.239 INFO:tasks.workunit.client.1.smithi163.stdout:4/511: dwrite d11/d18/d1f/d29/d52/f80 [0,4194304] 0 2024-02-16T19:24:30.242 INFO:tasks.workunit.client.1.smithi163.stdout:2/706: sync 2024-02-16T19:24:30.249 INFO:tasks.workunit.client.1.smithi163.stdout:1/608: sync 2024-02-16T19:24:30.250 INFO:tasks.workunit.client.1.smithi163.stdout:5/608: dwrite d0/d2/d31/fb3 [0,4194304] 0 2024-02-16T19:24:30.251 INFO:tasks.workunit.client.1.smithi163.stdout:1/609: mkdir d7/d20/d93/dd2 0 2024-02-16T19:24:30.254 INFO:tasks.workunit.client.1.smithi163.stdout:5/609: rename d0/d22/d28/d36/fa6 to d0/d2/d5c/fcb 0 2024-02-16T19:24:30.254 INFO:tasks.workunit.client.1.smithi163.stdout:1/610: mknod d7/dc/d78/cd3 0 2024-02-16T19:24:30.255 INFO:tasks.workunit.client.1.smithi163.stdout:2/707: dread d2/f1a [0,4194304] 0 2024-02-16T19:24:30.255 INFO:tasks.workunit.client.1.smithi163.stdout:1/611: symlink d7/d20/d2a/ld4 0 2024-02-16T19:24:30.256 INFO:tasks.workunit.client.1.smithi163.stdout:1/612: creat d7/d20/d96/fd5 x:0 0 0 2024-02-16T19:24:30.256 INFO:tasks.workunit.client.1.smithi163.stdout:1/613: readlink d7/l16 0 2024-02-16T19:24:30.269 INFO:tasks.workunit.client.1.smithi163.stdout:2/708: dread d2/d2a/d39/d66/d6c/fa0 [0,4194304] 0 2024-02-16T19:24:30.270 INFO:tasks.workunit.client.1.smithi163.stdout:2/709: unlink d2/d2a/d2d/d73/d94/dc9/lf3 0 2024-02-16T19:24:30.270 INFO:tasks.workunit.client.1.smithi163.stdout:2/710: stat d2/d26/d54/d58/d98/lfa 0 2024-02-16T19:24:30.271 INFO:tasks.workunit.client.1.smithi163.stdout:2/711: mknod d2/d26/d54/d58/d7d/daf/cfe 0 2024-02-16T19:24:30.272 INFO:tasks.workunit.client.1.smithi163.stdout:2/712: truncate d2/d2a/d39/fae 809813 0 2024-02-16T19:24:30.277 INFO:tasks.workunit.client.1.smithi163.stdout:2/713: dread d2/fb [8388608,4194304] 0 2024-02-16T19:24:30.277 INFO:tasks.workunit.client.1.smithi163.stdout:2/714: chown d2/d26/fd9 184595 1 2024-02-16T19:24:30.277 INFO:tasks.workunit.client.1.smithi163.stdout:8/567: dwrite d0/d7/d19/d1f/d32/f34 [0,4194304] 0 2024-02-16T19:24:30.289 INFO:tasks.workunit.client.1.smithi163.stdout:2/715: dread d2/d26/d54/d58/f69 [0,4194304] 0 2024-02-16T19:24:30.289 INFO:tasks.workunit.client.1.smithi163.stdout:9/850: dwrite d5/de/d16/d46/fe2 [0,4194304] 0 2024-02-16T19:24:30.289 INFO:tasks.workunit.client.1.smithi163.stdout:0/844: dwrite d11/d12/f10b [0,4194304] 0 2024-02-16T19:24:30.289 INFO:tasks.workunit.client.1.smithi163.stdout:0/845: write d11/d22/d7f/f95 [958845,67442] 0 2024-02-16T19:24:30.289 INFO:tasks.workunit.client.1.smithi163.stdout:0/846: readlink d11/d12/d67/d6a/dae/ld1 0 2024-02-16T19:24:30.289 INFO:tasks.workunit.client.1.smithi163.stdout:4/512: dwrite d11/f2f [0,4194304] 0 2024-02-16T19:24:30.289 INFO:tasks.workunit.client.1.smithi163.stdout:4/513: read - d11/d18/d1f/d29/d52/d57/f84 zero size 2024-02-16T19:24:30.293 INFO:tasks.workunit.client.1.smithi163.stdout:4/514: mknod d11/d18/d1f/d29/d52/d57/d6c/d9c/ca5 0 2024-02-16T19:24:30.305 INFO:tasks.workunit.client.1.smithi163.stdout:1/614: dwrite d7/d20/d96/fce [0,4194304] 0 2024-02-16T19:24:30.310 INFO:tasks.workunit.client.1.smithi163.stdout:1/615: write d7/dc/d78/d7c/f41 [1216655,30934] 0 2024-02-16T19:24:30.310 INFO:tasks.workunit.client.1.smithi163.stdout:1/616: fdatasync d7/dc/d2d/d32/d3b/f50 0 2024-02-16T19:24:30.310 INFO:tasks.workunit.client.1.smithi163.stdout:1/617: chown d7/d20/d2a/f9f 182251426 1 2024-02-16T19:24:30.311 INFO:tasks.workunit.client.1.smithi163.stdout:4/515: write d11/d18/d1f/d29/d52/f80 [3551770,56147] 0 2024-02-16T19:24:30.311 INFO:tasks.workunit.client.1.smithi163.stdout:1/618: creat d7/dc/d2d/d32/d87/da6/fd6 x:0 0 0 2024-02-16T19:24:30.313 INFO:tasks.workunit.client.1.smithi163.stdout:4/516: creat d11/d2d/d4f/d5f/d70/d74/fa6 x:0 0 0 2024-02-16T19:24:30.313 INFO:tasks.workunit.client.1.smithi163.stdout:4/517: chown f2 10246 1 2024-02-16T19:24:30.314 INFO:tasks.workunit.client.1.smithi163.stdout:1/619: dread d7/d20/d2a/f40 [0,4194304] 0 2024-02-16T19:24:30.314 INFO:tasks.workunit.client.1.smithi163.stdout:4/518: rename d11/d18/d1f/d29/d52/d57/d6c/f95 to d11/d18/d1f/d34/fa7 0 2024-02-16T19:24:30.315 INFO:tasks.workunit.client.1.smithi163.stdout:1/620: mknod d7/d20/d93/dd2/cd7 0 2024-02-16T19:24:30.321 INFO:tasks.workunit.client.1.smithi163.stdout:0/847: dwrite d11/d21/d50/d58/d72/d9e/ffe [0,4194304] 0 2024-02-16T19:24:30.322 INFO:tasks.workunit.client.1.smithi163.stdout:8/568: dwrite d0/fa9 [0,4194304] 0 2024-02-16T19:24:30.326 INFO:tasks.workunit.client.1.smithi163.stdout:0/848: truncate d11/d21/fdb 3460786 0 2024-02-16T19:24:30.326 INFO:tasks.workunit.client.1.smithi163.stdout:0/849: creat d11/d22/f125 x:0 0 0 2024-02-16T19:24:30.332 INFO:tasks.workunit.client.1.smithi163.stdout:0/850: unlink d11/c18 0 2024-02-16T19:24:30.332 INFO:tasks.workunit.client.1.smithi163.stdout:0/851: chown d11/d12/d20/l2e 70986669 1 2024-02-16T19:24:30.334 INFO:tasks.workunit.client.1.smithi163.stdout:0/852: getdents d11/d12/d20 0 2024-02-16T19:24:30.363 INFO:tasks.workunit.client.1.smithi163.stdout:8/569: dwrite d0/d24/d78/fb9 [0,4194304] 0 2024-02-16T19:24:30.363 INFO:tasks.workunit.client.1.smithi163.stdout:8/570: read d0/d7/d19/d1f/d32/d70/f8b [560433,15872] 0 2024-02-16T19:24:30.363 INFO:tasks.workunit.client.1.smithi163.stdout:4/519: dwrite d11/d18/d1f/d29/f3e [4194304,4194304] 0 2024-02-16T19:24:30.363 INFO:tasks.workunit.client.1.smithi163.stdout:0/853: dwrite d11/d21/d50/f116 [0,4194304] 0 2024-02-16T19:24:30.363 INFO:tasks.workunit.client.1.smithi163.stdout:1/621: dwrite d7/d5b/d63/f7b [8388608,4194304] 0 2024-02-16T19:24:30.363 INFO:tasks.workunit.client.1.smithi163.stdout:1/622: write d7/dc/d78/d7c/f81 [422310,38151] 0 2024-02-16T19:24:30.368 INFO:tasks.workunit.client.1.smithi163.stdout:4/520: link d11/d18/d1f/d29/f85 d11/d5b/fa8 0 2024-02-16T19:24:30.369 INFO:tasks.workunit.client.1.smithi163.stdout:1/623: symlink d7/d20/d21/d4e/dc6/ld8 0 2024-02-16T19:24:30.370 INFO:tasks.workunit.client.1.smithi163.stdout:4/521: mkdir d11/d18/d1f/da9 0 2024-02-16T19:24:30.372 INFO:tasks.workunit.client.1.smithi163.stdout:1/624: rename d7/dc/d2d/d32/d3b/l83 to d7/dc/d2d/ld9 0 2024-02-16T19:24:30.373 INFO:tasks.workunit.client.1.smithi163.stdout:4/522: creat d11/d2d/d4f/d5f/faa x:0 0 0 2024-02-16T19:24:30.374 INFO:tasks.workunit.client.1.smithi163.stdout:1/625: symlink d7/d20/lda 0 2024-02-16T19:24:30.375 INFO:tasks.workunit.client.1.smithi163.stdout:4/523: mknod d11/d18/d1f/d29/d52/d57/d6c/d7c/cab 0 2024-02-16T19:24:30.376 INFO:tasks.workunit.client.1.smithi163.stdout:1/626: creat d7/dc/d2d/d32/d3b/fdb x:0 0 0 2024-02-16T19:24:30.377 INFO:tasks.workunit.client.1.smithi163.stdout:6/504: sync 2024-02-16T19:24:30.377 INFO:tasks.workunit.client.1.smithi163.stdout:4/524: creat d11/d2d/d4f/d5f/d70/d74/d75/fac x:0 0 0 2024-02-16T19:24:30.381 INFO:tasks.workunit.client.1.smithi163.stdout:0/854: getdents d11/d22 0 2024-02-16T19:24:30.381 INFO:tasks.workunit.client.1.smithi163.stdout:0/855: chown d11/d12/dc6/d102/l115 5422185 1 2024-02-16T19:24:30.384 INFO:tasks.workunit.client.1.smithi163.stdout:0/856: creat d11/d21/d50/d58/d72/d9e/dd8/f126 x:0 0 0 2024-02-16T19:24:30.418 INFO:tasks.workunit.client.1.smithi163.stdout:1/627: dwrite d7/dc/d2d/d32/d3b/f58 [0,4194304] 0 2024-02-16T19:24:30.419 INFO:tasks.workunit.client.1.smithi163.stdout:1/628: dread - d7/d20/d96/fcb zero size 2024-02-16T19:24:30.419 INFO:tasks.workunit.client.1.smithi163.stdout:6/505: dwrite d0/dd/d7f/d2e/f4f [0,4194304] 0 2024-02-16T19:24:30.420 INFO:tasks.workunit.client.1.smithi163.stdout:0/857: dwrite d11/d21/d50/f52 [0,4194304] 0 2024-02-16T19:24:30.420 INFO:tasks.workunit.client.1.smithi163.stdout:0/858: stat d11/d12/d67/d6a/dae 0 2024-02-16T19:24:30.422 INFO:tasks.workunit.client.1.smithi163.stdout:1/629: symlink d7/dc/d2d/d90/ldc 0 2024-02-16T19:24:30.422 INFO:tasks.workunit.client.1.smithi163.stdout:1/630: creat d7/dc/d2d/d32/d3b/fdd x:0 0 0 2024-02-16T19:24:30.424 INFO:tasks.workunit.client.0.smithi040.stdout:4/981: sync 2024-02-16T19:24:30.425 INFO:tasks.workunit.client.0.smithi040.stdout:4/982: read dd/de/d10/d36/f63 [1834960,110916] 0 2024-02-16T19:24:30.425 INFO:tasks.workunit.client.1.smithi163.stdout:0/859: getdents d11/d21/d50/d58 0 2024-02-16T19:24:30.431 INFO:tasks.workunit.client.1.smithi163.stdout:1/631: write d7/f6b [5638305,37462] 0 2024-02-16T19:24:30.431 INFO:tasks.workunit.client.1.smithi163.stdout:5/610: sync 2024-02-16T19:24:30.431 INFO:tasks.workunit.client.1.smithi163.stdout:2/716: sync 2024-02-16T19:24:30.431 INFO:tasks.workunit.client.1.smithi163.stdout:3/585: sync 2024-02-16T19:24:30.432 INFO:tasks.workunit.client.1.smithi163.stdout:7/586: sync 2024-02-16T19:24:30.432 INFO:tasks.workunit.client.1.smithi163.stdout:9/851: sync 2024-02-16T19:24:30.432 INFO:tasks.workunit.client.1.smithi163.stdout:1/632: creat d7/dc/dc1/fde x:0 0 0 2024-02-16T19:24:30.432 INFO:tasks.workunit.client.0.smithi040.stdout:4/983: read dd/de/d45/f74 [6696639,11475] 0 2024-02-16T19:24:30.433 INFO:tasks.workunit.client.0.smithi040.stdout:4/984: truncate dd/dd2/f122 348789 0 2024-02-16T19:24:30.434 INFO:tasks.workunit.client.1.smithi163.stdout:3/586: dread d0/f92 [0,4194304] 0 2024-02-16T19:24:30.434 INFO:tasks.workunit.client.1.smithi163.stdout:3/587: write d0/dc/d96/fbb [89630,96371] 0 2024-02-16T19:24:30.434 INFO:tasks.workunit.client.1.smithi163.stdout:5/611: rename d0/d22/d28/l41 to d0/d2/d34/d44/lcc 0 2024-02-16T19:24:30.435 INFO:tasks.workunit.client.1.smithi163.stdout:9/852: symlink d5/de/d16/d47/l127 0 2024-02-16T19:24:30.435 INFO:tasks.workunit.client.0.smithi040.stdout:4/985: rename dd/d58/d5e/f10d to dd/de/daa/f135 0 2024-02-16T19:24:30.435 INFO:tasks.workunit.client.0.smithi040.stdout:4/986: chown dd/de/d10/d36/d9e/l126 36657 1 2024-02-16T19:24:30.436 INFO:tasks.workunit.client.1.smithi163.stdout:6/506: dread d0/dd/d7f/d15/d1c/d65/d63/f7c [0,4194304] 0 2024-02-16T19:24:30.436 INFO:tasks.workunit.client.1.smithi163.stdout:3/588: rmdir d0/dc/d1d/d29/d95/dd6 0 2024-02-16T19:24:30.438 INFO:tasks.workunit.client.1.smithi163.stdout:1/633: getdents d7/dc/d2d/d32/d87/da6 0 2024-02-16T19:24:30.438 INFO:tasks.workunit.client.1.smithi163.stdout:1/634: dread - d7/dc/d2d/d32/fbe zero size 2024-02-16T19:24:30.438 INFO:tasks.workunit.client.1.smithi163.stdout:0/860: dread d11/d12/d67/d6a/d6d/fcb [0,4194304] 0 2024-02-16T19:24:30.439 INFO:tasks.workunit.client.1.smithi163.stdout:5/612: symlink d0/d2/d34/d44/d53/lcd 0 2024-02-16T19:24:30.440 INFO:tasks.workunit.client.1.smithi163.stdout:6/507: mknod d0/dd/d7f/d15/d1c/d65/d63/d86/c9d 0 2024-02-16T19:24:30.441 INFO:tasks.workunit.client.1.smithi163.stdout:3/589: unlink d0/dc/d1d/f1f 0 2024-02-16T19:24:30.441 INFO:tasks.workunit.client.0.smithi040.stdout:4/987: link dd/de/d10/d36/d9e/d77/f7b dd/de/d10/d36/d9e/d32/f136 0 2024-02-16T19:24:30.441 INFO:tasks.workunit.client.0.smithi040.stdout:4/988: chown dd/f66 190703869 1 2024-02-16T19:24:30.442 INFO:tasks.workunit.client.1.smithi163.stdout:6/508: creat d0/d18/d77/f9e x:0 0 0 2024-02-16T19:24:30.443 INFO:tasks.workunit.client.1.smithi163.stdout:2/717: dread d2/d26/f2f [0,4194304] 0 2024-02-16T19:24:30.443 INFO:tasks.workunit.client.1.smithi163.stdout:2/718: chown d2/d2a/d39/l85 10738 1 2024-02-16T19:24:30.443 INFO:tasks.workunit.client.0.smithi040.stdout:4/989: unlink dd/de/d10/d36/dad/f105 0 2024-02-16T19:24:30.444 INFO:tasks.workunit.client.1.smithi163.stdout:3/590: creat d0/dc/d50/d66/da3/fd7 x:0 0 0 2024-02-16T19:24:30.444 INFO:tasks.workunit.client.1.smithi163.stdout:3/591: write d0/f13 [2584741,47976] 0 2024-02-16T19:24:30.446 INFO:tasks.workunit.client.1.smithi163.stdout:6/509: unlink d0/dd/l82 0 2024-02-16T19:24:30.447 INFO:tasks.workunit.client.1.smithi163.stdout:0/861: dread d11/d22/d7f/fb6 [0,4194304] 0 2024-02-16T19:24:30.447 INFO:tasks.workunit.client.1.smithi163.stdout:5/613: dread d0/d1c/f74 [0,4194304] 0 2024-02-16T19:24:30.447 INFO:tasks.workunit.client.1.smithi163.stdout:5/614: creat d0/d2/d34/fce x:0 0 0 2024-02-16T19:24:30.447 INFO:tasks.workunit.client.1.smithi163.stdout:2/719: unlink d2/ddb/d7c/d9c/ldc 0 2024-02-16T19:24:30.448 INFO:tasks.workunit.client.0.smithi040.stdout:4/990: mknod dd/de/d10/d36/d9e/d32/df9/c137 0 2024-02-16T19:24:30.448 INFO:tasks.workunit.client.0.smithi040.stdout:4/991: fsync dd/de/db0/f132 0 2024-02-16T19:24:30.448 INFO:tasks.workunit.client.0.smithi040.stdout:4/992: chown dd/d58/d5e/dd1/d9d/d123/d12e/d107 1332861 1 2024-02-16T19:24:30.448 INFO:tasks.workunit.client.1.smithi163.stdout:3/592: truncate d0/dc/f6e 513589 0 2024-02-16T19:24:30.449 INFO:tasks.workunit.client.1.smithi163.stdout:1/635: dread d7/d20/d2a/f9f [0,4194304] 0 2024-02-16T19:24:30.449 INFO:tasks.workunit.client.1.smithi163.stdout:1/636: readlink d7/dc/l98 0 2024-02-16T19:24:30.450 INFO:tasks.workunit.client.1.smithi163.stdout:6/510: creat d0/dd/d7f/d2e/d33/f9f x:0 0 0 2024-02-16T19:24:30.451 INFO:tasks.workunit.client.1.smithi163.stdout:0/862: stat d11/d4f/c53 0 2024-02-16T19:24:30.451 INFO:tasks.workunit.client.1.smithi163.stdout:7/587: dwrite d11/d50/d5d/d8b/d67/f9b [0,4194304] 0 2024-02-16T19:24:30.451 INFO:tasks.workunit.client.1.smithi163.stdout:7/588: read - d11/fa8 zero size 2024-02-16T19:24:30.451 INFO:tasks.workunit.client.1.smithi163.stdout:0/863: write d11/d22/fa3 [4364948,63332] 0 2024-02-16T19:24:30.451 INFO:tasks.workunit.client.1.smithi163.stdout:7/589: fdatasync d11/d50/d5d/d8b/d67/f9b 0 2024-02-16T19:24:30.454 INFO:tasks.workunit.client.1.smithi163.stdout:2/720: rename d2/d2a/d39/d66/dbc to d2/d26/d54/d58/dff 0 2024-02-16T19:24:30.456 INFO:tasks.workunit.client.1.smithi163.stdout:2/721: dread d2/d2a/d39/d66/f9a [0,4194304] 0 2024-02-16T19:24:30.456 INFO:tasks.workunit.client.1.smithi163.stdout:1/637: creat d7/d20/d21/d4e/dc6/fdf x:0 0 0 2024-02-16T19:24:30.456 INFO:tasks.workunit.client.1.smithi163.stdout:1/638: write d7/dc/d2d/d32/d87/f88 [734561,67608] 0 2024-02-16T19:24:30.456 INFO:tasks.workunit.client.1.smithi163.stdout:1/639: chown d7/dc/l11 474755 1 2024-02-16T19:24:30.456 INFO:tasks.workunit.client.1.smithi163.stdout:1/640: stat d7/d20/d2a/f9f 0 2024-02-16T19:24:30.457 INFO:tasks.workunit.client.1.smithi163.stdout:6/511: symlink d0/dd/d7f/d15/la0 0 2024-02-16T19:24:30.457 INFO:tasks.workunit.client.1.smithi163.stdout:6/512: fsync d0/dd/d7f/d2e/d33/f38 0 2024-02-16T19:24:30.457 INFO:tasks.workunit.client.1.smithi163.stdout:6/513: chown d0/d6/df/d20/d40/d17/f4b 1 1 2024-02-16T19:24:30.459 INFO:tasks.workunit.client.1.smithi163.stdout:3/593: dread d0/d3e/f78 [0,4194304] 0 2024-02-16T19:24:30.460 INFO:tasks.workunit.client.1.smithi163.stdout:2/722: mknod d2/d2a/d2d/d73/d84/dba/c100 0 2024-02-16T19:24:30.460 INFO:tasks.workunit.client.1.smithi163.stdout:2/723: readlink d2/l20 0 2024-02-16T19:24:30.461 INFO:tasks.workunit.client.1.smithi163.stdout:6/514: rename d0/dd/d7f/d15/c1f to d0/d6/df/ca1 0 2024-02-16T19:24:30.461 INFO:tasks.workunit.client.1.smithi163.stdout:6/515: chown d0/l14 44448 1 2024-02-16T19:24:30.461 INFO:tasks.workunit.client.1.smithi163.stdout:6/516: creat d0/d6/df/d20/d40/d17/fa2 x:0 0 0 2024-02-16T19:24:30.462 INFO:tasks.workunit.client.1.smithi163.stdout:6/517: chown d0/d6/f92 58 1 2024-02-16T19:24:30.462 INFO:tasks.workunit.client.1.smithi163.stdout:6/518: creat d0/d6/fa3 x:0 0 0 2024-02-16T19:24:30.462 INFO:tasks.workunit.client.1.smithi163.stdout:7/590: dread d11/d50/d5d/d8b/f4e [4194304,4194304] 0 2024-02-16T19:24:30.463 INFO:tasks.workunit.client.1.smithi163.stdout:2/724: rmdir d2/d26/d54/dca 0 2024-02-16T19:24:30.464 INFO:tasks.workunit.client.1.smithi163.stdout:7/591: mknod d11/d50/d5d/d8b/d67/ccc 0 2024-02-16T19:24:30.465 INFO:tasks.workunit.client.1.smithi163.stdout:3/594: getdents d0/dc/d50 0 2024-02-16T19:24:30.466 INFO:tasks.workunit.client.1.smithi163.stdout:2/725: mkdir d2/d26/d54/d58/d7d/daf/d101 0 2024-02-16T19:24:30.468 INFO:tasks.workunit.client.1.smithi163.stdout:3/595: creat d0/dc/d1d/fd8 x:0 0 0 2024-02-16T19:24:30.476 INFO:tasks.workunit.client.1.smithi163.stdout:2/726: write d2/d2a/d2d/d73/d94/fbd [488916,52660] 0 2024-02-16T19:24:30.476 INFO:tasks.workunit.client.1.smithi163.stdout:2/727: stat d2/ddb/dc8 0 2024-02-16T19:24:30.477 INFO:tasks.workunit.client.1.smithi163.stdout:2/728: truncate d2/d2a/d39/d66/f77 18496 0 2024-02-16T19:24:30.477 INFO:tasks.workunit.client.0.smithi040.stdout:4/993: dwrite dd/de/f12 [0,4194304] 0 2024-02-16T19:24:30.478 INFO:tasks.workunit.client.0.smithi040.stdout:4/994: fsync dd/de/d71/ff2 0 2024-02-16T19:24:30.484 INFO:tasks.workunit.client.1.smithi163.stdout:0/864: dwrite d11/d12/d67/d6a/d6d/f65 [0,4194304] 0 2024-02-16T19:24:30.485 INFO:tasks.workunit.client.1.smithi163.stdout:0/865: write d11/d12/d20/fe4 [4441859,22702] 0 2024-02-16T19:24:30.485 INFO:tasks.workunit.client.1.smithi163.stdout:6/519: dwrite d0/d6/f45 [0,4194304] 0 2024-02-16T19:24:30.485 INFO:tasks.workunit.client.1.smithi163.stdout:6/520: fdatasync d0/dd/d7f/d2e/d33/f60 0 2024-02-16T19:24:30.489 INFO:tasks.workunit.client.1.smithi163.stdout:2/729: dread d2/d2a/d39/f42 [0,4194304] 0 2024-02-16T19:24:30.489 INFO:tasks.workunit.client.1.smithi163.stdout:2/730: fdatasync d2/f1f 0 2024-02-16T19:24:30.489 INFO:tasks.workunit.client.1.smithi163.stdout:2/731: dread - d2/d26/fd9 zero size 2024-02-16T19:24:30.494 INFO:tasks.workunit.client.1.smithi163.stdout:6/521: dread d0/d6/df/d20/d40/f89 [0,4194304] 0 2024-02-16T19:24:30.494 INFO:tasks.workunit.client.1.smithi163.stdout:6/522: chown d0/d6/df/d20/d40/d17 3649 1 2024-02-16T19:24:30.495 INFO:tasks.workunit.client.1.smithi163.stdout:2/732: creat d2/dce/dee/f102 x:0 0 0 2024-02-16T19:24:30.495 INFO:tasks.workunit.client.1.smithi163.stdout:2/733: chown d2/d2a/d39 1838629 1 2024-02-16T19:24:30.498 INFO:tasks.workunit.client.1.smithi163.stdout:0/866: rmdir d11/d12/d67/d6a/d6d 39 2024-02-16T19:24:30.500 INFO:tasks.workunit.client.1.smithi163.stdout:2/734: creat d2/d2a/d2d/d73/d94/dc9/f103 x:0 0 0 2024-02-16T19:24:30.500 INFO:tasks.workunit.client.1.smithi163.stdout:2/735: dread - d2/d2a/d39/d66/d6c/f79 zero size 2024-02-16T19:24:30.503 INFO:tasks.workunit.client.1.smithi163.stdout:0/867: creat d11/d21/f127 x:0 0 0 2024-02-16T19:24:30.503 INFO:tasks.workunit.client.1.smithi163.stdout:0/868: creat d11/d21/d50/d58/d72/d9e/dd8/f128 x:0 0 0 2024-02-16T19:24:30.503 INFO:tasks.workunit.client.1.smithi163.stdout:0/869: fsync d11/d21/d50/f52 0 2024-02-16T19:24:30.504 INFO:tasks.workunit.client.1.smithi163.stdout:2/736: getdents d2/d2a/d2d 0 2024-02-16T19:24:30.508 INFO:tasks.workunit.client.1.smithi163.stdout:2/737: rename d2/d2a/d2d/d73/db5/de6 to d2/d26/d54/dfb/d104 0 2024-02-16T19:24:30.510 INFO:tasks.workunit.client.1.smithi163.stdout:2/738: mkdir d2/d26/d54/d58/d7d/d105 0 2024-02-16T19:24:30.511 INFO:tasks.workunit.client.1.smithi163.stdout:2/739: link d2/d2a/d39/d66/d6c/lb4 d2/d2a/d39/d66/d6c/dcb/l106 0 2024-02-16T19:24:30.513 INFO:tasks.workunit.client.1.smithi163.stdout:6/523: dwrite d0/dd/d7f/f2d [0,4194304] 0 2024-02-16T19:24:30.513 INFO:tasks.workunit.client.1.smithi163.stdout:2/740: rename d2/ddb/la9 to d2/d26/d54/d58/d8c/dc5/l107 0 2024-02-16T19:24:30.514 INFO:tasks.workunit.client.1.smithi163.stdout:2/741: write d2/d26/d54/f64 [1572686,109651] 0 2024-02-16T19:24:30.514 INFO:tasks.workunit.client.1.smithi163.stdout:2/742: write d2/f1a [5246965,104275] 0 2024-02-16T19:24:30.516 INFO:tasks.workunit.client.1.smithi163.stdout:6/524: rmdir d0/dd/d7f/d81/d98 0 2024-02-16T19:24:30.516 INFO:tasks.workunit.client.1.smithi163.stdout:7/592: rmdir d11/d50/d5d 39 2024-02-16T19:24:30.518 INFO:tasks.workunit.client.1.smithi163.stdout:2/743: creat d2/d26/d54/d58/d8c/dc5/f108 x:0 0 0 2024-02-16T19:24:30.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/593: mkdir d11/d50/d5d/dcd 0 2024-02-16T19:24:30.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/594: readlink d11/d50/d5d/d8b/l41 0 2024-02-16T19:24:30.522 INFO:tasks.workunit.client.1.smithi163.stdout:2/744: unlink d2/f19 0 2024-02-16T19:24:30.526 INFO:tasks.workunit.client.1.smithi163.stdout:2/745: unlink d2/d26/f35 0 2024-02-16T19:24:30.526 INFO:tasks.workunit.client.1.smithi163.stdout:2/746: fdatasync d2/d2a/d39/f57 0 2024-02-16T19:24:30.530 INFO:tasks.workunit.client.1.smithi163.stdout:7/595: dread d11/f15 [0,4194304] 0 2024-02-16T19:24:30.533 INFO:tasks.workunit.client.1.smithi163.stdout:6/525: dwrite d0/d6/f92 [0,4194304] 0 2024-02-16T19:24:30.536 INFO:tasks.workunit.client.1.smithi163.stdout:6/526: mkdir d0/dd/d7f/d81/d85/da4 0 2024-02-16T19:24:30.538 INFO:tasks.workunit.client.1.smithi163.stdout:6/527: mknod d0/d18/d77/ca5 0 2024-02-16T19:24:30.540 INFO:tasks.workunit.client.1.smithi163.stdout:6/528: dread d0/d6/df/f83 [0,4194304] 0 2024-02-16T19:24:30.542 INFO:tasks.workunit.client.1.smithi163.stdout:6/529: rmdir d0/dd/d7f/d2e/d33 39 2024-02-16T19:24:30.544 INFO:tasks.workunit.client.1.smithi163.stdout:6/530: rmdir d0/dd/d7f/d2e 39 2024-02-16T19:24:30.546 INFO:tasks.workunit.client.1.smithi163.stdout:7/596: dwrite d11/d24/d34/fab [0,4194304] 0 2024-02-16T19:24:30.546 INFO:tasks.workunit.client.1.smithi163.stdout:7/597: dread - d11/d24/d25/f9a zero size 2024-02-16T19:24:30.548 INFO:tasks.workunit.client.1.smithi163.stdout:2/747: truncate d2/d2a/d2d/f6e 2947156 0 2024-02-16T19:24:30.548 INFO:tasks.workunit.client.1.smithi163.stdout:2/748: write d2/d26/d54/fd5 [421903,110568] 0 2024-02-16T19:24:30.548 INFO:tasks.workunit.client.1.smithi163.stdout:2/749: chown d2/d26/d54/fd5 8871 1 2024-02-16T19:24:30.548 INFO:tasks.workunit.client.1.smithi163.stdout:2/750: fdatasync d2/d26/d54/d58/fcc 0 2024-02-16T19:24:30.551 INFO:tasks.workunit.client.1.smithi163.stdout:7/598: creat d11/d50/d5d/fce x:0 0 0 2024-02-16T19:24:30.552 INFO:tasks.workunit.client.1.smithi163.stdout:2/751: rmdir d2/d44/de4 0 2024-02-16T19:24:30.554 INFO:tasks.workunit.client.1.smithi163.stdout:2/752: symlink d2/d26/d54/d58/d7d/daf/l109 0 2024-02-16T19:24:30.554 INFO:tasks.workunit.client.1.smithi163.stdout:6/531: write d0/d18/f48 [847866,82347] 0 2024-02-16T19:24:30.554 INFO:tasks.workunit.client.1.smithi163.stdout:6/532: creat d0/d6/df/d20/d69/fa6 x:0 0 0 2024-02-16T19:24:30.554 INFO:tasks.workunit.client.1.smithi163.stdout:6/533: dread - d0/d6/df/d20/d69/fa6 zero size 2024-02-16T19:24:30.556 INFO:tasks.workunit.client.1.smithi163.stdout:2/753: truncate d2/d2a/d39/f57 560014 0 2024-02-16T19:24:30.560 INFO:tasks.workunit.client.1.smithi163.stdout:7/599: dread d11/d50/fa3 [0,4194304] 0 2024-02-16T19:24:30.561 INFO:tasks.workunit.client.1.smithi163.stdout:6/534: rename d0/d6/df/d20/d40/d17/f5e to d0/d6/fa7 0 2024-02-16T19:24:30.561 INFO:tasks.workunit.client.1.smithi163.stdout:6/535: dread - d0/dd/d7f/d15/d1c/f99 zero size 2024-02-16T19:24:30.561 INFO:tasks.workunit.client.1.smithi163.stdout:6/536: chown d0/dd/d7f/d15/d1c/f99 1 1 2024-02-16T19:24:30.563 INFO:tasks.workunit.client.1.smithi163.stdout:7/600: creat d11/d24/d25/d4b/d8c/fcf x:0 0 0 2024-02-16T19:24:30.564 INFO:tasks.workunit.client.1.smithi163.stdout:6/537: symlink d0/d6/df/d20/d40/la8 0 2024-02-16T19:24:30.564 INFO:tasks.workunit.client.1.smithi163.stdout:6/538: dread - d0/d6/df/d8f/f91 zero size 2024-02-16T19:24:30.564 INFO:tasks.workunit.client.1.smithi163.stdout:6/539: fsync d0/d6/f92 0 2024-02-16T19:24:30.565 INFO:tasks.workunit.client.1.smithi163.stdout:6/540: mkdir d0/d6/df/d8f/da9 0 2024-02-16T19:24:30.584 INFO:tasks.workunit.client.1.smithi163.stdout:2/754: dwrite d2/d2a/d39/f75 [0,4194304] 0 2024-02-16T19:24:30.586 INFO:tasks.workunit.client.1.smithi163.stdout:2/755: mknod d2/d44/c10a 0 2024-02-16T19:24:30.587 INFO:tasks.workunit.client.1.smithi163.stdout:2/756: write d2/d2a/d9f/fb9 [1004558,8156] 0 2024-02-16T19:24:30.587 INFO:tasks.workunit.client.1.smithi163.stdout:2/757: read d2/d2a/d2d/d73/d84/f8e [1987928,45928] 0 2024-02-16T19:24:30.587 INFO:tasks.workunit.client.1.smithi163.stdout:2/758: write d2/d2a/d9f/fb9 [1214651,58578] 0 2024-02-16T19:24:30.589 INFO:tasks.workunit.client.1.smithi163.stdout:2/759: creat d2/d2a/d39/d66/d6c/d7f/f10b x:0 0 0 2024-02-16T19:24:30.589 INFO:tasks.workunit.client.1.smithi163.stdout:2/760: stat d2/d2a/d39/d66/d6c/dcb/l106 0 2024-02-16T19:24:30.598 INFO:tasks.workunit.client.1.smithi163.stdout:6/541: dwrite d0/d6/df/d20/d40/d17/f88 [0,4194304] 0 2024-02-16T19:24:30.598 INFO:tasks.workunit.client.1.smithi163.stdout:7/601: dwrite fc [0,4194304] 0 2024-02-16T19:24:30.604 INFO:tasks.workunit.client.1.smithi163.stdout:7/602: symlink d11/d30/ld0 0 2024-02-16T19:24:30.604 INFO:tasks.workunit.client.1.smithi163.stdout:7/603: fdatasync d11/d24/d34/d8d/f92 0 2024-02-16T19:24:30.612 INFO:tasks.workunit.client.1.smithi163.stdout:2/761: dwrite d2/d26/d54/d58/d8c/dc5/f108 [0,4194304] 0 2024-02-16T19:24:30.616 INFO:tasks.workunit.client.1.smithi163.stdout:6/542: write d0/d6/f31 [3517264,100531] 0 2024-02-16T19:24:30.616 INFO:tasks.workunit.client.1.smithi163.stdout:2/762: truncate d2/d2a/d39/d3a/f7b 799663 0 2024-02-16T19:24:30.617 INFO:tasks.workunit.client.1.smithi163.stdout:6/543: rmdir d0/dd/d7f/d81/d85 39 2024-02-16T19:24:30.619 INFO:tasks.workunit.client.1.smithi163.stdout:6/544: mkdir d0/dd/d7f/d81/daa 0 2024-02-16T19:24:30.620 INFO:tasks.workunit.client.1.smithi163.stdout:6/545: symlink d0/dd/d7f/d2e/lab 0 2024-02-16T19:24:30.625 INFO:tasks.workunit.client.1.smithi163.stdout:7/604: dwrite d11/d30/f6a [0,4194304] 0 2024-02-16T19:24:30.628 INFO:tasks.workunit.client.1.smithi163.stdout:7/605: link d11/d50/d5d/d8b/l38 d11/d24/d49/ld1 0 2024-02-16T19:24:30.633 INFO:tasks.workunit.client.1.smithi163.stdout:6/546: write d0/dd/d7f/d2e/d33/f5f [3993003,125933] 0 2024-02-16T19:24:30.633 INFO:tasks.workunit.client.1.smithi163.stdout:6/547: creat d0/d18/d94/fac x:0 0 0 2024-02-16T19:24:30.633 INFO:tasks.workunit.client.1.smithi163.stdout:2/763: dwrite d2/ddb/dc8/fd4 [0,4194304] 0 2024-02-16T19:24:30.633 INFO:tasks.workunit.client.1.smithi163.stdout:2/764: dread - d2/f9d zero size 2024-02-16T19:24:30.637 INFO:tasks.workunit.client.1.smithi163.stdout:6/548: rename d0/d6/df/d20/d40/d53/d74/d84 to d0/dd/d7f/d81/d85/da4/dad 0 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:8/571: sync 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:4/525: sync 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:9/853: sync 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:5/615: sync 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:1/641: sync 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:9/854: write d5/de/d30/d3d/f6f [969918,98086] 0 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:4/526: write d11/f4b [4714030,109970] 0 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:5/616: write d0/d2/d34/fce [307718,9854] 0 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:9/855: readlink d5/de/d30/d31/d52/ld0 0 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:9/856: chown d5/de/d30/d3d/d49/d6e/f8d 72 1 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:9/857: write d5/de/d16/d46/f9c [609956,82339] 0 2024-02-16T19:24:30.638 INFO:tasks.workunit.client.1.smithi163.stdout:9/858: chown d5/de/d16/d47 8 1 2024-02-16T19:24:30.639 INFO:tasks.workunit.client.1.smithi163.stdout:5/617: creat d0/d1c/d5a/fcf x:0 0 0 2024-02-16T19:24:30.639 INFO:tasks.workunit.client.1.smithi163.stdout:2/765: symlink d2/d2a/d2d/d73/d94/dc9/l10c 0 2024-02-16T19:24:30.639 INFO:tasks.workunit.client.1.smithi163.stdout:3/596: sync 2024-02-16T19:24:30.639 INFO:tasks.workunit.client.1.smithi163.stdout:0/870: sync 2024-02-16T19:24:30.640 INFO:tasks.workunit.client.1.smithi163.stdout:0/871: write d11/d22/d7f/f83 [1821215,10832] 0 2024-02-16T19:24:30.643 INFO:tasks.workunit.client.1.smithi163.stdout:9/859: symlink d5/dc0/d102/d11e/l128 0 2024-02-16T19:24:30.643 INFO:tasks.workunit.client.1.smithi163.stdout:9/860: dread - d5/de/d84/fe0 zero size 2024-02-16T19:24:30.644 INFO:tasks.workunit.client.1.smithi163.stdout:5/618: mknod d0/d1c/dbe/cd0 0 2024-02-16T19:24:30.644 INFO:tasks.workunit.client.1.smithi163.stdout:5/619: chown d0/d22/d3b/f42 240 1 2024-02-16T19:24:30.644 INFO:tasks.workunit.client.1.smithi163.stdout:5/620: dread - d0/d22/d3b/dc0/fb0 zero size 2024-02-16T19:24:30.645 INFO:tasks.workunit.client.1.smithi163.stdout:5/621: chown d0/d2/d34/d44/d53/d93/f7a 0 1 2024-02-16T19:24:30.646 INFO:tasks.workunit.client.1.smithi163.stdout:0/872: getdents d11/d12 0 2024-02-16T19:24:30.650 INFO:tasks.workunit.client.1.smithi163.stdout:4/527: write d11/d2d/d4f/f64 [4823900,42378] 0 2024-02-16T19:24:30.650 INFO:tasks.workunit.client.1.smithi163.stdout:4/528: readlink d11/d18/d1f/d29/d52/d57/d6c/l87 0 2024-02-16T19:24:30.652 INFO:tasks.workunit.client.1.smithi163.stdout:1/642: dread d7/f4f [0,4194304] 0 2024-02-16T19:24:30.656 INFO:tasks.workunit.client.0.smithi040.stdout:4/995: sync 2024-02-16T19:24:30.658 INFO:tasks.workunit.client.1.smithi163.stdout:0/873: mknod d11/d21/d50/d58/d72/d9e/dd8/c129 0 2024-02-16T19:24:30.658 INFO:tasks.workunit.client.1.smithi163.stdout:8/572: dwrite d0/d7/d19/d1f/d30/d6a/d7b/f89 [0,4194304] 0 2024-02-16T19:24:30.658 INFO:tasks.workunit.client.1.smithi163.stdout:8/573: stat d0/d7/d18 0 2024-02-16T19:24:30.660 INFO:tasks.workunit.client.0.smithi040.stdout:4/996: mknod dd/c138 0 2024-02-16T19:24:30.662 INFO:tasks.workunit.client.1.smithi163.stdout:0/874: rmdir d11/d4f 39 2024-02-16T19:24:30.662 INFO:tasks.workunit.client.1.smithi163.stdout:0/875: write d11/d21/d50/d58/d72/d9e/dd8/f126 [32959,5105] 0 2024-02-16T19:24:30.662 INFO:tasks.workunit.client.1.smithi163.stdout:0/876: write d11/d12/d67/d6a/d6d/f93 [4504169,37368] 0 2024-02-16T19:24:30.663 INFO:tasks.workunit.client.1.smithi163.stdout:1/643: rmdir d7/d20/d96 39 2024-02-16T19:24:30.663 INFO:tasks.workunit.client.1.smithi163.stdout:1/644: stat d7/dc/d78/d7c/fae 0 2024-02-16T19:24:30.663 INFO:tasks.workunit.client.1.smithi163.stdout:1/645: chown d7/d20/d21/d74/dab/lb4 12741 1 2024-02-16T19:24:30.663 INFO:tasks.workunit.client.1.smithi163.stdout:4/529: dread f10 [0,4194304] 0 2024-02-16T19:24:30.664 INFO:tasks.workunit.client.1.smithi163.stdout:0/877: mkdir d11/d12/d67/d6a/dae/d118/d12a 0 2024-02-16T19:24:30.665 INFO:tasks.workunit.client.1.smithi163.stdout:4/530: mkdir d11/d18/d1f/d29/d52/d57/d6c/d7c/dad 0 2024-02-16T19:24:30.667 INFO:tasks.workunit.client.1.smithi163.stdout:4/531: link d11/d2d/d4f/c69 d11/d18/d1f/d29/d52/d76/da2/cae 0 2024-02-16T19:24:30.671 INFO:tasks.workunit.client.1.smithi163.stdout:4/532: mkdir d11/d2d/d4f/daf 0 2024-02-16T19:24:30.673 INFO:tasks.workunit.client.1.smithi163.stdout:4/533: mknod d11/d18/d1f/d29/d52/d57/cb0 0 2024-02-16T19:24:30.673 INFO:tasks.workunit.client.1.smithi163.stdout:4/534: chown d11/d2d/d4f/f72 664751 1 2024-02-16T19:24:30.676 INFO:tasks.workunit.client.0.smithi040.stdout:4/997: write dd/de/d10/d36/d9e/d32/f111 [685166,47949] 0 2024-02-16T19:24:30.676 INFO:tasks.workunit.client.0.smithi040.stdout:4/998: truncate dd/de/d71/fa3 878423 0 2024-02-16T19:24:30.682 INFO:tasks.workunit.client.1.smithi163.stdout:9/861: dwrite d5/de/d30/d3d/f3b [0,4194304] 0 2024-02-16T19:24:30.682 INFO:tasks.workunit.client.1.smithi163.stdout:9/862: chown d5/de/d16/d46/d9b/de4 25696 1 2024-02-16T19:24:30.684 INFO:tasks.workunit.client.1.smithi163.stdout:9/863: mknod d5/de/d30/d8a/da8/c129 0 2024-02-16T19:24:30.685 INFO:tasks.workunit.client.1.smithi163.stdout:9/864: getdents d5/de/d30/d43 0 2024-02-16T19:24:30.685 INFO:tasks.workunit.client.1.smithi163.stdout:9/865: dread - d5/de/d16/d46/d9b/fee zero size 2024-02-16T19:24:30.685 INFO:tasks.workunit.client.1.smithi163.stdout:9/866: chown d5/de/d30/d3d/d49/d6e/f73 2263387 1 2024-02-16T19:24:30.685 INFO:tasks.workunit.client.1.smithi163.stdout:9/867: truncate d5/de/d16/d46/d9b/fc6 420175 0 2024-02-16T19:24:30.700 INFO:tasks.workunit.client.1.smithi163.stdout:2/766: dwrite d2/d2a/d39/f42 [0,4194304] 0 2024-02-16T19:24:30.700 INFO:tasks.workunit.client.1.smithi163.stdout:2/767: dread - d2/dce/dee/f102 zero size 2024-02-16T19:24:30.700 INFO:tasks.workunit.client.1.smithi163.stdout:2/768: dread - d2/d26/d54/d58/d98/fab zero size 2024-02-16T19:24:30.701 INFO:tasks.workunit.client.1.smithi163.stdout:2/769: rmdir d2/d26/d54/dfb 39 2024-02-16T19:24:30.703 INFO:tasks.workunit.client.1.smithi163.stdout:2/770: dread d2/d2a/d2d/f6e [0,4194304] 0 2024-02-16T19:24:30.703 INFO:tasks.workunit.client.1.smithi163.stdout:2/771: dread - d2/d26/fd9 zero size 2024-02-16T19:24:30.703 INFO:tasks.workunit.client.1.smithi163.stdout:2/772: chown d2/d2a/d2d/d73/d94/dc9/f103 4852 1 2024-02-16T19:24:30.704 INFO:tasks.workunit.client.1.smithi163.stdout:2/773: dread d2/d2a/d39/fae [0,4194304] 0 2024-02-16T19:24:30.704 INFO:tasks.workunit.client.1.smithi163.stdout:2/774: fsync d2/d2a/d2d/f5e 0 2024-02-16T19:24:30.705 INFO:tasks.workunit.client.1.smithi163.stdout:2/775: creat d2/d26/d54/d58/d98/f10d x:0 0 0 2024-02-16T19:24:30.706 INFO:tasks.workunit.client.1.smithi163.stdout:5/622: dwrite d0/d2/f57 [0,4194304] 0 2024-02-16T19:24:30.708 INFO:tasks.workunit.client.1.smithi163.stdout:5/623: creat d0/d2/d34/d44/d53/d93/da8/fd1 x:0 0 0 2024-02-16T19:24:30.708 INFO:tasks.workunit.client.1.smithi163.stdout:5/624: truncate d0/d2/d34/fc6 489256 0 2024-02-16T19:24:30.709 INFO:tasks.workunit.client.1.smithi163.stdout:5/625: truncate d0/d2/d34/d44/f7f 2070765 0 2024-02-16T19:24:30.710 INFO:tasks.workunit.client.1.smithi163.stdout:5/626: symlink d0/d2/d34/d44/d53/d93/ld2 0 2024-02-16T19:24:30.721 INFO:tasks.workunit.client.1.smithi163.stdout:4/535: dwrite d11/d2d/d4f/d60/d79/f90 [0,4194304] 0 2024-02-16T19:24:30.722 INFO:tasks.workunit.client.1.smithi163.stdout:1/646: dwrite d7/d20/d21/d4e/dc6/fdf [0,4194304] 0 2024-02-16T19:24:30.723 INFO:tasks.workunit.client.1.smithi163.stdout:8/574: dwrite d0/d7/d19/d1f/f4e [0,4194304] 0 2024-02-16T19:24:30.723 INFO:tasks.workunit.client.1.smithi163.stdout:0/878: dwrite d11/d22/fa7 [4194304,4194304] 0 2024-02-16T19:24:30.723 INFO:tasks.workunit.client.1.smithi163.stdout:0/879: fsync d11/d22/d7f/f112 0 2024-02-16T19:24:30.723 INFO:tasks.workunit.client.1.smithi163.stdout:2/776: dwrite d2/d26/d54/fe1 [0,4194304] 0 2024-02-16T19:24:30.724 INFO:tasks.workunit.client.0.smithi040.stdout:4/999: dwrite dd/de/d10/d36/d9e/d32/f41 [0,4194304] 0 2024-02-16T19:24:30.725 INFO:tasks.workunit.client.1.smithi163.stdout:4/536: link d11/d18/d1f/d29/d52/d57/d6c/d7c/cab d11/d18/d1f/d23/d3a/cb1 0 2024-02-16T19:24:30.728 INFO:tasks.workunit.client.0.smithi040.stderr:+ rm -rf -- ./tmp.MFeDXfBFbL 2024-02-16T19:24:30.731 INFO:tasks.workunit.client.1.smithi163.stdout:0/880: mknod d11/d12/d67/d6a/c12b 0 2024-02-16T19:24:30.734 INFO:tasks.workunit.client.1.smithi163.stdout:5/627: fsync d0/d2/d34/d44/f7f 0 2024-02-16T19:24:30.738 INFO:tasks.workunit.client.1.smithi163.stdout:8/575: write d0/d7/d18/d53/f9d [3606742,47302] 0 2024-02-16T19:24:30.738 INFO:tasks.workunit.client.1.smithi163.stdout:9/868: dwrite d5/de/d16/d46/f11c [0,4194304] 0 2024-02-16T19:24:30.740 INFO:tasks.workunit.client.1.smithi163.stdout:0/881: dread d11/d4f/fb0 [0,4194304] 0 2024-02-16T19:24:30.742 INFO:tasks.workunit.client.1.smithi163.stdout:4/537: creat d11/d2d/d4f/daf/fb2 x:0 0 0 2024-02-16T19:24:30.742 INFO:tasks.workunit.client.1.smithi163.stdout:4/538: dread - d11/d18/d1f/d29/d52/d57/f84 zero size 2024-02-16T19:24:30.744 INFO:tasks.workunit.client.1.smithi163.stdout:5/628: creat d0/d22/d28/d36/d3c/d4f/fd3 x:0 0 0 2024-02-16T19:24:30.746 INFO:tasks.workunit.client.1.smithi163.stdout:8/576: rename d0/d7/d18/cb4 to d0/d24/d27/cc1 0 2024-02-16T19:24:30.749 INFO:tasks.workunit.client.1.smithi163.stdout:9/869: symlink d5/de/d30/d3d/d93/l12a 0 2024-02-16T19:24:30.750 INFO:tasks.workunit.client.1.smithi163.stdout:4/539: getdents d11/d2d/d4f/d5f/d70 0 2024-02-16T19:24:30.750 INFO:tasks.workunit.client.1.smithi163.stdout:9/870: chown d5/de/d16/d47/f86 1023403 1 2024-02-16T19:24:30.750 INFO:tasks.workunit.client.1.smithi163.stdout:4/540: fdatasync d11/f4b 0 2024-02-16T19:24:30.750 INFO:tasks.workunit.client.1.smithi163.stdout:2/777: dwrite d2/d2a/d2d/f6e [0,4194304] 0 2024-02-16T19:24:30.750 INFO:tasks.workunit.client.1.smithi163.stdout:1/647: dwrite d7/d20/d21/d74/f76 [0,4194304] 0 2024-02-16T19:24:30.753 INFO:tasks.workunit.client.1.smithi163.stdout:8/577: mknod d0/d24/d78/db1/cc2 0 2024-02-16T19:24:30.754 INFO:tasks.workunit.client.1.smithi163.stdout:9/871: rmdir d5/de/df6 39 2024-02-16T19:24:30.754 INFO:tasks.workunit.client.1.smithi163.stdout:9/872: creat d5/de/d30/d3d/d49/d6d/d97/f12b x:0 0 0 2024-02-16T19:24:30.754 INFO:tasks.workunit.client.1.smithi163.stdout:9/873: write d5/fc8 [2469014,13077] 0 2024-02-16T19:24:30.754 INFO:tasks.workunit.client.1.smithi163.stdout:4/541: mknod d11/d18/d1f/d29/cb3 0 2024-02-16T19:24:30.754 INFO:tasks.workunit.client.1.smithi163.stdout:4/542: creat d11/d3d/fb4 x:0 0 0 2024-02-16T19:24:30.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/648: link d7/fb d7/d20/d96/fe0 0 2024-02-16T19:24:30.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/649: read - d7/dc/d2d/d3f/fad zero size 2024-02-16T19:24:30.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/650: stat d7/dc/d78/d7c/c34 0 2024-02-16T19:24:30.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/651: fdatasync d7/d20/d21/d4e/f79 0 2024-02-16T19:24:30.757 INFO:tasks.workunit.client.1.smithi163.stdout:1/652: read d7/d5b/fa4 [558815,104316] 0 2024-02-16T19:24:30.763 INFO:tasks.workunit.client.1.smithi163.stdout:5/629: rmdir d0/d2 39 2024-02-16T19:24:30.765 INFO:tasks.workunit.client.1.smithi163.stdout:0/882: dwrite d11/d22/f24 [0,4194304] 0 2024-02-16T19:24:30.765 INFO:tasks.workunit.client.1.smithi163.stdout:0/883: stat d11/d21/d50/d58/d72/d9e/ffe 0 2024-02-16T19:24:30.766 INFO:tasks.workunit.client.1.smithi163.stdout:0/884: stat d11/f98 0 2024-02-16T19:24:30.766 INFO:tasks.workunit.client.1.smithi163.stdout:0/885: chown d11/d12/d20/f3a 228774 1 2024-02-16T19:24:30.766 INFO:tasks.workunit.client.1.smithi163.stdout:0/886: write d11/d22/d61/f78 [452679,26904] 0 2024-02-16T19:24:30.766 INFO:tasks.workunit.client.1.smithi163.stdout:8/578: write d0/d7/d19/d1f/f4e [4037305,115197] 0 2024-02-16T19:24:30.768 INFO:tasks.workunit.client.1.smithi163.stdout:9/874: rename d5/l27 to d5/de/d30/l12c 0 2024-02-16T19:24:30.768 INFO:tasks.workunit.client.1.smithi163.stdout:9/875: truncate d5/de/d16/d46/d9b/fc6 1235917 0 2024-02-16T19:24:30.771 INFO:tasks.workunit.client.1.smithi163.stdout:4/543: unlink d11/d18/d1f/l37 0 2024-02-16T19:24:30.773 INFO:tasks.workunit.client.1.smithi163.stdout:1/653: mkdir d7/dc/d2d/d32/d87/de1 0 2024-02-16T19:24:30.773 INFO:tasks.workunit.client.1.smithi163.stdout:1/654: stat d7/dc/c10 0 2024-02-16T19:24:30.773 INFO:tasks.workunit.client.1.smithi163.stdout:1/655: write d7/d20/d2a/f5c [269698,6349] 0 2024-02-16T19:24:30.774 INFO:tasks.workunit.client.1.smithi163.stdout:5/630: mkdir d0/d2/d34/d44/dd4 0 2024-02-16T19:24:30.776 INFO:tasks.workunit.client.1.smithi163.stdout:0/887: mkdir d11/d22/d7f/dcf/d12c 0 2024-02-16T19:24:30.777 INFO:tasks.workunit.client.1.smithi163.stdout:8/579: link d0/d7/d18/c1a d0/d7/d55/cc3 0 2024-02-16T19:24:30.779 INFO:tasks.workunit.client.1.smithi163.stdout:5/631: unlink d0/d2/d34/d44/f7f 0 2024-02-16T19:24:30.780 INFO:tasks.workunit.client.1.smithi163.stdout:8/580: link d0/d7/c2a d0/d7/d19/d1f/d32/cc4 0 2024-02-16T19:24:30.780 INFO:tasks.workunit.client.1.smithi163.stdout:8/581: chown d0/l28 62 1 2024-02-16T19:24:30.781 INFO:tasks.workunit.client.1.smithi163.stdout:5/632: link d0/d22/d28/d36/f6b d0/d2/fd5 0 2024-02-16T19:24:30.782 INFO:tasks.workunit.client.1.smithi163.stdout:5/633: mknod d0/d1c/d60/cd6 0 2024-02-16T19:24:30.782 INFO:tasks.workunit.client.1.smithi163.stdout:5/634: creat d0/d1c/fd7 x:0 0 0 2024-02-16T19:24:30.785 INFO:tasks.workunit.client.1.smithi163.stdout:5/635: mknod d0/cd8 0 2024-02-16T19:24:30.791 INFO:tasks.workunit.client.1.smithi163.stdout:8/582: write d0/d7/d19/d1f/d32/f34 [2748446,77504] 0 2024-02-16T19:24:30.795 INFO:tasks.workunit.client.1.smithi163.stdout:9/876: dwrite d5/de/d30/d3d/d49/d6e/f73 [0,4194304] 0 2024-02-16T19:24:30.796 INFO:tasks.workunit.client.1.smithi163.stdout:9/877: dread - d5/de/d72/f109 zero size 2024-02-16T19:24:30.798 INFO:tasks.workunit.client.1.smithi163.stdout:9/878: creat d5/dc0/d102/d11e/f12d x:0 0 0 2024-02-16T19:24:30.798 INFO:tasks.workunit.client.1.smithi163.stdout:9/879: stat d5/de/d30/le6 0 2024-02-16T19:24:30.798 INFO:tasks.workunit.client.1.smithi163.stdout:9/880: fsync d5/de/d30/d3d/d49/d6e/f8d 0 2024-02-16T19:24:30.798 INFO:tasks.workunit.client.1.smithi163.stdout:5/636: dread d0/d2/d5c/f8c [0,4194304] 0 2024-02-16T19:24:30.798 INFO:tasks.workunit.client.1.smithi163.stdout:5/637: chown d0/d22/d28/d36/faa 373 1 2024-02-16T19:24:30.799 INFO:tasks.workunit.client.1.smithi163.stdout:9/881: creat d5/de/d84/f12e x:0 0 0 2024-02-16T19:24:30.799 INFO:tasks.workunit.client.1.smithi163.stdout:9/882: readlink d5/de/d30/d3d/l100 0 2024-02-16T19:24:30.800 INFO:tasks.workunit.client.1.smithi163.stdout:5/638: mkdir d0/d2/d34/d44/d64/dd9 0 2024-02-16T19:24:30.800 INFO:tasks.workunit.client.1.smithi163.stdout:5/639: write d0/f8e [366877,77404] 0 2024-02-16T19:24:30.801 INFO:tasks.workunit.client.1.smithi163.stdout:8/583: dread d0/d7/d19/d1f/d32/f34 [0,4194304] 0 2024-02-16T19:24:30.801 INFO:tasks.workunit.client.1.smithi163.stdout:8/584: creat d0/d7/d18/d53/fc5 x:0 0 0 2024-02-16T19:24:30.801 INFO:tasks.workunit.client.1.smithi163.stdout:9/883: mknod d5/de/d72/da9/c12f 0 2024-02-16T19:24:30.801 INFO:tasks.workunit.client.1.smithi163.stdout:5/640: creat d0/d2/d34/d44/d53/d93/da8/fda x:0 0 0 2024-02-16T19:24:30.802 INFO:tasks.workunit.client.1.smithi163.stdout:9/884: write d5/de/d30/d3d/d49/d6d/d97/fe1 [514450,13886] 0 2024-02-16T19:24:30.802 INFO:tasks.workunit.client.1.smithi163.stdout:9/885: creat d5/de/d84/f130 x:0 0 0 2024-02-16T19:24:30.803 INFO:tasks.workunit.client.1.smithi163.stdout:9/886: mknod d5/de/d16/d46/d9b/d121/c131 0 2024-02-16T19:24:30.804 INFO:tasks.workunit.client.1.smithi163.stdout:9/887: creat d5/de/d16/d46/d9b/d106/f132 x:0 0 0 2024-02-16T19:24:30.811 INFO:tasks.workunit.client.1.smithi163.stdout:8/585: dread d0/dc/f6d [0,4194304] 0 2024-02-16T19:24:30.811 INFO:tasks.workunit.client.1.smithi163.stdout:8/586: readlink d0/d7/d18/l3e 0 2024-02-16T19:24:30.811 INFO:tasks.workunit.client.1.smithi163.stdout:8/587: stat d0/d7/d18 0 2024-02-16T19:24:30.812 INFO:tasks.workunit.client.1.smithi163.stdout:8/588: truncate d0/d24/d27/f64 1659856 0 2024-02-16T19:24:30.812 INFO:tasks.workunit.client.1.smithi163.stdout:8/589: readlink d0/dc/l4f 0 2024-02-16T19:24:30.812 INFO:tasks.workunit.client.1.smithi163.stdout:8/590: chown d0/d7/f68 1033332411 1 2024-02-16T19:24:30.816 INFO:tasks.workunit.client.1.smithi163.stdout:8/591: rename d0/d7/d19/d1f/d32/fa2 to d0/d7/d19/fc6 0 2024-02-16T19:24:30.816 INFO:tasks.workunit.client.1.smithi163.stdout:8/592: dread - d0/d24/d3c/f8a zero size 2024-02-16T19:24:30.816 INFO:tasks.workunit.client.1.smithi163.stdout:8/593: chown d0/d7/d18/d2e/f5d 5453 1 2024-02-16T19:24:30.833 INFO:tasks.workunit.client.1.smithi163.stdout:0/888: dwrite f3 [0,4194304] 0 2024-02-16T19:24:30.834 INFO:tasks.workunit.client.1.smithi163.stdout:0/889: creat d11/d22/d3e/f12d x:0 0 0 2024-02-16T19:24:30.834 INFO:tasks.workunit.client.1.smithi163.stdout:8/594: dread d0/d7/d19/d1f/d30/f92 [0,4194304] 0 2024-02-16T19:24:30.834 INFO:tasks.workunit.client.1.smithi163.stdout:0/890: chown d11/d22/d7f/d8e/fa4 15399 1 2024-02-16T19:24:30.835 INFO:tasks.workunit.client.1.smithi163.stdout:0/891: fdatasync d11/d21/d50/d58/d6b/f7a 0 2024-02-16T19:24:30.836 INFO:tasks.workunit.client.1.smithi163.stdout:1/656: dwrite d7/d20/d21/d4e/f9e [0,4194304] 0 2024-02-16T19:24:30.836 INFO:tasks.workunit.client.1.smithi163.stdout:0/892: getdents d11/d12/d67/d6a/dae/dcc 0 2024-02-16T19:24:30.836 INFO:tasks.workunit.client.1.smithi163.stdout:0/893: stat d11/d22/d61/ca0 0 2024-02-16T19:24:30.836 INFO:tasks.workunit.client.1.smithi163.stdout:0/894: fdatasync d11/d12/d67/d6a/dae/f105 0 2024-02-16T19:24:30.836 INFO:tasks.workunit.client.1.smithi163.stdout:4/544: dwrite d11/f65 [0,4194304] 0 2024-02-16T19:24:30.836 INFO:tasks.workunit.client.1.smithi163.stdout:9/888: dwrite d5/de/d30/d31/d52/fe7 [0,4194304] 0 2024-02-16T19:24:30.836 INFO:tasks.workunit.client.1.smithi163.stdout:9/889: fdatasync d5/de/d30/d43/d8f/f90 0 2024-02-16T19:24:30.838 INFO:tasks.workunit.client.1.smithi163.stdout:5/641: dwrite d0/d2/d55/f91 [0,4194304] 0 2024-02-16T19:24:30.838 INFO:tasks.workunit.client.1.smithi163.stdout:5/642: dread - d0/d2/f29 zero size 2024-02-16T19:24:30.838 INFO:tasks.workunit.client.1.smithi163.stdout:5/643: fsync d0/d22/d28/d36/d3c/d4f/f62 0 2024-02-16T19:24:30.838 INFO:tasks.workunit.client.1.smithi163.stdout:5/644: chown d0/d22/d28/d36/d3c/d4f/lac 3402829 1 2024-02-16T19:24:30.838 INFO:tasks.workunit.client.1.smithi163.stdout:1/657: mkdir d7/d20/d21/d74/de2 0 2024-02-16T19:24:30.840 INFO:tasks.workunit.client.1.smithi163.stdout:0/895: link d11/d12/d20/fb9 d11/d12/d67/d6a/d6d/f12e 0 2024-02-16T19:24:30.842 INFO:tasks.workunit.client.1.smithi163.stdout:4/545: read d11/d18/d1f/d34/f51 [1576382,51233] 0 2024-02-16T19:24:30.842 INFO:tasks.workunit.client.1.smithi163.stdout:4/546: dread - d11/d18/d66/f98 zero size 2024-02-16T19:24:30.845 INFO:tasks.workunit.client.1.smithi163.stdout:9/890: truncate d5/de/d16/d47/f86 3372537 0 2024-02-16T19:24:30.846 INFO:tasks.workunit.client.1.smithi163.stdout:5/645: dread d0/d22/d28/d36/db7/fb8 [0,4194304] 0 2024-02-16T19:24:30.851 INFO:tasks.workunit.client.1.smithi163.stdout:8/595: dwrite d0/d7/d19/d1f/d30/f52 [4194304,4194304] 0 2024-02-16T19:24:30.858 INFO:tasks.workunit.client.1.smithi163.stdout:0/896: mkdir d11/d11d/d12f 0 2024-02-16T19:24:30.858 INFO:tasks.workunit.client.1.smithi163.stdout:0/897: dread - d11/d22/f125 zero size 2024-02-16T19:24:30.858 INFO:tasks.workunit.client.1.smithi163.stdout:0/898: fdatasync d11/d22/fa3 0 2024-02-16T19:24:30.859 INFO:tasks.workunit.client.1.smithi163.stdout:4/547: rename d11/d2d/f99 to d11/d5b/d8f/fb5 0 2024-02-16T19:24:30.860 INFO:tasks.workunit.client.1.smithi163.stdout:9/891: unlink d5/de/d30/d8a/da8/l125 0 2024-02-16T19:24:30.862 INFO:tasks.workunit.client.1.smithi163.stdout:1/658: dwrite d7/dc/d2d/fba [0,4194304] 0 2024-02-16T19:24:30.863 INFO:tasks.workunit.client.1.smithi163.stdout:5/646: creat d0/d1c/d60/fdb x:0 0 0 2024-02-16T19:24:30.867 INFO:tasks.workunit.client.1.smithi163.stdout:8/596: rename d0/d7/f68 to d0/d7/d19/d1f/fc7 0 2024-02-16T19:24:30.867 INFO:tasks.workunit.client.1.smithi163.stdout:8/597: dread - d0/d24/d27/d45/fae zero size 2024-02-16T19:24:30.869 INFO:tasks.workunit.client.1.smithi163.stdout:4/548: write d11/d2d/d4f/f64 [3712629,36200] 0 2024-02-16T19:24:30.871 INFO:tasks.workunit.client.1.smithi163.stdout:0/899: dwrite d11/d22/d7f/d8e/fa4 [4194304,4194304] 0 2024-02-16T19:24:30.872 INFO:tasks.workunit.client.1.smithi163.stdout:5/647: creat d0/d2/d34/d44/d64/dd9/fdc x:0 0 0 2024-02-16T19:24:30.873 INFO:tasks.workunit.client.1.smithi163.stdout:0/900: dread d11/d21/d50/d58/d72/fa8 [4194304,4194304] 0 2024-02-16T19:24:30.873 INFO:tasks.workunit.client.1.smithi163.stdout:0/901: creat d11/d21/f130 x:0 0 0 2024-02-16T19:24:30.874 INFO:tasks.workunit.client.1.smithi163.stdout:8/598: unlink d0/d7/d19/f2f 0 2024-02-16T19:24:30.874 INFO:tasks.workunit.client.1.smithi163.stdout:8/599: truncate d0/d7/d19/fa8 292882 0 2024-02-16T19:24:30.874 INFO:tasks.workunit.client.1.smithi163.stdout:8/600: creat d0/d6f/fc8 x:0 0 0 2024-02-16T19:24:30.875 INFO:tasks.workunit.client.1.smithi163.stdout:9/892: mkdir d5/de/d16/d133 0 2024-02-16T19:24:30.875 INFO:tasks.workunit.client.1.smithi163.stdout:9/893: truncate d5/de/d30/d3d/d49/d6d/d97/fd8 264235 0 2024-02-16T19:24:30.877 INFO:tasks.workunit.client.1.smithi163.stdout:4/549: unlink d11/d18/d1f/d29/cb3 0 2024-02-16T19:24:30.882 INFO:tasks.workunit.client.1.smithi163.stdout:0/902: mkdir d11/d12/d67/d6a/d6d/d131 0 2024-02-16T19:24:30.884 INFO:tasks.workunit.client.1.smithi163.stdout:8/601: creat d0/d7/d19/d1f/d30/d6a/db8/fc9 x:0 0 0 2024-02-16T19:24:30.886 INFO:tasks.workunit.client.1.smithi163.stdout:9/894: creat d5/de/d72/da9/d104/f134 x:0 0 0 2024-02-16T19:24:30.886 INFO:tasks.workunit.client.1.smithi163.stdout:9/895: chown d5/l10 25880 1 2024-02-16T19:24:30.887 INFO:tasks.workunit.client.1.smithi163.stdout:4/550: creat d11/d5b/d8f/fb6 x:0 0 0 2024-02-16T19:24:30.888 INFO:tasks.workunit.client.1.smithi163.stdout:1/659: dwrite d7/d20/d2a/f40 [0,4194304] 0 2024-02-16T19:24:30.888 INFO:tasks.workunit.client.1.smithi163.stdout:1/660: chown d7/dc/d2d/d3f/lb2 14726389 1 2024-02-16T19:24:30.889 INFO:tasks.workunit.client.1.smithi163.stdout:0/903: link d11/d12/c4d d11/d22/d7f/d8e/c132 0 2024-02-16T19:24:30.891 INFO:tasks.workunit.client.1.smithi163.stdout:9/896: symlink d5/de/d16/d46/d9b/d121/l135 0 2024-02-16T19:24:30.891 INFO:tasks.workunit.client.1.smithi163.stdout:9/897: stat d5/de/d30/d8a/da8/fcb 0 2024-02-16T19:24:30.891 INFO:tasks.workunit.client.1.smithi163.stdout:4/551: dread d11/f14 [0,4194304] 0 2024-02-16T19:24:30.892 INFO:tasks.workunit.client.1.smithi163.stdout:0/904: creat d11/d4f/f133 x:0 0 0 2024-02-16T19:24:30.893 INFO:tasks.workunit.client.1.smithi163.stdout:5/648: dwrite d0/d22/d3b/d8d/fab [0,4194304] 0 2024-02-16T19:24:30.894 INFO:tasks.workunit.client.1.smithi163.stdout:9/898: truncate d5/de/d30/d31/f54 58804 0 2024-02-16T19:24:30.894 INFO:tasks.workunit.client.1.smithi163.stdout:9/899: creat d5/de/d16/d46/f136 x:0 0 0 2024-02-16T19:24:30.898 INFO:tasks.workunit.client.1.smithi163.stdout:7/606: sync 2024-02-16T19:24:30.898 INFO:tasks.workunit.client.1.smithi163.stdout:6/549: sync 2024-02-16T19:24:30.899 INFO:tasks.workunit.client.1.smithi163.stdout:2/778: sync 2024-02-16T19:24:30.899 INFO:tasks.workunit.client.1.smithi163.stdout:3/597: sync 2024-02-16T19:24:30.900 INFO:tasks.workunit.client.1.smithi163.stdout:4/552: mknod d11/d2d/d4f/d5f/d70/cb7 0 2024-02-16T19:24:30.900 INFO:tasks.workunit.client.1.smithi163.stdout:5/649: creat d0/d2/d31/fdd x:0 0 0 2024-02-16T19:24:30.900 INFO:tasks.workunit.client.1.smithi163.stdout:9/900: creat d5/de/d30/d3d/d49/f137 x:0 0 0 2024-02-16T19:24:30.900 INFO:tasks.workunit.client.1.smithi163.stdout:8/602: dread d0/d24/d2c/f4d [0,4194304] 0 2024-02-16T19:24:30.902 INFO:tasks.workunit.client.1.smithi163.stdout:7/607: unlink ff 0 2024-02-16T19:24:30.902 INFO:tasks.workunit.client.1.smithi163.stdout:8/603: write d0/d7/d18/d2e/f8f [675585,83603] 0 2024-02-16T19:24:30.903 INFO:tasks.workunit.client.1.smithi163.stdout:3/598: rmdir d0/dc/d50/d66/d93 39 2024-02-16T19:24:30.903 INFO:tasks.workunit.client.1.smithi163.stdout:8/604: write d0/d7/d18/d53/f9d [4549787,41213] 0 2024-02-16T19:24:30.903 INFO:tasks.workunit.client.1.smithi163.stdout:1/661: dread d7/dc/d2d/d32/f64 [0,4194304] 0 2024-02-16T19:24:30.903 INFO:tasks.workunit.client.1.smithi163.stdout:8/605: rename d0/d7 to d0/d7/d19/d1f/d32/dca 22 2024-02-16T19:24:30.904 INFO:tasks.workunit.client.1.smithi163.stdout:1/662: creat d7/d20/d21/fe3 x:0 0 0 2024-02-16T19:24:30.905 INFO:tasks.workunit.client.1.smithi163.stdout:2/779: link d2/d2a/d2d/d73/lbf d2/d2a/d9f/df8/l10e 0 2024-02-16T19:24:30.907 INFO:tasks.workunit.client.1.smithi163.stdout:5/650: symlink d0/d1c/dbe/lde 0 2024-02-16T19:24:30.907 INFO:tasks.workunit.client.1.smithi163.stdout:6/550: write d0/d6/df/f2a [4071024,78045] 0 2024-02-16T19:24:30.909 INFO:tasks.workunit.client.1.smithi163.stdout:3/599: read d0/dc/d94/d9e/dac/fd0 [60528,63464] 0 2024-02-16T19:24:30.910 INFO:tasks.workunit.client.1.smithi163.stdout:9/901: link d5/de/d16/d46/d9b/fc6 d5/de/d84/f138 0 2024-02-16T19:24:30.910 INFO:tasks.workunit.client.1.smithi163.stdout:9/902: chown d5/de/d16/d47/l48 1430525 1 2024-02-16T19:24:30.910 INFO:tasks.workunit.client.1.smithi163.stdout:9/903: write d5/de/d30/d8a/da8/fcb [5375502,90056] 0 2024-02-16T19:24:30.910 INFO:tasks.workunit.client.1.smithi163.stdout:9/904: write d5/de/d30/d3d/d49/df3/ffc [339204,76050] 0 2024-02-16T19:24:30.910 INFO:tasks.workunit.client.1.smithi163.stdout:9/905: readlink d5/de/l26 0 2024-02-16T19:24:30.915 INFO:tasks.workunit.client.1.smithi163.stdout:8/606: mkdir d0/d24/d2c/d74/dcb 0 2024-02-16T19:24:30.916 INFO:tasks.workunit.client.1.smithi163.stdout:8/607: write d0/d24/d2c/f3b [4635181,72670] 0 2024-02-16T19:24:30.916 INFO:tasks.workunit.client.1.smithi163.stdout:7/608: dread d11/f1f [0,4194304] 0 2024-02-16T19:24:30.917 INFO:tasks.workunit.client.1.smithi163.stdout:7/609: chown d11/d50/l57 310907 1 2024-02-16T19:24:30.917 INFO:tasks.workunit.client.1.smithi163.stdout:7/610: dread - d11/d24/d49/f84 zero size 2024-02-16T19:24:30.917 INFO:tasks.workunit.client.1.smithi163.stdout:7/611: truncate d11/d26/f6d 2681342 0 2024-02-16T19:24:30.917 INFO:tasks.workunit.client.1.smithi163.stdout:1/663: creat d7/dc/d2d/d32/d87/da6/fe4 x:0 0 0 2024-02-16T19:24:30.918 INFO:tasks.workunit.client.1.smithi163.stdout:5/651: getdents d0/d2/d5c 0 2024-02-16T19:24:30.919 INFO:tasks.workunit.client.1.smithi163.stdout:5/652: getdents d0/d2/d5c 0 2024-02-16T19:24:30.920 INFO:tasks.workunit.client.1.smithi163.stdout:6/551: unlink d0/d18/d77/f6a 0 2024-02-16T19:24:30.921 INFO:tasks.workunit.client.1.smithi163.stdout:3/600: symlink d0/dc/d1d/d49/d98/ld9 0 2024-02-16T19:24:30.924 INFO:tasks.workunit.client.1.smithi163.stdout:8/608: mknod d0/d7/d19/d1f/d30/d6a/ccc 0 2024-02-16T19:24:30.924 INFO:tasks.workunit.client.1.smithi163.stdout:8/609: creat d0/d7/d19/d1f/d30/d6a/db8/fcd x:0 0 0 2024-02-16T19:24:30.925 INFO:tasks.workunit.client.1.smithi163.stdout:5/653: creat d0/d2/d34/d44/d53/fdf x:0 0 0 2024-02-16T19:24:30.925 INFO:tasks.workunit.client.1.smithi163.stdout:5/654: stat d0/d2/d55/fae 0 2024-02-16T19:24:30.927 INFO:tasks.workunit.client.1.smithi163.stdout:8/610: getdents d0/dc 0 2024-02-16T19:24:30.927 INFO:tasks.workunit.client.1.smithi163.stdout:8/611: fsync d0/faa 0 2024-02-16T19:24:30.928 INFO:tasks.workunit.client.1.smithi163.stdout:9/906: write d5/de/d16/d47/f86 [40448,10376] 0 2024-02-16T19:24:30.928 INFO:tasks.workunit.client.1.smithi163.stdout:8/612: mknod d0/d7/d19/d1f/d32/cce 0 2024-02-16T19:24:30.930 INFO:tasks.workunit.client.1.smithi163.stdout:1/664: dread d7/dc/d2d/d32/d87/f88 [0,4194304] 0 2024-02-16T19:24:30.930 INFO:tasks.workunit.client.1.smithi163.stdout:1/665: write d7/d20/f24 [1304164,129429] 0 2024-02-16T19:24:30.930 INFO:tasks.workunit.client.1.smithi163.stdout:8/613: mkdir d0/d7/d19/d1f/d30/d6a/db8/dcf 0 2024-02-16T19:24:30.932 INFO:tasks.workunit.client.1.smithi163.stdout:1/666: stat d7/dc/l13 0 2024-02-16T19:24:30.940 INFO:tasks.workunit.client.1.smithi163.stdout:6/552: dread d0/d6/df/d20/d40/d17/f4b [0,4194304] 0 2024-02-16T19:24:30.940 INFO:tasks.workunit.client.1.smithi163.stdout:1/667: dread d7/dc/d2d/d32/f64 [0,4194304] 0 2024-02-16T19:24:30.942 INFO:tasks.workunit.client.1.smithi163.stdout:6/553: rename d0/dd/d7f/d2e/l5d to d0/dd/d7f/d81/daa/lae 0 2024-02-16T19:24:30.943 INFO:tasks.workunit.client.1.smithi163.stdout:0/905: dwrite d11/d12/d67/d6a/d6d/f93 [0,4194304] 0 2024-02-16T19:24:30.943 INFO:tasks.workunit.client.1.smithi163.stdout:4/553: dwrite d11/d18/d1f/d29/d52/d57/d6c/d7c/f96 [4194304,4194304] 0 2024-02-16T19:24:30.944 INFO:tasks.workunit.client.1.smithi163.stdout:1/668: mkdir d7/dc/d2d/d32/d87/da6/de5 0 2024-02-16T19:24:30.944 INFO:tasks.workunit.client.1.smithi163.stdout:6/554: symlink d0/d18/d77/d97/laf 0 2024-02-16T19:24:30.946 INFO:tasks.workunit.client.1.smithi163.stdout:6/555: truncate d0/d6/df/f32 921033 0 2024-02-16T19:24:30.947 INFO:tasks.workunit.client.1.smithi163.stdout:4/554: rmdir d11/d3d 39 2024-02-16T19:24:30.947 INFO:tasks.workunit.client.1.smithi163.stdout:4/555: fsync d11/f41 0 2024-02-16T19:24:30.949 INFO:tasks.workunit.client.1.smithi163.stdout:6/556: mknod d0/dd/d7f/d15/d1c/d65/d63/cb0 0 2024-02-16T19:24:30.950 INFO:tasks.workunit.client.1.smithi163.stdout:6/557: chown d0/dd/d7f/d15/d1c/d65/c64 1113229 1 2024-02-16T19:24:30.950 INFO:tasks.workunit.client.1.smithi163.stdout:6/558: write d0/d6/df/d20/d40/d1e/f52 [3797759,14807] 0 2024-02-16T19:24:30.952 INFO:tasks.workunit.client.1.smithi163.stdout:2/780: dwrite d2/d26/d54/d58/d8c/ff9 [0,4194304] 0 2024-02-16T19:24:30.952 INFO:tasks.workunit.client.1.smithi163.stdout:2/781: write d2/d2a/d2d/d73/db5/fe2 [354634,63943] 0 2024-02-16T19:24:30.953 INFO:tasks.workunit.client.1.smithi163.stdout:2/782: fsync d2/d2a/d2d/d73/d84/ff5 0 2024-02-16T19:24:30.958 INFO:tasks.workunit.client.1.smithi163.stdout:2/783: mkdir d2/d26/d97/d10f 0 2024-02-16T19:24:30.958 INFO:tasks.workunit.client.1.smithi163.stdout:2/784: stat d2/d26/fd9 0 2024-02-16T19:24:30.960 INFO:tasks.workunit.client.1.smithi163.stdout:2/785: mknod d2/d2a/d2d/d73/db5/c110 0 2024-02-16T19:24:30.965 INFO:tasks.workunit.client.1.smithi163.stdout:7/612: dwrite d11/d24/d25/d4b/d66/dc1/fcb [0,4194304] 0 2024-02-16T19:24:30.965 INFO:tasks.workunit.client.1.smithi163.stdout:7/613: creat d11/d24/d25/d4b/d8c/fd2 x:0 0 0 2024-02-16T19:24:30.969 INFO:tasks.workunit.client.1.smithi163.stdout:5/655: dwrite d0/d2/f16 [0,4194304] 0 2024-02-16T19:24:30.969 INFO:tasks.workunit.client.1.smithi163.stdout:7/614: getdents d11/d24/d49/dc4 0 2024-02-16T19:24:30.969 INFO:tasks.workunit.client.1.smithi163.stdout:7/615: fdatasync d11/d24/d49/f51 0 2024-02-16T19:24:30.971 INFO:tasks.workunit.client.1.smithi163.stdout:5/656: creat d0/d1c/d5a/da5/fe0 x:0 0 0 2024-02-16T19:24:30.971 INFO:tasks.workunit.client.1.smithi163.stdout:5/657: chown d0/d2/f4 130358 1 2024-02-16T19:24:30.971 INFO:tasks.workunit.client.1.smithi163.stdout:5/658: truncate d0/d22/d28/d36/f5e 141778 0 2024-02-16T19:24:30.971 INFO:tasks.workunit.client.1.smithi163.stdout:5/659: dread - d0/d22/d3b/f56 zero size 2024-02-16T19:24:30.971 INFO:tasks.workunit.client.1.smithi163.stdout:5/660: stat d0/c4e 0 2024-02-16T19:24:30.971 INFO:tasks.workunit.client.1.smithi163.stdout:5/661: chown d0/f8e 59993 1 2024-02-16T19:24:30.972 INFO:tasks.workunit.client.1.smithi163.stdout:7/616: mknod d11/d50/d5d/dcd/cd3 0 2024-02-16T19:24:30.972 INFO:tasks.workunit.client.1.smithi163.stdout:7/617: chown d11/d50/c53 50436444 1 2024-02-16T19:24:30.972 INFO:tasks.workunit.client.1.smithi163.stdout:5/662: write d0/f81 [77673,83920] 0 2024-02-16T19:24:30.972 INFO:tasks.workunit.client.1.smithi163.stdout:5/663: readlink d0/d2/l94 0 2024-02-16T19:24:30.974 INFO:tasks.workunit.client.1.smithi163.stdout:5/664: dread d0/d22/d28/d36/d3c/d4f/f7b [0,4194304] 0 2024-02-16T19:24:30.974 INFO:tasks.workunit.client.1.smithi163.stdout:5/665: stat d0/d2/c9a 0 2024-02-16T19:24:30.974 INFO:tasks.workunit.client.1.smithi163.stdout:7/618: truncate d11/d30/f5b 3197819 0 2024-02-16T19:24:30.981 INFO:tasks.workunit.client.1.smithi163.stdout:3/601: dwrite d0/dc/d50/d66/da3/f86 [0,4194304] 0 2024-02-16T19:24:30.981 INFO:tasks.workunit.client.1.smithi163.stdout:9/907: dwrite d5/de/d16/f6b [4194304,4194304] 0 2024-02-16T19:24:30.981 INFO:tasks.workunit.client.1.smithi163.stdout:9/908: creat d5/de/d72/f139 x:0 0 0 2024-02-16T19:24:30.981 INFO:tasks.workunit.client.1.smithi163.stdout:9/909: stat d5/f6 0 2024-02-16T19:24:30.994 INFO:tasks.workunit.client.1.smithi163.stdout:0/906: dwrite d11/f85 [0,4194304] 0 2024-02-16T19:24:30.996 INFO:tasks.workunit.client.1.smithi163.stdout:0/907: chown d11/d4f/lc2 118 1 2024-02-16T19:24:30.996 INFO:tasks.workunit.client.1.smithi163.stdout:0/908: chown d11/d22/d3e/fdf 6 1 2024-02-16T19:24:30.999 INFO:tasks.workunit.client.1.smithi163.stdout:9/910: write d5/de/d16/d46/d9b/f9d [1421404,49461] 0 2024-02-16T19:24:31.000 INFO:tasks.workunit.client.1.smithi163.stdout:6/559: dwrite d0/dd/f50 [0,4194304] 0 2024-02-16T19:24:31.000 INFO:tasks.workunit.client.1.smithi163.stdout:4/556: dwrite d11/d5b/f78 [0,4194304] 0 2024-02-16T19:24:31.003 INFO:tasks.workunit.client.1.smithi163.stdout:3/602: dwrite d0/d3e/f78 [0,4194304] 0 2024-02-16T19:24:31.003 INFO:tasks.workunit.client.1.smithi163.stdout:1/669: dwrite d7/d20/d21/f6c [4194304,4194304] 0 2024-02-16T19:24:31.003 INFO:tasks.workunit.client.1.smithi163.stdout:1/670: creat d7/dc/d2d/d32/d87/d97/fe6 x:0 0 0 2024-02-16T19:24:31.004 INFO:tasks.workunit.client.1.smithi163.stdout:6/560: fsync d0/d6/df/d20/d40/d1e/f52 0 2024-02-16T19:24:31.004 INFO:tasks.workunit.client.1.smithi163.stdout:4/557: creat d11/d18/d1f/d29/d52/d76/fb8 x:0 0 0 2024-02-16T19:24:31.004 INFO:tasks.workunit.client.1.smithi163.stdout:8/614: dwrite d0/d7/d18/d2e/f7f [0,4194304] 0 2024-02-16T19:24:31.004 INFO:tasks.workunit.client.1.smithi163.stdout:8/615: write d0/d24/d3c/f8a [1042579,10978] 0 2024-02-16T19:24:31.008 INFO:tasks.workunit.client.1.smithi163.stdout:3/603: symlink d0/dc/d96/da7/lda 0 2024-02-16T19:24:31.008 INFO:tasks.workunit.client.1.smithi163.stdout:3/604: dread - d0/dc/d94/d9e/dac/fc0 zero size 2024-02-16T19:24:31.008 INFO:tasks.workunit.client.1.smithi163.stdout:3/605: dread - d0/dc/f8c zero size 2024-02-16T19:24:31.012 INFO:tasks.workunit.client.1.smithi163.stdout:1/671: truncate d7/dc/d78/d7c/f81 350218 0 2024-02-16T19:24:31.012 INFO:tasks.workunit.client.1.smithi163.stdout:1/672: write d7/dc/d2d/d32/d87/d97/fe6 [273027,43054] 0 2024-02-16T19:24:31.012 INFO:tasks.workunit.client.1.smithi163.stdout:1/673: fdatasync d7/d20/f24 0 2024-02-16T19:24:31.018 INFO:tasks.workunit.client.1.smithi163.stdout:6/561: symlink d0/dd/d7f/d81/lb1 0 2024-02-16T19:24:31.019 INFO:tasks.workunit.client.1.smithi163.stdout:4/558: symlink d11/d18/d1f/d29/d52/d57/lb9 0 2024-02-16T19:24:31.020 INFO:tasks.workunit.client.1.smithi163.stdout:0/909: dwrite d11/d21/d50/d58/d72/f104 [0,4194304] 0 2024-02-16T19:24:31.020 INFO:tasks.workunit.client.1.smithi163.stdout:0/910: fsync d11/d22/f2d 0 2024-02-16T19:24:31.021 INFO:tasks.workunit.client.1.smithi163.stdout:0/911: dread d11/d12/d67/d6a/dae/dcc/dd9/ff0 [0,4194304] 0 2024-02-16T19:24:31.021 INFO:tasks.workunit.client.1.smithi163.stdout:0/912: creat d11/d22/d3e/f134 x:0 0 0 2024-02-16T19:24:31.022 INFO:tasks.workunit.client.1.smithi163.stdout:9/911: dwrite d5/de/d30/d43/d8f/f90 [0,4194304] 0 2024-02-16T19:24:31.023 INFO:tasks.workunit.client.1.smithi163.stdout:9/912: write d5/dc0/fcc [323551,4647] 0 2024-02-16T19:24:31.028 INFO:tasks.workunit.client.1.smithi163.stdout:7/619: dread d11/d30/f5b [0,4194304] 0 2024-02-16T19:24:31.028 INFO:tasks.workunit.client.1.smithi163.stdout:7/620: dread - d11/f3f zero size 2024-02-16T19:24:31.035 INFO:tasks.workunit.client.1.smithi163.stdout:3/606: mkdir d0/dc/d1c/d32/ddb 0 2024-02-16T19:24:31.042 INFO:tasks.workunit.client.1.smithi163.stdout:8/616: rmdir d0/d7/d19/d1f/d30/d48 39 2024-02-16T19:24:31.042 INFO:tasks.workunit.client.1.smithi163.stdout:8/617: write d0/dc/f2d [4079245,55767] 0 2024-02-16T19:24:31.045 INFO:tasks.workunit.client.1.smithi163.stdout:0/913: mknod d11/d12/d67/d6a/dae/c135 0 2024-02-16T19:24:31.045 INFO:tasks.workunit.client.1.smithi163.stdout:0/914: chown d11/d12/d67/lb1 1220 1 2024-02-16T19:24:31.045 INFO:tasks.workunit.client.1.smithi163.stdout:0/915: dread - d11/d4f/f133 zero size 2024-02-16T19:24:31.045 INFO:tasks.workunit.client.1.smithi163.stdout:0/916: readlink d11/d21/l82 0 2024-02-16T19:24:31.045 INFO:tasks.workunit.client.1.smithi163.stdout:0/917: stat d11/d12/d20/ca2 0 2024-02-16T19:24:31.046 INFO:tasks.workunit.client.1.smithi163.stdout:9/913: creat d5/de/d30/f13a x:0 0 0 2024-02-16T19:24:31.046 INFO:tasks.workunit.client.1.smithi163.stdout:4/559: dread d11/f16 [0,4194304] 0 2024-02-16T19:24:31.048 INFO:tasks.workunit.client.1.smithi163.stdout:7/621: mkdir d11/d26/dd4 0 2024-02-16T19:24:31.048 INFO:tasks.workunit.client.1.smithi163.stdout:7/622: rename d11/d24/d25 to d11/d24/d25/dd5 22 2024-02-16T19:24:31.049 INFO:tasks.workunit.client.1.smithi163.stdout:3/607: mkdir d0/dc/d96/ddc 0 2024-02-16T19:24:31.050 INFO:tasks.workunit.client.1.smithi163.stdout:6/562: getdents d0/d6/df/d20/d40/d53/d74 0 2024-02-16T19:24:31.050 INFO:tasks.workunit.client.1.smithi163.stdout:6/563: truncate d0/dd/d7f/f73 16224 0 2024-02-16T19:24:31.052 INFO:tasks.workunit.client.1.smithi163.stdout:8/618: mkdir d0/d7/d18/d2e/dd0 0 2024-02-16T19:24:31.053 INFO:tasks.workunit.client.1.smithi163.stdout:9/914: mkdir d5/de/d30/d43/d13b 0 2024-02-16T19:24:31.055 INFO:tasks.workunit.client.1.smithi163.stdout:4/560: symlink d11/d2d/d4f/lba 0 2024-02-16T19:24:31.057 INFO:tasks.workunit.client.1.smithi163.stdout:3/608: rename d0/f79 to d0/dc/d96/fdd 0 2024-02-16T19:24:31.057 INFO:tasks.workunit.client.1.smithi163.stdout:6/564: mkdir d0/d6/df/d20/d40/d1e/db2 0 2024-02-16T19:24:31.059 INFO:tasks.workunit.client.1.smithi163.stdout:4/561: symlink d11/d18/d1f/d29/lbb 0 2024-02-16T19:24:31.059 INFO:tasks.workunit.client.1.smithi163.stdout:9/915: symlink d5/de/d16/d46/d9b/d121/l13c 0 2024-02-16T19:24:31.060 INFO:tasks.workunit.client.1.smithi163.stdout:3/609: symlink d0/d83/da5/lde 0 2024-02-16T19:24:31.061 INFO:tasks.workunit.client.1.smithi163.stdout:3/610: truncate d0/dc/d50/fd3 583910 0 2024-02-16T19:24:31.061 INFO:tasks.workunit.client.1.smithi163.stdout:6/565: rename d0/d6/df/d7b/l93 to d0/d6/df/d7b/lb3 0 2024-02-16T19:24:31.061 INFO:tasks.workunit.client.1.smithi163.stdout:6/566: write d0/d6/df/d20/f90 [911319,66597] 0 2024-02-16T19:24:31.062 INFO:tasks.workunit.client.1.smithi163.stdout:6/567: fdatasync d0/d18/d77/d70/f8c 0 2024-02-16T19:24:31.062 INFO:tasks.workunit.client.1.smithi163.stdout:9/916: symlink d5/de/d30/d8a/da8/l13d 0 2024-02-16T19:24:31.062 INFO:tasks.workunit.client.1.smithi163.stdout:3/611: creat d0/dc/d1c/d32/ddb/fdf x:0 0 0 2024-02-16T19:24:31.063 INFO:tasks.workunit.client.1.smithi163.stdout:6/568: mkdir d0/d6/df/d20/d69/db4 0 2024-02-16T19:24:31.063 INFO:tasks.workunit.client.1.smithi163.stdout:6/569: creat d0/d6/fb5 x:0 0 0 2024-02-16T19:24:31.063 INFO:tasks.workunit.client.1.smithi163.stdout:8/619: dread d0/d24/d2c/d74/f79 [4194304,4194304] 0 2024-02-16T19:24:31.064 INFO:tasks.workunit.client.1.smithi163.stdout:9/917: unlink d5/de/d30/d3d/d49/d6d/d97/fd8 0 2024-02-16T19:24:31.065 INFO:tasks.workunit.client.1.smithi163.stdout:9/918: read d5/fed [10446,49250] 0 2024-02-16T19:24:31.065 INFO:tasks.workunit.client.1.smithi163.stdout:3/612: creat d0/dc/d50/d66/da3/d71/fe0 x:0 0 0 2024-02-16T19:24:31.065 INFO:tasks.workunit.client.1.smithi163.stdout:3/613: truncate d0/dc/d96/da7/fb1 999985 0 2024-02-16T19:24:31.067 INFO:tasks.workunit.client.1.smithi163.stdout:6/570: mknod d0/dd/d7f/d15/d1c/d65/cb6 0 2024-02-16T19:24:31.067 INFO:tasks.workunit.client.1.smithi163.stdout:6/571: fsync d0/d6/df/d20/d40/d17/f88 0 2024-02-16T19:24:31.068 INFO:tasks.workunit.client.1.smithi163.stdout:6/572: dread d0/dd/d7f/d15/d1c/d65/d63/f79 [0,4194304] 0 2024-02-16T19:24:31.068 INFO:tasks.workunit.client.1.smithi163.stdout:6/573: write d0/d6/df/d20/d40/d53/f72 [939075,61734] 0 2024-02-16T19:24:31.068 INFO:tasks.workunit.client.1.smithi163.stdout:3/614: getdents d0/d83/da5/db9 0 2024-02-16T19:24:31.070 INFO:tasks.workunit.client.1.smithi163.stdout:6/574: creat d0/dd/d7f/d2e/d33/fb7 x:0 0 0 2024-02-16T19:24:31.074 INFO:tasks.workunit.client.1.smithi163.stdout:3/615: unlink d0/d3e/f9f 0 2024-02-16T19:24:31.076 INFO:tasks.workunit.client.1.smithi163.stdout:3/616: link d0/dc/d1c/d32/l5b d0/dc/d1c/d91/le1 0 2024-02-16T19:24:31.077 INFO:tasks.workunit.client.1.smithi163.stdout:3/617: rmdir d0/dc/d1d/d29/d38/d46 39 2024-02-16T19:24:31.078 INFO:tasks.workunit.client.1.smithi163.stdout:7/623: dwrite d11/d24/d49/f69 [0,4194304] 0 2024-02-16T19:24:31.080 INFO:tasks.workunit.client.1.smithi163.stdout:0/918: dwrite d11/d21/d50/d58/d6b/fcd [0,4194304] 0 2024-02-16T19:24:31.081 INFO:tasks.workunit.client.1.smithi163.stdout:7/624: rename d11/d50/da2 to d11/d50/d5d/d8b/d67/dd6 0 2024-02-16T19:24:31.093 INFO:tasks.workunit.client.1.smithi163.stdout:7/625: dread d11/f1f [0,4194304] 0 2024-02-16T19:24:31.093 INFO:tasks.workunit.client.1.smithi163.stdout:7/626: dread - d11/d26/fc3 zero size 2024-02-16T19:24:31.095 INFO:tasks.workunit.client.1.smithi163.stdout:7/627: creat d11/d24/d25/d4b/d66/dc1/fd7 x:0 0 0 2024-02-16T19:24:31.095 INFO:tasks.workunit.client.1.smithi163.stdout:7/628: fsync f5 0 2024-02-16T19:24:31.096 INFO:tasks.workunit.client.1.smithi163.stdout:7/629: getdents d11/d24/d25 0 2024-02-16T19:24:31.097 INFO:tasks.workunit.client.1.smithi163.stdout:7/630: unlink d11/d24/d25/d4b/d8c/fd2 0 2024-02-16T19:24:31.097 INFO:tasks.workunit.client.1.smithi163.stdout:7/631: chown d11/d50/d5d/d8b/d67/f9b 715082 1 2024-02-16T19:24:31.097 INFO:tasks.workunit.client.1.smithi163.stdout:7/632: fdatasync d11/d50/d5d/fce 0 2024-02-16T19:24:31.098 INFO:tasks.workunit.client.1.smithi163.stdout:7/633: symlink d11/d50/d5d/dcd/ld8 0 2024-02-16T19:24:31.098 INFO:tasks.workunit.client.1.smithi163.stdout:7/634: chown d11/d24/d25/d4b/d66/fb9 69 1 2024-02-16T19:24:31.098 INFO:tasks.workunit.client.1.smithi163.stdout:7/635: chown d11/d26/f33 3703841 1 2024-02-16T19:24:31.098 INFO:tasks.workunit.client.1.smithi163.stdout:7/636: mkdir d11/d24/d34/d8d/dd9 0 2024-02-16T19:24:31.099 INFO:tasks.workunit.client.1.smithi163.stdout:8/620: dwrite d0/d6f/f98 [0,4194304] 0 2024-02-16T19:24:31.101 INFO:tasks.workunit.client.1.smithi163.stdout:8/621: symlink d0/d7/d19/d1f/d30/d6a/d7b/ld1 0 2024-02-16T19:24:31.101 INFO:tasks.workunit.client.1.smithi163.stdout:8/622: chown d0/d7/d18/fad 59911 1 2024-02-16T19:24:31.110 INFO:tasks.workunit.client.1.smithi163.stdout:8/623: dread d0/d7/d19/d1f/d30/f52 [0,4194304] 0 2024-02-16T19:24:31.110 INFO:tasks.workunit.client.1.smithi163.stdout:8/624: truncate d0/d24/d2c/f90 967632 0 2024-02-16T19:24:31.111 INFO:tasks.workunit.client.1.smithi163.stdout:8/625: read d0/d24/d2c/f90 [587164,48391] 0 2024-02-16T19:24:31.111 INFO:tasks.workunit.client.1.smithi163.stdout:8/626: readlink d0/d24/d3c/d3d/d41/lbc 0 2024-02-16T19:24:31.112 INFO:tasks.workunit.client.1.smithi163.stdout:0/919: dwrite d11/fb4 [4194304,4194304] 0 2024-02-16T19:24:31.112 INFO:tasks.workunit.client.1.smithi163.stdout:4/562: dwrite d11/d18/d1f/d29/d52/d57/d6c/d7c/f92 [0,4194304] 0 2024-02-16T19:24:31.112 INFO:tasks.workunit.client.1.smithi163.stdout:4/563: fdatasync d11/d2d/d4f/d5f/d70/d74/d75/fac 0 2024-02-16T19:24:31.113 INFO:tasks.workunit.client.1.smithi163.stdout:0/920: write d11/d21/fad [244892,45046] 0 2024-02-16T19:24:31.113 INFO:tasks.workunit.client.1.smithi163.stdout:0/921: write d11/d12/dc6/ff1 [522664,126152] 0 2024-02-16T19:24:31.113 INFO:tasks.workunit.client.1.smithi163.stdout:9/919: dwrite d5/de/fdb [0,4194304] 0 2024-02-16T19:24:31.113 INFO:tasks.workunit.client.1.smithi163.stdout:8/627: getdents d0/d24/d2c 0 2024-02-16T19:24:31.114 INFO:tasks.workunit.client.1.smithi163.stdout:8/628: truncate d0/d24/d2c/f3b 3728992 0 2024-02-16T19:24:31.115 INFO:tasks.workunit.client.1.smithi163.stdout:8/629: creat d0/d6f/fd2 x:0 0 0 2024-02-16T19:24:31.116 INFO:tasks.workunit.client.1.smithi163.stdout:8/630: mknod d0/d24/d3c/d3d/d41/cd3 0 2024-02-16T19:24:31.118 INFO:tasks.workunit.client.1.smithi163.stdout:8/631: rename d0/d24/d2c/c50 to d0/dc/cd4 0 2024-02-16T19:24:31.120 INFO:tasks.workunit.client.1.smithi163.stdout:8/632: symlink d0/d24/d3c/d3d/d41/ld5 0 2024-02-16T19:24:31.120 INFO:tasks.workunit.client.1.smithi163.stdout:8/633: write d0/d7/d19/d1f/d30/d48/f65 [1692929,76202] 0 2024-02-16T19:24:31.120 INFO:tasks.workunit.client.1.smithi163.stdout:8/634: dread - d0/d7/d19/d1f/d30/d6a/d7b/fac zero size 2024-02-16T19:24:31.121 INFO:tasks.workunit.client.1.smithi163.stdout:0/922: dread d11/d22/fd7 [0,4194304] 0 2024-02-16T19:24:31.121 INFO:tasks.workunit.client.1.smithi163.stdout:0/923: write fd [2601993,35193] 0 2024-02-16T19:24:31.121 INFO:tasks.workunit.client.1.smithi163.stdout:8/635: write d0/dc/f7d [671347,123336] 0 2024-02-16T19:24:31.123 INFO:tasks.workunit.client.1.smithi163.stdout:0/924: write d11/d21/d7c/fce [32187,115426] 0 2024-02-16T19:24:31.124 INFO:tasks.workunit.client.1.smithi163.stdout:7/637: write d11/d24/d34/fc0 [677515,16255] 0 2024-02-16T19:24:31.124 INFO:tasks.workunit.client.1.smithi163.stdout:3/618: dwrite d0/dc/f31 [0,4194304] 0 2024-02-16T19:24:31.127 INFO:tasks.workunit.client.1.smithi163.stdout:7/638: mkdir d11/d30/dda 0 2024-02-16T19:24:31.130 INFO:tasks.workunit.client.1.smithi163.stdout:7/639: dread d11/d24/d25/d4b/d66/f6f [0,4194304] 0 2024-02-16T19:24:31.133 INFO:tasks.workunit.client.1.smithi163.stdout:3/619: rename d0/dc/l8f to d0/d83/da5/db9/le2 0 2024-02-16T19:24:31.138 INFO:tasks.workunit.client.1.smithi163.stdout:8/636: write d0/d24/d27/f64 [802460,48976] 0 2024-02-16T19:24:31.142 INFO:tasks.workunit.client.1.smithi163.stdout:4/564: dwrite d11/d18/d1f/d29/f3e [4194304,4194304] 0 2024-02-16T19:24:31.150 INFO:tasks.workunit.client.1.smithi163.stdout:3/620: write d0/dc/d1d/d29/f45 [2019846,111969] 0 2024-02-16T19:24:31.150 INFO:tasks.workunit.client.1.smithi163.stdout:3/621: write d0/dc/d50/d66/da3/d71/f87 [456568,39708] 0 2024-02-16T19:24:31.150 INFO:tasks.workunit.client.1.smithi163.stdout:3/622: stat d0/dc/d1c/lcd 0 2024-02-16T19:24:31.152 INFO:tasks.workunit.client.1.smithi163.stdout:3/623: rename d0/dc/cca to d0/dc/ce3 0 2024-02-16T19:24:31.153 INFO:tasks.workunit.client.1.smithi163.stdout:3/624: symlink d0/dc/d94/d9e/dac/le4 0 2024-02-16T19:24:31.155 INFO:tasks.workunit.client.1.smithi163.stdout:4/565: write d11/f30 [2666619,19586] 0 2024-02-16T19:24:31.158 INFO:tasks.workunit.client.1.smithi163.stdout:7/640: dwrite d11/d24/d34/f58 [0,4194304] 0 2024-02-16T19:24:31.158 INFO:tasks.workunit.client.1.smithi163.stdout:9/920: dwrite d5/f1a [0,4194304] 0 2024-02-16T19:24:31.163 INFO:tasks.workunit.client.1.smithi163.stdout:9/921: rename d5/de/d16/d46/d9b/d106/f126 to d5/de/d16/d133/f13e 0 2024-02-16T19:24:31.164 INFO:tasks.workunit.client.1.smithi163.stdout:9/922: mkdir d5/de/d30/d3d/d49/d13f 0 2024-02-16T19:24:31.164 INFO:tasks.workunit.client.1.smithi163.stdout:9/923: truncate d5/de/d16/d47/f86 4031451 0 2024-02-16T19:24:31.167 INFO:tasks.workunit.client.1.smithi163.stdout:4/566: write d11/d2d/d4f/f53 [5974773,9808] 0 2024-02-16T19:24:31.168 INFO:tasks.workunit.client.1.smithi163.stdout:4/567: creat d11/d3d/fbc x:0 0 0 2024-02-16T19:24:31.170 INFO:tasks.workunit.client.1.smithi163.stdout:4/568: rename d11/d18/d1f/d23/d3a/d4a/f4e to d11/d5b/d8f/fbd 0 2024-02-16T19:24:31.170 INFO:tasks.workunit.client.1.smithi163.stdout:3/625: dread d0/d3e/f78 [0,4194304] 0 2024-02-16T19:24:31.170 INFO:tasks.workunit.client.1.smithi163.stdout:3/626: truncate d0/dc/d50/d66/d80/fa0 112433 0 2024-02-16T19:24:31.170 INFO:tasks.workunit.client.1.smithi163.stdout:0/925: dwrite d11/d12/d67/f91 [0,4194304] 0 2024-02-16T19:24:31.171 INFO:tasks.workunit.client.1.smithi163.stdout:4/569: truncate d11/d18/d1f/d29/d52/d57/d6c/d7c/f96 3757084 0 2024-02-16T19:24:31.173 INFO:tasks.workunit.client.1.smithi163.stdout:3/627: creat d0/dc/d1d/d29/d38/fe5 x:0 0 0 2024-02-16T19:24:31.175 INFO:tasks.workunit.client.1.smithi163.stdout:3/628: creat d0/dc/d50/d66/fe6 x:0 0 0 2024-02-16T19:24:31.176 INFO:tasks.workunit.client.1.smithi163.stdout:3/629: rename d0/dc/d50/d66/l8d to d0/dc/d94/d9e/le7 0 2024-02-16T19:24:31.178 INFO:tasks.workunit.client.1.smithi163.stdout:3/630: creat d0/dc/d96/ddc/fe8 x:0 0 0 2024-02-16T19:24:31.178 INFO:tasks.workunit.client.1.smithi163.stdout:7/641: dwrite d11/d26/fbb [0,4194304] 0 2024-02-16T19:24:31.178 INFO:tasks.workunit.client.1.smithi163.stdout:7/642: chown d11/d50/d5d/d8b/lb6 179709937 1 2024-02-16T19:24:31.178 INFO:tasks.workunit.client.1.smithi163.stdout:7/643: chown d11/d24/d49/f69 145255348 1 2024-02-16T19:24:31.178 INFO:tasks.workunit.client.1.smithi163.stdout:7/644: fsync d11/d24/d49/fb3 0 2024-02-16T19:24:31.180 INFO:tasks.workunit.client.1.smithi163.stdout:7/645: creat d11/d24/d49/fdb x:0 0 0 2024-02-16T19:24:31.181 INFO:tasks.workunit.client.1.smithi163.stdout:3/631: getdents d0/dc/d1d/d49 0 2024-02-16T19:24:31.182 INFO:tasks.workunit.client.1.smithi163.stdout:7/646: creat d11/d24/d25/d4b/fdc x:0 0 0 2024-02-16T19:24:31.185 INFO:tasks.workunit.client.1.smithi163.stdout:7/647: rename d11/d24/d25/d4b/d8c/f96 to d11/d24/d49/fdd 0 2024-02-16T19:24:31.185 INFO:tasks.workunit.client.1.smithi163.stdout:7/648: readlink d11/d24/d34/l45 0 2024-02-16T19:24:31.185 INFO:tasks.workunit.client.1.smithi163.stdout:7/649: readlink d11/d24/d34/l8f 0 2024-02-16T19:24:31.185 INFO:tasks.workunit.client.1.smithi163.stdout:7/650: write d11/d24/d25/d4b/d66/f6f [829640,129965] 0 2024-02-16T19:24:31.185 INFO:tasks.workunit.client.1.smithi163.stdout:7/651: truncate d11/d24/d25/d4b/d66/f70 719285 0 2024-02-16T19:24:31.185 INFO:tasks.workunit.client.1.smithi163.stdout:7/652: fsync d11/d24/d25/f2d 0 2024-02-16T19:24:31.190 INFO:tasks.workunit.client.1.smithi163.stdout:4/570: dwrite d11/d2d/d4f/f63 [0,4194304] 0 2024-02-16T19:24:31.197 INFO:tasks.workunit.client.1.smithi163.stdout:2/786: sync 2024-02-16T19:24:31.197 INFO:tasks.workunit.client.1.smithi163.stdout:7/653: write d11/d50/d5d/d8b/f90 [1979380,40417] 0 2024-02-16T19:24:31.198 INFO:tasks.workunit.client.1.smithi163.stdout:7/654: stat d11/f19 0 2024-02-16T19:24:31.198 INFO:tasks.workunit.client.1.smithi163.stdout:7/655: chown d11/d50/d5d/d8b/f4e 324 1 2024-02-16T19:24:31.200 INFO:tasks.workunit.client.1.smithi163.stdout:3/632: dwrite d0/dc/d1d/d29/f33 [0,4194304] 0 2024-02-16T19:24:31.200 INFO:tasks.workunit.client.1.smithi163.stdout:3/633: creat d0/dc/d1d/d49/d98/fe9 x:0 0 0 2024-02-16T19:24:31.200 INFO:tasks.workunit.client.1.smithi163.stdout:3/634: truncate d0/dc/f8c 240141 0 2024-02-16T19:24:31.200 INFO:tasks.workunit.client.1.smithi163.stdout:3/635: truncate d0/dc/d1c/d32/db8/fc7 646592 0 2024-02-16T19:24:31.212 INFO:tasks.workunit.client.1.smithi163.stdout:3/636: dread d0/dc/d1d/d49/f56 [0,4194304] 0 2024-02-16T19:24:31.212 INFO:tasks.workunit.client.1.smithi163.stdout:3/637: readlink d0/dc/d1c/d32/db8/d99/l9b 0 2024-02-16T19:24:31.212 INFO:tasks.workunit.client.1.smithi163.stdout:3/638: mkdir d0/d3e/dea 0 2024-02-16T19:24:31.213 INFO:tasks.workunit.client.1.smithi163.stdout:4/571: dwrite d11/d18/d1f/d34/f51 [0,4194304] 0 2024-02-16T19:24:31.213 INFO:tasks.workunit.client.1.smithi163.stdout:4/572: dread - d11/d2d/d4f/daf/fb2 zero size 2024-02-16T19:24:31.213 INFO:tasks.workunit.client.1.smithi163.stdout:3/639: mkdir d0/dc/d1c/d21/deb 0 2024-02-16T19:24:31.214 INFO:tasks.workunit.client.1.smithi163.stdout:4/573: mknod d11/d18/d1f/d29/d52/d57/d6c/d9c/cbe 0 2024-02-16T19:24:31.214 INFO:tasks.workunit.client.1.smithi163.stdout:4/574: fdatasync d11/d2d/d4f/d60/d79/f90 0 2024-02-16T19:24:31.215 INFO:tasks.workunit.client.1.smithi163.stdout:3/640: link d0/dc/f24 d0/dc/d50/d66/fec 0 2024-02-16T19:24:31.215 INFO:tasks.workunit.client.1.smithi163.stdout:3/641: write d0/dc/d50/fd3 [1074292,117399] 0 2024-02-16T19:24:31.216 INFO:tasks.workunit.client.1.smithi163.stdout:3/642: chown d0/dc/d1d/d49/d98 2341421 1 2024-02-16T19:24:31.216 INFO:tasks.workunit.client.1.smithi163.stdout:3/643: fdatasync d0/dc/d1c/d21/f42 0 2024-02-16T19:24:31.216 INFO:tasks.workunit.client.1.smithi163.stdout:2/787: write d2/d2a/d39/d66/f77 [904055,101295] 0 2024-02-16T19:24:31.217 INFO:tasks.workunit.client.1.smithi163.stdout:3/644: creat d0/d83/da5/db9/fed x:0 0 0 2024-02-16T19:24:31.218 INFO:tasks.workunit.client.1.smithi163.stdout:2/788: rmdir d2/d2a/d39/d66/d6c/dcb 39 2024-02-16T19:24:31.219 INFO:tasks.workunit.client.1.smithi163.stdout:3/645: mknod d0/dc/d50/d66/cee 0 2024-02-16T19:24:31.219 INFO:tasks.workunit.client.1.smithi163.stdout:3/646: write d0/dc/d1d/fd8 [181692,53511] 0 2024-02-16T19:24:31.219 INFO:tasks.workunit.client.1.smithi163.stdout:3/647: fsync d0/f39 0 2024-02-16T19:24:31.229 INFO:tasks.workunit.client.1.smithi163.stdout:2/789: write d2/d26/d54/d58/d8c/ff1 [3726840,15319] 0 2024-02-16T19:24:31.230 INFO:tasks.workunit.client.1.smithi163.stdout:4/575: dwrite d11/d18/d1f/d29/fa1 [4194304,4194304] 0 2024-02-16T19:24:31.231 INFO:tasks.workunit.client.1.smithi163.stdout:4/576: stat d11/d18/d1f/d29/d52/d76/d89/d94 0 2024-02-16T19:24:31.231 INFO:tasks.workunit.client.1.smithi163.stdout:4/577: fdatasync d11/d18/d1f/d29/d52/d57/d6c/d7c/f96 0 2024-02-16T19:24:31.231 INFO:tasks.workunit.client.1.smithi163.stdout:7/656: dwrite d11/d24/d49/f51 [0,4194304] 0 2024-02-16T19:24:31.231 INFO:tasks.workunit.client.1.smithi163.stdout:3/648: dread d0/dc/d1d/d49/f56 [4194304,4194304] 0 2024-02-16T19:24:31.231 INFO:tasks.workunit.client.1.smithi163.stdout:3/649: creat d0/dc/d1d/d49/d98/fef x:0 0 0 2024-02-16T19:24:31.232 INFO:tasks.workunit.client.1.smithi163.stdout:4/578: mkdir d11/d5b/dbf 0 2024-02-16T19:24:31.233 INFO:tasks.workunit.client.1.smithi163.stdout:4/579: write d11/d2d/d4f/f8c [201218,32055] 0 2024-02-16T19:24:31.235 INFO:tasks.workunit.client.1.smithi163.stdout:7/657: creat d11/d24/d49/dac/fde x:0 0 0 2024-02-16T19:24:31.237 INFO:tasks.workunit.client.1.smithi163.stdout:3/650: unlink d0/d83/c75 0 2024-02-16T19:24:31.237 INFO:tasks.workunit.client.1.smithi163.stdout:3/651: write d0/dc/d94/d9e/fad [215188,109744] 0 2024-02-16T19:24:31.237 INFO:tasks.workunit.client.1.smithi163.stdout:3/652: fdatasync d0/f8 0 2024-02-16T19:24:31.240 INFO:tasks.workunit.client.1.smithi163.stdout:7/658: link d11/d24/l9e d11/d24/d25/d4b/d66/ldf 0 2024-02-16T19:24:31.241 INFO:tasks.workunit.client.1.smithi163.stdout:2/790: dread d2/ddb/d7c/f99 [0,4194304] 0 2024-02-16T19:24:31.241 INFO:tasks.workunit.client.1.smithi163.stdout:2/791: chown d2/d2a/d39/d3a 1060269 1 2024-02-16T19:24:31.241 INFO:tasks.workunit.client.1.smithi163.stdout:3/653: dread d0/dc/d1d/d29/f45 [0,4194304] 0 2024-02-16T19:24:31.242 INFO:tasks.workunit.client.1.smithi163.stdout:3/654: truncate d0/dc/d50/d66/da3/d71/f89 1256013 0 2024-02-16T19:24:31.242 INFO:tasks.workunit.client.1.smithi163.stdout:2/792: symlink d2/d26/d54/d58/d8c/l111 0 2024-02-16T19:24:31.246 INFO:tasks.workunit.client.1.smithi163.stdout:4/580: write d11/d18/f43 [2756488,23019] 0 2024-02-16T19:24:31.246 INFO:tasks.workunit.client.1.smithi163.stdout:4/581: write d11/d5b/d8f/fb5 [235697,30895] 0 2024-02-16T19:24:31.246 INFO:tasks.workunit.client.1.smithi163.stdout:4/582: dread - d11/d2d/d4f/d5f/d70/d74/fa6 zero size 2024-02-16T19:24:31.248 INFO:tasks.workunit.client.1.smithi163.stdout:4/583: rename d11/d3d/l44 to d11/d18/d1f/d29/lc0 0 2024-02-16T19:24:31.248 INFO:tasks.workunit.client.1.smithi163.stdout:2/793: dread d2/d26/f2f [0,4194304] 0 2024-02-16T19:24:31.248 INFO:tasks.workunit.client.1.smithi163.stdout:2/794: read - d2/d2a/d2d/fc6 zero size 2024-02-16T19:24:31.248 INFO:tasks.workunit.client.1.smithi163.stdout:2/795: fsync d2/f37 0 2024-02-16T19:24:31.251 INFO:tasks.workunit.client.1.smithi163.stdout:4/584: rename d11/d2d/d4f/d5f/d70/d74/fa6 to d11/d5b/fc1 0 2024-02-16T19:24:31.253 INFO:tasks.workunit.client.1.smithi163.stdout:2/796: symlink d2/d26/d54/d58/d98/l112 0 2024-02-16T19:24:31.253 INFO:tasks.workunit.client.1.smithi163.stdout:2/797: creat d2/d2a/d2d/d73/f113 x:0 0 0 2024-02-16T19:24:31.254 INFO:tasks.workunit.client.1.smithi163.stdout:4/585: link d11/d18/d1f/d29/f36 d11/d2d/d4f/d5f/d70/fc2 0 2024-02-16T19:24:31.254 INFO:tasks.workunit.client.1.smithi163.stdout:4/586: fdatasync d11/d18/f1d 0 2024-02-16T19:24:31.257 INFO:tasks.workunit.client.1.smithi163.stdout:4/587: rename d11/d2d/d4f/daf to d11/d18/d1f/d29/d52/d57/d6c/dc3 0 2024-02-16T19:24:31.263 INFO:tasks.workunit.client.1.smithi163.stdout:3/655: dwrite d0/f92 [0,4194304] 0 2024-02-16T19:24:31.263 INFO:tasks.workunit.client.1.smithi163.stdout:3/656: fsync d0/dc/d50/d66/d80/fb7 0 2024-02-16T19:24:31.263 INFO:tasks.workunit.client.1.smithi163.stdout:2/798: dread d2/d26/f28 [0,4194304] 0 2024-02-16T19:24:31.266 INFO:tasks.workunit.client.1.smithi163.stdout:2/799: rename d2/ddb/dc8 to d2/d26/d54/dfb/d114 0 2024-02-16T19:24:31.266 INFO:tasks.workunit.client.1.smithi163.stdout:2/800: creat d2/d2a/d2d/d73/d94/dc9/f115 x:0 0 0 2024-02-16T19:24:31.268 INFO:tasks.workunit.client.1.smithi163.stdout:2/801: rmdir d2/d26/d54/d58/dff 39 2024-02-16T19:24:31.268 INFO:tasks.workunit.client.1.smithi163.stdout:2/802: fdatasync d2/ff 0 2024-02-16T19:24:31.268 INFO:tasks.workunit.client.1.smithi163.stdout:2/803: chown d2/d2a/d2d/d73 1102302 1 2024-02-16T19:24:31.269 INFO:tasks.workunit.client.1.smithi163.stdout:2/804: write d2/d26/d54/d58/d98/f10d [36959,102669] 0 2024-02-16T19:24:31.270 INFO:tasks.workunit.client.1.smithi163.stdout:3/657: write d0/dc/d1d/d29/f30 [7781625,20034] 0 2024-02-16T19:24:31.272 INFO:tasks.workunit.client.1.smithi163.stdout:3/658: mkdir d0/dc/d1c/d32/df0 0 2024-02-16T19:24:31.273 INFO:tasks.workunit.client.1.smithi163.stdout:2/805: read d2/d2a/d39/f3b [150471,53138] 0 2024-02-16T19:24:31.273 INFO:tasks.workunit.client.1.smithi163.stdout:4/588: dwrite d11/d18/d1f/f20 [0,4194304] 0 2024-02-16T19:24:31.274 INFO:tasks.workunit.client.1.smithi163.stdout:2/806: getdents d2/d2a/d9f 0 2024-02-16T19:24:31.276 INFO:tasks.workunit.client.1.smithi163.stdout:4/589: symlink d11/d18/d1f/d23/d3a/lc4 0 2024-02-16T19:24:31.277 INFO:tasks.workunit.client.1.smithi163.stdout:2/807: unlink d2/f16 0 2024-02-16T19:24:31.278 INFO:tasks.workunit.client.1.smithi163.stdout:2/808: dread - d2/d2a/d2d/d73/fa2 zero size 2024-02-16T19:24:31.278 INFO:tasks.workunit.client.1.smithi163.stdout:2/809: write d2/d26/d54/d58/d7d/fd2 [405167,28489] 0 2024-02-16T19:24:31.280 INFO:tasks.workunit.client.1.smithi163.stdout:2/810: unlink d2/d2a/d2d/d73/db5/fe2 0 2024-02-16T19:24:31.286 INFO:tasks.workunit.client.1.smithi163.stdout:4/590: dread d11/d18/d1f/f20 [0,4194304] 0 2024-02-16T19:24:31.286 INFO:tasks.workunit.client.1.smithi163.stdout:4/591: fsync d11/d18/d1f/d29/d52/d57/f7e 0 2024-02-16T19:24:31.286 INFO:tasks.workunit.client.1.smithi163.stdout:4/592: readlink d11/d18/d1f/d23/l2c 0 2024-02-16T19:24:31.288 INFO:tasks.workunit.client.1.smithi163.stdout:3/659: dwrite d0/dc/d1d/d49/fd4 [0,4194304] 0 2024-02-16T19:24:31.296 INFO:tasks.workunit.client.1.smithi163.stdout:3/660: truncate d0/dc/f24 2580932 0 2024-02-16T19:24:31.296 INFO:tasks.workunit.client.1.smithi163.stdout:3/661: chown d0/dc/f31 60261934 1 2024-02-16T19:24:31.296 INFO:tasks.workunit.client.1.smithi163.stdout:3/662: write d0/dc/d1c/d32/f3f [5448116,51747] 0 2024-02-16T19:24:31.300 INFO:tasks.workunit.client.1.smithi163.stdout:3/663: dread d0/dc/d1c/d21/f2d [0,4194304] 0 2024-02-16T19:24:31.300 INFO:tasks.workunit.client.1.smithi163.stdout:3/664: chown d0/dc/d1d/d29/d38/d46/c68 89 1 2024-02-16T19:24:31.300 INFO:tasks.workunit.client.1.smithi163.stdout:3/665: write d0/dc/d1d/d49/d98/fcc [957750,125192] 0 2024-02-16T19:24:31.303 INFO:tasks.workunit.client.1.smithi163.stdout:3/666: dread d0/d3e/f78 [0,4194304] 0 2024-02-16T19:24:31.304 INFO:tasks.workunit.client.1.smithi163.stdout:2/811: dwrite d2/d2a/d2d/f6e [0,4194304] 0 2024-02-16T19:24:31.304 INFO:tasks.workunit.client.1.smithi163.stdout:3/667: creat d0/dc/d1c/d32/df0/ff1 x:0 0 0 2024-02-16T19:24:31.304 INFO:tasks.workunit.client.1.smithi163.stdout:3/668: read - d0/dc/d50/d66/fe6 zero size 2024-02-16T19:24:31.305 INFO:tasks.workunit.client.1.smithi163.stdout:2/812: write d2/d2a/d39/d66/f77 [608243,24969] 0 2024-02-16T19:24:31.306 INFO:tasks.workunit.client.1.smithi163.stdout:3/669: unlink d0/dc/d96/da7/lda 0 2024-02-16T19:24:31.307 INFO:tasks.workunit.client.1.smithi163.stdout:2/813: unlink d2/d2a/d9f/cf7 0 2024-02-16T19:24:31.308 INFO:tasks.workunit.client.1.smithi163.stdout:4/593: dwrite d11/d18/d1f/d29/d52/d57/f73 [0,4194304] 0 2024-02-16T19:24:31.308 INFO:tasks.workunit.client.1.smithi163.stdout:4/594: fsync d11/d18/d1f/d23/d3a/f58 0 2024-02-16T19:24:31.311 INFO:tasks.workunit.client.1.smithi163.stdout:2/814: rename d2/d2a/d39/d3a/f5b to d2/d26/d54/f116 0 2024-02-16T19:24:31.315 INFO:tasks.workunit.client.1.smithi163.stdout:4/595: getdents d11/d18/d1f/d29/d52/d57/d6c/d9d 0 2024-02-16T19:24:31.317 INFO:tasks.workunit.client.1.smithi163.stdout:4/596: mkdir d11/d18/d1f/d29/d52/d57/dc5 0 2024-02-16T19:24:31.318 INFO:tasks.workunit.client.1.smithi163.stdout:4/597: creat d11/d2d/d4f/d60/fc6 x:0 0 0 2024-02-16T19:24:31.320 INFO:tasks.workunit.client.1.smithi163.stdout:4/598: link d11/d18/d1f/d29/d52/d57/f7e d11/d18/d1f/d29/d52/d76/da2/fc7 0 2024-02-16T19:24:31.340 INFO:tasks.workunit.client.1.smithi163.stdout:4/599: dwrite d11/d18/f9f [0,4194304] 0 2024-02-16T19:24:31.340 INFO:tasks.workunit.client.1.smithi163.stdout:4/600: fsync f10 0 2024-02-16T19:24:31.342 INFO:tasks.workunit.client.1.smithi163.stdout:4/601: symlink d11/d2d/d4f/d5f/d70/d74/d75/lc8 0 2024-02-16T19:24:31.343 INFO:tasks.workunit.client.1.smithi163.stdout:5/666: sync 2024-02-16T19:24:31.344 INFO:tasks.workunit.client.1.smithi163.stdout:5/667: rename d0/cd8 to d0/d1c/d5a/da5/ce1 0 2024-02-16T19:24:31.346 INFO:tasks.workunit.client.1.smithi163.stdout:5/668: mkdir d0/d2/d34/d44/d53/de2 0 2024-02-16T19:24:31.346 INFO:tasks.workunit.client.1.smithi163.stdout:5/669: fsync d0/d1c/f40 0 2024-02-16T19:24:31.351 INFO:tasks.workunit.client.1.smithi163.stdout:4/602: write d11/d2d/d4f/f63 [943786,73262] 0 2024-02-16T19:24:31.352 INFO:tasks.workunit.client.1.smithi163.stdout:4/603: truncate d11/f65 2964842 0 2024-02-16T19:24:31.353 INFO:tasks.workunit.client.1.smithi163.stdout:4/604: mkdir d11/d2d/d4f/d60/d79/dc9 0 2024-02-16T19:24:31.356 INFO:tasks.workunit.client.1.smithi163.stdout:4/605: rename d11/d5b to d11/d2d/d4f/d5f/d70/d74/d75/dca 0 2024-02-16T19:24:31.357 INFO:tasks.workunit.client.1.smithi163.stdout:4/606: creat d11/d18/d1f/d29/d52/d76/da2/fcb x:0 0 0 2024-02-16T19:24:31.361 INFO:tasks.workunit.client.1.smithi163.stdout:5/670: dwrite d0/d2/f16 [4194304,4194304] 0 2024-02-16T19:24:31.364 INFO:tasks.workunit.client.1.smithi163.stdout:5/671: mkdir d0/d1c/dbe/de3 0 2024-02-16T19:24:31.364 INFO:tasks.workunit.client.1.smithi163.stdout:5/672: write d0/d22/d3b/f66 [5210548,46288] 0 2024-02-16T19:24:31.364 INFO:tasks.workunit.client.1.smithi163.stdout:5/673: write d0/d22/d28/d36/f98 [51130,47125] 0 2024-02-16T19:24:31.364 INFO:tasks.workunit.client.1.smithi163.stdout:5/674: chown d0/d22/d28/d36/d3c/d4f/f50 235450 1 2024-02-16T19:24:31.367 INFO:tasks.workunit.client.1.smithi163.stdout:5/675: dread d0/d2/d5c/fcb [0,4194304] 0 2024-02-16T19:24:31.368 INFO:tasks.workunit.client.1.smithi163.stdout:5/676: dread d0/d2/d5c/fcb [0,4194304] 0 2024-02-16T19:24:31.370 INFO:tasks.workunit.client.1.smithi163.stdout:5/677: link d0/d2/d34/d44/f51 d0/d2/fe4 0 2024-02-16T19:24:31.375 INFO:tasks.workunit.client.1.smithi163.stdout:4/607: dwrite d11/d18/d1f/d29/d52/d76/da2/fc7 [0,4194304] 0 2024-02-16T19:24:31.378 INFO:tasks.workunit.client.1.smithi163.stdout:4/608: mknod d11/d2d/ccc 0 2024-02-16T19:24:31.379 INFO:tasks.workunit.client.1.smithi163.stdout:4/609: symlink d11/d18/d1f/d23/d3a/d4a/lcd 0 2024-02-16T19:24:31.380 INFO:tasks.workunit.client.1.smithi163.stdout:4/610: unlink d11/f65 0 2024-02-16T19:24:31.380 INFO:tasks.workunit.client.1.smithi163.stdout:4/611: fsync d11/d18/d1f/d29/d52/d76/da2/fcb 0 2024-02-16T19:24:31.381 INFO:tasks.workunit.client.1.smithi163.stdout:4/612: creat d11/d18/d1f/d29/d52/d57/d6c/d7c/fce x:0 0 0 2024-02-16T19:24:31.384 INFO:tasks.workunit.client.1.smithi163.stdout:4/613: dread d11/d2d/d4f/f64 [4194304,4194304] 0 2024-02-16T19:24:31.387 INFO:tasks.workunit.client.1.smithi163.stdout:4/614: write d11/f41 [1002179,58008] 0 2024-02-16T19:24:31.387 INFO:tasks.workunit.client.1.smithi163.stdout:4/615: fsync d11/d2d/d4f/f55 0 2024-02-16T19:24:31.387 INFO:tasks.workunit.client.1.smithi163.stdout:4/616: creat d11/d2d/d4f/d5f/d70/d74/d75/dca/dbf/fcf x:0 0 0 2024-02-16T19:24:31.388 INFO:tasks.workunit.client.1.smithi163.stdout:4/617: mknod d11/d2d/d4f/d60/cd0 0 2024-02-16T19:24:31.388 INFO:tasks.workunit.client.1.smithi163.stdout:5/678: dwrite d0/d2/f4 [0,4194304] 0 2024-02-16T19:24:31.390 INFO:tasks.workunit.client.1.smithi163.stdout:5/679: rename d0/d22/d28/d36/ca4 to d0/d22/d3b/ce5 0 2024-02-16T19:24:31.390 INFO:tasks.workunit.client.1.smithi163.stdout:5/680: chown d0/d2/d34/d44/f72 50290701 1 2024-02-16T19:24:31.390 INFO:tasks.workunit.client.1.smithi163.stdout:5/681: write d0/d22/d3b/dc0/fb0 [55370,110557] 0 2024-02-16T19:24:31.390 INFO:tasks.workunit.client.1.smithi163.stdout:5/682: write d0/d2/d34/d44/d53/fdf [272378,33961] 0 2024-02-16T19:24:31.390 INFO:tasks.workunit.client.1.smithi163.stdout:5/683: write d0/d22/d3b/f4c [1186563,110361] 0 2024-02-16T19:24:31.391 INFO:tasks.workunit.client.1.smithi163.stdout:5/684: getdents d0/d1c/d5a 0 2024-02-16T19:24:31.393 INFO:tasks.workunit.client.1.smithi163.stdout:5/685: mkdir d0/d22/d28/d36/d3c/d4f/de6 0 2024-02-16T19:24:31.395 INFO:tasks.workunit.client.1.smithi163.stdout:5/686: rename d0/d1c/f40 to d0/d2/fe7 0 2024-02-16T19:24:31.395 INFO:tasks.workunit.client.1.smithi163.stdout:5/687: chown d0/d2/d34/d44/d53/f87 37438 1 2024-02-16T19:24:31.396 INFO:tasks.workunit.client.1.smithi163.stdout:5/688: creat d0/d2/d34/d44/fe8 x:0 0 0 2024-02-16T19:24:31.397 INFO:tasks.workunit.client.1.smithi163.stdout:5/689: truncate d0/d2/fe4 2143481 0 2024-02-16T19:24:31.398 INFO:tasks.workunit.client.1.smithi163.stdout:5/690: rename d0/d1c/d5a/l97 to d0/d22/d3b/dc0/le9 0 2024-02-16T19:24:31.403 INFO:tasks.workunit.client.1.smithi163.stdout:4/618: dwrite d11/d18/d1f/d29/d52/d57/f84 [0,4194304] 0 2024-02-16T19:24:31.407 INFO:tasks.workunit.client.1.smithi163.stdout:4/619: link d11/d2d/d4f/f64 d11/d18/d1f/d29/d52/d76/fd1 0 2024-02-16T19:24:31.409 INFO:tasks.workunit.client.1.smithi163.stdout:4/620: creat d11/d18/d1f/d23/d3a/d4a/fd2 x:0 0 0 2024-02-16T19:24:31.418 INFO:tasks.workunit.client.1.smithi163.stdout:5/691: dwrite d0/d22/f35 [0,4194304] 0 2024-02-16T19:24:31.418 INFO:tasks.workunit.client.1.smithi163.stdout:5/692: readlink d0/d22/d28/d36/d3c/d4f/lac 0 2024-02-16T19:24:31.418 INFO:tasks.workunit.client.1.smithi163.stdout:5/693: write d0/d2/fc [3745543,113872] 0 2024-02-16T19:24:31.427 INFO:tasks.workunit.client.1.smithi163.stdout:4/621: dwrite d11/d18/d1f/d29/d52/d76/da2/fc7 [0,4194304] 0 2024-02-16T19:24:31.427 INFO:tasks.workunit.client.1.smithi163.stdout:5/694: dread d0/d2/f30 [0,4194304] 0 2024-02-16T19:24:31.427 INFO:tasks.workunit.client.1.smithi163.stdout:5/695: truncate d0/d2/d34/d44/d53/d93/fa0 557700 0 2024-02-16T19:24:31.434 INFO:tasks.workunit.client.1.smithi163.stdout:4/622: unlink d11/d18/d1f/d23/l2a 0 2024-02-16T19:24:31.434 INFO:tasks.workunit.client.1.smithi163.stdout:4/623: read d11/d18/d1f/d29/f36 [2666633,55109] 0 2024-02-16T19:24:31.434 INFO:tasks.workunit.client.1.smithi163.stdout:4/624: dread - d11/d18/d1f/d23/d3a/d4a/fd2 zero size 2024-02-16T19:24:31.437 INFO:tasks.workunit.client.1.smithi163.stdout:5/696: rename d0/d2/f7 to d0/d2/d34/d44/d64/dd9/fea 0 2024-02-16T19:24:31.437 INFO:tasks.workunit.client.1.smithi163.stdout:5/697: dread - d0/d2/d34/d44/d53/d93/da8/fda zero size 2024-02-16T19:24:31.437 INFO:tasks.workunit.client.1.smithi163.stdout:5/698: write d0/d2/d34/d44/f51 [2429966,107367] 0 2024-02-16T19:24:31.446 INFO:tasks.workunit.client.1.smithi163.stdout:5/699: dread d0/d22/d28/d36/d3c/d4f/f61 [0,4194304] 0 2024-02-16T19:24:31.446 INFO:tasks.workunit.client.1.smithi163.stdout:5/700: stat d0/d2/d34/fce 0 2024-02-16T19:24:31.446 INFO:tasks.workunit.client.1.smithi163.stdout:5/701: fsync d0/d22/f3a 0 2024-02-16T19:24:31.447 INFO:tasks.workunit.client.1.smithi163.stdout:5/702: rmdir d0/d22/d28/d36/d3c/d4f 39 2024-02-16T19:24:31.448 INFO:tasks.workunit.client.1.smithi163.stdout:5/703: read - d0/d22/d28/fad zero size 2024-02-16T19:24:31.448 INFO:tasks.workunit.client.1.smithi163.stdout:5/704: symlink d0/d2/d5c/leb 0 2024-02-16T19:24:31.456 INFO:tasks.workunit.client.1.smithi163.stdout:5/705: dread d0/d2/d34/d44/d64/dd9/fea [0,4194304] 0 2024-02-16T19:24:31.456 INFO:tasks.workunit.client.1.smithi163.stdout:5/706: read - d0/d2/f29 zero size 2024-02-16T19:24:31.471 INFO:tasks.workunit.client.1.smithi163.stdout:5/707: dwrite d0/d2/d34/d44/d53/f7d [0,4194304] 0 2024-02-16T19:24:31.473 INFO:tasks.workunit.client.1.smithi163.stdout:5/708: creat d0/d1c/dbe/de3/fec x:0 0 0 2024-02-16T19:24:31.474 INFO:tasks.workunit.client.1.smithi163.stdout:5/709: creat d0/d22/d28/d36/db7/fed x:0 0 0 2024-02-16T19:24:31.474 INFO:tasks.workunit.client.1.smithi163.stdout:5/710: write d0/d2/d34/d44/d53/d93/f7a [812747,58246] 0 2024-02-16T19:24:31.475 INFO:tasks.workunit.client.1.smithi163.stdout:5/711: symlink d0/d1c/d5a/da5/lee 0 2024-02-16T19:24:31.476 INFO:tasks.workunit.client.1.smithi163.stdout:5/712: link d0/d2/d34/lbd d0/d22/lef 0 2024-02-16T19:24:31.476 INFO:tasks.workunit.client.1.smithi163.stdout:5/713: creat d0/d2/d34/d44/d53/ff0 x:0 0 0 2024-02-16T19:24:31.476 INFO:tasks.workunit.client.1.smithi163.stdout:5/714: fdatasync d0/d2/f67 0 2024-02-16T19:24:31.477 INFO:tasks.workunit.client.1.smithi163.stdout:5/715: symlink d0/d22/lf1 0 2024-02-16T19:24:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:31 smithi163 ceph-mon[36452]: pgmap v254: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 227 MiB/s rd, 276 MiB/s wr, 394 op/s 2024-02-16T19:24:31.493 INFO:tasks.workunit.client.1.smithi163.stdout:5/716: dwrite d0/d22/d3b/fb2 [0,4194304] 0 2024-02-16T19:24:31.493 INFO:tasks.workunit.client.1.smithi163.stdout:5/717: creat d0/d2/d34/d44/d53/ff2 x:0 0 0 2024-02-16T19:24:31.506 INFO:tasks.workunit.client.1.smithi163.stdout:5/718: dwrite d0/d2/d34/d44/d53/ff0 [0,4194304] 0 2024-02-16T19:24:31.506 INFO:tasks.workunit.client.1.smithi163.stdout:5/719: creat d0/d2/d31/ff3 x:0 0 0 2024-02-16T19:24:31.506 INFO:tasks.workunit.client.1.smithi163.stdout:5/720: fsync d0/d22/d28/d36/d3c/d4f/f7b 0 2024-02-16T19:24:31.508 INFO:tasks.workunit.client.1.smithi163.stdout:5/721: getdents d0/d2/d34/d44/d64/dd9 0 2024-02-16T19:24:31.509 INFO:tasks.workunit.client.1.smithi163.stdout:5/722: truncate d0/d1c/dbe/de3/fec 516304 0 2024-02-16T19:24:31.509 INFO:tasks.workunit.client.1.smithi163.stdout:5/723: chown d0/d1c/d5a/da5/lee 1028 1 2024-02-16T19:24:31.510 INFO:tasks.workunit.client.1.smithi163.stdout:1/674: sync 2024-02-16T19:24:31.510 INFO:tasks.workunit.client.1.smithi163.stdout:6/575: sync 2024-02-16T19:24:31.510 INFO:tasks.workunit.client.1.smithi163.stdout:5/724: rmdir d0/d2/d34/d44/d53/d93/da8 39 2024-02-16T19:24:31.510 INFO:tasks.workunit.client.1.smithi163.stdout:5/725: chown d0/d22/d3b/f42 1 1 2024-02-16T19:24:31.511 INFO:tasks.workunit.client.1.smithi163.stdout:1/675: symlink d7/dc/d78/le7 0 2024-02-16T19:24:31.512 INFO:tasks.workunit.client.1.smithi163.stdout:1/676: write d7/d20/d21/f45 [1156473,111640] 0 2024-02-16T19:24:31.512 INFO:tasks.workunit.client.1.smithi163.stdout:1/677: creat d7/dc/d2d/d32/d87/da6/fe8 x:0 0 0 2024-02-16T19:24:31.513 INFO:tasks.workunit.client.1.smithi163.stdout:5/726: link d0/d22/d3b/f42 d0/d2/d34/d44/d64/dd9/ff4 0 2024-02-16T19:24:31.513 INFO:tasks.workunit.client.1.smithi163.stdout:5/727: readlink d0/d22/d3b/l90 0 2024-02-16T19:24:31.513 INFO:tasks.workunit.client.1.smithi163.stdout:5/728: readlink d0/d22/d28/l88 0 2024-02-16T19:24:31.515 INFO:tasks.workunit.client.1.smithi163.stdout:5/729: link d0/d22/d28/d36/db7/lba d0/d22/d28/d36/db7/lf5 0 2024-02-16T19:24:31.515 INFO:tasks.workunit.client.1.smithi163.stdout:5/730: fdatasync d0/d2/d34/d44/d53/f87 0 2024-02-16T19:24:31.516 INFO:tasks.workunit.client.1.smithi163.stdout:1/678: getdents d7/dc/d2d/d32/d87 0 2024-02-16T19:24:31.519 INFO:tasks.workunit.client.1.smithi163.stdout:5/731: unlink d0/d22/d28/d36/d3c/cc2 0 2024-02-16T19:24:31.520 INFO:tasks.workunit.client.1.smithi163.stdout:1/679: symlink d7/d20/d21/d4e/le9 0 2024-02-16T19:24:31.522 INFO:tasks.workunit.client.1.smithi163.stdout:1/680: getdents d7/dc/dc1 0 2024-02-16T19:24:31.525 INFO:tasks.workunit.client.1.smithi163.stdout:1/681: creat d7/dc/d2d/d90/fea x:0 0 0 2024-02-16T19:24:31.525 INFO:tasks.workunit.client.1.smithi163.stdout:1/682: chown d7/dc/d2d/d32/d87/da6/lb1 44 1 2024-02-16T19:24:31.525 INFO:tasks.workunit.client.1.smithi163.stdout:1/683: creat d7/d20/d21/d4e/feb x:0 0 0 2024-02-16T19:24:31.525 INFO:tasks.workunit.client.1.smithi163.stdout:1/684: write d7/f6b [7966143,8372] 0 2024-02-16T19:24:31.525 INFO:tasks.workunit.client.1.smithi163.stdout:1/685: chown d7/dc/d2d/d32/d3b/fdd 1467617 1 2024-02-16T19:24:31.525 INFO:tasks.workunit.client.1.smithi163.stdout:1/686: fdatasync d7/d20/f38 0 2024-02-16T19:24:31.535 INFO:tasks.workunit.client.1.smithi163.stdout:1/687: dread d7/dc/d2d/d32/d3b/f50 [0,4194304] 0 2024-02-16T19:24:31.536 INFO:tasks.workunit.client.1.smithi163.stdout:1/688: symlink d7/d20/d93/dd2/lec 0 2024-02-16T19:24:31.538 INFO:tasks.workunit.client.1.smithi163.stdout:6/576: dwrite d0/d18/f48 [0,4194304] 0 2024-02-16T19:24:31.538 INFO:tasks.workunit.client.1.smithi163.stdout:5/732: dwrite d0/f4d [0,4194304] 0 2024-02-16T19:24:31.538 INFO:tasks.workunit.client.1.smithi163.stdout:1/689: link d7/dc/d2d/d3f/f67 d7/dc/dc1/fed 0 2024-02-16T19:24:31.539 INFO:tasks.workunit.client.1.smithi163.stdout:6/577: mkdir d0/d18/d77/d97/db8 0 2024-02-16T19:24:31.544 INFO:tasks.workunit.client.1.smithi163.stdout:5/733: link d0/d2/d34/d44/d53/l9e d0/d1c/d5a/da5/lf6 0 2024-02-16T19:24:31.544 INFO:tasks.workunit.client.1.smithi163.stdout:5/734: chown d0/d22/f35 575 1 2024-02-16T19:24:31.544 INFO:tasks.workunit.client.1.smithi163.stdout:5/735: truncate d0/d2/d34/f6d 4339371 0 2024-02-16T19:24:31.545 INFO:tasks.workunit.client.1.smithi163.stdout:6/578: dread d0/d6/df/d20/d40/d53/f72 [0,4194304] 0 2024-02-16T19:24:31.545 INFO:tasks.workunit.client.1.smithi163.stdout:5/736: dread d0/d2/d34/fc6 [0,4194304] 0 2024-02-16T19:24:31.545 INFO:tasks.workunit.client.1.smithi163.stdout:5/737: fsync d0/d2/f30 0 2024-02-16T19:24:31.545 INFO:tasks.workunit.client.1.smithi163.stdout:5/738: chown d0/c2d 0 1 2024-02-16T19:24:31.545 INFO:tasks.workunit.client.1.smithi163.stdout:5/739: write d0/d22/d28/f38 [6015416,76185] 0 2024-02-16T19:24:31.552 INFO:tasks.workunit.client.1.smithi163.stdout:1/690: mkdir d7/dc/d78/dee 0 2024-02-16T19:24:31.553 INFO:tasks.workunit.client.1.smithi163.stdout:6/579: creat d0/d6/df/d20/d69/db4/fb9 x:0 0 0 2024-02-16T19:24:31.556 INFO:tasks.workunit.client.1.smithi163.stdout:5/740: mknod d0/d1c/dbe/cf7 0 2024-02-16T19:24:31.557 INFO:tasks.workunit.client.1.smithi163.stdout:6/580: write d0/d6/df/d20/d40/f89 [3841130,54056] 0 2024-02-16T19:24:31.560 INFO:tasks.workunit.client.1.smithi163.stdout:6/581: creat d0/d6/df/d7b/d87/fba x:0 0 0 2024-02-16T19:24:31.560 INFO:tasks.workunit.client.1.smithi163.stdout:5/741: getdents d0/d22/d28/d36/d3c 0 2024-02-16T19:24:31.561 INFO:tasks.workunit.client.1.smithi163.stdout:5/742: write d0/d22/f33 [2210880,35103] 0 2024-02-16T19:24:31.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/582: rmdir d0/d18 39 2024-02-16T19:24:31.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/583: dread - d0/dd/d7f/d2e/d33/f3c zero size 2024-02-16T19:24:31.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/584: stat d0/d6/df/d20/f90 0 2024-02-16T19:24:31.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/585: readlink d0/d6/df/d30/l6e 0 2024-02-16T19:24:31.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/586: dread - d0/dd/d7f/d15/d1c/f99 zero size 2024-02-16T19:24:31.562 INFO:tasks.workunit.client.1.smithi163.stdout:1/691: read d7/d20/d21/d4e/f9e [2695776,112922] 0 2024-02-16T19:24:31.562 INFO:tasks.workunit.client.1.smithi163.stdout:1/692: write d7/f22 [635935,56654] 0 2024-02-16T19:24:31.564 INFO:tasks.workunit.client.1.smithi163.stdout:6/587: unlink d0/d6/df/d30/l43 0 2024-02-16T19:24:31.564 INFO:tasks.workunit.client.1.smithi163.stdout:1/693: symlink d7/d20/d93/lef 0 2024-02-16T19:24:31.565 INFO:tasks.workunit.client.1.smithi163.stdout:6/588: symlink d0/d18/d94/lbb 0 2024-02-16T19:24:31.566 INFO:tasks.workunit.client.1.smithi163.stdout:5/743: read d0/d22/d28/d36/d3c/d4f/f61 [4010311,37468] 0 2024-02-16T19:24:31.576 INFO:tasks.workunit.client.1.smithi163.stdout:1/694: dread d7/dc/f33 [0,4194304] 0 2024-02-16T19:24:31.578 INFO:tasks.workunit.client.1.smithi163.stdout:6/589: write d0/d18/f48 [4155910,128074] 0 2024-02-16T19:24:31.578 INFO:tasks.workunit.client.1.smithi163.stdout:6/590: write d0/d18/d77/d70/f8c [105791,17837] 0 2024-02-16T19:24:31.578 INFO:tasks.workunit.client.1.smithi163.stdout:1/695: rename d7/d20/d96/fa1 to d7/dc/d2d/d32/ff0 0 2024-02-16T19:24:31.579 INFO:tasks.workunit.client.1.smithi163.stdout:6/591: mkdir d0/dd/dbc 0 2024-02-16T19:24:31.580 INFO:tasks.workunit.client.1.smithi163.stdout:1/696: symlink d7/d20/d21/d4e/lf1 0 2024-02-16T19:24:31.580 INFO:tasks.workunit.client.1.smithi163.stdout:1/697: truncate d7/dc/d2d/d32/f46 544458 0 2024-02-16T19:24:31.581 INFO:tasks.workunit.client.1.smithi163.stdout:5/744: dwrite d0/d2/d34/fc6 [0,4194304] 0 2024-02-16T19:24:31.582 INFO:tasks.workunit.client.1.smithi163.stdout:5/745: readlink d0/l19 0 2024-02-16T19:24:31.584 INFO:tasks.workunit.client.1.smithi163.stdout:5/746: mkdir d0/d2/d34/df8 0 2024-02-16T19:24:31.586 INFO:tasks.workunit.client.1.smithi163.stdout:5/747: creat d0/d2/d34/d44/d53/d93/da8/ff9 x:0 0 0 2024-02-16T19:24:31.594 INFO:tasks.workunit.client.1.smithi163.stdout:6/592: dread d0/dd/d7f/d15/d1c/d65/d63/f6b [0,4194304] 0 2024-02-16T19:24:31.594 INFO:tasks.workunit.client.1.smithi163.stdout:6/593: write d0/d6/f8 [556924,107191] 0 2024-02-16T19:24:31.594 INFO:tasks.workunit.client.1.smithi163.stdout:6/594: dread - d0/d6/df/d30/f5c zero size 2024-02-16T19:24:31.594 INFO:tasks.workunit.client.1.smithi163.stdout:6/595: readlink d0/d6/l67 0 2024-02-16T19:24:31.594 INFO:tasks.workunit.client.1.smithi163.stdout:6/596: truncate d0/d6/df/d20/d69/fa6 230062 0 2024-02-16T19:24:31.595 INFO:tasks.workunit.client.1.smithi163.stdout:6/597: write d0/dd/d7f/d15/d1c/d65/d63/f79 [711845,87568] 0 2024-02-16T19:24:31.595 INFO:tasks.workunit.client.1.smithi163.stdout:6/598: creat d0/dd/d7f/d2e/fbd x:0 0 0 2024-02-16T19:24:31.600 INFO:tasks.workunit.client.1.smithi163.stdout:6/599: dread d0/d6/df/d20/d40/d1e/f52 [0,4194304] 0 2024-02-16T19:24:31.600 INFO:tasks.workunit.client.1.smithi163.stdout:6/600: dread - d0/d6/df/d30/f5a zero size 2024-02-16T19:24:31.605 INFO:tasks.workunit.client.1.smithi163.stdout:1/698: dwrite d7/dc/d2d/d32/f43 [4194304,4194304] 0 2024-02-16T19:24:31.606 INFO:tasks.workunit.client.1.smithi163.stdout:5/748: dwrite d0/d1c/f74 [4194304,4194304] 0 2024-02-16T19:24:31.607 INFO:tasks.workunit.client.1.smithi163.stdout:1/699: link d7/d20/d2a/c7a d7/dc/d2d/d32/d87/da6/de5/cf2 0 2024-02-16T19:24:31.607 INFO:tasks.workunit.client.1.smithi163.stdout:1/700: fsync d7/d20/f35 0 2024-02-16T19:24:31.611 INFO:tasks.workunit.client.1.smithi163.stdout:6/601: write d0/d18/f58 [576670,46034] 0 2024-02-16T19:24:31.614 INFO:tasks.workunit.client.1.smithi163.stdout:5/749: unlink d0/d2/d34/d44/d53/lcd 0 2024-02-16T19:24:31.616 INFO:tasks.workunit.client.1.smithi163.stdout:1/701: unlink d7/d20/d2a/ld4 0 2024-02-16T19:24:31.620 INFO:tasks.workunit.client.1.smithi163.stdout:5/750: mkdir d0/d22/d3b/dc0/dfa 0 2024-02-16T19:24:31.621 INFO:tasks.workunit.client.1.smithi163.stdout:6/602: link d0/f5 d0/dd/d7f/d15/d1c/d65/d63/fbe 0 2024-02-16T19:24:31.622 INFO:tasks.workunit.client.1.smithi163.stdout:5/751: mknod d0/d22/d3b/dc0/dfa/cfb 0 2024-02-16T19:24:31.625 INFO:tasks.workunit.client.1.smithi163.stdout:1/702: dread d7/d20/d21/f55 [0,4194304] 0 2024-02-16T19:24:31.626 INFO:tasks.workunit.client.1.smithi163.stdout:5/752: creat d0/d2/d34/df8/ffc x:0 0 0 2024-02-16T19:24:31.627 INFO:tasks.workunit.client.1.smithi163.stdout:1/703: mknod d7/d20/cf3 0 2024-02-16T19:24:31.630 INFO:tasks.workunit.client.1.smithi163.stdout:5/753: rename d0/d1c/dbe/lde to d0/d2/d34/d44/d53/d93/lfd 0 2024-02-16T19:24:31.632 INFO:tasks.workunit.client.1.smithi163.stdout:1/704: unlink d7/dc/d78/d7c/c6f 0 2024-02-16T19:24:31.632 INFO:tasks.workunit.client.1.smithi163.stdout:1/705: fsync d7/dc/d78/d7c/f81 0 2024-02-16T19:24:31.632 INFO:tasks.workunit.client.1.smithi163.stdout:1/706: fdatasync d7/d5b/f91 0 2024-02-16T19:24:31.632 INFO:tasks.workunit.client.1.smithi163.stdout:1/707: rename d7/d20 to d7/d20/d93/dd2/df4 22 2024-02-16T19:24:31.633 INFO:tasks.workunit.client.1.smithi163.stdout:5/754: mknod d0/d2/d34/d44/cfe 0 2024-02-16T19:24:31.633 INFO:tasks.workunit.client.1.smithi163.stdout:1/708: stat d7/dc/d2d/d32/d87/lcd 0 2024-02-16T19:24:31.633 INFO:tasks.workunit.client.1.smithi163.stdout:1/709: chown d7/l9b 853022564 1 2024-02-16T19:24:31.633 INFO:tasks.workunit.client.1.smithi163.stdout:1/710: stat d7/dc/d2d/fc7 0 2024-02-16T19:24:31.634 INFO:tasks.workunit.client.1.smithi163.stdout:1/711: write d7/dc/d2d/d32/d87/d97/fe6 [947321,11506] 0 2024-02-16T19:24:31.634 INFO:tasks.workunit.client.1.smithi163.stdout:6/603: write d0/d6/df/f32 [682107,107773] 0 2024-02-16T19:24:31.634 INFO:tasks.workunit.client.1.smithi163.stdout:6/604: write d0/d6/df/f32 [398437,45814] 0 2024-02-16T19:24:31.634 INFO:tasks.workunit.client.1.smithi163.stdout:5/755: creat d0/d22/d28/d36/d3c/fff x:0 0 0 2024-02-16T19:24:31.634 INFO:tasks.workunit.client.1.smithi163.stdout:5/756: chown d0/d22/d3b/f56 830120 1 2024-02-16T19:24:31.634 INFO:tasks.workunit.client.1.smithi163.stdout:5/757: truncate d0/d22/d28/d36/d3c/d4f/fd3 248084 0 2024-02-16T19:24:31.635 INFO:tasks.workunit.client.1.smithi163.stdout:1/712: creat d7/d20/d93/ff5 x:0 0 0 2024-02-16T19:24:31.635 INFO:tasks.workunit.client.1.smithi163.stdout:1/713: dread - d7/d5b/d63/fb8 zero size 2024-02-16T19:24:31.635 INFO:tasks.workunit.client.1.smithi163.stdout:6/605: truncate d0/dd/d7f/d15/f4d 399902 0 2024-02-16T19:24:31.635 INFO:tasks.workunit.client.1.smithi163.stdout:6/606: creat d0/d18/d77/fbf x:0 0 0 2024-02-16T19:24:31.637 INFO:tasks.workunit.client.1.smithi163.stdout:6/607: rename d0/d6/df/d20/d40/d17/c39 to d0/d6/cc0 0 2024-02-16T19:24:31.644 INFO:tasks.workunit.client.1.smithi163.stdout:1/714: dread d7/d20/d2a/f40 [0,4194304] 0 2024-02-16T19:24:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:31 smithi040 ceph-mon[29201]: pgmap v254: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 227 MiB/s rd, 276 MiB/s wr, 394 op/s 2024-02-16T19:24:31.647 INFO:tasks.workunit.client.1.smithi163.stdout:6/608: write d0/dd/d7f/d15/d1c/f56 [910852,101613] 0 2024-02-16T19:24:31.651 INFO:tasks.workunit.client.1.smithi163.stdout:5/758: dwrite d0/d2/d34/d44/d53/d93/fa0 [0,4194304] 0 2024-02-16T19:24:31.651 INFO:tasks.workunit.client.1.smithi163.stdout:5/759: write d0/d1c/d5a/fcf [5504,63116] 0 2024-02-16T19:24:31.654 INFO:tasks.workunit.client.1.smithi163.stdout:5/760: creat d0/d2/d34/f100 x:0 0 0 2024-02-16T19:24:31.656 INFO:tasks.workunit.client.1.smithi163.stdout:5/761: creat d0/d22/d28/d36/dc4/f101 x:0 0 0 2024-02-16T19:24:31.656 INFO:tasks.workunit.client.1.smithi163.stdout:5/762: fdatasync d0/d2/d34/d44/f72 0 2024-02-16T19:24:31.658 INFO:tasks.workunit.client.1.smithi163.stdout:5/763: rename d0/d22/d3b/l90 to d0/d2/d34/d44/d64/dd9/l102 0 2024-02-16T19:24:31.659 INFO:tasks.workunit.client.1.smithi163.stdout:1/715: write d7/fb [1150453,100267] 0 2024-02-16T19:24:31.659 INFO:tasks.workunit.client.1.smithi163.stdout:1/716: write d7/dc/d2d/d32/fbe [777161,109592] 0 2024-02-16T19:24:31.665 INFO:tasks.workunit.client.1.smithi163.stdout:8/637: sync 2024-02-16T19:24:31.670 INFO:tasks.workunit.client.1.smithi163.stdout:8/638: getdents d0/d7/d18 0 2024-02-16T19:24:31.670 INFO:tasks.workunit.client.1.smithi163.stdout:8/639: fsync d0/d24/d2c/d74/f93 0 2024-02-16T19:24:31.670 INFO:tasks.workunit.client.1.smithi163.stdout:8/640: write d0/dc/f7d [913334,91661] 0 2024-02-16T19:24:31.673 INFO:tasks.workunit.client.1.smithi163.stdout:5/764: dread d0/d2/d34/d44/d53/f7d [0,4194304] 0 2024-02-16T19:24:31.674 INFO:tasks.workunit.client.1.smithi163.stdout:5/765: chown d0/d2/d34/d44/d53/f7d 18535 1 2024-02-16T19:24:31.674 INFO:tasks.workunit.client.1.smithi163.stdout:5/766: dread - d0/d2/d5c/f83 zero size 2024-02-16T19:24:31.675 INFO:tasks.workunit.client.1.smithi163.stdout:5/767: rename d0/d22/d28/d36/l52 to d0/d1c/dbe/l103 0 2024-02-16T19:24:31.676 INFO:tasks.workunit.client.1.smithi163.stdout:5/768: mkdir d0/d22/d28/d36/d3c/d104 0 2024-02-16T19:24:31.677 INFO:tasks.workunit.client.1.smithi163.stdout:5/769: truncate d0/d22/d3b/fb2 1660466 0 2024-02-16T19:24:31.680 INFO:tasks.workunit.client.1.smithi163.stdout:6/609: dwrite d0/dd/d7f/d15/f4d [0,4194304] 0 2024-02-16T19:24:31.690 INFO:tasks.workunit.client.1.smithi163.stdout:1/717: dwrite d7/d20/f38 [0,4194304] 0 2024-02-16T19:24:31.694 INFO:tasks.workunit.client.1.smithi163.stdout:6/610: dread d0/d6/df/d20/d40/d17/f88 [0,4194304] 0 2024-02-16T19:24:31.698 INFO:tasks.workunit.client.1.smithi163.stdout:8/641: dwrite d0/d24/d27/d45/f5a [0,4194304] 0 2024-02-16T19:24:31.698 INFO:tasks.workunit.client.1.smithi163.stdout:5/770: dwrite d0/d2/f57 [0,4194304] 0 2024-02-16T19:24:31.698 INFO:tasks.workunit.client.1.smithi163.stdout:5/771: chown d0/d2/fe7 0 1 2024-02-16T19:24:31.699 INFO:tasks.workunit.client.1.smithi163.stdout:8/642: write d0/d7/d18/d53/fc5 [1044544,13394] 0 2024-02-16T19:24:31.710 INFO:tasks.workunit.client.1.smithi163.stdout:8/643: dread d0/d7/d19/d1f/f4e [0,4194304] 0 2024-02-16T19:24:31.711 INFO:tasks.workunit.client.1.smithi163.stdout:8/644: chown d0/d24/d3c/d3d/l2b 172 1 2024-02-16T19:24:31.711 INFO:tasks.workunit.client.1.smithi163.stdout:8/645: truncate d0/d6f/fc8 361688 0 2024-02-16T19:24:31.716 INFO:tasks.workunit.client.1.smithi163.stdout:8/646: rmdir d0/d7 39 2024-02-16T19:24:31.716 INFO:tasks.workunit.client.1.smithi163.stdout:8/647: getdents d0/d24/d3c 0 2024-02-16T19:24:31.717 INFO:tasks.workunit.client.1.smithi163.stdout:8/648: write d0/d7/d18/d53/fc5 [887071,63524] 0 2024-02-16T19:24:31.718 INFO:tasks.workunit.client.1.smithi163.stdout:8/649: mknod d0/d7/d19/d1f/d30/d6a/cd6 0 2024-02-16T19:24:31.718 INFO:tasks.workunit.client.1.smithi163.stdout:8/650: creat d0/d24/d2c/d74/fd7 x:0 0 0 2024-02-16T19:24:31.718 INFO:tasks.workunit.client.1.smithi163.stdout:8/651: fsync d0/d7/d19/d1f/d30/fb2 0 2024-02-16T19:24:31.719 INFO:tasks.workunit.client.1.smithi163.stdout:8/652: truncate d0/d7/d18/d2e/f5d 2404525 0 2024-02-16T19:24:31.720 INFO:tasks.workunit.client.1.smithi163.stdout:8/653: mknod d0/d24/d2c/d74/dcb/cd8 0 2024-02-16T19:24:31.720 INFO:tasks.workunit.client.1.smithi163.stdout:8/654: fdatasync d0/d7/d19/d1f/d30/d6a/db8/fcd 0 2024-02-16T19:24:31.721 INFO:tasks.workunit.client.1.smithi163.stdout:8/655: rename d0/d24/d2c/d74/f93 to d0/d7/d18/fd9 0 2024-02-16T19:24:31.722 INFO:tasks.workunit.client.1.smithi163.stdout:6/611: dwrite d0/d6/df/d20/f68 [0,4194304] 0 2024-02-16T19:24:31.722 INFO:tasks.workunit.client.1.smithi163.stdout:1/718: dwrite d7/dc/d78/fbd [0,4194304] 0 2024-02-16T19:24:31.723 INFO:tasks.workunit.client.1.smithi163.stdout:5/772: dwrite d0/d2/d34/d44/d53/ff0 [0,4194304] 0 2024-02-16T19:24:31.726 INFO:tasks.workunit.client.1.smithi163.stdout:8/656: truncate d0/d7/d19/d1f/d32/d70/f8b 563844 0 2024-02-16T19:24:31.731 INFO:tasks.workunit.client.1.smithi163.stdout:5/773: mknod d0/d1c/d5a/c105 0 2024-02-16T19:24:31.731 INFO:tasks.workunit.client.1.smithi163.stdout:5/774: fsync d0/d2/d34/d44/d53/f87 0 2024-02-16T19:24:31.732 INFO:tasks.workunit.client.1.smithi163.stdout:5/775: mkdir d0/d2/db1/d106 0 2024-02-16T19:24:31.734 INFO:tasks.workunit.client.1.smithi163.stdout:5/776: symlink d0/d1c/d5a/l107 0 2024-02-16T19:24:31.739 INFO:tasks.workunit.client.1.smithi163.stdout:5/777: rmdir d0/d22/d3b 39 2024-02-16T19:24:31.739 INFO:tasks.workunit.client.1.smithi163.stdout:5/778: write d0/d2/d5c/f5d [1024479,101509] 0 2024-02-16T19:24:31.739 INFO:tasks.workunit.client.1.smithi163.stdout:8/657: dread d0/d7/d19/d1f/d30/f92 [0,4194304] 0 2024-02-16T19:24:31.739 INFO:tasks.workunit.client.1.smithi163.stdout:9/924: sync 2024-02-16T19:24:31.740 INFO:tasks.workunit.client.1.smithi163.stdout:9/925: chown d5/de/d30/d8a/da8/fad 14500 1 2024-02-16T19:24:31.740 INFO:tasks.workunit.client.1.smithi163.stdout:9/926: chown d5/de/d16/d46/d9b/f9d 1598064 1 2024-02-16T19:24:31.740 INFO:tasks.workunit.client.1.smithi163.stdout:0/926: sync 2024-02-16T19:24:31.741 INFO:tasks.workunit.client.1.smithi163.stdout:8/658: symlink d0/lda 0 2024-02-16T19:24:31.741 INFO:tasks.workunit.client.1.smithi163.stdout:9/927: creat d5/de/d30/d43/d8f/f140 x:0 0 0 2024-02-16T19:24:31.741 INFO:tasks.workunit.client.1.smithi163.stdout:9/928: write d5/de/d16/d47/fb4 [1042771,128072] 0 2024-02-16T19:24:31.742 INFO:tasks.workunit.client.1.smithi163.stdout:9/929: mknod d5/de/d16/d46/d9b/d106/c141 0 2024-02-16T19:24:31.744 INFO:tasks.workunit.client.1.smithi163.stdout:1/719: dwrite d7/d20/d21/d4e/dc6/fdf [4194304,4194304] 0 2024-02-16T19:24:31.749 INFO:tasks.workunit.client.1.smithi163.stdout:6/612: dwrite d0/dd/d7f/f4c [0,4194304] 0 2024-02-16T19:24:31.750 INFO:tasks.workunit.client.1.smithi163.stdout:1/720: dread d7/fa [0,4194304] 0 2024-02-16T19:24:31.750 INFO:tasks.workunit.client.1.smithi163.stdout:1/721: chown d7/dc/c10 248870370 1 2024-02-16T19:24:31.752 INFO:tasks.workunit.client.1.smithi163.stdout:7/659: sync 2024-02-16T19:24:31.752 INFO:tasks.workunit.client.1.smithi163.stdout:7/660: fsync d11/d24/d25/d4b/fdc 0 2024-02-16T19:24:31.753 INFO:tasks.workunit.client.1.smithi163.stdout:7/661: readlink d11/d26/l9c 0 2024-02-16T19:24:31.753 INFO:tasks.workunit.client.1.smithi163.stdout:3/670: sync 2024-02-16T19:24:31.753 INFO:tasks.workunit.client.1.smithi163.stdout:2/815: sync 2024-02-16T19:24:31.753 INFO:tasks.workunit.client.1.smithi163.stdout:4/625: sync 2024-02-16T19:24:31.753 INFO:tasks.workunit.client.1.smithi163.stdout:4/626: creat d11/d2d/d4f/d60/d79/fd3 x:0 0 0 2024-02-16T19:24:31.753 INFO:tasks.workunit.client.1.smithi163.stdout:4/627: readlink d11/d18/d66/l77 0 2024-02-16T19:24:31.753 INFO:tasks.workunit.client.1.smithi163.stdout:4/628: fsync d11/d3d/f62 0 2024-02-16T19:24:31.753 INFO:tasks.workunit.client.1.smithi163.stdout:4/629: fsync d11/d18/d1f/d29/d52/d57/d6c/d9d/fa0 0 2024-02-16T19:24:31.754 INFO:tasks.workunit.client.1.smithi163.stdout:6/613: mknod d0/d6/df/d30/cc1 0 2024-02-16T19:24:31.754 INFO:tasks.workunit.client.1.smithi163.stdout:8/659: dread d0/d7/d55/f80 [4194304,4194304] 0 2024-02-16T19:24:31.754 INFO:tasks.workunit.client.1.smithi163.stdout:8/660: fdatasync d0/d24/d27/d45/f75 0 2024-02-16T19:24:31.754 INFO:tasks.workunit.client.1.smithi163.stdout:7/662: unlink d11/d26/f98 0 2024-02-16T19:24:31.755 INFO:tasks.workunit.client.1.smithi163.stdout:3/671: readlink d0/dc/l35 0 2024-02-16T19:24:31.755 INFO:tasks.workunit.client.1.smithi163.stdout:3/672: chown d0/d3e/c47 14398 1 2024-02-16T19:24:31.758 INFO:tasks.workunit.client.1.smithi163.stdout:6/614: symlink d0/d18/lc2 0 2024-02-16T19:24:31.760 INFO:tasks.workunit.client.1.smithi163.stdout:7/663: truncate fa 3891843 0 2024-02-16T19:24:31.760 INFO:tasks.workunit.client.1.smithi163.stdout:3/673: symlink d0/dc/d1d/d49/lf2 0 2024-02-16T19:24:31.762 INFO:tasks.workunit.client.1.smithi163.stdout:6/615: mkdir d0/d6/df/d20/d40/d17/dc3 0 2024-02-16T19:24:31.762 INFO:tasks.workunit.client.1.smithi163.stdout:6/616: write d0/d6/df/f32 [295515,59951] 0 2024-02-16T19:24:31.764 INFO:tasks.workunit.client.1.smithi163.stdout:1/722: dread d7/d20/f38 [0,4194304] 0 2024-02-16T19:24:31.767 INFO:tasks.workunit.client.1.smithi163.stdout:7/664: creat d11/d50/d5d/dcd/fe0 x:0 0 0 2024-02-16T19:24:31.767 INFO:tasks.workunit.client.1.smithi163.stdout:7/665: write d11/d24/d25/fbf [77163,105678] 0 2024-02-16T19:24:31.767 INFO:tasks.workunit.client.1.smithi163.stdout:7/666: dread - d11/d24/d25/d4b/d66/dc1/fd7 zero size 2024-02-16T19:24:31.769 INFO:tasks.workunit.client.1.smithi163.stdout:3/674: mknod d0/dc/d1d/d49/d98/cf3 0 2024-02-16T19:24:31.770 INFO:tasks.workunit.client.1.smithi163.stdout:4/630: write d11/d18/d1f/d29/d52/d57/f84 [2176030,91188] 0 2024-02-16T19:24:31.772 INFO:tasks.workunit.client.1.smithi163.stdout:8/661: dread d0/d6f/f98 [0,4194304] 0 2024-02-16T19:24:31.772 INFO:tasks.workunit.client.1.smithi163.stdout:8/662: fdatasync d0/d24/d27/d45/fae 0 2024-02-16T19:24:31.772 INFO:tasks.workunit.client.1.smithi163.stdout:8/663: truncate d0/d24/d2c/f71 623506 0 2024-02-16T19:24:31.772 INFO:tasks.workunit.client.1.smithi163.stdout:8/664: chown d0/d7/d19/d1f/f4e 11 1 2024-02-16T19:24:31.772 INFO:tasks.workunit.client.1.smithi163.stdout:8/665: write d0/d7/d18/d2e/f5f [4851962,90314] 0 2024-02-16T19:24:31.773 INFO:tasks.workunit.client.1.smithi163.stdout:2/816: rename d2/d2a/d39/d66/d6c to d2/d2a/d39/d117 0 2024-02-16T19:24:31.776 INFO:tasks.workunit.client.1.smithi163.stdout:2/817: mknod d2/c118 0 2024-02-16T19:24:31.776 INFO:tasks.workunit.client.1.smithi163.stdout:2/818: chown d2/d2a/d9f/df8 5 1 2024-02-16T19:24:31.777 INFO:tasks.workunit.client.1.smithi163.stdout:2/819: read - d2/d2a/d2d/d73/d94/dc9/f115 zero size 2024-02-16T19:24:31.780 INFO:tasks.workunit.client.1.smithi163.stdout:4/631: getdents d11/d2d/d4f/d5f/d70/d74/d75 0 2024-02-16T19:24:31.780 INFO:tasks.workunit.client.1.smithi163.stdout:4/632: truncate d11/d18/d1f/f33 1279524 0 2024-02-16T19:24:31.780 INFO:tasks.workunit.client.1.smithi163.stdout:4/633: chown d11/d2d/d4f/d60/d79/f90 586768750 1 2024-02-16T19:24:31.781 INFO:tasks.workunit.client.1.smithi163.stdout:9/930: dwrite d5/dc0/d102/d11e/f11f [0,4194304] 0 2024-02-16T19:24:31.781 INFO:tasks.workunit.client.1.smithi163.stdout:9/931: chown d5/de/d30/d31/c37 21219 1 2024-02-16T19:24:31.781 INFO:tasks.workunit.client.1.smithi163.stdout:2/820: truncate d2/f1f 2205248 0 2024-02-16T19:24:31.782 INFO:tasks.workunit.client.1.smithi163.stdout:0/927: dwrite d11/d21/d50/f79 [0,4194304] 0 2024-02-16T19:24:31.782 INFO:tasks.workunit.client.1.smithi163.stdout:0/928: dread - d11/d21/d50/d58/d74/df8/f106 zero size 2024-02-16T19:24:31.782 INFO:tasks.workunit.client.1.smithi163.stdout:0/929: chown d11/f62 790397 1 2024-02-16T19:24:31.788 INFO:tasks.workunit.client.1.smithi163.stdout:3/675: read d0/dc/d96/fdd [698598,25552] 0 2024-02-16T19:24:31.788 INFO:tasks.workunit.client.1.smithi163.stdout:5/779: dwrite d0/d22/f33 [4194304,4194304] 0 2024-02-16T19:24:31.788 INFO:tasks.workunit.client.1.smithi163.stdout:9/932: unlink d5/fed 0 2024-02-16T19:24:31.789 INFO:tasks.workunit.client.1.smithi163.stdout:0/930: mkdir d11/d12/d67/d6a/dae/d118/d12a/d136 0 2024-02-16T19:24:31.791 INFO:tasks.workunit.client.1.smithi163.stdout:3/676: truncate d0/dc/d1c/f2c 3125507 0 2024-02-16T19:24:31.791 INFO:tasks.workunit.client.1.smithi163.stdout:3/677: fdatasync d0/dc/d94/d9e/dac/fd0 0 2024-02-16T19:24:31.791 INFO:tasks.workunit.client.1.smithi163.stdout:5/780: mknod d0/d22/d28/d36/d3c/d104/c108 0 2024-02-16T19:24:31.791 INFO:tasks.workunit.client.1.smithi163.stdout:5/781: dread - d0/d2/d34/d44/d53/d93/da8/fda zero size 2024-02-16T19:24:31.792 INFO:tasks.workunit.client.1.smithi163.stdout:9/933: truncate d5/de/d30/d3d/d49/d50/f63 3935530 0 2024-02-16T19:24:31.795 INFO:tasks.workunit.client.1.smithi163.stdout:0/931: creat d11/d21/d7c/d11c/f137 x:0 0 0 2024-02-16T19:24:31.795 INFO:tasks.workunit.client.1.smithi163.stdout:0/932: fdatasync d11/d21/f130 0 2024-02-16T19:24:31.796 INFO:tasks.workunit.client.1.smithi163.stdout:5/782: rmdir d0/d2/d55 39 2024-02-16T19:24:31.798 INFO:tasks.workunit.client.1.smithi163.stdout:0/933: creat d11/d11d/f138 x:0 0 0 2024-02-16T19:24:31.798 INFO:tasks.workunit.client.1.smithi163.stdout:3/678: getdents d0/dc/d1c/d32/db8/d99 0 2024-02-16T19:24:31.798 INFO:tasks.workunit.client.1.smithi163.stdout:3/679: chown d0/dc/d1d/d29/d38/l3a 2 1 2024-02-16T19:24:31.802 INFO:tasks.workunit.client.1.smithi163.stdout:0/934: mknod d11/d21/d50/d58/d72/d9e/dd8/c139 0 2024-02-16T19:24:31.802 INFO:tasks.workunit.client.1.smithi163.stdout:0/935: fdatasync d11/d21/d7c/f81 0 2024-02-16T19:24:31.805 INFO:tasks.workunit.client.1.smithi163.stdout:7/667: dwrite d11/d24/d34/d8d/f92 [0,4194304] 0 2024-02-16T19:24:31.805 INFO:tasks.workunit.client.1.smithi163.stdout:7/668: write d11/f1b [4291331,114545] 0 2024-02-16T19:24:31.805 INFO:tasks.workunit.client.1.smithi163.stdout:7/669: write d11/d24/d49/fa5 [5056820,123393] 0 2024-02-16T19:24:31.807 INFO:tasks.workunit.client.1.smithi163.stdout:3/680: getdents d0/dc/d94/d9e/dac 0 2024-02-16T19:24:31.809 INFO:tasks.workunit.client.1.smithi163.stdout:8/666: dwrite d0/d7/d19/d1f/d30/d48/fa7 [0,4194304] 0 2024-02-16T19:24:31.810 INFO:tasks.workunit.client.1.smithi163.stdout:6/617: dwrite d0/d6/df/d20/d69/fa6 [0,4194304] 0 2024-02-16T19:24:31.811 INFO:tasks.workunit.client.1.smithi163.stdout:6/618: read d0/dd/d7f/d2e/d33/f38 [221685,45246] 0 2024-02-16T19:24:31.816 INFO:tasks.workunit.client.1.smithi163.stdout:0/936: dread d11/d12/f15 [0,4194304] 0 2024-02-16T19:24:31.818 INFO:tasks.workunit.client.1.smithi163.stdout:7/670: rename d11/d24/d34/fc0 to d11/d30/dda/fe1 0 2024-02-16T19:24:31.819 INFO:tasks.workunit.client.1.smithi163.stdout:4/634: dwrite d11/d2d/d4f/d60/d79/fd3 [0,4194304] 0 2024-02-16T19:24:31.819 INFO:tasks.workunit.client.1.smithi163.stdout:4/635: creat d11/d2d/d4f/d5f/d70/d74/fd4 x:0 0 0 2024-02-16T19:24:31.820 INFO:tasks.workunit.client.1.smithi163.stdout:4/636: write d11/f8a [685232,39270] 0 2024-02-16T19:24:31.820 INFO:tasks.workunit.client.1.smithi163.stdout:8/667: creat d0/d7/fdb x:0 0 0 2024-02-16T19:24:31.822 INFO:tasks.workunit.client.1.smithi163.stdout:6/619: symlink d0/lc4 0 2024-02-16T19:24:31.822 INFO:tasks.workunit.client.1.smithi163.stdout:9/934: dwrite d5/de/d16/d133/f13e [0,4194304] 0 2024-02-16T19:24:31.823 INFO:tasks.workunit.client.1.smithi163.stdout:9/935: creat d5/de/d16/d46/d9b/d106/f142 x:0 0 0 2024-02-16T19:24:31.823 INFO:tasks.workunit.client.1.smithi163.stdout:2/821: dwrite d2/fb [4194304,4194304] 0 2024-02-16T19:24:31.823 INFO:tasks.workunit.client.1.smithi163.stdout:2/822: chown d2/d26/d54/d58/d98 130945977 1 2024-02-16T19:24:31.823 INFO:tasks.workunit.client.1.smithi163.stdout:0/937: symlink d11/d12/d67/d6a/dae/d118/l13a 0 2024-02-16T19:24:31.828 INFO:tasks.workunit.client.1.smithi163.stdout:7/671: dread d11/d26/f6d [0,4194304] 0 2024-02-16T19:24:31.829 INFO:tasks.workunit.client.1.smithi163.stdout:4/637: creat d11/d18/d1f/d29/d52/d76/d89/d94/fd5 x:0 0 0 2024-02-16T19:24:31.829 INFO:tasks.workunit.client.1.smithi163.stdout:4/638: chown d11/d18/d1f/d29/la4 59028 1 2024-02-16T19:24:31.830 INFO:tasks.workunit.client.1.smithi163.stdout:5/783: rmdir d0/d22/d28/d36 39 2024-02-16T19:24:31.830 INFO:tasks.workunit.client.1.smithi163.stdout:5/784: write d0/d22/d28/d36/dc4/f101 [573808,104206] 0 2024-02-16T19:24:31.830 INFO:tasks.workunit.client.1.smithi163.stdout:5/785: creat d0/d1c/dbe/de3/f109 x:0 0 0 2024-02-16T19:24:31.831 INFO:tasks.workunit.client.1.smithi163.stdout:6/620: symlink d0/d6/df/d20/d40/lc5 0 2024-02-16T19:24:31.831 INFO:tasks.workunit.client.1.smithi163.stdout:6/621: readlink d0/d6/le 0 2024-02-16T19:24:31.831 INFO:tasks.workunit.client.1.smithi163.stdout:6/622: fsync d0/dd/d7f/d2e/d33/fb7 0 2024-02-16T19:24:31.834 INFO:tasks.workunit.client.1.smithi163.stdout:9/936: dread d5/de/d30/d3d/d49/d6d/fc3 [0,4194304] 0 2024-02-16T19:24:31.835 INFO:tasks.workunit.client.1.smithi163.stdout:2/823: creat d2/d26/d97/d10f/f119 x:0 0 0 2024-02-16T19:24:31.836 INFO:tasks.workunit.client.1.smithi163.stdout:0/938: symlink d11/d21/d50/d58/d72/d9e/l13b 0 2024-02-16T19:24:31.837 INFO:tasks.workunit.client.1.smithi163.stdout:7/672: mknod d11/d50/d5d/d8b/ce2 0 2024-02-16T19:24:31.840 INFO:tasks.workunit.client.1.smithi163.stdout:5/786: link d0/d2/fd5 d0/d2/d34/d44/d53/d93/da8/f10a 0 2024-02-16T19:24:31.842 INFO:tasks.workunit.client.1.smithi163.stdout:6/623: mknod d0/dd/d7f/d2e/d33/cc6 0 2024-02-16T19:24:31.847 INFO:tasks.workunit.client.1.smithi163.stdout:7/673: unlink d11/d24/d25/f62 0 2024-02-16T19:24:31.848 INFO:tasks.workunit.client.1.smithi163.stdout:0/939: dread d11/d21/fdb [0,4194304] 0 2024-02-16T19:24:31.849 INFO:tasks.workunit.client.1.smithi163.stdout:5/787: truncate d0/d1c/d5a/f96 1588990 0 2024-02-16T19:24:31.849 INFO:tasks.workunit.client.1.smithi163.stdout:5/788: stat d0/d22/lf1 0 2024-02-16T19:24:31.849 INFO:tasks.workunit.client.1.smithi163.stdout:3/681: dread d0/dc/d1d/d29/d38/d46/f90 [0,4194304] 0 2024-02-16T19:24:31.850 INFO:tasks.workunit.client.1.smithi163.stdout:8/668: dwrite d0/d24/d2c/d74/fd7 [0,4194304] 0 2024-02-16T19:24:31.850 INFO:tasks.workunit.client.1.smithi163.stdout:8/669: write d0/d7/d18/d53/fc5 [1861082,359] 0 2024-02-16T19:24:31.850 INFO:tasks.workunit.client.1.smithi163.stdout:6/624: creat d0/d18/d77/d97/db8/fc7 x:0 0 0 2024-02-16T19:24:31.850 INFO:tasks.workunit.client.1.smithi163.stdout:6/625: fdatasync d0/dd/d7f/d15/d1c/f7d 0 2024-02-16T19:24:31.851 INFO:tasks.workunit.client.1.smithi163.stdout:9/937: rename d5/de/d16/f53 to d5/de/f143 0 2024-02-16T19:24:31.852 INFO:tasks.workunit.client.1.smithi163.stdout:3/682: mknod d0/d83/da5/db9/cf4 0 2024-02-16T19:24:31.852 INFO:tasks.workunit.client.1.smithi163.stdout:4/639: dwrite d11/d18/d1f/d29/d52/d57/d6c/d7c/f96 [0,4194304] 0 2024-02-16T19:24:31.854 INFO:tasks.workunit.client.1.smithi163.stdout:8/670: getdents d0/d7/d18/d53 0 2024-02-16T19:24:31.856 INFO:tasks.workunit.client.1.smithi163.stdout:6/626: creat d0/dd/d7f/d81/fc8 x:0 0 0 2024-02-16T19:24:31.856 INFO:tasks.workunit.client.1.smithi163.stdout:6/627: fdatasync d0/d6/df/d20/d69/fa6 0 2024-02-16T19:24:31.858 INFO:tasks.workunit.client.1.smithi163.stdout:2/824: rename d2/l8 to d2/d2a/d9f/df8/l11a 0 2024-02-16T19:24:31.859 INFO:tasks.workunit.client.1.smithi163.stdout:9/938: symlink d5/de/d30/d3d/d49/dea/d11d/l144 0 2024-02-16T19:24:31.859 INFO:tasks.workunit.client.1.smithi163.stdout:9/939: write d5/dc0/ffa [50577,22565] 0 2024-02-16T19:24:31.859 INFO:tasks.workunit.client.1.smithi163.stdout:3/683: mknod d0/dc/d1d/d49/d98/cf5 0 2024-02-16T19:24:31.860 INFO:tasks.workunit.client.1.smithi163.stdout:3/684: chown d0/dc/f24 5520 1 2024-02-16T19:24:31.861 INFO:tasks.workunit.client.1.smithi163.stdout:0/940: dread d11/d12/d20/f5c [0,4194304] 0 2024-02-16T19:24:31.862 INFO:tasks.workunit.client.1.smithi163.stdout:5/789: dread d0/d2/d34/d44/d53/ff0 [0,4194304] 0 2024-02-16T19:24:31.862 INFO:tasks.workunit.client.1.smithi163.stdout:5/790: getdents d0/d2/db1/d106 0 2024-02-16T19:24:31.862 INFO:tasks.workunit.client.1.smithi163.stdout:8/671: dread d0/d24/d2c/f3b [0,4194304] 0 2024-02-16T19:24:31.862 INFO:tasks.workunit.client.1.smithi163.stdout:8/672: fdatasync d0/d7/d19/d1f/d30/d6a/d7b/fac 0 2024-02-16T19:24:31.866 INFO:tasks.workunit.client.1.smithi163.stdout:2/825: dread d2/f5 [0,4194304] 0 2024-02-16T19:24:31.872 INFO:tasks.workunit.client.1.smithi163.stdout:9/940: dread d5/de/d16/f98 [0,4194304] 0 2024-02-16T19:24:31.873 INFO:tasks.workunit.client.1.smithi163.stdout:5/791: read d0/d2/d34/d44/d53/f87 [231052,125519] 0 2024-02-16T19:24:31.875 INFO:tasks.workunit.client.1.smithi163.stdout:7/674: rename d11/d50/cb7 to d11/d50/d5d/d8b/dc8/ce3 0 2024-02-16T19:24:31.876 INFO:tasks.workunit.client.1.smithi163.stdout:3/685: mknod d0/d83/da5/cf6 0 2024-02-16T19:24:31.877 INFO:tasks.workunit.client.1.smithi163.stdout:0/941: mknod d11/d12/d67/d6a/dae/c13c 0 2024-02-16T19:24:31.878 INFO:tasks.workunit.client.1.smithi163.stdout:8/673: truncate d0/d7/d18/d53/f9d 2576214 0 2024-02-16T19:24:31.879 INFO:tasks.workunit.client.1.smithi163.stdout:9/941: symlink d5/de/d16/d46/d9b/d106/l145 0 2024-02-16T19:24:31.882 INFO:tasks.workunit.client.1.smithi163.stdout:5/792: creat d0/d2/d34/d44/d53/de2/f10b x:0 0 0 2024-02-16T19:24:31.883 INFO:tasks.workunit.client.1.smithi163.stdout:2/826: dwrite d2/f37 [0,4194304] 0 2024-02-16T19:24:31.886 INFO:tasks.workunit.client.1.smithi163.stdout:7/675: mknod d11/d50/d5d/dcd/ce4 0 2024-02-16T19:24:31.888 INFO:tasks.workunit.client.1.smithi163.stdout:0/942: unlink d11/d12/d20/fe4 0 2024-02-16T19:24:31.888 INFO:tasks.workunit.client.1.smithi163.stdout:0/943: truncate d11/d12/d67/fa6 1585262 0 2024-02-16T19:24:31.890 INFO:tasks.workunit.client.1.smithi163.stdout:9/942: symlink d5/de/d30/d3d/d49/dea/d11d/l146 0 2024-02-16T19:24:31.892 INFO:tasks.workunit.client.1.smithi163.stdout:5/793: getdents d0/d2/d5c 0 2024-02-16T19:24:31.895 INFO:tasks.workunit.client.1.smithi163.stdout:7/676: truncate d11/d24/d25/f5a 943762 0 2024-02-16T19:24:31.897 INFO:tasks.workunit.client.1.smithi163.stdout:9/943: mknod d5/de/d72/c147 0 2024-02-16T19:24:31.898 INFO:tasks.workunit.client.1.smithi163.stdout:5/794: getdents d0/d2/d34/d44/d53/d93 0 2024-02-16T19:24:31.898 INFO:tasks.workunit.client.1.smithi163.stdout:5/795: fsync d0/d2/d55/f91 0 2024-02-16T19:24:31.899 INFO:tasks.workunit.client.1.smithi163.stdout:2/827: mknod d2/d26/c11b 0 2024-02-16T19:24:31.899 INFO:tasks.workunit.client.1.smithi163.stdout:2/828: chown d2/d2a/d39/fae 630404636 1 2024-02-16T19:24:31.899 INFO:tasks.workunit.client.1.smithi163.stdout:2/829: write d2/d26/d54/fd5 [900475,53032] 0 2024-02-16T19:24:31.899 INFO:tasks.workunit.client.1.smithi163.stdout:2/830: dread - d2/d2a/d2d/d73/d94/dc9/f103 zero size 2024-02-16T19:24:31.900 INFO:tasks.workunit.client.1.smithi163.stdout:3/686: dwrite d0/dc/d1c/d32/d7f/fbd [0,4194304] 0 2024-02-16T19:24:31.900 INFO:tasks.workunit.client.1.smithi163.stdout:6/628: rename d0/d6/df/d20/d40/d53 to d0/d18/d77/dc9 0 2024-02-16T19:24:31.901 INFO:tasks.workunit.client.1.smithi163.stdout:2/831: getdents d2/d2a/d39/d117 0 2024-02-16T19:24:31.904 INFO:tasks.workunit.client.1.smithi163.stdout:7/677: rename d11/d26/c5f to d11/d24/d49/dc4/ce5 0 2024-02-16T19:24:31.904 INFO:tasks.workunit.client.1.smithi163.stdout:6/629: mkdir d0/dd/d7f/d15/d1c/dca 0 2024-02-16T19:24:31.905 INFO:tasks.workunit.client.1.smithi163.stdout:2/832: link d2/d2a/d39/d3a/fe0 d2/d26/d54/d58/d98/f11c 0 2024-02-16T19:24:31.905 INFO:tasks.workunit.client.1.smithi163.stdout:2/833: truncate d2/d26/d54/dfb/d114/fe3 96902 0 2024-02-16T19:24:31.906 INFO:tasks.workunit.client.1.smithi163.stdout:3/687: getdents d0/d83 0 2024-02-16T19:24:31.906 INFO:tasks.workunit.client.1.smithi163.stdout:3/688: fdatasync d0/dc/d1d/d49/d98/fe9 0 2024-02-16T19:24:31.907 INFO:tasks.workunit.client.1.smithi163.stdout:8/674: dwrite d0/d7/d18/d53/fc5 [0,4194304] 0 2024-02-16T19:24:31.908 INFO:tasks.workunit.client.1.smithi163.stdout:9/944: rename d5/f1a to d5/de/d30/d3d/d49/df3/f148 0 2024-02-16T19:24:31.910 INFO:tasks.workunit.client.1.smithi163.stdout:5/796: write d0/d2/d34/d44/d53/d93/fa0 [3464219,93838] 0 2024-02-16T19:24:31.910 INFO:tasks.workunit.client.1.smithi163.stdout:5/797: write d0/d2/d34/d44/d64/dd9/fdc [397009,111382] 0 2024-02-16T19:24:31.910 INFO:tasks.workunit.client.1.smithi163.stdout:7/678: creat d11/d24/d49/dc4/fe6 x:0 0 0 2024-02-16T19:24:31.911 INFO:tasks.workunit.client.1.smithi163.stdout:2/834: creat d2/d26/f11d x:0 0 0 2024-02-16T19:24:31.912 INFO:tasks.workunit.client.1.smithi163.stdout:2/835: dread - d2/d26/d54/d58/d98/feb zero size 2024-02-16T19:24:31.913 INFO:tasks.workunit.client.1.smithi163.stdout:6/630: read d0/d6/df/d20/d40/d17/f88 [3885798,130625] 0 2024-02-16T19:24:31.913 INFO:tasks.workunit.client.1.smithi163.stdout:3/689: symlink d0/d3e/dea/lf7 0 2024-02-16T19:24:31.916 INFO:tasks.workunit.client.1.smithi163.stdout:8/675: mkdir d0/ddc 0 2024-02-16T19:24:31.919 INFO:tasks.workunit.client.1.smithi163.stdout:7/679: chown d11/d50/d5d/d8b/d67/db2/lb4 48377336 1 2024-02-16T19:24:31.921 INFO:tasks.workunit.client.1.smithi163.stdout:6/631: mkdir d0/d6/df/d20/d40/d17/dcb 0 2024-02-16T19:24:31.921 INFO:tasks.workunit.client.1.smithi163.stdout:6/632: readlink d0/dd/d7f/d81/lb1 0 2024-02-16T19:24:31.921 INFO:tasks.workunit.client.1.smithi163.stdout:6/633: chown d0/dd/d7f/d15/d1c/d65/d63/d86/c9d 119447 1 2024-02-16T19:24:31.922 INFO:tasks.workunit.client.1.smithi163.stdout:3/690: symlink d0/dc/d96/ddc/lf8 0 2024-02-16T19:24:31.925 INFO:tasks.workunit.client.1.smithi163.stdout:9/945: rename d5/de/d16/lb1 to d5/de/d30/d3d/d49/dea/l149 0 2024-02-16T19:24:31.926 INFO:tasks.workunit.client.1.smithi163.stdout:6/634: mknod d0/dd/d7f/d15/d1c/d65/d63/d86/ccc 0 2024-02-16T19:24:31.929 INFO:tasks.workunit.client.1.smithi163.stdout:9/946: truncate d5/de/d16/d46/f9c 3865172 0 2024-02-16T19:24:31.933 INFO:tasks.workunit.client.1.smithi163.stdout:5/798: rename d0/d2/d34/d44/d64/l77 to d0/d22/d3b/dc0/dfa/l10c 0 2024-02-16T19:24:31.934 INFO:tasks.workunit.client.1.smithi163.stdout:6/635: mkdir d0/d6/df/d8f/dcd 0 2024-02-16T19:24:31.936 INFO:tasks.workunit.client.1.smithi163.stdout:9/947: mknod d5/de/d30/d3d/d49/dea/d11d/c14a 0 2024-02-16T19:24:31.937 INFO:tasks.workunit.client.1.smithi163.stdout:6/636: unlink d0/dd/d7f/d15/c19 0 2024-02-16T19:24:31.945 INFO:tasks.workunit.client.1.smithi163.stdout:2/836: dwrite d2/d2a/f5f [0,4194304] 0 2024-02-16T19:24:31.948 INFO:tasks.workunit.client.1.smithi163.stdout:2/837: rename d2/d26/d54/d58/d8c/dc5/f108 to d2/dce/dee/f11e 0 2024-02-16T19:24:31.949 INFO:tasks.workunit.client.1.smithi163.stdout:2/838: symlink d2/d2a/d9f/df8/l11f 0 2024-02-16T19:24:31.949 INFO:tasks.workunit.client.1.smithi163.stdout:2/839: fdatasync d2/f9d 0 2024-02-16T19:24:31.949 INFO:tasks.workunit.client.1.smithi163.stdout:2/840: symlink d2/d26/d54/d58/d7d/daf/d101/l120 0 2024-02-16T19:24:31.950 INFO:tasks.workunit.client.1.smithi163.stdout:9/948: dread d5/f77 [0,4194304] 0 2024-02-16T19:24:31.951 INFO:tasks.workunit.client.1.smithi163.stdout:9/949: fdatasync d5/de/d30/d3d/d49/df3/f148 0 2024-02-16T19:24:31.951 INFO:tasks.workunit.client.1.smithi163.stdout:9/950: mkdir d5/de/d30/d3d/d49/d6e/d14b 0 2024-02-16T19:24:31.952 INFO:tasks.workunit.client.1.smithi163.stdout:9/951: write d5/de/d72/f7a [607048,113888] 0 2024-02-16T19:24:31.952 INFO:tasks.workunit.client.1.smithi163.stdout:9/952: fsync d5/de/d16/d47/f87 0 2024-02-16T19:24:31.952 INFO:tasks.workunit.client.1.smithi163.stdout:9/953: mkdir d5/de/d16/d46/d9b/d106/d14c 0 2024-02-16T19:24:31.953 INFO:tasks.workunit.client.1.smithi163.stdout:9/954: truncate d5/de/d30/d31/f69 1631798 0 2024-02-16T19:24:31.953 INFO:tasks.workunit.client.1.smithi163.stdout:9/955: mknod d5/de/d30/d31/c14d 0 2024-02-16T19:24:31.954 INFO:tasks.workunit.client.1.smithi163.stdout:7/680: dwrite d11/d24/d25/d4b/d66/dc1/fd7 [0,4194304] 0 2024-02-16T19:24:31.955 INFO:tasks.workunit.client.1.smithi163.stdout:8/676: dwrite d0/d7/d19/d1f/d32/f39 [0,4194304] 0 2024-02-16T19:24:31.963 INFO:tasks.workunit.client.1.smithi163.stdout:2/841: write d2/fe5 [1092547,76522] 0 2024-02-16T19:24:31.964 INFO:tasks.workunit.client.1.smithi163.stdout:3/691: dwrite d0/dc/d1c/d32/ddb/fdf [0,4194304] 0 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:3/692: readlink d0/dc/d50/d66/d93/lc5 0 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:3/693: chown d0/dc/d1d/d29/d38/fe5 2873 1 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:3/694: write d0/dc/d94/d9e/fb6 [1108301,115630] 0 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:3/695: fdatasync d0/d83/da5/db9/fd1 0 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:3/696: readlink d0/dc/d1d/d49/l67 0 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:3/697: creat d0/dc/d1c/d91/ff9 x:0 0 0 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:3/698: creat d0/dc/d94/ffa x:0 0 0 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:2/842: creat d2/d44/f121 x:0 0 0 2024-02-16T19:24:31.965 INFO:tasks.workunit.client.1.smithi163.stdout:6/637: dwrite d0/dd/d7f/d81/fc8 [0,4194304] 0 2024-02-16T19:24:31.970 INFO:tasks.workunit.client.1.smithi163.stdout:7/681: write d11/d24/d49/f69 [3111307,4127] 0 2024-02-16T19:24:31.973 INFO:tasks.workunit.client.1.smithi163.stdout:9/956: dwrite d5/de/f5f [0,4194304] 0 2024-02-16T19:24:31.974 INFO:tasks.workunit.client.1.smithi163.stdout:8/677: write d0/d7/d19/d1f/d30/d48/fa7 [205313,61271] 0 2024-02-16T19:24:31.974 INFO:tasks.workunit.client.1.smithi163.stdout:2/843: write d2/ddb/d7c/f99 [4112568,116454] 0 2024-02-16T19:24:31.984 INFO:tasks.workunit.client.1.smithi163.stdout:7/682: write d11/d24/d34/d8d/f92 [2191822,45720] 0 2024-02-16T19:24:31.986 INFO:tasks.workunit.client.1.smithi163.stdout:3/699: rename d0/dc/l7e to d0/d83/lfb 0 2024-02-16T19:24:31.987 INFO:tasks.workunit.client.1.smithi163.stdout:7/683: mkdir d11/d24/d25/d4b/d66/dc1/de7 0 2024-02-16T19:24:31.988 INFO:tasks.workunit.client.1.smithi163.stdout:3/700: symlink d0/dc/d94/lfc 0 2024-02-16T19:24:31.989 INFO:tasks.workunit.client.1.smithi163.stdout:3/701: mknod d0/d3e/cfd 0 2024-02-16T19:24:31.990 INFO:tasks.workunit.client.1.smithi163.stdout:3/702: rmdir d0/dc/d94/d9e/dac 39 2024-02-16T19:24:31.992 INFO:tasks.workunit.client.1.smithi163.stdout:3/703: dread d0/dc/d50/d66/da3/d71/f89 [0,4194304] 0 2024-02-16T19:24:31.997 INFO:tasks.workunit.client.1.smithi163.stdout:6/638: dwrite d0/dd/d7f/d15/d1c/d65/d63/fbe [8388608,4194304] 0 2024-02-16T19:24:31.997 INFO:tasks.workunit.client.1.smithi163.stdout:6/639: fsync d0/d6/df/d20/f90 0 2024-02-16T19:24:31.998 INFO:tasks.workunit.client.1.smithi163.stdout:6/640: fsync d0/dd/d7f/d15/d1c/f56 0 2024-02-16T19:24:32.002 INFO:tasks.workunit.client.1.smithi163.stdout:6/641: link d0/f5 d0/dd/d7f/d81/d85/fce 0 2024-02-16T19:24:32.002 INFO:tasks.workunit.client.1.smithi163.stdout:6/642: write d0/dd/d7f/d15/d1c/f2b [2683159,15548] 0 2024-02-16T19:24:32.008 INFO:tasks.workunit.client.1.smithi163.stdout:3/704: read d0/dc/d50/d66/da3/f86 [582978,16577] 0 2024-02-16T19:24:32.009 INFO:tasks.workunit.client.1.smithi163.stdout:3/705: chown d0/dc/d1c/d32/db8/d99/l9b 46138477 1 2024-02-16T19:24:32.009 INFO:tasks.workunit.client.1.smithi163.stdout:3/706: creat d0/dc/d94/ffe x:0 0 0 2024-02-16T19:24:32.009 INFO:tasks.workunit.client.1.smithi163.stdout:3/707: chown d0/dc/d50/d66/lb4 112 1 2024-02-16T19:24:32.009 INFO:tasks.workunit.client.1.smithi163.stdout:3/708: mkdir d0/dc/d96/dff 0 2024-02-16T19:24:32.011 INFO:tasks.workunit.client.1.smithi163.stdout:3/709: truncate d0/dc/d1d/d29/f30 2230786 0 2024-02-16T19:24:32.013 INFO:tasks.workunit.client.1.smithi163.stdout:8/678: dwrite d0/faa [0,4194304] 0 2024-02-16T19:24:32.016 INFO:tasks.workunit.client.1.smithi163.stdout:7/684: dwrite d11/f1f [0,4194304] 0 2024-02-16T19:24:32.018 INFO:tasks.workunit.client.1.smithi163.stdout:8/679: rmdir d0/ddc 0 2024-02-16T19:24:32.018 INFO:tasks.workunit.client.1.smithi163.stdout:9/957: dwrite d5/dc0/fc5 [0,4194304] 0 2024-02-16T19:24:32.021 INFO:tasks.workunit.client.1.smithi163.stdout:8/680: dread d0/d24/d3c/f8a [0,4194304] 0 2024-02-16T19:24:32.027 INFO:tasks.workunit.client.1.smithi163.stdout:9/958: dread d5/de/d16/f6b [0,4194304] 0 2024-02-16T19:24:32.030 INFO:tasks.workunit.client.1.smithi163.stdout:9/959: link d5/de/d30/f7b d5/de/d30/d8a/d10a/f14e 0 2024-02-16T19:24:32.030 INFO:tasks.workunit.client.1.smithi163.stdout:9/960: fdatasync d5/de/d16/d46/d9b/d106/f132 0 2024-02-16T19:24:32.036 INFO:tasks.workunit.client.1.smithi163.stdout:8/681: write d0/d24/d2c/d74/fd7 [625278,39569] 0 2024-02-16T19:24:32.036 INFO:tasks.workunit.client.1.smithi163.stdout:8/682: stat d0/d7/d18/d53/f9d 0 2024-02-16T19:24:32.036 INFO:tasks.workunit.client.1.smithi163.stdout:8/683: write d0/d24/d3c/f8a [1527615,10352] 0 2024-02-16T19:24:32.037 INFO:tasks.workunit.client.1.smithi163.stdout:8/684: rmdir d0/d7/d18 39 2024-02-16T19:24:32.038 INFO:tasks.workunit.client.1.smithi163.stdout:8/685: mkdir d0/d7/d18/d2e/dd0/ddd 0 2024-02-16T19:24:32.039 INFO:tasks.workunit.client.1.smithi163.stdout:8/686: truncate d0/d24/d27/fa4 2994229 0 2024-02-16T19:24:32.039 INFO:tasks.workunit.client.1.smithi163.stdout:8/687: write d0/d24/d3c/f4c [632543,40523] 0 2024-02-16T19:24:32.041 INFO:tasks.workunit.client.1.smithi163.stdout:8/688: creat d0/d24/d3c/d3d/fde x:0 0 0 2024-02-16T19:24:32.041 INFO:tasks.workunit.client.1.smithi163.stdout:8/689: mkdir d0/d24/d2c/ddf 0 2024-02-16T19:24:32.043 INFO:tasks.workunit.client.1.smithi163.stdout:8/690: link d0/d24/d3c/f84 d0/d7/d19/d1f/d30/d6a/fe0 0 2024-02-16T19:24:32.052 INFO:tasks.workunit.client.1.smithi163.stdout:6/643: dwrite d0/d6/df/d20/f90 [0,4194304] 0 2024-02-16T19:24:32.052 INFO:tasks.workunit.client.1.smithi163.stdout:6/644: chown d0/d6/df/d20/f90 26023235 1 2024-02-16T19:24:32.052 INFO:tasks.workunit.client.1.smithi163.stdout:7/685: dwrite d11/d30/f32 [4194304,4194304] 0 2024-02-16T19:24:32.053 INFO:tasks.workunit.client.1.smithi163.stdout:3/710: dwrite d0/f8a [0,4194304] 0 2024-02-16T19:24:32.053 INFO:tasks.workunit.client.1.smithi163.stdout:3/711: dread - d0/dc/d50/d66/fe6 zero size 2024-02-16T19:24:32.053 INFO:tasks.workunit.client.1.smithi163.stdout:9/961: dwrite d5/dc0/fcc [0,4194304] 0 2024-02-16T19:24:32.056 INFO:tasks.workunit.client.1.smithi163.stdout:6/645: rename d0/f5 to d0/d6/df/d20/d69/fcf 0 2024-02-16T19:24:32.057 INFO:tasks.workunit.client.1.smithi163.stdout:7/686: rmdir d11/d50/d5d/d8b/d67/dd6/da6 39 2024-02-16T19:24:32.060 INFO:tasks.workunit.client.1.smithi163.stdout:8/691: dwrite d0/d7/d18/d2e/f5d [0,4194304] 0 2024-02-16T19:24:32.060 INFO:tasks.workunit.client.1.smithi163.stdout:8/692: write d0/d7/d19/d1f/d30/fb5 [197584,47886] 0 2024-02-16T19:24:32.066 INFO:tasks.workunit.client.1.smithi163.stdout:9/962: creat d5/de/d30/d31/f14f x:0 0 0 2024-02-16T19:24:32.070 INFO:tasks.workunit.client.1.smithi163.stdout:6/646: truncate d0/dd/d7f/d15/d1c/f56 376094 0 2024-02-16T19:24:32.072 INFO:tasks.workunit.client.1.smithi163.stdout:8/693: mknod d0/d24/d3c/d3d/d41/ce1 0 2024-02-16T19:24:32.073 INFO:tasks.workunit.client.1.smithi163.stdout:3/712: getdents d0/dc/d1c/d32/df0 0 2024-02-16T19:24:32.074 INFO:tasks.workunit.client.1.smithi163.stdout:9/963: rmdir d5/de/d16/d133 39 2024-02-16T19:24:32.075 INFO:tasks.workunit.client.1.smithi163.stdout:7/687: dwrite d11/d26/fbb [4194304,4194304] 0 2024-02-16T19:24:32.075 INFO:tasks.workunit.client.1.smithi163.stdout:8/694: mkdir d0/d24/d78/d85/de2 0 2024-02-16T19:24:32.075 INFO:tasks.workunit.client.1.smithi163.stdout:8/695: fsync d0/dc/f7d 0 2024-02-16T19:24:32.075 INFO:tasks.workunit.client.1.smithi163.stdout:8/696: readlink d0/lda 0 2024-02-16T19:24:32.078 INFO:tasks.workunit.client.1.smithi163.stdout:7/688: rename d11/d24/d25/d4b/d66/lc2 to d11/d24/d34/d8d/dd9/le8 0 2024-02-16T19:24:32.078 INFO:tasks.workunit.client.1.smithi163.stdout:7/689: read - d11/d24/d25/d4b/fdc zero size 2024-02-16T19:24:32.079 INFO:tasks.workunit.client.1.smithi163.stdout:7/690: creat d11/d24/d49/dac/fe9 x:0 0 0 2024-02-16T19:24:32.079 INFO:tasks.workunit.client.1.smithi163.stdout:8/697: mkdir d0/dc/de3 0 2024-02-16T19:24:32.080 INFO:tasks.workunit.client.1.smithi163.stdout:9/964: symlink d5/de/d30/d3d/l150 0 2024-02-16T19:24:32.084 INFO:tasks.workunit.client.1.smithi163.stdout:7/691: creat d11/d30/dda/fea x:0 0 0 2024-02-16T19:24:32.084 INFO:tasks.workunit.client.1.smithi163.stdout:7/692: truncate d11/d24/d49/fb3 951306 0 2024-02-16T19:24:32.084 INFO:tasks.workunit.client.1.smithi163.stdout:7/693: creat d11/d24/d49/dac/feb x:0 0 0 2024-02-16T19:24:32.084 INFO:tasks.workunit.client.1.smithi163.stdout:7/694: fsync d11/d24/d25/f9a 0 2024-02-16T19:24:32.085 INFO:tasks.workunit.client.1.smithi163.stdout:8/698: creat d0/d7/d19/fe4 x:0 0 0 2024-02-16T19:24:32.086 INFO:tasks.workunit.client.1.smithi163.stdout:9/965: creat d5/dc0/d102/f151 x:0 0 0 2024-02-16T19:24:32.086 INFO:tasks.workunit.client.1.smithi163.stdout:9/966: chown d5/de/d30/d3d/d93/cc1 256 1 2024-02-16T19:24:32.089 INFO:tasks.workunit.client.1.smithi163.stdout:7/695: mknod d11/d24/d25/d4b/d66/dc1/de7/cec 0 2024-02-16T19:24:32.089 INFO:tasks.workunit.client.1.smithi163.stdout:7/696: dread - d11/d30/dda/fea zero size 2024-02-16T19:24:32.090 INFO:tasks.workunit.client.1.smithi163.stdout:9/967: mknod d5/de/d30/d8a/d10a/c152 0 2024-02-16T19:24:32.090 INFO:tasks.workunit.client.1.smithi163.stdout:9/968: truncate d5/de/d30/d3d/d49/d6d/d97/fe1 1739363 0 2024-02-16T19:24:32.090 INFO:tasks.workunit.client.1.smithi163.stdout:9/969: readlink d5/de/d30/d3d/d49/dea/lfd 0 2024-02-16T19:24:32.098 INFO:tasks.workunit.client.1.smithi163.stdout:6/647: dwrite d0/dd/d7f/f73 [0,4194304] 0 2024-02-16T19:24:32.103 INFO:tasks.workunit.client.1.smithi163.stdout:7/697: read fc [1580177,103645] 0 2024-02-16T19:24:32.103 INFO:tasks.workunit.client.1.smithi163.stdout:7/698: write d11/d50/d5d/d8b/f3c [4886401,4475] 0 2024-02-16T19:24:32.105 INFO:tasks.workunit.client.1.smithi163.stdout:7/699: mknod d11/d24/d25/d4b/d66/ced 0 2024-02-16T19:24:32.109 INFO:tasks.workunit.client.1.smithi163.stdout:7/700: mkdir d11/d24/d25/d4b/d8c/dee 0 2024-02-16T19:24:32.125 INFO:tasks.workunit.client.1.smithi163.stdout:8/699: dwrite d0/d7/d18/fad [0,4194304] 0 2024-02-16T19:24:32.125 INFO:tasks.workunit.client.1.smithi163.stdout:8/700: write d0/d7/d19/f26 [4517151,92420] 0 2024-02-16T19:24:32.127 INFO:tasks.workunit.client.1.smithi163.stdout:8/701: mkdir d0/d24/d2c/ddf/de5 0 2024-02-16T19:24:32.129 INFO:tasks.workunit.client.1.smithi163.stdout:9/970: dwrite d5/de/d30/f13a [0,4194304] 0 2024-02-16T19:24:32.131 INFO:tasks.workunit.client.1.smithi163.stdout:6/648: dwrite d0/d18/d77/dc9/f80 [0,4194304] 0 2024-02-16T19:24:32.133 INFO:tasks.workunit.client.1.smithi163.stdout:9/971: rename d5/de/d30/d3d/d49/d6e/l78 to d5/de/d30/d3d/d49/dea/l153 0 2024-02-16T19:24:32.133 INFO:tasks.workunit.client.1.smithi163.stdout:7/701: dwrite d11/d24/d25/d4b/d66/fb9 [0,4194304] 0 2024-02-16T19:24:32.137 INFO:tasks.workunit.client.1.smithi163.stdout:9/972: dread d5/de/d16/d46/d9b/fc6 [0,4194304] 0 2024-02-16T19:24:32.161 INFO:tasks.workunit.client.1.smithi163.stdout:8/702: dwrite d0/d7/d19/d1f/d32/d70/f8b [0,4194304] 0 2024-02-16T19:24:32.163 INFO:tasks.workunit.client.1.smithi163.stdout:8/703: rename d0/d24/d2c/d74/f79 to d0/d24/d78/fe6 0 2024-02-16T19:24:32.164 INFO:tasks.workunit.client.1.smithi163.stdout:8/704: fdatasync d0/d24/d27/fa4 0 2024-02-16T19:24:32.164 INFO:tasks.workunit.client.1.smithi163.stdout:6/649: dwrite d0/d18/d77/fbf [0,4194304] 0 2024-02-16T19:24:32.164 INFO:tasks.workunit.client.1.smithi163.stdout:9/973: dwrite d5/de/d16/d46/d9b/de4/ff0 [0,4194304] 0 2024-02-16T19:24:32.164 INFO:tasks.workunit.client.1.smithi163.stdout:9/974: chown d5/de/d30/d3d/d49/d6e/ffe 866687554 1 2024-02-16T19:24:32.168 INFO:tasks.workunit.client.1.smithi163.stdout:9/975: creat d5/de/d72/da9/d104/f154 x:0 0 0 2024-02-16T19:24:32.169 INFO:tasks.workunit.client.1.smithi163.stdout:8/705: mknod d0/d7/d19/ce7 0 2024-02-16T19:24:32.169 INFO:tasks.workunit.client.1.smithi163.stdout:8/706: chown d0/d7/d18/fad 407882249 1 2024-02-16T19:24:32.173 INFO:tasks.workunit.client.1.smithi163.stdout:9/976: rename d5/de/d16/d46/d9b/fee to d5/de/d84/f155 0 2024-02-16T19:24:32.173 INFO:tasks.workunit.client.1.smithi163.stdout:9/977: write d5/de/d30/d3d/d49/df3/ffc [1058378,113539] 0 2024-02-16T19:24:32.174 INFO:tasks.workunit.client.1.smithi163.stdout:8/707: mkdir d0/d6f/de8 0 2024-02-16T19:24:32.174 INFO:tasks.workunit.client.1.smithi163.stdout:8/708: chown d0/dc/f2d 90615 1 2024-02-16T19:24:32.176 INFO:tasks.workunit.client.1.smithi163.stdout:9/978: rename d5/de/d30/d3d/d93/cd6 to d5/dc0/d102/c156 0 2024-02-16T19:24:32.176 INFO:tasks.workunit.client.1.smithi163.stdout:9/979: truncate d5/de/d30/d3d/d49/f137 270154 0 2024-02-16T19:24:32.177 INFO:tasks.workunit.client.1.smithi163.stdout:8/709: dread d0/d7/d18/d2e/f8f [0,4194304] 0 2024-02-16T19:24:32.177 INFO:tasks.workunit.client.1.smithi163.stdout:9/980: getdents d5/de/d30/d31/d52 0 2024-02-16T19:24:32.190 INFO:tasks.workunit.client.1.smithi163.stdout:1/723: sync 2024-02-16T19:24:32.190 INFO:tasks.workunit.client.1.smithi163.stdout:1/724: read - d7/dc/d2d/d3f/fc8 zero size 2024-02-16T19:24:32.191 INFO:tasks.workunit.client.1.smithi163.stdout:9/981: dread d5/de/d16/d46/f8c [0,4194304] 0 2024-02-16T19:24:32.191 INFO:tasks.workunit.client.1.smithi163.stdout:9/982: readlink d5/l10 0 2024-02-16T19:24:32.191 INFO:tasks.workunit.client.1.smithi163.stdout:9/983: truncate d5/de/d30/d3d/d49/d6e/fb9 1166637 0 2024-02-16T19:24:32.191 INFO:tasks.workunit.client.1.smithi163.stdout:9/984: readlink d5/de/d30/d3d/d49/d6d/lb7 0 2024-02-16T19:24:32.191 INFO:tasks.workunit.client.1.smithi163.stdout:9/985: stat d5/de/d30/d3d/d49/d50 0 2024-02-16T19:24:32.192 INFO:tasks.workunit.client.1.smithi163.stdout:6/650: dwrite d0/dd/d7f/f4c [0,4194304] 0 2024-02-16T19:24:32.192 INFO:tasks.workunit.client.1.smithi163.stdout:6/651: stat d0/dd/d7f/d15/d1c/f2b 0 2024-02-16T19:24:32.193 INFO:tasks.workunit.client.1.smithi163.stdout:9/986: rmdir d5/de/d16/d46/d9b/d106/d14c 0 2024-02-16T19:24:32.193 INFO:tasks.workunit.client.1.smithi163.stdout:9/987: dread - d5/dc0/d102/f151 zero size 2024-02-16T19:24:32.196 INFO:tasks.workunit.client.1.smithi163.stdout:6/652: mkdir d0/d6/df/d7b/d87/dd0 0 2024-02-16T19:24:32.198 INFO:tasks.workunit.client.1.smithi163.stdout:9/988: rmdir d5/de/d30/d3d/d49/d6e/d14b 0 2024-02-16T19:24:32.198 INFO:tasks.workunit.client.1.smithi163.stdout:9/989: fsync d5/de/d30/d3d/d49/d50/df9/f111 0 2024-02-16T19:24:32.199 INFO:tasks.workunit.client.1.smithi163.stdout:6/653: unlink d0/d18/lc2 0 2024-02-16T19:24:32.260 INFO:tasks.workunit.client.1.smithi163.stdout:1/725: dwrite d7/d5b/d63/f7b [4194304,4194304] 0 2024-02-16T19:24:32.261 INFO:tasks.workunit.client.1.smithi163.stdout:8/710: dwrite d0/d24/d3c/f8a [0,4194304] 0 2024-02-16T19:24:32.261 INFO:tasks.workunit.client.1.smithi163.stdout:8/711: truncate d0/d7/d19/d1f/d30/fb2 567399 0 2024-02-16T19:24:32.263 INFO:tasks.workunit.client.1.smithi163.stdout:1/726: link d7/dc/d2d/d3f/lb2 d7/d20/d96/lf6 0 2024-02-16T19:24:32.265 INFO:tasks.workunit.client.1.smithi163.stdout:8/712: creat d0/d7/d19/d1f/d32/fe9 x:0 0 0 2024-02-16T19:24:32.266 INFO:tasks.workunit.client.1.smithi163.stdout:1/727: mkdir d7/d20/df7 0 2024-02-16T19:24:32.266 INFO:tasks.workunit.client.1.smithi163.stdout:1/728: chown d7/d5b/l5e 40 1 2024-02-16T19:24:32.266 INFO:tasks.workunit.client.1.smithi163.stdout:1/729: dread - d7/d20/d21/d4e/feb zero size 2024-02-16T19:24:32.267 INFO:tasks.workunit.client.1.smithi163.stdout:9/990: dwrite d5/de/d16/f6b [0,4194304] 0 2024-02-16T19:24:32.267 INFO:tasks.workunit.client.1.smithi163.stdout:8/713: link d0/d7/d19/d1f/d32/fe9 d0/fea 0 2024-02-16T19:24:32.267 INFO:tasks.workunit.client.1.smithi163.stdout:8/714: chown d0/d7/d19/d1f/d32/d70 3216 1 2024-02-16T19:24:32.267 INFO:tasks.workunit.client.1.smithi163.stdout:9/991: write d5/de/d30/d43/f4b [721219,120993] 0 2024-02-16T19:24:32.267 INFO:tasks.workunit.client.1.smithi163.stdout:9/992: chown d5/de/d84/ce3 971 1 2024-02-16T19:24:32.268 INFO:tasks.workunit.client.1.smithi163.stdout:9/993: fsync d5/de/d30/d31/f69 0 2024-02-16T19:24:32.268 INFO:tasks.workunit.client.1.smithi163.stdout:9/994: chown d5/de/d30/d3d/f66 164 1 2024-02-16T19:24:32.269 INFO:tasks.workunit.client.1.smithi163.stdout:8/715: getdents d0/d24/d2c/d74/dcb 0 2024-02-16T19:24:32.269 INFO:tasks.workunit.client.1.smithi163.stdout:8/716: dread - d0/d7/d18/fa0 zero size 2024-02-16T19:24:32.297 INFO:tasks.workunit.client.1.smithi163.stdout:1/730: dwrite d7/dc/d2d/d32/f46 [0,4194304] 0 2024-02-16T19:24:32.297 INFO:tasks.workunit.client.1.smithi163.stdout:9/995: dwrite d5/de/d16/d46/d9b/d106/f142 [0,4194304] 0 2024-02-16T19:24:32.297 INFO:tasks.workunit.client.1.smithi163.stdout:9/996: readlink d5/de/d30/d31/lbe 0 2024-02-16T19:24:32.299 INFO:tasks.workunit.client.1.smithi163.stdout:9/997: dread d5/de/d84/f138 [0,4194304] 0 2024-02-16T19:24:32.299 INFO:tasks.workunit.client.1.smithi163.stdout:9/998: creat d5/de/d30/d3d/d49/d6d/d97/f157 x:0 0 0 2024-02-16T19:24:32.302 INFO:tasks.workunit.client.1.smithi163.stdout:1/731: creat d7/d20/d21/d74/dab/ff8 x:0 0 0 2024-02-16T19:24:32.305 INFO:tasks.workunit.client.1.smithi163.stdout:9/999: rename d5/de/d30/d43/fda to d5/de/d30/d8a/da8/f158 0 2024-02-16T19:24:32.306 INFO:tasks.workunit.client.1.smithi163.stdout:1/732: symlink d7/d20/d21/d74/de2/lf9 0 2024-02-16T19:24:32.308 INFO:tasks.workunit.client.1.smithi163.stdout:1/733: symlink d7/d20/d21/d74/lfa 0 2024-02-16T19:24:32.308 INFO:tasks.workunit.client.1.smithi163.stdout:1/734: chown d7/dc/d2d/d3f/d85 6420 1 2024-02-16T19:24:32.308 INFO:tasks.workunit.client.1.smithi163.stdout:1/735: write d7/d20/d96/fd5 [286633,87960] 0 2024-02-16T19:24:32.332 INFO:tasks.workunit.client.1.smithi163.stdout:1/736: dwrite d7/d20/d21/fe3 [0,4194304] 0 2024-02-16T19:24:32.333 INFO:tasks.workunit.client.1.smithi163.stdout:0/944: sync 2024-02-16T19:24:32.333 INFO:tasks.workunit.client.1.smithi163.stdout:0/945: write d11/d12/d67/d6a/dae/f105 [591136,88744] 0 2024-02-16T19:24:32.333 INFO:tasks.workunit.client.1.smithi163.stdout:0/946: creat d11/d21/d50/d58/d6b/f13d x:0 0 0 2024-02-16T19:24:32.333 INFO:tasks.workunit.client.1.smithi163.stdout:4/640: sync 2024-02-16T19:24:32.333 INFO:tasks.workunit.client.1.smithi163.stdout:4/641: write d11/d2d/d4f/d5f/faa [655147,126888] 0 2024-02-16T19:24:32.337 INFO:tasks.workunit.client.1.smithi163.stdout:5/799: sync 2024-02-16T19:24:32.337 INFO:tasks.workunit.client.1.smithi163.stdout:5/800: fdatasync d0/d2/d55/fae 0 2024-02-16T19:24:32.338 INFO:tasks.workunit.client.1.smithi163.stdout:0/947: symlink d11/d22/d61/d10f/l13e 0 2024-02-16T19:24:32.338 INFO:tasks.workunit.client.1.smithi163.stdout:0/948: chown d11/d21/d50/d58/d72/d9e/dbf/lff 211 1 2024-02-16T19:24:32.340 INFO:tasks.workunit.client.1.smithi163.stdout:4/642: mkdir d11/d18/d1f/d29/d52/dd6 0 2024-02-16T19:24:32.347 INFO:tasks.workunit.client.1.smithi163.stdout:0/949: creat d11/d12/f13f x:0 0 0 2024-02-16T19:24:32.348 INFO:tasks.workunit.client.1.smithi163.stdout:4/643: creat d11/d2d/d81/fd7 x:0 0 0 2024-02-16T19:24:32.351 INFO:tasks.workunit.client.1.smithi163.stdout:0/950: creat d11/d22/f140 x:0 0 0 2024-02-16T19:24:32.351 INFO:tasks.workunit.client.1.smithi163.stdout:0/951: write d11/d21/d50/d58/f8b [2631346,11665] 0 2024-02-16T19:24:32.353 INFO:tasks.workunit.client.1.smithi163.stdout:5/801: rename d0/d2/d34/df8 to d0/d10d 0 2024-02-16T19:24:32.354 INFO:tasks.workunit.client.1.smithi163.stdout:5/802: getdents d0/d1c/d5a 0 2024-02-16T19:24:32.354 INFO:tasks.workunit.client.1.smithi163.stdout:5/803: read - d0/d10d/ffc zero size 2024-02-16T19:24:32.355 INFO:tasks.workunit.client.1.smithi163.stdout:5/804: mkdir d0/d2/d34/d44/d64/dd9/d10e 0 2024-02-16T19:24:32.357 INFO:tasks.workunit.client.1.smithi163.stdout:1/737: dwrite d7/dc/d78/d7c/f41 [0,4194304] 0 2024-02-16T19:24:32.359 INFO:tasks.workunit.client.1.smithi163.stdout:1/738: dread d7/f7d [0,4194304] 0 2024-02-16T19:24:32.362 INFO:tasks.workunit.client.1.smithi163.stdout:0/952: dread d11/d21/fdb [0,4194304] 0 2024-02-16T19:24:32.362 INFO:tasks.workunit.client.1.smithi163.stdout:0/953: write d11/d22/d7f/f112 [8207,86668] 0 2024-02-16T19:24:32.363 INFO:tasks.workunit.client.1.smithi163.stdout:1/739: mkdir d7/d20/d21/d74/de2/dfb 0 2024-02-16T19:24:32.364 INFO:tasks.workunit.client.1.smithi163.stdout:4/644: dwrite d11/f91 [0,4194304] 0 2024-02-16T19:24:32.364 INFO:tasks.workunit.client.1.smithi163.stdout:0/954: mknod d11/d12/d20/c141 0 2024-02-16T19:24:32.371 INFO:tasks.workunit.client.1.smithi163.stdout:4/645: creat d11/d2d/d4f/d5f/d70/d74/d75/dca/fd8 x:0 0 0 2024-02-16T19:24:32.371 INFO:tasks.workunit.client.1.smithi163.stdout:4/646: dread - d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/fb6 zero size 2024-02-16T19:24:32.371 INFO:tasks.workunit.client.1.smithi163.stdout:4/647: chown d11/d18/d66 20 1 2024-02-16T19:24:32.372 INFO:tasks.workunit.client.1.smithi163.stdout:1/740: mknod d7/dc/d2d/d32/cfc 0 2024-02-16T19:24:32.378 INFO:tasks.workunit.client.1.smithi163.stdout:0/955: dread d11/d12/d67/f91 [0,4194304] 0 2024-02-16T19:24:32.378 INFO:tasks.workunit.client.1.smithi163.stdout:0/956: truncate d11/d22/d7f/f95 1645659 0 2024-02-16T19:24:32.378 INFO:tasks.workunit.client.1.smithi163.stdout:0/957: fsync d11/d21/fdb 0 2024-02-16T19:24:32.380 INFO:tasks.workunit.client.1.smithi163.stdout:5/805: dwrite d0/d2/d5c/f8c [4194304,4194304] 0 2024-02-16T19:24:32.384 INFO:tasks.workunit.client.1.smithi163.stdout:1/741: write d7/d20/d2a/f9f [3067628,94050] 0 2024-02-16T19:24:32.384 INFO:tasks.workunit.client.1.smithi163.stdout:5/806: creat d0/d22/d3b/f10f x:0 0 0 2024-02-16T19:24:32.385 INFO:tasks.workunit.client.1.smithi163.stdout:5/807: readlink d0/d2/l2a 0 2024-02-16T19:24:32.385 INFO:tasks.workunit.client.1.smithi163.stdout:5/808: stat d0/d22/d28/d36/d3c/d104 0 2024-02-16T19:24:32.385 INFO:tasks.workunit.client.1.smithi163.stdout:4/648: dread d11/f14 [0,4194304] 0 2024-02-16T19:24:32.385 INFO:tasks.workunit.client.1.smithi163.stdout:1/742: mkdir d7/dc/d2d/d32/d87/de1/dfd 0 2024-02-16T19:24:32.388 INFO:tasks.workunit.client.1.smithi163.stdout:3/713: sync 2024-02-16T19:24:32.388 INFO:tasks.workunit.client.1.smithi163.stdout:2/844: sync 2024-02-16T19:24:32.389 INFO:tasks.workunit.client.1.smithi163.stdout:5/809: fsync d0/d22/d3b/f10f 0 2024-02-16T19:24:32.394 INFO:tasks.workunit.client.1.smithi163.stdout:4/649: write d11/d18/d1f/f33 [952351,124744] 0 2024-02-16T19:24:32.394 INFO:tasks.workunit.client.1.smithi163.stdout:4/650: fdatasync d11/d18/d1f/d29/f49 0 2024-02-16T19:24:32.394 INFO:tasks.workunit.client.1.smithi163.stdout:4/651: fsync d11/d2d/d4f/f63 0 2024-02-16T19:24:32.395 INFO:tasks.workunit.client.1.smithi163.stdout:4/652: mknod d11/d18/d1f/d29/d52/d57/d6c/d9d/cd9 0 2024-02-16T19:24:32.395 INFO:tasks.workunit.client.1.smithi163.stdout:4/653: write d11/d18/d1f/d29/d52/f5a [1044109,84407] 0 2024-02-16T19:24:32.397 INFO:tasks.workunit.client.1.smithi163.stdout:1/743: write d7/d5b/d63/f7b [7085352,122720] 0 2024-02-16T19:24:32.397 INFO:tasks.workunit.client.1.smithi163.stdout:4/654: rename d11/d18/d66/c6b to d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/cda 0 2024-02-16T19:24:32.398 INFO:tasks.workunit.client.1.smithi163.stdout:4/655: mknod d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/cdb 0 2024-02-16T19:24:32.398 INFO:tasks.workunit.client.1.smithi163.stdout:4/656: readlink d11/d18/d1f/d29/l3c 0 2024-02-16T19:24:32.401 INFO:tasks.workunit.client.1.smithi163.stdout:1/744: dread d7/d5b/fa4 [0,4194304] 0 2024-02-16T19:24:32.402 INFO:tasks.workunit.client.1.smithi163.stdout:3/714: dread d0/f13 [0,4194304] 0 2024-02-16T19:24:32.402 INFO:tasks.workunit.client.1.smithi163.stdout:1/745: mkdir d7/d20/d93/dfe 0 2024-02-16T19:24:32.402 INFO:tasks.workunit.client.1.smithi163.stdout:3/715: truncate d0/dc/d94/ffe 952338 0 2024-02-16T19:24:32.407 INFO:tasks.workunit.client.1.smithi163.stdout:3/716: unlink d0/dc/d1c/d32/db8/l6c 0 2024-02-16T19:24:32.407 INFO:tasks.workunit.client.1.smithi163.stdout:3/717: chown d0/dc/d1c/d91/fc3 403935528 1 2024-02-16T19:24:32.408 INFO:tasks.workunit.client.1.smithi163.stdout:1/746: dread d7/d20/f24 [0,4194304] 0 2024-02-16T19:24:32.409 INFO:tasks.workunit.client.1.smithi163.stdout:1/747: mknod d7/dc/d2d/d3f/d85/cff 0 2024-02-16T19:24:32.410 INFO:tasks.workunit.client.1.smithi163.stdout:1/748: creat d7/dc/d2d/d32/d87/f100 x:0 0 0 2024-02-16T19:24:32.410 INFO:tasks.workunit.client.1.smithi163.stdout:1/749: creat d7/d20/d21/f101 x:0 0 0 2024-02-16T19:24:32.410 INFO:tasks.workunit.client.1.smithi163.stdout:1/750: fsync d7/dc/d2d/d32/f51 0 2024-02-16T19:24:32.411 INFO:tasks.workunit.client.1.smithi163.stdout:1/751: stat d7 0 2024-02-16T19:24:32.411 INFO:tasks.workunit.client.1.smithi163.stdout:5/810: dwrite d0/d2/d34/d44/d64/dd9/fdc [0,4194304] 0 2024-02-16T19:24:32.411 INFO:tasks.workunit.client.1.smithi163.stdout:5/811: truncate d0/f4d 4528954 0 2024-02-16T19:24:32.411 INFO:tasks.workunit.client.1.smithi163.stdout:5/812: creat d0/d2/d34/d44/d53/de2/f110 x:0 0 0 2024-02-16T19:24:32.412 INFO:tasks.workunit.client.1.smithi163.stdout:5/813: write d0/d2/d5c/fcb [1060261,90413] 0 2024-02-16T19:24:32.412 INFO:tasks.workunit.client.1.smithi163.stdout:1/752: unlink d7/d20/d21/f6c 0 2024-02-16T19:24:32.412 INFO:tasks.workunit.client.1.smithi163.stdout:1/753: dread - d7/dc/d2d/d90/fea zero size 2024-02-16T19:24:32.416 INFO:tasks.workunit.client.1.smithi163.stdout:1/754: rename d7/d20/d21/d4e/f68 to d7/d20/d21/d74/dab/f102 0 2024-02-16T19:24:32.418 INFO:tasks.workunit.client.1.smithi163.stdout:1/755: getdents d7/dc/d2d/d32 0 2024-02-16T19:24:32.418 INFO:tasks.workunit.client.1.smithi163.stdout:1/756: rmdir d7/d5b/d63 39 2024-02-16T19:24:32.418 INFO:tasks.workunit.client.1.smithi163.stdout:1/757: creat d7/dc/d2d/d32/d87/da6/f103 x:0 0 0 2024-02-16T19:24:32.420 INFO:tasks.workunit.client.1.smithi163.stdout:1/758: symlink d7/d20/d21/d74/dab/l104 0 2024-02-16T19:24:32.420 INFO:tasks.workunit.client.1.smithi163.stdout:1/759: write d7/d20/d21/d4e/f79 [1914238,28347] 0 2024-02-16T19:24:32.421 INFO:tasks.workunit.client.1.smithi163.stdout:1/760: creat d7/d20/d96/f105 x:0 0 0 2024-02-16T19:24:32.421 INFO:tasks.workunit.client.1.smithi163.stdout:1/761: chown d7/dc/d78/d7c/c7f 13 1 2024-02-16T19:24:32.421 INFO:tasks.workunit.client.1.smithi163.stdout:1/762: chown d7/d20/d96/lf6 6350 1 2024-02-16T19:24:32.424 INFO:tasks.workunit.client.1.smithi163.stdout:2/845: dwrite d2/d44/ff4 [0,4194304] 0 2024-02-16T19:24:32.424 INFO:tasks.workunit.client.1.smithi163.stdout:2/846: chown d2/d2a/fb2 99 1 2024-02-16T19:24:32.424 INFO:tasks.workunit.client.1.smithi163.stdout:2/847: dread - d2/d26/f11d zero size 2024-02-16T19:24:32.424 INFO:tasks.workunit.client.1.smithi163.stdout:2/848: chown d2/f15 104 1 2024-02-16T19:24:32.427 INFO:tasks.workunit.client.1.smithi163.stdout:2/849: link d2/d2a/d2d/d73/ld1 d2/d26/d54/d58/d7d/d105/l122 0 2024-02-16T19:24:32.427 INFO:tasks.workunit.client.1.smithi163.stdout:2/850: fdatasync d2/d2a/d39/d117/f6d 0 2024-02-16T19:24:32.427 INFO:tasks.workunit.client.1.smithi163.stdout:2/851: truncate d2/dce/ffc 291245 0 2024-02-16T19:24:32.431 INFO:tasks.workunit.client.1.smithi163.stdout:5/814: dread d0/f32 [0,4194304] 0 2024-02-16T19:24:32.431 INFO:tasks.workunit.client.1.smithi163.stdout:5/815: chown d0/d2/d34/d44/d53/ff0 785 1 2024-02-16T19:24:32.435 INFO:tasks.workunit.client.1.smithi163.stdout:4/657: dwrite d11/f8a [0,4194304] 0 2024-02-16T19:24:32.438 INFO:tasks.workunit.client.1.smithi163.stdout:2/852: dread d2/d2a/d2d/d73/d94/dc9/fcf [0,4194304] 0 2024-02-16T19:24:32.439 INFO:tasks.workunit.client.1.smithi163.stdout:2/853: creat d2/d26/d54/d58/dff/f123 x:0 0 0 2024-02-16T19:24:32.439 INFO:tasks.workunit.client.1.smithi163.stdout:2/854: write d2/d2a/d39/f42 [5098629,24405] 0 2024-02-16T19:24:32.440 INFO:tasks.workunit.client.1.smithi163.stdout:2/855: mknod d2/d2a/d2d/d73/d94/c124 0 2024-02-16T19:24:32.441 INFO:tasks.workunit.client.1.smithi163.stdout:3/718: dwrite d0/dc/d1d/d49/d98/fef [0,4194304] 0 2024-02-16T19:24:32.441 INFO:tasks.workunit.client.1.smithi163.stdout:1/763: dwrite d7/d20/d96/fcb [0,4194304] 0 2024-02-16T19:24:32.443 INFO:tasks.workunit.client.1.smithi163.stdout:2/856: rename d2/d26/d54/fd5 to d2/d26/d54/d58/d8c/f125 0 2024-02-16T19:24:32.443 INFO:tasks.workunit.client.1.smithi163.stdout:2/857: write d2/d2a/d2d/d73/d94/dc9/f103 [142073,118269] 0 2024-02-16T19:24:32.446 INFO:tasks.workunit.client.1.smithi163.stdout:4/658: dread d11/d2d/d4f/d5f/d70/d74/d75/dca/f78 [0,4194304] 0 2024-02-16T19:24:32.451 INFO:tasks.workunit.client.1.smithi163.stdout:3/719: dread d0/dc/f24 [0,4194304] 0 2024-02-16T19:24:32.453 INFO:tasks.workunit.client.1.smithi163.stdout:1/764: creat d7/dc/d2d/d32/f106 x:0 0 0 2024-02-16T19:24:32.457 INFO:tasks.workunit.client.1.smithi163.stdout:2/858: rename d2/lfd to d2/d44/l126 0 2024-02-16T19:24:32.461 INFO:tasks.workunit.client.1.smithi163.stdout:4/659: mknod d11/d18/d1f/d29/d52/d57/dc5/cdc 0 2024-02-16T19:24:32.462 INFO:tasks.workunit.client.1.smithi163.stdout:3/720: symlink d0/dc/d94/l100 0 2024-02-16T19:24:32.465 INFO:tasks.workunit.client.1.smithi163.stdout:2/859: symlink d2/d26/d54/d58/d7d/d105/l127 0 2024-02-16T19:24:32.465 INFO:tasks.workunit.client.1.smithi163.stdout:2/860: chown d2/d2a/d39/d117/d7f/f10b 25073 1 2024-02-16T19:24:32.467 INFO:tasks.workunit.client.1.smithi163.stdout:4/660: link d11/d3d/f62 d11/d18/d1f/d29/d52/d76/fdd 0 2024-02-16T19:24:32.469 INFO:tasks.workunit.client.1.smithi163.stdout:3/721: creat d0/dc/d1c/f101 x:0 0 0 2024-02-16T19:24:32.469 INFO:tasks.workunit.client.1.smithi163.stdout:3/722: readlink d0/dc/d1d/d49/l4c 0 2024-02-16T19:24:32.470 INFO:tasks.workunit.client.1.smithi163.stdout:1/765: getdents d7/d5b 0 2024-02-16T19:24:32.472 INFO:tasks.workunit.client.1.smithi163.stdout:2/861: getdents d2/d26/d54/d58/d7d/daf 0 2024-02-16T19:24:32.476 INFO:tasks.workunit.client.1.smithi163.stdout:4/661: unlink d11/d18/d1f/d29/d52/d57/d6c/d7c/cab 0 2024-02-16T19:24:32.478 INFO:tasks.workunit.client.1.smithi163.stdout:3/723: creat d0/f102 x:0 0 0 2024-02-16T19:24:32.480 INFO:tasks.workunit.client.1.smithi163.stdout:1/766: rename d7/d5b/d63/c82 to d7/d20/c107 0 2024-02-16T19:24:32.480 INFO:tasks.workunit.client.1.smithi163.stdout:1/767: fsync d7/dc/d2d/d32/f51 0 2024-02-16T19:24:32.482 INFO:tasks.workunit.client.1.smithi163.stdout:2/862: mkdir d2/d26/d54/d58/d8c/d128 0 2024-02-16T19:24:32.483 INFO:tasks.workunit.client.1.smithi163.stdout:4/662: dread d11/f26 [0,4194304] 0 2024-02-16T19:24:32.483 INFO:tasks.workunit.client.1.smithi163.stdout:4/663: write d11/d2d/d81/fd7 [356502,72022] 0 2024-02-16T19:24:32.485 INFO:tasks.workunit.client.1.smithi163.stdout:3/724: symlink d0/d83/da5/db9/l103 0 2024-02-16T19:24:32.488 INFO:tasks.workunit.client.1.smithi163.stdout:2/863: creat d2/d2a/d9f/df8/f129 x:0 0 0 2024-02-16T19:24:32.488 INFO:tasks.workunit.client.1.smithi163.stdout:2/864: write d2/d26/d54/d58/d98/fab [642970,41433] 0 2024-02-16T19:24:32.489 INFO:tasks.workunit.client.1.smithi163.stdout:3/725: mkdir d0/d83/d104 0 2024-02-16T19:24:32.489 INFO:tasks.workunit.client.1.smithi163.stdout:2/865: getdents d2/d2a/d39/d117/d7f 0 2024-02-16T19:24:32.491 INFO:tasks.workunit.client.1.smithi163.stdout:4/664: dread d11/f14 [0,4194304] 0 2024-02-16T19:24:32.491 INFO:tasks.workunit.client.1.smithi163.stdout:2/866: link d2/d26/d54/dfb/d114/cdd d2/d2a/d39/d117/dcb/c12a 0 2024-02-16T19:24:32.491 INFO:tasks.workunit.client.1.smithi163.stdout:2/867: fsync d2/d2a/f5f 0 2024-02-16T19:24:32.492 INFO:tasks.workunit.client.1.smithi163.stdout:4/665: truncate d11/d18/d1f/d34/f51 729220 0 2024-02-16T19:24:32.493 INFO:tasks.workunit.client.1.smithi163.stdout:4/666: mknod d11/d18/d1f/d29/cde 0 2024-02-16T19:24:32.494 INFO:tasks.workunit.client.1.smithi163.stdout:2/868: getdents d2/d2a/d2d/d73/d94/dc9 0 2024-02-16T19:24:32.495 INFO:tasks.workunit.client.1.smithi163.stdout:4/667: symlink d11/d18/d1f/d29/d52/ldf 0 2024-02-16T19:24:32.496 INFO:tasks.workunit.client.1.smithi163.stdout:3/726: dread d0/dc/d1d/d29/f33 [4194304,4194304] 0 2024-02-16T19:24:32.497 INFO:tasks.workunit.client.1.smithi163.stdout:3/727: creat d0/d83/da5/f105 x:0 0 0 2024-02-16T19:24:32.499 INFO:tasks.workunit.client.1.smithi163.stdout:3/728: mknod d0/dc/d50/c106 0 2024-02-16T19:24:32.500 INFO:tasks.workunit.client.1.smithi163.stdout:3/729: truncate d0/f92 3711518 0 2024-02-16T19:24:32.500 INFO:tasks.workunit.client.1.smithi163.stdout:3/730: write d0/f2 [5432632,122722] 0 2024-02-16T19:24:32.501 INFO:tasks.workunit.client.1.smithi163.stdout:3/731: creat d0/d83/d104/f107 x:0 0 0 2024-02-16T19:24:32.502 INFO:tasks.workunit.client.1.smithi163.stdout:3/732: creat d0/d83/f108 x:0 0 0 2024-02-16T19:24:32.502 INFO:tasks.workunit.client.1.smithi163.stdout:1/768: dwrite d7/f14 [0,4194304] 0 2024-02-16T19:24:32.503 INFO:tasks.workunit.client.1.smithi163.stdout:3/733: dread d0/dc/d1c/d32/db8/fc7 [0,4194304] 0 2024-02-16T19:24:32.504 INFO:tasks.workunit.client.1.smithi163.stdout:3/734: write d0/dc/d96/da7/fb1 [169535,61870] 0 2024-02-16T19:24:32.510 INFO:tasks.workunit.client.1.smithi163.stdout:3/735: dread d0/dc/d50/fd3 [0,4194304] 0 2024-02-16T19:24:32.510 INFO:tasks.workunit.client.1.smithi163.stdout:3/736: write d0/dc/d94/d9e/dac/fc0 [785653,61770] 0 2024-02-16T19:24:32.515 INFO:tasks.workunit.client.1.smithi163.stdout:3/737: symlink d0/dc/d1c/d32/d7f/l109 0 2024-02-16T19:24:32.520 INFO:tasks.workunit.client.1.smithi163.stdout:4/668: dwrite d11/d2d/d4f/d60/d79/f90 [0,4194304] 0 2024-02-16T19:24:32.522 INFO:tasks.workunit.client.1.smithi163.stdout:1/769: dwrite d7/d20/d2a/f5c [0,4194304] 0 2024-02-16T19:24:32.522 INFO:tasks.workunit.client.1.smithi163.stdout:1/770: write d7/d20/d96/fd5 [405733,24447] 0 2024-02-16T19:24:32.522 INFO:tasks.workunit.client.1.smithi163.stdout:1/771: write d7/d20/d96/fce [4750181,3046] 0 2024-02-16T19:24:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.522+0000 7f4e7ec07700 1 -- 172.21.15.40:0/3676830223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e78066c50 msgr2=0x7f4e780ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.522+0000 7f4e7ec07700 1 --2- 172.21.15.40:0/3676830223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e78066c50 0x7f4e780ffb10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4e68004940 tx=0x7f4e68009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.523+0000 7f4e7ec07700 1 -- 172.21.15.40:0/3676830223 shutdown_connections 2024-02-16T19:24:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.523+0000 7f4e7ec07700 1 --2- 172.21.15.40:0/3676830223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e781000e0 0x7f4e78102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.523+0000 7f4e7ec07700 1 --2- 172.21.15.40:0/3676830223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e78066c50 0x7f4e780ffb10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.523+0000 7f4e7ec07700 1 -- 172.21.15.40:0/3676830223 >> 172.21.15.40:0/3676830223 conn(0x7f4e780f7500 msgr2=0x7f4e780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.523+0000 7f4e7ec07700 1 -- 172.21.15.40:0/3676830223 shutdown_connections 2024-02-16T19:24:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.523+0000 7f4e7ec07700 1 -- 172.21.15.40:0/3676830223 wait complete. 2024-02-16T19:24:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.523+0000 7f4e7ec07700 1 Processor -- start 2024-02-16T19:24:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.524+0000 7f4e7ec07700 1 -- start start 2024-02-16T19:24:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.524+0000 7f4e7ec07700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e78066c50 0x7f4e780fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.524+0000 7f4e7ec07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e781000e0 0x7f4e780fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.524+0000 7f4e7ec07700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7811b340 con 0x7f4e781000e0 2024-02-16T19:24:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.524+0000 7f4e7ec07700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7811b4b0 con 0x7f4e78066c50 2024-02-16T19:24:32.527 INFO:tasks.workunit.client.1.smithi163.stdout:4/669: dread d11/d2d/d4f/f72 [0,4194304] 0 2024-02-16T19:24:32.527 INFO:tasks.workunit.client.1.smithi163.stdout:4/670: write d11/d2d/d81/fd7 [426934,62475] 0 2024-02-16T19:24:32.527 INFO:tasks.workunit.client.1.smithi163.stdout:4/671: dread - d11/d18/d1f/d23/d3a/d4a/f6e zero size 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.525+0000 7f4e77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e781000e0 0x7f4e780fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.525+0000 7f4e77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e781000e0 0x7f4e780fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37108/0 (socket says 172.21.15.40:37108) 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.525+0000 7f4e7c9a3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e78066c50 0x7f4e780fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.525+0000 7f4e7c9a3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e78066c50 0x7f4e780fcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.40:50428/0 (socket says 172.21.15.40:50428) 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.525+0000 7f4e7c9a3700 1 -- 172.21.15.40:0/84900564 learned_addr learned my addr 172.21.15.40:0/84900564 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.525+0000 7f4e77fff700 1 -- 172.21.15.40:0/84900564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e78066c50 msgr2=0x7f4e780fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.525+0000 7f4e77fff700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e78066c50 0x7f4e780fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.525+0000 7f4e77fff700 1 -- 172.21.15.40:0/84900564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68005040 con 0x7f4e781000e0 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.526+0000 7f4e77fff700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e781000e0 0x7f4e780fd490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e78066990 tx=0x7f4e6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.526+0000 7f4e757fa700 1 -- 172.21.15.40:0/84900564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6c004e50 con 0x7f4e781000e0 2024-02-16T19:24:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.526+0000 7f4e757fa700 1 -- 172.21.15.40:0/84900564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e6c00f040 con 0x7f4e781000e0 2024-02-16T19:24:32.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.526+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7811b790 con 0x7f4e781000e0 2024-02-16T19:24:32.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.526+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7811bcd0 con 0x7f4e781000e0 2024-02-16T19:24:32.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.527+0000 7f4e757fa700 1 -- 172.21.15.40:0/84900564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6c0095c0 con 0x7f4e781000e0 2024-02-16T19:24:32.529 INFO:tasks.workunit.client.1.smithi163.stdout:4/672: creat d11/d18/d1f/d29/d52/d57/dc5/fe0 x:0 0 0 2024-02-16T19:24:32.529 INFO:tasks.workunit.client.1.smithi163.stdout:4/673: dread - d11/d3d/fb4 zero size 2024-02-16T19:24:32.530 INFO:tasks.workunit.client.1.smithi163.stdout:4/674: creat d11/d18/d1f/d29/fe1 x:0 0 0 2024-02-16T19:24:32.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.530+0000 7f4e757fa700 1 -- 172.21.15.40:0/84900564 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4e6c009720 con 0x7f4e781000e0 2024-02-16T19:24:32.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.531+0000 7f4e757fa700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4e60069440 0x7f4e6006b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:32.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.531+0000 7f4e757fa700 1 -- 172.21.15.40:0/84900564 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4e6c086dc0 con 0x7f4e781000e0 2024-02-16T19:24:32.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.531+0000 7f4e7c9a3700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4e60069440 0x7f4e6006b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:32.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.531+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e640052e0 con 0x7f4e781000e0 2024-02-16T19:24:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.532+0000 7f4e7c9a3700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4e60069440 0x7f4e6006b8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4e68004f90 tx=0x7f4e68009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:32.534 INFO:tasks.workunit.client.1.smithi163.stdout:4/675: readlink d11/d18/d1f/d23/d3a/lc4 0 2024-02-16T19:24:32.534 INFO:tasks.workunit.client.1.smithi163.stdout:3/738: dwrite d0/dc/d1c/d32/f36 [0,4194304] 0 2024-02-16T19:24:32.534 INFO:tasks.workunit.client.1.smithi163.stdout:4/676: truncate d11/d2d/d4f/d60/fc6 371966 0 2024-02-16T19:24:32.534 INFO:tasks.workunit.client.1.smithi163.stdout:3/739: fdatasync d0/dc/d1d/d29/f45 0 2024-02-16T19:24:32.534 INFO:tasks.workunit.client.1.smithi163.stdout:4/677: dread - d11/d18/d1f/d29/d52/d76/d89/d94/fd5 zero size 2024-02-16T19:24:32.534 INFO:tasks.workunit.client.1.smithi163.stdout:3/740: write d0/dc/d94/d9e/dac/fbf [240518,63133] 0 2024-02-16T19:24:32.536 INFO:tasks.workunit.client.1.smithi163.stdout:7/702: sync 2024-02-16T19:24:32.536 INFO:tasks.workunit.client.1.smithi163.stdout:7/703: readlink d11/d24/d34/d8d/lad 0 2024-02-16T19:24:32.537 INFO:tasks.workunit.client.1.smithi163.stdout:4/678: write f2 [1892041,78572] 0 2024-02-16T19:24:32.537 INFO:tasks.workunit.client.1.smithi163.stdout:4/679: read d11/f30 [3668793,68114] 0 2024-02-16T19:24:32.537 INFO:tasks.workunit.client.1.smithi163.stdout:4/680: write d11/d2d/d81/fd7 [542057,66026] 0 2024-02-16T19:24:32.538 INFO:tasks.workunit.client.1.smithi163.stdout:3/741: unlink d0/dc/d1d/d29/d38/c5f 0 2024-02-16T19:24:32.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.537+0000 7f4e757fa700 1 -- 172.21.15.40:0/84900564 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4e6c054140 con 0x7f4e781000e0 2024-02-16T19:24:32.538 INFO:tasks.workunit.client.1.smithi163.stdout:6/654: sync 2024-02-16T19:24:32.542 INFO:tasks.workunit.client.1.smithi163.stdout:1/772: dwrite d7/dc/d2d/fba [0,4194304] 0 2024-02-16T19:24:32.543 INFO:tasks.workunit.client.1.smithi163.stdout:1/773: stat d7/dc/d2d 0 2024-02-16T19:24:32.544 INFO:tasks.workunit.client.1.smithi163.stdout:7/704: symlink d11/d24/d49/dc4/lef 0 2024-02-16T19:24:32.544 INFO:tasks.workunit.client.1.smithi163.stdout:7/705: readlink d11/d50/d5d/d8b/d67/db2/lb4 0 2024-02-16T19:24:32.547 INFO:tasks.workunit.client.1.smithi163.stdout:4/681: mknod d11/d18/d1f/d29/d52/d76/da2/ce2 0 2024-02-16T19:24:32.547 INFO:tasks.workunit.client.1.smithi163.stdout:4/682: readlink d11/d18/d1f/d29/d52/d57/lb9 0 2024-02-16T19:24:32.550 INFO:tasks.workunit.client.1.smithi163.stdout:6/655: mkdir d0/dd1 0 2024-02-16T19:24:32.553 INFO:tasks.workunit.client.1.smithi163.stdout:1/774: creat d7/dc/dc1/f108 x:0 0 0 2024-02-16T19:24:32.553 INFO:tasks.workunit.client.1.smithi163.stdout:1/775: chown d7/dc/d78/c84 519602 1 2024-02-16T19:24:32.553 INFO:tasks.workunit.client.1.smithi163.stdout:1/776: stat d7/d20/d21/f101 0 2024-02-16T19:24:32.554 INFO:tasks.workunit.client.1.smithi163.stdout:4/683: read d11/d18/d1f/f38 [3045144,81573] 0 2024-02-16T19:24:32.554 INFO:tasks.workunit.client.1.smithi163.stdout:4/684: fsync d11/d2d/d4f/d5f/d70/d74/d75/dca/fd8 0 2024-02-16T19:24:32.554 INFO:tasks.workunit.client.1.smithi163.stdout:4/685: creat d11/d18/d1f/d29/d52/d76/fe3 x:0 0 0 2024-02-16T19:24:32.555 INFO:tasks.workunit.client.1.smithi163.stdout:7/706: rename d11/f3f to d11/ff0 0 2024-02-16T19:24:32.557 INFO:tasks.workunit.client.1.smithi163.stdout:1/777: write d7/d20/d2a/f40 [1397623,95553] 0 2024-02-16T19:24:32.559 INFO:tasks.workunit.client.1.smithi163.stdout:6/656: unlink d0/dd/d7f/d81/lb1 0 2024-02-16T19:24:32.559 INFO:tasks.workunit.client.1.smithi163.stdout:6/657: read d0/d6/df/d20/d40/d17/f88 [4061099,12676] 0 2024-02-16T19:24:32.559 INFO:tasks.workunit.client.1.smithi163.stdout:6/658: truncate d0/d6/df/d20/d40/d17/f4b 4391594 0 2024-02-16T19:24:32.563 INFO:tasks.workunit.client.1.smithi163.stdout:4/686: unlink d11/d2d/d4f/d5f/d70/d74/d75/dca/dbf/fcf 0 2024-02-16T19:24:32.565 INFO:tasks.workunit.client.1.smithi163.stdout:6/659: mknod d0/dd/d7f/d81/daa/cd2 0 2024-02-16T19:24:32.565 INFO:tasks.workunit.client.1.smithi163.stdout:6/660: read d0/dd/d7f/d15/d1c/f56 [17581,109664] 0 2024-02-16T19:24:32.567 INFO:tasks.workunit.client.1.smithi163.stdout:6/661: creat d0/d6/df/fd3 x:0 0 0 2024-02-16T19:24:32.567 INFO:tasks.workunit.client.1.smithi163.stdout:6/662: stat d0/d6/df 0 2024-02-16T19:24:32.577 INFO:tasks.workunit.client.1.smithi163.stdout:6/663: dread d0/d6/df/d20/d69/fa6 [0,4194304] 0 2024-02-16T19:24:32.579 INFO:tasks.workunit.client.1.smithi163.stdout:6/664: creat d0/dd/dbc/fd4 x:0 0 0 2024-02-16T19:24:32.579 INFO:tasks.workunit.client.1.smithi163.stdout:1/778: dread d7/d20/d21/d4e/f9e [0,4194304] 0 2024-02-16T19:24:32.580 INFO:tasks.workunit.client.1.smithi163.stdout:6/665: mknod d0/dd/d7f/d2e/cd5 0 2024-02-16T19:24:32.581 INFO:tasks.workunit.client.1.smithi163.stdout:6/666: unlink d0/dd/d7f/d2e/lab 0 2024-02-16T19:24:32.587 INFO:tasks.workunit.client.1.smithi163.stdout:7/707: dwrite d11/d24/d34/f58 [0,4194304] 0 2024-02-16T19:24:32.587 INFO:tasks.workunit.client.1.smithi163.stdout:4/687: dwrite d11/d2d/d4f/d5f/d70/d74/fd4 [0,4194304] 0 2024-02-16T19:24:32.588 INFO:tasks.workunit.client.1.smithi163.stdout:3/742: dwrite d0/dc/d50/d66/da3/d71/f89 [0,4194304] 0 2024-02-16T19:24:32.599 INFO:tasks.workunit.client.1.smithi163.stdout:1/779: dwrite d7/dc/d2d/d32/d87/d97/fb0 [0,4194304] 0 2024-02-16T19:24:32.599 INFO:tasks.workunit.client.1.smithi163.stdout:1/780: stat d7/d20/d21/d74/de2/dfb 0 2024-02-16T19:24:32.599 INFO:tasks.workunit.client.1.smithi163.stdout:4/688: rmdir d11/d18/d1f/d29/d52/d76/da2 39 2024-02-16T19:24:32.601 INFO:tasks.workunit.client.1.smithi163.stdout:3/743: getdents d0/dc/d1c/d32/d7f 0 2024-02-16T19:24:32.601 INFO:tasks.workunit.client.1.smithi163.stdout:3/744: dread - d0/d3e/f60 zero size 2024-02-16T19:24:32.601 INFO:tasks.workunit.client.1.smithi163.stdout:3/745: write d0/dc/d94/d9e/dac/fd0 [1367923,76781] 0 2024-02-16T19:24:32.601 INFO:tasks.workunit.client.1.smithi163.stdout:3/746: chown d0/d83/da5/db9 6 1 2024-02-16T19:24:32.607 INFO:tasks.workunit.client.1.smithi163.stdout:1/781: rename d7/dc/d2d/d32/d87/lcd to d7/d20/d21/l109 0 2024-02-16T19:24:32.608 INFO:tasks.workunit.client.1.smithi163.stdout:4/689: creat d11/d18/d1f/d29/d52/d57/d6c/d9d/fe4 x:0 0 0 2024-02-16T19:24:32.608 INFO:tasks.workunit.client.1.smithi163.stdout:4/690: readlink d11/d18/d1f/d29/lbb 0 2024-02-16T19:24:32.608 INFO:tasks.workunit.client.1.smithi163.stdout:4/691: write d11/d18/f1d [6959243,73362] 0 2024-02-16T19:24:32.609 INFO:tasks.workunit.client.1.smithi163.stdout:1/782: symlink d7/dc/d2d/d90/l10a 0 2024-02-16T19:24:32.609 INFO:tasks.workunit.client.1.smithi163.stdout:1/783: write d7/d5b/d63/f7b [3788920,109757] 0 2024-02-16T19:24:32.611 INFO:tasks.workunit.client.1.smithi163.stdout:4/692: mkdir d11/d18/d1f/d34/de5 0 2024-02-16T19:24:32.616 INFO:tasks.workunit.client.1.smithi163.stdout:1/784: mknod d7/d5b/c10b 0 2024-02-16T19:24:32.616 INFO:tasks.workunit.client.1.smithi163.stdout:6/667: dwrite d0/d6/f31 [0,4194304] 0 2024-02-16T19:24:32.616 INFO:tasks.workunit.client.1.smithi163.stdout:6/668: creat d0/dd/d7f/d2e/fd6 x:0 0 0 2024-02-16T19:24:32.623 INFO:tasks.workunit.client.1.smithi163.stdout:7/708: dwrite d11/d24/d25/d4b/fdc [0,4194304] 0 2024-02-16T19:24:32.623 INFO:tasks.workunit.client.1.smithi163.stdout:7/709: chown d11/d24/d49/dac/feb 26659 1 2024-02-16T19:24:32.623 INFO:tasks.workunit.client.1.smithi163.stdout:3/747: dwrite d0/dc/d96/fdd [0,4194304] 0 2024-02-16T19:24:32.623 INFO:tasks.workunit.client.1.smithi163.stdout:4/693: creat d11/d18/d1f/d29/d52/d57/d6c/dc3/fe6 x:0 0 0 2024-02-16T19:24:32.623 INFO:tasks.workunit.client.1.smithi163.stdout:4/694: getdents d11/d2d/d4f/d5f/d70/d74/d75 0 2024-02-16T19:24:32.623 INFO:tasks.workunit.client.1.smithi163.stdout:4/695: chown d11/d18/d1f/d29/d52/d76 7 1 2024-02-16T19:24:32.624 INFO:tasks.workunit.client.1.smithi163.stdout:1/785: symlink d7/d20/d21/d74/l10c 0 2024-02-16T19:24:32.624 INFO:tasks.workunit.client.1.smithi163.stdout:1/786: write d7/dc/d2d/d32/d3b/fdd [981691,100339] 0 2024-02-16T19:24:32.624 INFO:tasks.workunit.client.1.smithi163.stdout:1/787: fdatasync d7/dc/f30 0 2024-02-16T19:24:32.624 INFO:tasks.workunit.client.1.smithi163.stdout:1/788: creat d7/d20/d96/f10d x:0 0 0 2024-02-16T19:24:32.630 INFO:tasks.workunit.client.1.smithi163.stdout:1/789: dread d7/d20/d21/d4e/f9e [0,4194304] 0 2024-02-16T19:24:32.630 INFO:tasks.workunit.client.1.smithi163.stdout:6/669: dread d0/dd/d7f/d2e/d33/f3d [0,4194304] 0 2024-02-16T19:24:32.630 INFO:tasks.workunit.client.1.smithi163.stdout:6/670: chown d0/d18/d77/d97/db8 0 1 2024-02-16T19:24:32.630 INFO:tasks.workunit.client.1.smithi163.stdout:6/671: fsync d0/dd/d7f/d2e/d33/f3d 0 2024-02-16T19:24:32.636 INFO:tasks.workunit.client.1.smithi163.stdout:3/748: symlink d0/d3e/l10a 0 2024-02-16T19:24:32.636 INFO:tasks.workunit.client.1.smithi163.stdout:3/749: chown d0/dc/d94/lfc 1665 1 2024-02-16T19:24:32.636 INFO:tasks.workunit.client.1.smithi163.stdout:4/696: creat d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/fe7 x:0 0 0 2024-02-16T19:24:32.636 INFO:tasks.workunit.client.1.smithi163.stdout:7/710: rmdir d11/d24/d25/d4b/d8c 39 2024-02-16T19:24:32.637 INFO:tasks.workunit.client.1.smithi163.stdout:6/672: mkdir d0/d18/d77/d97/db8/dd7 0 2024-02-16T19:24:32.639 INFO:tasks.workunit.client.1.smithi163.stdout:8/717: sync 2024-02-16T19:24:32.644 INFO:tasks.workunit.client.1.smithi163.stdout:1/790: rmdir d7/dc/d2d 39 2024-02-16T19:24:32.644 INFO:tasks.workunit.client.1.smithi163.stdout:1/791: write d7/f26 [2604483,41670] 0 2024-02-16T19:24:32.644 INFO:tasks.workunit.client.1.smithi163.stdout:1/792: fsync d7/d20/f35 0 2024-02-16T19:24:32.644 INFO:tasks.workunit.client.1.smithi163.stdout:1/793: write d7/dc/d2d/d3f/fad [835531,23673] 0 2024-02-16T19:24:32.646 INFO:tasks.workunit.client.1.smithi163.stdout:6/673: mknod d0/d6/df/d20/d40/d17/dcb/cd8 0 2024-02-16T19:24:32.646 INFO:tasks.workunit.client.1.smithi163.stdout:2/869: sync 2024-02-16T19:24:32.646 INFO:tasks.workunit.client.1.smithi163.stdout:0/958: sync 2024-02-16T19:24:32.646 INFO:tasks.workunit.client.1.smithi163.stdout:5/816: sync 2024-02-16T19:24:32.647 INFO:tasks.workunit.client.1.smithi163.stdout:3/750: rmdir d0/dc/d94 39 2024-02-16T19:24:32.647 INFO:tasks.workunit.client.1.smithi163.stdout:3/751: readlink d0/dc/d1d/d49/l67 0 2024-02-16T19:24:32.647 INFO:tasks.workunit.client.1.smithi163.stdout:8/718: getdents d0/d24/d3c/d3d/d41 0 2024-02-16T19:24:32.649 INFO:tasks.workunit.client.1.smithi163.stdout:2/870: mkdir d2/d26/d54/d58/d8c/dc5/d12b 0 2024-02-16T19:24:32.649 INFO:tasks.workunit.client.1.smithi163.stdout:2/871: truncate d2/d2a/d39/d3a/f7b 1218197 0 2024-02-16T19:24:32.649 INFO:tasks.workunit.client.1.smithi163.stdout:2/872: dread - d2/d2a/d39/d117/f79 zero size 2024-02-16T19:24:32.650 INFO:tasks.workunit.client.1.smithi163.stdout:4/697: read d11/d18/d1f/d29/d52/d57/f7e [2894370,57805] 0 2024-02-16T19:24:32.651 INFO:tasks.workunit.client.1.smithi163.stdout:1/794: dread d7/dc/d2d/d32/d3b/f50 [0,4194304] 0 2024-02-16T19:24:32.651 INFO:tasks.workunit.client.1.smithi163.stdout:1/795: dread - d7/d20/d96/f10d zero size 2024-02-16T19:24:32.652 INFO:tasks.workunit.client.1.smithi163.stdout:2/873: creat d2/d2a/d9f/f12c x:0 0 0 2024-02-16T19:24:32.655 INFO:tasks.workunit.client.1.smithi163.stdout:5/817: rename d0/d2/d34/d44/d53/d93 to d0/d22/d28/d111 0 2024-02-16T19:24:32.657 INFO:tasks.workunit.client.1.smithi163.stdout:0/959: dread d11/d21/d50/d58/f92 [0,4194304] 0 2024-02-16T19:24:32.658 INFO:tasks.workunit.client.1.smithi163.stdout:8/719: rename d0/d24/d78/db1 to d0/d7/d19/d1f/d30/d6a/d7b/deb 0 2024-02-16T19:24:32.661 INFO:tasks.workunit.client.1.smithi163.stdout:8/720: creat d0/d24/d2c/d74/dcb/fec x:0 0 0 2024-02-16T19:24:32.661 INFO:tasks.workunit.client.1.smithi163.stdout:8/721: write d0/d24/d3c/d3d/fde [665676,87304] 0 2024-02-16T19:24:32.661 INFO:tasks.workunit.client.1.smithi163.stdout:4/698: dread d11/d18/d1f/d29/f31 [0,4194304] 0 2024-02-16T19:24:32.661 INFO:tasks.workunit.client.1.smithi163.stdout:4/699: fdatasync d11/d18/d1f/d29/d52/d76/fe3 0 2024-02-16T19:24:32.661 INFO:tasks.workunit.client.1.smithi163.stdout:4/700: dread - d11/d18/d1f/d29/d52/d57/d6c/d7c/fce zero size 2024-02-16T19:24:32.661 INFO:tasks.workunit.client.1.smithi163.stdout:4/701: write d11/d2d/d4f/d5f/d70/fc2 [3876270,15404] 0 2024-02-16T19:24:32.664 INFO:tasks.workunit.client.1.smithi163.stdout:0/960: dread d11/d21/d7c/fce [0,4194304] 0 2024-02-16T19:24:32.664 INFO:tasks.workunit.client.1.smithi163.stdout:3/752: dread d0/dc/d50/f8b [0,4194304] 0 2024-02-16T19:24:32.666 INFO:tasks.workunit.client.1.smithi163.stdout:4/702: link d11/d18/c1b d11/d18/d1f/d29/d52/d57/d6c/d7c/ce8 0 2024-02-16T19:24:32.666 INFO:tasks.workunit.client.1.smithi163.stdout:4/703: chown d11/c1c 93899896 1 2024-02-16T19:24:32.668 INFO:tasks.workunit.client.1.smithi163.stdout:3/753: unlink d0/dc/d50/d66/cee 0 2024-02-16T19:24:32.668 INFO:tasks.workunit.client.1.smithi163.stdout:3/754: creat d0/dc/d1c/d32/ddb/f10b x:0 0 0 2024-02-16T19:24:32.670 INFO:tasks.workunit.client.1.smithi163.stdout:3/755: creat d0/dc/d1d/d49/f10c x:0 0 0 2024-02-16T19:24:32.670 INFO:tasks.workunit.client.1.smithi163.stdout:3/756: write d0/dc/d94/fae [617914,104621] 0 2024-02-16T19:24:32.671 INFO:tasks.workunit.client.1.smithi163.stdout:3/757: creat d0/dc/d50/d66/d80/f10d x:0 0 0 2024-02-16T19:24:32.680 INFO:tasks.workunit.client.1.smithi163.stdout:5/818: dwrite d0/d2/d34/d44/f51 [0,4194304] 0 2024-02-16T19:24:32.681 INFO:tasks.workunit.client.1.smithi163.stdout:1/796: dwrite d7/dc/d2d/d3f/fc8 [0,4194304] 0 2024-02-16T19:24:32.684 INFO:tasks.workunit.client.1.smithi163.stdout:1/797: rmdir d7/d20/d93/dfe 0 2024-02-16T19:24:32.688 INFO:tasks.workunit.client.1.smithi163.stdout:3/758: write d0/dc/d1d/d29/f33 [126269,113250] 0 2024-02-16T19:24:32.689 INFO:tasks.workunit.client.1.smithi163.stdout:1/798: unlink d7/d5b/l5e 0 2024-02-16T19:24:32.689 INFO:tasks.workunit.client.1.smithi163.stdout:6/674: dwrite d0/d6/fa3 [0,4194304] 0 2024-02-16T19:24:32.690 INFO:tasks.workunit.client.1.smithi163.stdout:3/759: creat d0/dc/d1c/d32/f10e x:0 0 0 2024-02-16T19:24:32.690 INFO:tasks.workunit.client.1.smithi163.stdout:3/760: read - d0/dc/d50/d66/da3/d71/fe0 zero size 2024-02-16T19:24:32.691 INFO:tasks.workunit.client.1.smithi163.stdout:1/799: mknod d7/d5b/c10e 0 2024-02-16T19:24:32.691 INFO:tasks.workunit.client.1.smithi163.stdout:1/800: chown d7/dc/d78 3980819 1 2024-02-16T19:24:32.692 INFO:tasks.workunit.client.1.smithi163.stdout:5/819: dread d0/d22/d28/d111/fa0 [0,4194304] 0 2024-02-16T19:24:32.694 INFO:tasks.workunit.client.1.smithi163.stdout:6/675: symlink d0/dd/ld9 0 2024-02-16T19:24:32.694 INFO:tasks.workunit.client.1.smithi163.stdout:6/676: read - d0/d6/df/d20/d69/f8d zero size 2024-02-16T19:24:32.695 INFO:tasks.workunit.client.1.smithi163.stdout:3/761: creat d0/dc/f10f x:0 0 0 2024-02-16T19:24:32.695 INFO:tasks.workunit.client.1.smithi163.stdout:3/762: fdatasync d0/dc/d50/d66/da3/d71/fe0 0 2024-02-16T19:24:32.696 INFO:tasks.workunit.client.1.smithi163.stdout:1/801: mkdir d7/dc/d2d/d32/d10f 0 2024-02-16T19:24:32.696 INFO:tasks.workunit.client.1.smithi163.stdout:1/802: write d7/dc/f5a [256236,62337] 0 2024-02-16T19:24:32.697 INFO:tasks.workunit.client.1.smithi163.stdout:5/820: creat d0/d22/d3b/d8d/f112 x:0 0 0 2024-02-16T19:24:32.699 INFO:tasks.workunit.client.1.smithi163.stdout:1/803: mkdir d7/dc/d2d/d32/d87/da6/de5/d110 0 2024-02-16T19:24:32.700 INFO:tasks.workunit.client.1.smithi163.stdout:1/804: fdatasync d7/dc/fca 0 2024-02-16T19:24:32.702 INFO:tasks.workunit.client.1.smithi163.stdout:5/821: mknod d0/d2/d34/d44/d64/dd9/d10e/c113 0 2024-02-16T19:24:32.704 INFO:tasks.workunit.client.1.smithi163.stdout:6/677: write d0/dd/d7f/d81/fc8 [3873359,33596] 0 2024-02-16T19:24:32.704 INFO:tasks.workunit.client.1.smithi163.stdout:6/678: write d0/d6/df/d20/d40/d17/fa2 [24461,589] 0 2024-02-16T19:24:32.704 INFO:tasks.workunit.client.1.smithi163.stdout:8/722: rename d0/d24/d78 to d0/d7/d18/d53/ded 0 2024-02-16T19:24:32.704 INFO:tasks.workunit.client.1.smithi163.stdout:8/723: chown d0/d24/d27/f64 201657 1 2024-02-16T19:24:32.709 INFO:tasks.workunit.client.1.smithi163.stdout:1/805: dread d7/f14 [0,4194304] 0 2024-02-16T19:24:32.709 INFO:tasks.workunit.client.1.smithi163.stdout:1/806: fdatasync d7/dc/d2d/d32/fbe 0 2024-02-16T19:24:32.710 INFO:tasks.workunit.client.1.smithi163.stdout:3/763: rename d0/d83/d104/f107 to d0/d83/f110 0 2024-02-16T19:24:32.710 INFO:tasks.workunit.client.1.smithi163.stdout:3/764: truncate d0/dc/d94/d9e/fb6 1402903 0 2024-02-16T19:24:32.710 INFO:tasks.workunit.client.1.smithi163.stdout:3/765: creat d0/dc/d50/d66/da3/d71/f111 x:0 0 0 2024-02-16T19:24:32.714 INFO:tasks.workunit.client.1.smithi163.stdout:8/724: dread d0/d7/d19/d1f/d30/d6a/d7b/f89 [0,4194304] 0 2024-02-16T19:24:32.715 INFO:tasks.workunit.client.1.smithi163.stdout:6/679: rmdir d0/d18 39 2024-02-16T19:24:32.717 INFO:tasks.workunit.client.1.smithi163.stdout:1/807: symlink d7/d20/d21/l111 0 2024-02-16T19:24:32.720 INFO:tasks.workunit.client.1.smithi163.stdout:1/808: dread d7/d20/d93/fb3 [0,4194304] 0 2024-02-16T19:24:32.720 INFO:tasks.workunit.client.1.smithi163.stdout:1/809: fsync d7/dc/d2d/d32/d87/f100 0 2024-02-16T19:24:32.720 INFO:tasks.workunit.client.1.smithi163.stdout:3/766: symlink d0/dc/d50/l112 0 2024-02-16T19:24:32.720 INFO:tasks.workunit.client.1.smithi163.stdout:3/767: fdatasync d0/f92 0 2024-02-16T19:24:32.720 INFO:tasks.workunit.client.1.smithi163.stdout:3/768: fdatasync d0/dc/d50/d66/fec 0 2024-02-16T19:24:32.722 INFO:tasks.workunit.client.1.smithi163.stdout:5/822: rename d0/d2/d34/d44/fe8 to d0/d22/d3b/dc0/f114 0 2024-02-16T19:24:32.723 INFO:tasks.workunit.client.1.smithi163.stdout:5/823: write d0/d2/d34/d44/d53/f87 [99142,44059] 0 2024-02-16T19:24:32.724 INFO:tasks.workunit.client.1.smithi163.stdout:6/680: mkdir d0/d6/df/d20/d40/d1e/db2/dda 0 2024-02-16T19:24:32.728 INFO:tasks.workunit.client.1.smithi163.stdout:3/769: creat d0/dc/d50/f113 x:0 0 0 2024-02-16T19:24:32.728 INFO:tasks.workunit.client.1.smithi163.stdout:3/770: write d0/f102 [789718,64296] 0 2024-02-16T19:24:32.730 INFO:tasks.workunit.client.1.smithi163.stdout:8/725: rename d0/d7/d18/d53/f7a to d0/d24/d2c/d74/dcb/fee 0 2024-02-16T19:24:32.730 INFO:tasks.workunit.client.1.smithi163.stdout:8/726: truncate d0/d7/d19/d1f/d30/d48/f65 4622289 0 2024-02-16T19:24:32.731 INFO:tasks.workunit.client.1.smithi163.stdout:6/681: creat d0/d18/d77/fdb x:0 0 0 2024-02-16T19:24:32.732 INFO:tasks.workunit.client.1.smithi163.stdout:5/824: write d0/d22/f35 [513353,128160] 0 2024-02-16T19:24:32.732 INFO:tasks.workunit.client.1.smithi163.stdout:5/825: chown d0/d2/d55 1 1 2024-02-16T19:24:32.732 INFO:tasks.workunit.client.1.smithi163.stdout:3/771: creat d0/dc/d50/d66/d93/f114 x:0 0 0 2024-02-16T19:24:32.732 INFO:tasks.workunit.client.1.smithi163.stdout:3/772: readlink d0/dc/d50/d66/la9 0 2024-02-16T19:24:32.734 INFO:tasks.workunit.client.1.smithi163.stdout:1/810: rename d7/dc/d2d/d32/la2 to d7/d5b/d63/l112 0 2024-02-16T19:24:32.735 INFO:tasks.workunit.client.1.smithi163.stdout:8/727: mknod d0/d24/d3c/d3d/d41/cef 0 2024-02-16T19:24:32.735 INFO:tasks.workunit.client.1.smithi163.stdout:8/728: write d0/d24/d27/f64 [816727,82702] 0 2024-02-16T19:24:32.736 INFO:tasks.workunit.client.1.smithi163.stdout:6/682: mkdir d0/dd/dbc/ddc 0 2024-02-16T19:24:32.736 INFO:tasks.workunit.client.1.smithi163.stdout:6/683: read d0/dd/d7f/d2e/d33/f5f [1076513,57171] 0 2024-02-16T19:24:32.736 INFO:tasks.workunit.client.1.smithi163.stdout:6/684: chown d0/d6/le 5986709 1 2024-02-16T19:24:32.736 INFO:tasks.workunit.client.1.smithi163.stdout:6/685: readlink d0/l14 0 2024-02-16T19:24:32.737 INFO:tasks.workunit.client.1.smithi163.stdout:1/811: dread d7/d5b/fa4 [0,4194304] 0 2024-02-16T19:24:32.737 INFO:tasks.workunit.client.1.smithi163.stdout:5/826: mknod d0/d22/d3b/c115 0 2024-02-16T19:24:32.739 INFO:tasks.workunit.client.1.smithi163.stdout:8/729: rename d0/d7/la to d0/d7/d18/lf0 0 2024-02-16T19:24:32.741 INFO:tasks.workunit.client.1.smithi163.stdout:6/686: creat d0/fdd x:0 0 0 2024-02-16T19:24:32.742 INFO:tasks.workunit.client.1.smithi163.stdout:8/730: symlink d0/d7/d18/d2e/dd0/lf1 0 2024-02-16T19:24:32.743 INFO:tasks.workunit.client.1.smithi163.stdout:6/687: truncate d0/d6/df/d20/d69/fcf 11862796 0 2024-02-16T19:24:32.743 INFO:tasks.workunit.client.1.smithi163.stdout:6/688: getdents d0/d18/d77/d97/db8/dd7 0 2024-02-16T19:24:32.744 INFO:tasks.workunit.client.1.smithi163.stdout:6/689: creat d0/dd/d7f/d81/daa/fde x:0 0 0 2024-02-16T19:24:32.746 INFO:tasks.workunit.client.1.smithi163.stdout:8/731: rename d0/d24/d27/d45 to d0/d24/d27/df2 0 2024-02-16T19:24:32.754 INFO:tasks.workunit.client.1.smithi163.stdout:1/812: write d7/dc/d78/d7c/f41 [1889449,39282] 0 2024-02-16T19:24:32.755 INFO:tasks.workunit.client.1.smithi163.stdout:1/813: chown d7/dc/d2d/d32/f43 533496841 1 2024-02-16T19:24:32.765 INFO:tasks.workunit.client.1.smithi163.stdout:5/827: dwrite d0/d22/d28/d36/dc4/f101 [0,4194304] 0 2024-02-16T19:24:32.766 INFO:tasks.workunit.client.1.smithi163.stdout:5/828: chown d0/d22/d3b/f10f 2 1 2024-02-16T19:24:32.766 INFO:tasks.workunit.client.1.smithi163.stdout:5/829: creat d0/d22/d3b/f116 x:0 0 0 2024-02-16T19:24:32.766 INFO:tasks.workunit.client.1.smithi163.stdout:3/773: dwrite d0/dc/f3c [0,4194304] 0 2024-02-16T19:24:32.769 INFO:tasks.workunit.client.1.smithi163.stdout:1/814: read d7/d20/d96/fcb [3221902,37676] 0 2024-02-16T19:24:32.770 INFO:tasks.workunit.client.1.smithi163.stdout:5/830: mknod d0/c117 0 2024-02-16T19:24:32.770 INFO:tasks.workunit.client.1.smithi163.stdout:5/831: fdatasync d0/d22/d28/d36/d3c/d4f/f62 0 2024-02-16T19:24:32.774 INFO:tasks.workunit.client.1.smithi163.stdout:1/815: creat d7/dc/d2d/d32/d10f/f113 x:0 0 0 2024-02-16T19:24:32.776 INFO:tasks.workunit.client.1.smithi163.stdout:3/774: getdents d0/dc/d50/d66 0 2024-02-16T19:24:32.780 INFO:tasks.workunit.client.1.smithi163.stdout:1/816: symlink d7/d20/d96/l114 0 2024-02-16T19:24:32.781 INFO:tasks.workunit.client.1.smithi163.stdout:3/775: mknod d0/d83/da5/c115 0 2024-02-16T19:24:32.781 INFO:tasks.workunit.client.1.smithi163.stdout:3/776: getdents d0/dc/d1c/d32/db8/d99 0 2024-02-16T19:24:32.782 INFO:tasks.workunit.client.1.smithi163.stdout:1/817: symlink d7/dc/d2d/d32/d87/da6/l115 0 2024-02-16T19:24:32.795 INFO:tasks.workunit.client.1.smithi163.stdout:8/732: rename d0/d24/d27 to d0/d7/d19/d1f/d30/d6a/df3 0 2024-02-16T19:24:32.795 INFO:tasks.workunit.client.1.smithi163.stdout:8/733: write d0/d7/d19/d1f/d30/fb2 [1177372,26926] 0 2024-02-16T19:24:32.799 INFO:tasks.workunit.client.1.smithi163.stdout:5/832: dwrite d0/d22/d28/d36/d3c/d4f/f61 [0,4194304] 0 2024-02-16T19:24:32.799 INFO:tasks.workunit.client.1.smithi163.stdout:8/734: symlink d0/d24/d3c/d3d/d41/lf4 0 2024-02-16T19:24:32.801 INFO:tasks.workunit.client.1.smithi163.stdout:3/777: dwrite d0/dc/d1c/d21/f42 [0,4194304] 0 2024-02-16T19:24:32.801 INFO:tasks.workunit.client.1.smithi163.stdout:5/833: symlink d0/d2/d34/d44/d53/de2/l118 0 2024-02-16T19:24:32.809 INFO:tasks.workunit.client.1.smithi163.stdout:5/834: rename d0/d1c/d5a/l107 to d0/d1c/dbe/de3/l119 0 2024-02-16T19:24:32.809 INFO:tasks.workunit.client.1.smithi163.stdout:5/835: write d0/d2/d34/d44/f72 [402615,12488] 0 2024-02-16T19:24:32.810 INFO:tasks.workunit.client.1.smithi163.stdout:3/778: mkdir d0/dc/d1c/d116 0 2024-02-16T19:24:32.815 INFO:tasks.workunit.client.1.smithi163.stdout:5/836: creat d0/d2/d34/d44/d64/f11a x:0 0 0 2024-02-16T19:24:32.816 INFO:tasks.workunit.client.1.smithi163.stdout:3/779: rename d0/f92 to d0/dc/d96/da7/f117 0 2024-02-16T19:24:32.818 INFO:tasks.workunit.client.1.smithi163.stdout:8/735: dwrite d0/d6f/fd2 [0,4194304] 0 2024-02-16T19:24:32.821 INFO:tasks.workunit.client.1.smithi163.stdout:5/837: unlink d0/ce 0 2024-02-16T19:24:32.828 INFO:tasks.workunit.client.1.smithi163.stdout:8/736: unlink d0/d24/d3c/f84 0 2024-02-16T19:24:32.829 INFO:tasks.workunit.client.1.smithi163.stdout:3/780: getdents d0/dc/d1c/d32/ddb 0 2024-02-16T19:24:32.833 INFO:tasks.workunit.client.1.smithi163.stdout:5/838: unlink d0/d22/d28/d36/d3c/d4f/fd3 0 2024-02-16T19:24:32.833 INFO:tasks.workunit.client.1.smithi163.stdout:5/839: write d0/d2/d34/f100 [976335,129366] 0 2024-02-16T19:24:32.840 INFO:tasks.workunit.client.1.smithi163.stdout:8/737: dread d0/d7/d19/d1f/d30/f52 [0,4194304] 0 2024-02-16T19:24:32.840 INFO:tasks.workunit.client.1.smithi163.stdout:8/738: fdatasync d0/fea 0 2024-02-16T19:24:32.841 INFO:tasks.workunit.client.1.smithi163.stdout:8/739: chown d0/d7/d19/d1f/d30/d6a/d7b/ld1 7392346 1 2024-02-16T19:24:32.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.846+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e64000bf0 con 0x7f4e60069440 2024-02-16T19:24:32.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.848+0000 7f4e757fa700 1 -- 172.21.15.40:0/84900564 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f4e64000bf0 con 0x7f4e60069440 2024-02-16T19:24:32.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4e60069440 msgr2=0x7f4e6006b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:32.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4e60069440 0x7f4e6006b8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4e68004f90 tx=0x7f4e68009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:32.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e781000e0 msgr2=0x7f4e780fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:32.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e781000e0 0x7f4e780fd490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e78066990 tx=0x7f4e6c004600 comp rx=0 tx=0).stop 2024-02-16T19:24:32.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 shutdown_connections 2024-02-16T19:24:32.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f4e60069440 0x7f4e6006b8f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:32.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e78066c50 0x7f4e780fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:32.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 --2- 172.21.15.40:0/84900564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e781000e0 0x7f4e780fd490 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:32.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 >> 172.21.15.40:0/84900564 conn(0x7f4e780f7500 msgr2=0x7f4e780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:32.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 shutdown_connections 2024-02-16T19:24:32.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:32.851+0000 7f4e7ec07700 1 -- 172.21.15.40:0/84900564 wait complete. 2024-02-16T19:24:32.853 INFO:tasks.workunit.client.1.smithi163.stdout:8/740: dread d0/d7/d19/d1f/d30/d48/f65 [0,4194304] 0 2024-02-16T19:24:32.856 INFO:tasks.workunit.client.1.smithi163.stdout:3/781: dwrite d0/d3e/f60 [0,4194304] 0 2024-02-16T19:24:32.856 INFO:tasks.workunit.client.1.smithi163.stdout:8/741: rename d0/dc/f22 to d0/d7/d19/d1f/d30/d6a/d7b/deb/ff5 0 2024-02-16T19:24:32.857 INFO:tasks.workunit.client.1.smithi163.stdout:8/742: read - d0/d24/d2c/f83 zero size 2024-02-16T19:24:32.857 INFO:tasks.workunit.client.1.smithi163.stdout:8/743: chown d0/d7/d19 34405797 1 2024-02-16T19:24:32.858 INFO:tasks.workunit.client.1.smithi163.stdout:8/744: creat d0/d6f/de8/ff6 x:0 0 0 2024-02-16T19:24:32.862 INFO:tasks.workunit.client.1.smithi163.stdout:5/840: dwrite d0/d22/d3b/f9b [0,4194304] 0 2024-02-16T19:24:32.867 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:24:32.871 INFO:tasks.workunit.client.1.smithi163.stdout:8/745: read d0/d7/d18/fad [3942160,69055] 0 2024-02-16T19:24:32.873 INFO:tasks.workunit.client.1.smithi163.stdout:3/782: dwrite d0/dc/d1c/d32/f3f [4194304,4194304] 0 2024-02-16T19:24:32.873 INFO:tasks.workunit.client.1.smithi163.stdout:3/783: stat d0/dc/d50/f8b 0 2024-02-16T19:24:32.875 INFO:tasks.workunit.client.1.smithi163.stdout:8/746: unlink d0/d7/d18/d53/ded/l76 0 2024-02-16T19:24:32.877 INFO:tasks.workunit.client.1.smithi163.stdout:3/784: symlink d0/dc/d1d/d29/d38/l118 0 2024-02-16T19:24:32.878 INFO:tasks.workunit.client.1.smithi163.stdout:3/785: chown d0/dc/fd2 3366 1 2024-02-16T19:24:32.878 INFO:tasks.workunit.client.1.smithi163.stdout:3/786: fsync d0/dc/d1d/d49/f10c 0 2024-02-16T19:24:32.878 INFO:tasks.workunit.client.1.smithi163.stdout:3/787: creat d0/dc/d1c/d21/f119 x:0 0 0 2024-02-16T19:24:32.878 INFO:tasks.workunit.client.1.smithi163.stdout:3/788: creat d0/dc/d50/f11a x:0 0 0 2024-02-16T19:24:32.882 INFO:tasks.workunit.client.1.smithi163.stdout:5/841: dwrite d0/d2/d5c/f75 [0,4194304] 0 2024-02-16T19:24:32.886 INFO:tasks.workunit.client.1.smithi163.stdout:3/789: link d0/dc/d50/d66/lb4 d0/dc/d1c/d21/l11b 0 2024-02-16T19:24:32.886 INFO:tasks.workunit.client.1.smithi163.stdout:3/790: dread - d0/dc/d1d/d29/d38/fe5 zero size 2024-02-16T19:24:32.886 INFO:tasks.workunit.client.1.smithi163.stdout:3/791: readlink d0/dc/d1d/d29/d38/l5e 0 2024-02-16T19:24:32.887 INFO:tasks.workunit.client.1.smithi163.stdout:8/747: getdents d0/d7/d19 0 2024-02-16T19:24:32.894 INFO:tasks.workunit.client.1.smithi163.stdout:5/842: mknod d0/d1c/d5a/c11b 0 2024-02-16T19:24:32.895 INFO:tasks.workunit.client.1.smithi163.stdout:8/748: getdents d0/d7/d18 0 2024-02-16T19:24:32.895 INFO:tasks.workunit.client.1.smithi163.stdout:8/749: write d0/d6f/fc8 [621124,50481] 0 2024-02-16T19:24:32.895 INFO:tasks.workunit.client.1.smithi163.stdout:8/750: chown d0/d7/c11 479 1 2024-02-16T19:24:32.898 INFO:tasks.workunit.client.1.smithi163.stdout:5/843: mkdir d0/d11c 0 2024-02-16T19:24:32.899 INFO:tasks.workunit.client.1.smithi163.stdout:8/751: symlink d0/d24/lf7 0 2024-02-16T19:24:32.899 INFO:tasks.workunit.client.1.smithi163.stdout:5/844: read d0/d2/d34/d44/d53/fdf [206611,13470] 0 2024-02-16T19:24:32.899 INFO:tasks.workunit.client.1.smithi163.stdout:5/845: chown d0/d22/d3b/dc0/f114 89413 1 2024-02-16T19:24:32.902 INFO:tasks.workunit.client.1.smithi163.stdout:8/752: creat d0/d7/d19/d1f/d30/d48/ff8 x:0 0 0 2024-02-16T19:24:32.902 INFO:tasks.workunit.client.1.smithi163.stdout:8/753: write d0/d7/d18/d2e/f7f [4879744,40476] 0 2024-02-16T19:24:32.929 INFO:tasks.workunit.client.1.smithi163.stdout:5/846: dwrite d0/f2c [8388608,4194304] 0 2024-02-16T19:24:32.930 INFO:tasks.workunit.client.1.smithi163.stdout:8/754: dwrite d0/dc/f6c [0,4194304] 0 2024-02-16T19:24:32.932 INFO:tasks.workunit.client.1.smithi163.stdout:5/847: mknod d0/d22/d28/d111/c11d 0 2024-02-16T19:24:32.932 INFO:tasks.workunit.client.1.smithi163.stdout:5/848: write d0/d22/f2e [1116602,101234] 0 2024-02-16T19:24:32.936 INFO:tasks.workunit.client.1.smithi163.stdout:5/849: creat d0/d2/d5c/f11e x:0 0 0 2024-02-16T19:24:32.936 INFO:tasks.workunit.client.1.smithi163.stdout:5/850: creat d0/d1c/f11f x:0 0 0 2024-02-16T19:24:32.947 INFO:tasks.workunit.client.1.smithi163.stdout:5/851: write d0/d22/d28/d36/d3c/d4f/f62 [2019410,103452] 0 2024-02-16T19:24:32.947 INFO:tasks.workunit.client.1.smithi163.stdout:5/852: chown d0/d2/d34/f6d 127655273 1 2024-02-16T19:24:32.947 INFO:tasks.workunit.client.1.smithi163.stdout:5/853: dread - d0/d22/d28/d111/da8/ff9 zero size 2024-02-16T19:24:32.949 INFO:tasks.workunit.client.1.smithi163.stdout:5/854: mknod d0/d2/d34/d44/d64/c120 0 2024-02-16T19:24:32.965 INFO:tasks.workunit.client.1.smithi163.stdout:5/855: dwrite d0/d2/d5c/f11e [0,4194304] 0 2024-02-16T19:24:32.965 INFO:tasks.workunit.client.1.smithi163.stdout:5/856: stat d0/d2/d34 0 2024-02-16T19:24:32.965 INFO:tasks.workunit.client.1.smithi163.stdout:5/857: write d0/d2/fe7 [93506,48852] 0 2024-02-16T19:24:32.965 INFO:tasks.workunit.client.1.smithi163.stdout:5/858: chown d0/d22/d28/d36/d3c/d4f 1222821187 1 2024-02-16T19:24:32.966 INFO:tasks.workunit.client.1.smithi163.stdout:5/859: write d0/d2/f57 [1198080,12690] 0 2024-02-16T19:24:32.984 INFO:tasks.workunit.client.1.smithi163.stdout:5/860: dwrite d0/d1c/dbe/de3/fec [0,4194304] 0 2024-02-16T19:24:33.002 INFO:tasks.workunit.client.1.smithi163.stdout:5/861: dwrite d0/d22/d28/fad [0,4194304] 0 2024-02-16T19:24:33.002 INFO:tasks.workunit.client.1.smithi163.stdout:5/862: chown d0/d22/d28/d36 4006119 1 2024-02-16T19:24:33.002 INFO:tasks.workunit.client.1.smithi163.stdout:5/863: fsync d0/d22/d28/d36/f98 0 2024-02-16T19:24:33.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.002+0000 7f7f9c425700 1 -- 172.21.15.40:0/1246451529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fb870 msgr2=0x7f7f940fdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.002+0000 7f7f9c425700 1 --2- 172.21.15.40:0/1246451529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fb870 0x7f7f940fdc90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7f90004940 tx=0x7f7f90009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:33.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.002+0000 7f7f9c425700 1 -- 172.21.15.40:0/1246451529 shutdown_connections 2024-02-16T19:24:33.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.002+0000 7f7f9c425700 1 --2- 172.21.15.40:0/1246451529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f940fe1d0 0x7f7f94100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.002+0000 7f7f9c425700 1 --2- 172.21.15.40:0/1246451529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fb870 0x7f7f940fdc90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.002+0000 7f7f9c425700 1 -- 172.21.15.40:0/1246451529 >> 172.21.15.40:0/1246451529 conn(0x7f7f940f7500 msgr2=0x7f7f940f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:33.005 INFO:tasks.workunit.client.1.smithi163.stdout:5/864: mkdir d0/d1c/d5a/d121 0 2024-02-16T19:24:33.005 INFO:tasks.workunit.client.1.smithi163.stdout:5/865: write d0/d2/d34/d44/d53/ff0 [4243888,62696] 0 2024-02-16T19:24:33.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.003+0000 7f7f9c425700 1 -- 172.21.15.40:0/1246451529 shutdown_connections 2024-02-16T19:24:33.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.003+0000 7f7f9c425700 1 -- 172.21.15.40:0/1246451529 wait complete. 2024-02-16T19:24:33.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.003+0000 7f7f9c425700 1 Processor -- start 2024-02-16T19:24:33.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.003+0000 7f7f9c425700 1 -- start start 2024-02-16T19:24:33.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f9c425700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f940fb870 0x7f7f94113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f9c425700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fe1d0 0x7f7f94114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f9c425700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94114a30 con 0x7f7f940fe1d0 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f9c425700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f941152d0 con 0x7f7f940fb870 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f999c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fe1d0 0x7f7f94114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f999c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fe1d0 0x7f7f94114430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37132/0 (socket says 172.21.15.40:37132) 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f999c0700 1 -- 172.21.15.40:0/3991910676 learned_addr learned my addr 172.21.15.40:0/3991910676 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f9a1c1700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f940fb870 0x7f7f94113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f999c0700 1 -- 172.21.15.40:0/3991910676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f940fb870 msgr2=0x7f7f94113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f999c0700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f940fb870 0x7f7f94113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f999c0700 1 -- 172.21.15.40:0/3991910676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90005040 con 0x7f7f940fe1d0 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f999c0700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fe1d0 0x7f7f94114430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800eb40 tx=0x7f7f8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.004+0000 7f7f86ffd700 1 -- 172.21.15.40:0/3991910676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f88007cc0 con 0x7f7f940fe1d0 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.005+0000 7f7f86ffd700 1 -- 172.21.15.40:0/3991910676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f8800f040 con 0x7f7f940fe1d0 2024-02-16T19:24:33.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.005+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f94115550 con 0x7f7f940fe1d0 2024-02-16T19:24:33.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.005+0000 7f7f86ffd700 1 -- 172.21.15.40:0/3991910676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f8800ca10 con 0x7f7f940fe1d0 2024-02-16T19:24:33.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.005+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f94115a90 con 0x7f7f940fe1d0 2024-02-16T19:24:33.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.006+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9410e050 con 0x7f7f940fe1d0 2024-02-16T19:24:33.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.006+0000 7f7f86ffd700 1 -- 172.21.15.40:0/3991910676 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7f8800cc80 con 0x7f7f940fe1d0 2024-02-16T19:24:33.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.006+0000 7f7f86ffd700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7f800692b0 0x7f7f8006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.006+0000 7f7f86ffd700 1 -- 172.21.15.40:0/3991910676 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7f88086dd0 con 0x7f7f940fe1d0 2024-02-16T19:24:33.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.007+0000 7f7f9a1c1700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7f800692b0 0x7f7f8006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.007+0000 7f7f9a1c1700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7f800692b0 0x7f7f8006b760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7f900136a0 tx=0x7f7f90009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:33.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.011+0000 7f7f86ffd700 1 -- 172.21.15.40:0/3991910676 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7f8804dbd0 con 0x7f7f940fe1d0 2024-02-16T19:24:33.023 INFO:tasks.workunit.client.1.smithi163.stdout:5/866: dwrite d0/d1c/f74 [0,4194304] 0 2024-02-16T19:24:33.025 INFO:tasks.workunit.client.1.smithi163.stdout:5/867: unlink d0/d22/d3b/f42 0 2024-02-16T19:24:33.025 INFO:tasks.workunit.client.1.smithi163.stdout:5/868: creat d0/d1c/dbe/de3/f122 x:0 0 0 2024-02-16T19:24:33.025 INFO:tasks.workunit.client.1.smithi163.stdout:5/869: fdatasync d0/d2/d34/d44/d53/fdf 0 2024-02-16T19:24:33.025 INFO:tasks.workunit.client.1.smithi163.stdout:5/870: fdatasync d0/d1c/fd7 0 2024-02-16T19:24:33.033 INFO:tasks.workunit.client.1.smithi163.stdout:5/871: dread d0/d2/fe4 [0,4194304] 0 2024-02-16T19:24:33.050 INFO:tasks.workunit.client.1.smithi163.stdout:5/872: dwrite d0/d2/d34/f100 [0,4194304] 0 2024-02-16T19:24:33.050 INFO:tasks.workunit.client.1.smithi163.stdout:5/873: readlink d0/d22/d28/d111/lc3 0 2024-02-16T19:24:33.050 INFO:tasks.workunit.client.1.smithi163.stdout:5/874: chown d0/d2/f29 1 1 2024-02-16T19:24:33.050 INFO:tasks.workunit.client.1.smithi163.stdout:5/875: dread - d0/d1c/d60/fdb zero size 2024-02-16T19:24:33.050 INFO:tasks.workunit.client.1.smithi163.stdout:5/876: chown d0/d10d/ffc 2 1 2024-02-16T19:24:33.050 INFO:tasks.workunit.client.1.smithi163.stdout:5/877: readlink d0/d2/d5c/l9c 0 2024-02-16T19:24:33.052 INFO:tasks.workunit.client.1.smithi163.stdout:5/878: write d0/d2/fd5 [419319,66422] 0 2024-02-16T19:24:33.053 INFO:tasks.workunit.client.1.smithi163.stdout:5/879: chown d0/d22/d28/d36/d3c/d4f/f62 882 1 2024-02-16T19:24:33.053 INFO:tasks.workunit.client.1.smithi163.stdout:5/880: creat d0/d1c/dbe/f123 x:0 0 0 2024-02-16T19:24:33.054 INFO:tasks.workunit.client.1.smithi163.stdout:5/881: mknod d0/d2/d34/c124 0 2024-02-16T19:24:33.055 INFO:tasks.workunit.client.1.smithi163.stdout:5/882: stat d0/d1c/f74 0 2024-02-16T19:24:33.056 INFO:tasks.workunit.client.1.smithi163.stdout:5/883: creat d0/d22/d28/d36/db7/f125 x:0 0 0 2024-02-16T19:24:33.056 INFO:tasks.workunit.client.1.smithi163.stdout:5/884: write d0/d22/d3b/d8d/fab [4363419,758] 0 2024-02-16T19:24:33.057 INFO:tasks.workunit.client.1.smithi163.stdout:5/885: dread d0/d22/d28/d36/d3c/d4f/f7b [0,4194304] 0 2024-02-16T19:24:33.060 INFO:tasks.workunit.client.1.smithi163.stdout:5/886: creat d0/d22/f126 x:0 0 0 2024-02-16T19:24:33.062 INFO:tasks.workunit.client.1.smithi163.stdout:5/887: creat d0/d22/d28/d36/d3c/d104/f127 x:0 0 0 2024-02-16T19:24:33.062 INFO:tasks.workunit.client.1.smithi163.stdout:5/888: write d0/d22/d28/d36/d3c/f54 [235531,71940] 0 2024-02-16T19:24:33.077 INFO:tasks.workunit.client.1.smithi163.stdout:5/889: dwrite d0/d22/d28/d111/da8/ff9 [0,4194304] 0 2024-02-16T19:24:33.077 INFO:tasks.workunit.client.1.smithi163.stdout:5/890: fsync d0/d1c/fd7 0 2024-02-16T19:24:33.078 INFO:tasks.workunit.client.1.smithi163.stdout:5/891: rmdir d0/d22/d28 39 2024-02-16T19:24:33.078 INFO:tasks.workunit.client.1.smithi163.stdout:5/892: mkdir d0/d22/d28/d36/d3c/d128 0 2024-02-16T19:24:33.080 INFO:tasks.workunit.client.1.smithi163.stdout:5/893: read d0/d22/f3a [1167209,100404] 0 2024-02-16T19:24:33.081 INFO:tasks.workunit.client.1.smithi163.stdout:5/894: link d0/d1c/f11f d0/d22/d28/d36/d3c/d128/f129 0 2024-02-16T19:24:33.082 INFO:tasks.workunit.client.1.smithi163.stdout:5/895: mkdir d0/d2/d34/d44/d64/dd9/d10e/d12a 0 2024-02-16T19:24:33.083 INFO:tasks.workunit.client.1.smithi163.stdout:5/896: symlink d0/d22/d3b/d8d/l12b 0 2024-02-16T19:24:33.083 INFO:tasks.workunit.client.1.smithi163.stdout:5/897: write d0/d2/f57 [4689251,52675] 0 2024-02-16T19:24:33.092 INFO:tasks.workunit.client.1.smithi163.stdout:5/898: write d0/d2/d34/f6d [5810643,101992] 0 2024-02-16T19:24:33.101 INFO:tasks.workunit.client.1.smithi163.stdout:5/899: read d0/d2/d5c/f8c [4890418,117917] 0 2024-02-16T19:24:33.101 INFO:tasks.workunit.client.1.smithi163.stdout:5/900: chown d0/d22/d3b/dc0/dfa/cfb 13102446 1 2024-02-16T19:24:33.130 INFO:tasks.workunit.client.1.smithi163.stdout:5/901: creat d0/d2/db1/f12c x:0 0 0 2024-02-16T19:24:33.131 INFO:tasks.workunit.client.1.smithi163.stdout:5/902: fdatasync d0/f2c 0 2024-02-16T19:24:33.131 INFO:tasks.workunit.client.1.smithi163.stdout:5/903: fsync d0/d2/f16 0 2024-02-16T19:24:33.131 INFO:tasks.workunit.client.1.smithi163.stdout:5/904: write d0/d22/d3b/f116 [829064,123950] 0 2024-02-16T19:24:33.151 INFO:tasks.workunit.client.1.smithi163.stdout:5/905: dwrite d0/d22/d28/d111/da8/f10a [0,4194304] 0 2024-02-16T19:24:33.207 INFO:tasks.workunit.client.1.smithi163.stdout:5/906: unlink d0/d2/d5c/f8c 0 2024-02-16T19:24:33.207 INFO:tasks.workunit.client.1.smithi163.stdout:5/907: chown d0/d22/d28/d36/d3c 44335 1 2024-02-16T19:24:33.210 INFO:tasks.workunit.client.1.smithi163.stdout:5/908: creat d0/d22/d28/d36/dc4/f12d x:0 0 0 2024-02-16T19:24:33.211 INFO:tasks.workunit.client.1.smithi163.stdout:5/909: mknod d0/d22/d28/d36/dc4/c12e 0 2024-02-16T19:24:33.211 INFO:tasks.workunit.client.1.smithi163.stdout:5/910: truncate d0/d22/d28/d36/d3c/f43 4382894 0 2024-02-16T19:24:33.224 INFO:tasks.workunit.client.1.smithi163.stdout:5/911: truncate d0/d22/d28/d111/da8/f10a 3459200 0 2024-02-16T19:24:33.226 INFO:tasks.workunit.client.1.smithi163.stdout:5/912: rename d0/d22/f3a to d0/d22/d28/d36/dc4/f12f 0 2024-02-16T19:24:33.228 INFO:tasks.workunit.client.1.smithi163.stdout:5/913: rename d0/d1c/f74 to d0/d2/d34/d44/d64/f130 0 2024-02-16T19:24:33.230 INFO:tasks.workunit.client.1.smithi163.stdout:5/914: creat d0/d22/d28/d36/d3c/f131 x:0 0 0 2024-02-16T19:24:33.233 INFO:tasks.workunit.client.1.smithi163.stdout:5/915: read d0/d22/f2e [468825,27667] 0 2024-02-16T19:24:33.233 INFO:tasks.workunit.client.1.smithi163.stdout:5/916: write d0/d1c/dbe/f123 [895164,14341] 0 2024-02-16T19:24:33.234 INFO:tasks.workunit.client.1.smithi163.stdout:5/917: dread d0/d2/d34/d44/f72 [0,4194304] 0 2024-02-16T19:24:33.234 INFO:tasks.workunit.client.1.smithi163.stdout:5/918: fsync d0/d2/fe4 0 2024-02-16T19:24:33.236 INFO:tasks.workunit.client.1.smithi163.stdout:5/919: unlink d0/d22/d3b/f10f 0 2024-02-16T19:24:33.236 INFO:tasks.workunit.client.1.smithi163.stdout:5/920: creat d0/d2/d34/d44/d64/f132 x:0 0 0 2024-02-16T19:24:33.243 INFO:tasks.workunit.client.1.smithi163.stdout:0/961: sync 2024-02-16T19:24:33.243 INFO:tasks.workunit.client.1.smithi163.stdout:2/874: sync 2024-02-16T19:24:33.244 INFO:tasks.workunit.client.1.smithi163.stdout:7/711: sync 2024-02-16T19:24:33.244 INFO:tasks.workunit.client.1.smithi163.stdout:2/875: stat d2/d26/d97/d10f/f119 0 2024-02-16T19:24:33.244 INFO:tasks.workunit.client.1.smithi163.stdout:2/876: write d2/d26/d97/d10f/f119 [395649,82972] 0 2024-02-16T19:24:33.244 INFO:tasks.workunit.client.1.smithi163.stdout:0/962: truncate d11/d22/fa3 4406103 0 2024-02-16T19:24:33.245 INFO:tasks.workunit.client.1.smithi163.stdout:7/712: symlink d11/d30/lf1 0 2024-02-16T19:24:33.247 INFO:tasks.workunit.client.1.smithi163.stdout:0/963: link d11/d21/d50/d58/d72/d9e/l13b d11/d12/d67/d6a/dae/d118/d12a/l142 0 2024-02-16T19:24:33.249 INFO:tasks.workunit.client.1.smithi163.stdout:7/713: creat d11/d24/d25/d4b/ff2 x:0 0 0 2024-02-16T19:24:33.250 INFO:tasks.workunit.client.1.smithi163.stdout:5/921: dwrite d0/d1c/d5a/fcf [0,4194304] 0 2024-02-16T19:24:33.251 INFO:tasks.workunit.client.1.smithi163.stdout:5/922: fdatasync d0/d2/d5c/fbb 0 2024-02-16T19:24:33.251 INFO:tasks.workunit.client.1.smithi163.stdout:5/923: getdents d0/d11c 0 2024-02-16T19:24:33.251 INFO:tasks.workunit.client.1.smithi163.stdout:7/714: mknod d11/d50/d5d/d8b/d67/cf3 0 2024-02-16T19:24:33.259 INFO:tasks.workunit.client.1.smithi163.stdout:7/715: rename d11/d24/d34/fab to d11/ff4 0 2024-02-16T19:24:33.261 INFO:tasks.workunit.client.1.smithi163.stdout:4/704: sync 2024-02-16T19:24:33.263 INFO:tasks.workunit.client.1.smithi163.stdout:0/964: write d11/d12/f15 [246704,98825] 0 2024-02-16T19:24:33.265 INFO:tasks.workunit.client.1.smithi163.stdout:0/965: link d11/d12/f10b d11/d22/d61/f143 0 2024-02-16T19:24:33.274 INFO:tasks.workunit.client.1.smithi163.stdout:0/966: read f3 [66975,19541] 0 2024-02-16T19:24:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.286+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f9402b310 con 0x7f7f800692b0 2024-02-16T19:24:33.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.287+0000 7f7f86ffd700 1 -- 172.21.15.40:0/3991910676 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f7f9402b310 con 0x7f7f800692b0 2024-02-16T19:24:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7f800692b0 msgr2=0x7f7f8006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7f800692b0 0x7f7f8006b760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7f900136a0 tx=0x7f7f90009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fe1d0 msgr2=0x7f7f94114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fe1d0 0x7f7f94114430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800eb40 tx=0x7f7f8800b040 comp rx=0 tx=0).stop 2024-02-16T19:24:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 shutdown_connections 2024-02-16T19:24:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f7f800692b0 0x7f7f8006b760 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f940fb870 0x7f7f94113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 --2- 172.21.15.40:0/3991910676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f940fe1d0 0x7f7f94114430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 >> 172.21.15.40:0/3991910676 conn(0x7f7f940f7500 msgr2=0x7f7f940f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 shutdown_connections 2024-02-16T19:24:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.289+0000 7f7f9c425700 1 -- 172.21.15.40:0/3991910676 wait complete. 2024-02-16T19:24:33.292 INFO:tasks.workunit.client.1.smithi163.stdout:7/716: dwrite d11/f15 [0,4194304] 0 2024-02-16T19:24:33.293 INFO:tasks.workunit.client.1.smithi163.stdout:4/705: dwrite d11/d18/d1f/d29/d52/f5a [0,4194304] 0 2024-02-16T19:24:33.293 INFO:tasks.workunit.client.1.smithi163.stdout:4/706: truncate d11/f91 4703546 0 2024-02-16T19:24:33.294 INFO:tasks.workunit.client.1.smithi163.stdout:7/717: link d11/d24/d25/c2a d11/d24/d49/dac/cf5 0 2024-02-16T19:24:33.295 INFO:tasks.workunit.client.1.smithi163.stdout:0/967: dwrite d11/d21/d7c/f81 [0,4194304] 0 2024-02-16T19:24:33.297 INFO:tasks.workunit.client.1.smithi163.stdout:4/707: mkdir d11/d2d/d4f/d60/d79/dc9/de9 0 2024-02-16T19:24:33.300 INFO:tasks.workunit.client.1.smithi163.stdout:7/718: symlink d11/d50/d5d/d8b/d67/dd6/lf6 0 2024-02-16T19:24:33.300 INFO:tasks.workunit.client.1.smithi163.stdout:7/719: write d11/d24/d49/dac/feb [914698,115063] 0 2024-02-16T19:24:33.301 INFO:tasks.workunit.client.1.smithi163.stdout:0/968: creat d11/d12/d67/f144 x:0 0 0 2024-02-16T19:24:33.302 INFO:tasks.workunit.client.1.smithi163.stdout:0/969: write d11/d22/d7f/f83 [502693,4955] 0 2024-02-16T19:24:33.303 INFO:tasks.workunit.client.1.smithi163.stdout:4/708: link d11/d18/d1f/d34/f39 d11/d18/fea 0 2024-02-16T19:24:33.304 INFO:tasks.workunit.client.1.smithi163.stdout:7/720: creat d11/d24/d25/d4b/d8c/ff7 x:0 0 0 2024-02-16T19:24:33.305 INFO:tasks.workunit.client.1.smithi163.stdout:0/970: symlink d11/d22/d61/l145 0 2024-02-16T19:24:33.305 INFO:tasks.workunit.client.1.smithi163.stdout:0/971: write d11/d21/d50/d58/d6b/fe9 [917741,87477] 0 2024-02-16T19:24:33.305 INFO:tasks.workunit.client.1.smithi163.stdout:4/709: getdents d11/d18/d1f/d29/d52/d76/da2 0 2024-02-16T19:24:33.306 INFO:tasks.workunit.client.1.smithi163.stdout:7/721: symlink d11/d50/d5d/lf8 0 2024-02-16T19:24:33.307 INFO:tasks.workunit.client.1.smithi163.stdout:7/722: mkdir d11/d24/d34/d8d/df9 0 2024-02-16T19:24:33.307 INFO:tasks.workunit.client.1.smithi163.stdout:7/723: chown d11/d24/d25/d4b/lc7 0 1 2024-02-16T19:24:33.308 INFO:tasks.workunit.client.1.smithi163.stdout:7/724: dread - d11/ff0 zero size 2024-02-16T19:24:33.308 INFO:tasks.workunit.client.1.smithi163.stdout:7/725: write d11/d50/d5d/f97 [496950,48809] 0 2024-02-16T19:24:33.309 INFO:tasks.workunit.client.1.smithi163.stdout:7/726: truncate d11/d50/d5d/d8b/f4e 90811 0 2024-02-16T19:24:33.314 INFO:tasks.workunit.client.1.smithi163.stdout:7/727: dread d11/d24/d34/d8d/f92 [0,4194304] 0 2024-02-16T19:24:33.316 INFO:tasks.workunit.client.1.smithi163.stdout:0/972: dread d11/d21/d50/d58/d74/df8/f11a [0,4194304] 0 2024-02-16T19:24:33.317 INFO:tasks.workunit.client.1.smithi163.stdout:4/710: dread d11/f2f [0,4194304] 0 2024-02-16T19:24:33.317 INFO:tasks.workunit.client.1.smithi163.stdout:7/728: unlink d11/d24/d49/dac/feb 0 2024-02-16T19:24:33.321 INFO:tasks.workunit.client.1.smithi163.stdout:7/729: mknod d11/d24/cfa 0 2024-02-16T19:24:33.324 INFO:tasks.workunit.client.1.smithi163.stdout:7/730: link d11/d24/d49/f51 d11/d50/d5d/d8b/d67/dd6/ffb 0 2024-02-16T19:24:33.324 INFO:tasks.workunit.client.1.smithi163.stdout:7/731: creat d11/d24/d49/dc4/ffc x:0 0 0 2024-02-16T19:24:33.324 INFO:tasks.workunit.client.1.smithi163.stdout:7/732: chown d11/d50/c59 0 1 2024-02-16T19:24:33.324 INFO:tasks.workunit.client.1.smithi163.stdout:7/733: chown d11/c1e 3 1 2024-02-16T19:24:33.331 INFO:tasks.workunit.client.1.smithi163.stdout:0/973: dwrite d11/d21/d7c/fb7 [0,4194304] 0 2024-02-16T19:24:33.333 INFO:tasks.workunit.client.1.smithi163.stdout:7/734: write d11/d30/f5b [2795504,35405] 0 2024-02-16T19:24:33.333 INFO:tasks.workunit.client.1.smithi163.stdout:7/735: chown d11 1084 1 2024-02-16T19:24:33.333 INFO:tasks.workunit.client.1.smithi163.stdout:7/736: fsync d11/ff4 0 2024-02-16T19:24:33.334 INFO:tasks.workunit.client.1.smithi163.stdout:7/737: symlink d11/d50/d5d/d8b/lfd 0 2024-02-16T19:24:33.336 INFO:tasks.workunit.client.1.smithi163.stdout:7/738: rename d11/d26/f6d to d11/d24/d34/d8d/ffe 0 2024-02-16T19:24:33.337 INFO:tasks.workunit.client.1.smithi163.stdout:7/739: mkdir d11/d24/d34/d8d/dff 0 2024-02-16T19:24:33.338 INFO:tasks.workunit.client.1.smithi163.stdout:7/740: rename d11/d24/cfa to d11/d50/d5d/d8b/d67/db2/c100 0 2024-02-16T19:24:33.340 INFO:tasks.workunit.client.1.smithi163.stdout:7/741: rename d11/d24/d25/c9d to d11/d24/d25/d4b/d8c/dee/c101 0 2024-02-16T19:24:33.340 INFO:tasks.workunit.client.1.smithi163.stdout:7/742: stat fb 0 2024-02-16T19:24:33.340 INFO:tasks.workunit.client.1.smithi163.stdout:7/743: stat d11/d24/d25/d4b/d66/dc1/de7 0 2024-02-16T19:24:33.341 INFO:tasks.workunit.client.1.smithi163.stdout:7/744: mknod d11/d50/d5d/d8b/d67/c102 0 2024-02-16T19:24:33.342 INFO:tasks.workunit.client.1.smithi163.stdout:7/745: symlink d11/d24/d25/d4b/d66/dc1/de7/l103 0 2024-02-16T19:24:33.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.405+0000 7fb6892d2700 1 -- 172.21.15.40:0/3665490159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 msgr2=0x7fb6840fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.405+0000 7fb6892d2700 1 --2- 172.21.15.40:0/3665490159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 0x7fb6840fc250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb674004940 tx=0x7fb674009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:33.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.405+0000 7fb6892d2700 1 -- 172.21.15.40:0/3665490159 shutdown_connections 2024-02-16T19:24:33.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.405+0000 7fb6892d2700 1 --2- 172.21.15.40:0/3665490159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6840fcfb0 0x7fb6840fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.405+0000 7fb6892d2700 1 --2- 172.21.15.40:0/3665490159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 0x7fb6840fc250 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.405+0000 7fb6892d2700 1 -- 172.21.15.40:0/3665490159 >> 172.21.15.40:0/3665490159 conn(0x7fb6840f7520 msgr2=0x7fb6840f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:33.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb6892d2700 1 -- 172.21.15.40:0/3665490159 shutdown_connections 2024-02-16T19:24:33.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb6892d2700 1 -- 172.21.15.40:0/3665490159 wait complete. 2024-02-16T19:24:33.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb6892d2700 1 Processor -- start 2024-02-16T19:24:33.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb6892d2700 1 -- start start 2024-02-16T19:24:33.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb6892d2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 0x7fb6840fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb6892d2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6840fcfb0 0x7fb6840fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb6892d2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840fe980 con 0x7fb6840fbe40 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb6892d2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840feaf0 con 0x7fb6840fcfb0 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb682ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 0x7fb6840fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb682ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 0x7fb6840fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37148/0 (socket says 172.21.15.40:37148) 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.406+0000 7fb682ffd700 1 -- 172.21.15.40:0/1227281087 learned_addr learned my addr 172.21.15.40:0/1227281087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb6827fc700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6840fcfb0 0x7fb6840fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb682ffd700 1 -- 172.21.15.40:0/1227281087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6840fcfb0 msgr2=0x7fb6840fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb682ffd700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6840fcfb0 0x7fb6840fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb682ffd700 1 -- 172.21.15.40:0/1227281087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674005040 con 0x7fb6840fbe40 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb682ffd700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 0x7fb6840fff00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb674000c00 tx=0x7fb674009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb66b7fe700 1 -- 172.21.15.40:0/1227281087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb674021070 con 0x7fb6840fbe40 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb66b7fe700 1 -- 172.21.15.40:0/1227281087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb67401c400 con 0x7fb6840fbe40 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6840fed70 con 0x7fb6840fbe40 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb66b7fe700 1 -- 172.21.15.40:0/1227281087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67400e070 con 0x7fb6840fbe40 2024-02-16T19:24:33.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.407+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6841158d0 con 0x7fb6840fbe40 2024-02-16T19:24:33.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.408+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68404bed0 con 0x7fb6840fbe40 2024-02-16T19:24:33.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.409+0000 7fb66b7fe700 1 -- 172.21.15.40:0/1227281087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb67401ca50 con 0x7fb6840fbe40 2024-02-16T19:24:33.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.409+0000 7fb66b7fe700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb66c0692a0 0x7fb66c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.409+0000 7fb66b7fe700 1 -- 172.21.15.40:0/1227281087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb6740883e0 con 0x7fb6840fbe40 2024-02-16T19:24:33.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.409+0000 7fb6827fc700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb66c0692a0 0x7fb66c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.410+0000 7fb6827fc700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb66c0692a0 0x7fb66c06b750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb678009b80 tx=0x7fb67800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:33.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.414+0000 7fb66b7fe700 1 -- 172.21.15.40:0/1227281087 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb674058150 con 0x7fb6840fbe40 2024-02-16T19:24:33.453 INFO:tasks.workunit.client.1.smithi163.stdout:3/792: sync 2024-02-16T19:24:33.453 INFO:tasks.workunit.client.1.smithi163.stdout:1/818: sync 2024-02-16T19:24:33.453 INFO:tasks.workunit.client.1.smithi163.stdout:8/755: sync 2024-02-16T19:24:33.453 INFO:tasks.workunit.client.1.smithi163.stdout:6/690: sync 2024-02-16T19:24:33.453 INFO:tasks.workunit.client.1.smithi163.stdout:1/819: chown d7/d5b/fa4 65 1 2024-02-16T19:24:33.453 INFO:tasks.workunit.client.1.smithi163.stdout:1/820: getdents d7/dc/d2d/d3f 0 2024-02-16T19:24:33.454 INFO:tasks.workunit.client.1.smithi163.stdout:6/691: mknod d0/dd/d7f/d81/d85/cdf 0 2024-02-16T19:24:33.455 INFO:tasks.workunit.client.1.smithi163.stdout:8/756: creat d0/d24/d2c/ddf/de5/ff9 x:0 0 0 2024-02-16T19:24:33.456 INFO:tasks.workunit.client.1.smithi163.stdout:1/821: creat d7/dc/d2d/d90/f116 x:0 0 0 2024-02-16T19:24:33.456 INFO:tasks.workunit.client.1.smithi163.stdout:1/822: creat d7/d20/d93/f117 x:0 0 0 2024-02-16T19:24:33.457 INFO:tasks.workunit.client.1.smithi163.stdout:6/692: symlink d0/le0 0 2024-02-16T19:24:33.457 INFO:tasks.workunit.client.1.smithi163.stdout:6/693: fsync d0/dd/d7f/d2e/fbd 0 2024-02-16T19:24:33.457 INFO:tasks.workunit.client.1.smithi163.stdout:3/793: dread d0/dc/d50/f8b [0,4194304] 0 2024-02-16T19:24:33.457 INFO:tasks.workunit.client.1.smithi163.stdout:3/794: chown d0/dc/d1c/d21/l2a 1 1 2024-02-16T19:24:33.460 INFO:tasks.workunit.client.1.smithi163.stdout:6/694: symlink d0/d6/df/d7b/le1 0 2024-02-16T19:24:33.460 INFO:tasks.workunit.client.1.smithi163.stdout:1/823: dread d7/d20/d93/fb3 [0,4194304] 0 2024-02-16T19:24:33.460 INFO:tasks.workunit.client.1.smithi163.stdout:1/824: stat d7/d20/d21/d74/dab/ff8 0 2024-02-16T19:24:33.462 INFO:tasks.workunit.client.1.smithi163.stdout:1/825: rename d7/dc/d2d/d32/d87/c9d to d7/d20/d93/dd2/c118 0 2024-02-16T19:24:33.462 INFO:tasks.workunit.client.1.smithi163.stdout:1/826: dread - d7/dc/d2d/d32/d87/da6/fd6 zero size 2024-02-16T19:24:33.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/827: link d7/dc/l13 d7/dc/d2d/d32/d10f/l119 0 2024-02-16T19:24:33.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/828: write d7/dc/d2d/d32/d3b/fdb [970962,52474] 0 2024-02-16T19:24:33.465 INFO:tasks.workunit.client.1.smithi163.stdout:1/829: read - d7/dc/d2d/d32/d87/da6/f103 zero size 2024-02-16T19:24:33.468 INFO:tasks.workunit.client.1.smithi163.stdout:1/830: creat d7/d20/f11a x:0 0 0 2024-02-16T19:24:33.468 INFO:tasks.workunit.client.1.smithi163.stdout:1/831: truncate d7/f7d 753919 0 2024-02-16T19:24:33.488 INFO:tasks.workunit.client.1.smithi163.stdout:6/695: dwrite d0/d18/d77/f9e [0,4194304] 0 2024-02-16T19:24:33.490 INFO:tasks.workunit.client.1.smithi163.stdout:3/795: dwrite d0/dc/d50/fd3 [0,4194304] 0 2024-02-16T19:24:33.490 INFO:tasks.workunit.client.1.smithi163.stdout:8/757: dwrite d0/d24/d2c/f90 [0,4194304] 0 2024-02-16T19:24:33.490 INFO:tasks.workunit.client.1.smithi163.stdout:8/758: stat d0/d6f/fc8 0 2024-02-16T19:24:33.490 INFO:tasks.workunit.client.1.smithi163.stdout:3/796: write d0/d83/da5/db9/fd1 [731706,62391] 0 2024-02-16T19:24:33.490 INFO:tasks.workunit.client.1.smithi163.stdout:3/797: fsync d0/dc/d50/d66/d80/f10d 0 2024-02-16T19:24:33.490 INFO:tasks.workunit.client.1.smithi163.stdout:3/798: truncate d0/dc/d50/d66/d93/f114 936576 0 2024-02-16T19:24:33.493 INFO:tasks.workunit.client.1.smithi163.stdout:8/759: truncate d0/faa 2504024 0 2024-02-16T19:24:33.493 INFO:tasks.workunit.client.1.smithi163.stdout:8/760: chown d0/d7/d18/l3e 74 1 2024-02-16T19:24:33.501 INFO:tasks.workunit.client.1.smithi163.stdout:6/696: write d0/d6/df/d20/d69/fa6 [4166850,20759] 0 2024-02-16T19:24:33.502 INFO:tasks.workunit.client.1.smithi163.stdout:6/697: read d0/d6/df/d20/d40/f59 [132615,32908] 0 2024-02-16T19:24:33.502 INFO:tasks.workunit.client.1.smithi163.stdout:1/832: dwrite d7/f7d [0,4194304] 0 2024-02-16T19:24:33.506 INFO:tasks.workunit.client.1.smithi163.stdout:1/833: creat d7/dc/d2d/d32/f11b x:0 0 0 2024-02-16T19:24:33.506 INFO:tasks.workunit.client.1.smithi163.stdout:1/834: chown d7/l69 8048013 1 2024-02-16T19:24:33.506 INFO:tasks.workunit.client.1.smithi163.stdout:1/835: creat d7/d20/d21/d4e/f11c x:0 0 0 2024-02-16T19:24:33.506 INFO:tasks.workunit.client.1.smithi163.stdout:1/836: read d7/d5b/d63/f7b [3838351,129352] 0 2024-02-16T19:24:33.506 INFO:tasks.workunit.client.1.smithi163.stdout:1/837: write d7/d20/d21/fbb [713362,20242] 0 2024-02-16T19:24:33.507 INFO:tasks.workunit.client.1.smithi163.stdout:8/761: write d0/d7/d18/d53/ded/fb9 [269601,88274] 0 2024-02-16T19:24:33.520 INFO:tasks.workunit.client.1.smithi163.stdout:6/698: dwrite d0/d6/f8 [4194304,4194304] 0 2024-02-16T19:24:33.522 INFO:tasks.workunit.client.1.smithi163.stdout:6/699: mknod d0/d6/df/d20/ce2 0 2024-02-16T19:24:33.524 INFO:tasks.workunit.client.1.smithi163.stdout:6/700: rmdir d0/dd/dbc 39 2024-02-16T19:24:33.524 INFO:tasks.workunit.client.1.smithi163.stdout:6/701: readlink d0/le0 0 2024-02-16T19:24:33.526 INFO:tasks.workunit.client.1.smithi163.stdout:6/702: mkdir d0/d18/d77/dc9/d74/de3 0 2024-02-16T19:24:33.526 INFO:tasks.workunit.client.1.smithi163.stdout:6/703: write d0/dd/d7f/d15/d1c/f56 [1120387,119806] 0 2024-02-16T19:24:33.526 INFO:tasks.workunit.client.1.smithi163.stdout:6/704: dread - d0/fdd zero size 2024-02-16T19:24:33.529 INFO:tasks.workunit.client.1.smithi163.stdout:6/705: rename d0/dd/d7f/d15/d1c/d65/d63/f6b to d0/dd/d7f/d81/fe4 0 2024-02-16T19:24:33.534 INFO:tasks.workunit.client.1.smithi163.stdout:6/706: rename d0/dd/d7f/d2e/d33 to d0/dd/dbc/ddc/de5 0 2024-02-16T19:24:33.542 INFO:tasks.workunit.client.1.smithi163.stdout:6/707: write d0/d6/f45 [929285,73045] 0 2024-02-16T19:24:33.544 INFO:tasks.workunit.client.1.smithi163.stdout:6/708: symlink d0/d6/df/d20/d69/le6 0 2024-02-16T19:24:33.544 INFO:tasks.workunit.client.1.smithi163.stdout:6/709: write d0/dd/d7f/d15/d1c/f99 [570140,24620] 0 2024-02-16T19:24:33.546 INFO:tasks.workunit.client.1.smithi163.stdout:6/710: link d0/d18/d77/fdb d0/d18/d77/d70/fe7 0 2024-02-16T19:24:33.548 INFO:tasks.workunit.client.1.smithi163.stdout:6/711: rename d0/d6/df/d7b/le1 to d0/dd/d7f/d81/le8 0 2024-02-16T19:24:33.548 INFO:tasks.workunit.client.1.smithi163.stdout:6/712: readlink d0/d6/df/d20/d40/lc5 0 2024-02-16T19:24:33.548 INFO:tasks.workunit.client.1.smithi163.stdout:6/713: getdents d0/dd/d7f/d15/d1c/d65/d63 0 2024-02-16T19:24:33.557 INFO:tasks.workunit.client.1.smithi163.stdout:6/714: write d0/d6/f8 [4978652,82317] 0 2024-02-16T19:24:33.557 INFO:tasks.workunit.client.1.smithi163.stdout:6/715: write d0/d18/d94/fac [894683,111599] 0 2024-02-16T19:24:33.558 INFO:tasks.workunit.client.1.smithi163.stdout:6/716: creat d0/dd/d7f/d15/d1c/dca/fe9 x:0 0 0 2024-02-16T19:24:33.560 INFO:tasks.workunit.client.1.smithi163.stdout:6/717: rename d0/d6/df/l35 to d0/d6/df/d20/d40/d1e/lea 0 2024-02-16T19:24:33.560 INFO:tasks.workunit.client.1.smithi163.stdout:6/718: mkdir d0/dd/dbc/ddc/deb 0 2024-02-16T19:24:33.561 INFO:tasks.workunit.client.1.smithi163.stdout:6/719: creat d0/d18/d77/dc9/fec x:0 0 0 2024-02-16T19:24:33.570 INFO:tasks.workunit.client.1.smithi163.stdout:6/720: dread d0/d18/f58 [0,4194304] 0 2024-02-16T19:24:33.579 INFO:tasks.workunit.client.1.smithi163.stdout:6/721: dread d0/d18/f48 [0,4194304] 0 2024-02-16T19:24:33.579 INFO:tasks.workunit.client.1.smithi163.stdout:6/722: dread - d0/d6/df/d20/d69/db4/fb9 zero size 2024-02-16T19:24:33.587 INFO:tasks.workunit.client.1.smithi163.stdout:6/723: dread d0/d18/d77/fbf [0,4194304] 0 2024-02-16T19:24:33.587 INFO:tasks.workunit.client.1.smithi163.stdout:6/724: readlink d0/lc4 0 2024-02-16T19:24:33.596 INFO:tasks.workunit.client.1.smithi163.stdout:6/725: write d0/d6/f92 [423551,10019] 0 2024-02-16T19:24:33.598 INFO:tasks.workunit.client.1.smithi163.stdout:6/726: fsync d0/d6/f8 0 2024-02-16T19:24:33.619 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:33 smithi163 ceph-mon[36452]: pgmap v255: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 280 MiB/s rd, 343 MiB/s wr, 455 op/s 2024-02-16T19:24:33.626 INFO:tasks.workunit.client.1.smithi163.stdout:2/877: sync 2024-02-16T19:24:33.627 INFO:tasks.workunit.client.1.smithi163.stdout:2/878: symlink d2/d26/l12d 0 2024-02-16T19:24:33.627 INFO:tasks.workunit.client.1.smithi163.stdout:2/879: chown d2/ddb/d7c/f99 165658339 1 2024-02-16T19:24:33.628 INFO:tasks.workunit.client.1.smithi163.stdout:2/880: truncate d2/d2a/d2d/f6a 4312171 0 2024-02-16T19:24:33.631 INFO:tasks.workunit.client.1.smithi163.stdout:2/881: link d2/d2a/d2d/d73/c95 d2/d26/d97/c12e 0 2024-02-16T19:24:33.632 INFO:tasks.workunit.client.1.smithi163.stdout:2/882: dread d2/d26/d54/d58/d98/fab [0,4194304] 0 2024-02-16T19:24:33.632 INFO:tasks.workunit.client.1.smithi163.stdout:2/883: symlink d2/d2a/d39/d117/d7f/l12f 0 2024-02-16T19:24:33.633 INFO:tasks.workunit.client.1.smithi163.stdout:2/884: mkdir d2/d2a/d39/d130 0 2024-02-16T19:24:33.633 INFO:tasks.workunit.client.1.smithi163.stdout:2/885: stat d2/d26/d54/d58 0 2024-02-16T19:24:33.633 INFO:tasks.workunit.client.1.smithi163.stdout:2/886: write d2/d26/d54/d58/fcc [4479637,124814] 0 2024-02-16T19:24:33.634 INFO:tasks.workunit.client.1.smithi163.stdout:2/887: symlink d2/d2a/d2d/l131 0 2024-02-16T19:24:33.635 INFO:tasks.workunit.client.1.smithi163.stdout:2/888: mkdir d2/ddb/d7c/d132 0 2024-02-16T19:24:33.636 INFO:tasks.workunit.client.1.smithi163.stdout:2/889: symlink d2/ddb/d7c/d9c/l133 0 2024-02-16T19:24:33.637 INFO:tasks.workunit.client.1.smithi163.stdout:2/890: chown d2/d26/d54/d58/dff/cd3 755809 1 2024-02-16T19:24:33.637 INFO:tasks.workunit.client.1.smithi163.stdout:2/891: creat d2/d26/d54/d58/dff/f134 x:0 0 0 2024-02-16T19:24:33.639 INFO:tasks.workunit.client.1.smithi163.stdout:2/892: link d2/d44/l126 d2/d2a/l135 0 2024-02-16T19:24:33.641 INFO:tasks.workunit.client.1.smithi163.stdout:2/893: dread d2/d2a/d2d/f92 [0,4194304] 0 2024-02-16T19:24:33.641 INFO:tasks.workunit.client.1.smithi163.stdout:2/894: write d2/d2a/d2d/d73/d84/f8e [2775007,36376] 0 2024-02-16T19:24:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:33 smithi040 ceph-mon[29201]: pgmap v255: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 280 MiB/s rd, 343 MiB/s wr, 455 op/s 2024-02-16T19:24:33.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.653+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb684066660 con 0x7fb66c0692a0 2024-02-16T19:24:33.654 INFO:tasks.workunit.client.1.smithi163.stdout:2/895: dwrite d2/d26/f2f [0,4194304] 0 2024-02-16T19:24:33.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.665+0000 7fb66b7fe700 1 -- 172.21.15.40:0/1227281087 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3344 (secure 0 0 0) 0x7fb684066660 con 0x7fb66c0692a0 2024-02-16T19:24:33.666 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:24:33.666 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (6m) 2m ago 8m 25.1M - 0.25.0 c8568f914cd2 aadccae1ccec 2024-02-16T19:24:33.666 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (8m) 2m ago 8m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:24:33.666 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (7m) 2m ago 7m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:24:33.666 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (6m) 2m ago 7m 84.5M - 9.4.7 954c08fa6188 8447b5580554 2024-02-16T19:24:33.666 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (2m) 2m ago 2m 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:24:33.666 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (2m) 2m ago 2m 10.4M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (2m) 2m ago 2m 14.4M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (2m) 2m ago 2m 14.3M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:9283 running (9m) 2m ago 9m 482M - 17.2.7-513-gb88fae66 870a0944ccb8 4e9e9bb4389a 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283 starting - - - - 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (9m) 2m ago 9m 44.9M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (7m) 2m ago 7m 35.4M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (8m) 2m ago 8m 14.0M - 1.5.0 0da6a335fe13 61d16f3f00d4 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (7m) 2m ago 7m 16.7M - 1.5.0 0da6a335fe13 c45ebf41f1b8 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (6m) 2m ago 6m 47.9M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (5m) 2m ago 5m 46.4M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (5m) 2m ago 5m 45.3M 4289M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:24:33.667 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (4m) 2m ago 4m 44.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:24:33.668 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (4m) 2m ago 4m 41.1M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:24:33.668 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (4m) 2m ago 4m 44.7M 5312M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:24:33.668 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (6m) 2m ago 7m 51.4M - 2.43.0 a07b618ecd1d 557bd885dbc5 2024-02-16T19:24:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb66c0692a0 msgr2=0x7fb66c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb66c0692a0 0x7fb66c06b750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb678009b80 tx=0x7fb67800b040 comp rx=0 tx=0).stop 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 msgr2=0x7fb6840fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 0x7fb6840fff00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb674000c00 tx=0x7fb674009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 shutdown_connections 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fb66c0692a0 0x7fb66c06b750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6840fbe40 0x7fb6840fff00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 --2- 172.21.15.40:0/1227281087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6840fcfb0 0x7fb6840fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 >> 172.21.15.40:0/1227281087 conn(0x7fb6840f7520 msgr2=0x7fb684065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 shutdown_connections 2024-02-16T19:24:33.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.667+0000 7fb6892d2700 1 -- 172.21.15.40:0/1227281087 wait complete. 2024-02-16T19:24:33.670 INFO:tasks.workunit.client.1.smithi163.stdout:2/896: dwrite d2/d2a/d39/f57 [0,4194304] 0 2024-02-16T19:24:33.678 INFO:tasks.workunit.client.1.smithi163.stdout:2/897: dread d2/d26/f28 [0,4194304] 0 2024-02-16T19:24:33.678 INFO:tasks.workunit.client.1.smithi163.stdout:2/898: creat d2/d26/d54/d58/da6/f136 x:0 0 0 2024-02-16T19:24:33.679 INFO:tasks.workunit.client.1.smithi163.stdout:2/899: write d2/d2a/d9f/fb9 [471525,90686] 0 2024-02-16T19:24:33.680 INFO:tasks.workunit.client.1.smithi163.stdout:2/900: creat d2/f137 x:0 0 0 2024-02-16T19:24:33.681 INFO:tasks.workunit.client.1.smithi163.stdout:2/901: dread d2/dce/ffc [0,4194304] 0 2024-02-16T19:24:33.682 INFO:tasks.workunit.client.1.smithi163.stdout:2/902: dread d2/d26/d54/f116 [0,4194304] 0 2024-02-16T19:24:33.683 INFO:tasks.workunit.client.1.smithi163.stdout:2/903: read d2/d2a/d39/fae [175807,70837] 0 2024-02-16T19:24:33.683 INFO:tasks.workunit.client.1.smithi163.stdout:2/904: creat d2/d44/f138 x:0 0 0 2024-02-16T19:24:33.684 INFO:tasks.workunit.client.1.smithi163.stdout:2/905: truncate d2/d26/d54/d58/d7d/f83 816904 0 2024-02-16T19:24:33.684 INFO:tasks.workunit.client.1.smithi163.stdout:2/906: fdatasync d2/d2a/d39/d66/f77 0 2024-02-16T19:24:33.720 INFO:tasks.workunit.client.1.smithi163.stdout:5/924: sync 2024-02-16T19:24:33.721 INFO:tasks.workunit.client.1.smithi163.stdout:5/925: rmdir d0/d2/d34/d44 39 2024-02-16T19:24:33.722 INFO:tasks.workunit.client.1.smithi163.stdout:5/926: rename d0/d1c/c8b to d0/d10d/c133 0 2024-02-16T19:24:33.723 INFO:tasks.workunit.client.1.smithi163.stdout:5/927: mkdir d0/d2/d134 0 2024-02-16T19:24:33.723 INFO:tasks.workunit.client.1.smithi163.stdout:5/928: symlink d0/d22/d28/d36/d3c/d4f/de6/l135 0 2024-02-16T19:24:33.725 INFO:tasks.workunit.client.1.smithi163.stdout:5/929: unlink d0/d2/d34/f6d 0 2024-02-16T19:24:33.726 INFO:tasks.workunit.client.1.smithi163.stdout:5/930: creat d0/d2/d34/d44/dd4/f136 x:0 0 0 2024-02-16T19:24:33.727 INFO:tasks.workunit.client.1.smithi163.stdout:5/931: rename d0/d22/d28/d36/f6b to d0/d1c/d5a/da5/f137 0 2024-02-16T19:24:33.729 INFO:tasks.workunit.client.1.smithi163.stdout:5/932: rename d0/d22/f33 to d0/d1c/d5a/da5/f138 0 2024-02-16T19:24:33.729 INFO:tasks.workunit.client.1.smithi163.stdout:5/933: stat d0/d22/d3b/f116 0 2024-02-16T19:24:33.730 INFO:tasks.workunit.client.1.smithi163.stdout:5/934: creat d0/d2/d34/d44/f139 x:0 0 0 2024-02-16T19:24:33.730 INFO:tasks.workunit.client.1.smithi163.stdout:5/935: stat d0/d1c/c20 0 2024-02-16T19:24:33.732 INFO:tasks.workunit.client.1.smithi163.stdout:5/936: rmdir d0/d2/d34/d44/d64/dd9/d10e/d12a 0 2024-02-16T19:24:33.732 INFO:tasks.workunit.client.1.smithi163.stdout:5/937: rmdir d0/d22/d28/d111/da8 39 2024-02-16T19:24:33.733 INFO:tasks.workunit.client.1.smithi163.stdout:5/938: rename d0/d2/d34/d44/d53/de2/f10b to d0/d1c/dbe/de3/f13a 0 2024-02-16T19:24:33.755 INFO:tasks.workunit.client.1.smithi163.stdout:5/939: dwrite d0/d22/d28/d36/db7/fb8 [0,4194304] 0 2024-02-16T19:24:33.755 INFO:tasks.workunit.client.1.smithi163.stdout:5/940: truncate d0/d22/d3b/f116 1749728 0 2024-02-16T19:24:33.755 INFO:tasks.workunit.client.1.smithi163.stdout:5/941: stat d0/d22/d28/d36/dc4/c12e 0 2024-02-16T19:24:33.755 INFO:tasks.workunit.client.1.smithi163.stdout:5/942: creat d0/d22/d28/d36/db7/f13b x:0 0 0 2024-02-16T19:24:33.757 INFO:tasks.workunit.client.1.smithi163.stdout:5/943: symlink d0/d1c/d5a/l13c 0 2024-02-16T19:24:33.757 INFO:tasks.workunit.client.1.smithi163.stdout:5/944: getdents d0/d11c 0 2024-02-16T19:24:33.764 INFO:tasks.workunit.client.1.smithi163.stdout:5/945: dread d0/d2/fd5 [0,4194304] 0 2024-02-16T19:24:33.764 INFO:tasks.workunit.client.1.smithi163.stdout:5/946: dread - d0/d22/d28/d36/d3c/fff zero size 2024-02-16T19:24:33.764 INFO:tasks.workunit.client.1.smithi163.stdout:5/947: readlink d0/d22/d3b/l46 0 2024-02-16T19:24:33.768 INFO:tasks.workunit.client.1.smithi163.stdout:5/948: creat d0/d2/d34/d44/d64/dd9/d10e/f13d x:0 0 0 2024-02-16T19:24:33.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.786+0000 7fbaa1c77700 1 -- 172.21.15.40:0/1687172842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 msgr2=0x7fba9c0fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.786+0000 7fbaa1c77700 1 --2- 172.21.15.40:0/1687172842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 0x7fba9c0fb630 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fba8c004940 tx=0x7fba8c009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:33.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.786+0000 7fbaa1c77700 1 -- 172.21.15.40:0/1687172842 shutdown_connections 2024-02-16T19:24:33.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.786+0000 7fbaa1c77700 1 --2- 172.21.15.40:0/1687172842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba9c0fe2b0 0x7fba9c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.786+0000 7fbaa1c77700 1 --2- 172.21.15.40:0/1687172842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 0x7fba9c0fb630 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.786+0000 7fbaa1c77700 1 -- 172.21.15.40:0/1687172842 >> 172.21.15.40:0/1687172842 conn(0x7fba9c0f6ca0 msgr2=0x7fba9c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.786+0000 7fbaa1c77700 1 -- 172.21.15.40:0/1687172842 shutdown_connections 2024-02-16T19:24:33.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.786+0000 7fbaa1c77700 1 -- 172.21.15.40:0/1687172842 wait complete. 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.787+0000 7fbaa1c77700 1 Processor -- start 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.787+0000 7fbaa1c77700 1 -- start start 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.787+0000 7fbaa1c77700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 0x7fba9c0fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.787+0000 7fbaa1c77700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba9c0fe2b0 0x7fba9c0fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.787+0000 7fbaa1c77700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0fc7c0 con 0x7fba9c0fb220 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.787+0000 7fbaa1c77700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0fc930 con 0x7fba9c0fe2b0 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.787+0000 7fba9b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 0x7fba9c0fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.788+0000 7fba9b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 0x7fba9c0fdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37158/0 (socket says 172.21.15.40:37158) 2024-02-16T19:24:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.788+0000 7fba9b7fe700 1 -- 172.21.15.40:0/448262867 learned_addr learned my addr 172.21.15.40:0/448262867 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.788+0000 7fba9affd700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba9c0fe2b0 0x7fba9c0fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.788+0000 7fba9b7fe700 1 -- 172.21.15.40:0/448262867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba9c0fe2b0 msgr2=0x7fba9c0fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.788+0000 7fba9b7fe700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba9c0fe2b0 0x7fba9c0fc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.788+0000 7fba9b7fe700 1 -- 172.21.15.40:0/448262867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c005040 con 0x7fba9c0fb220 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.788+0000 7fba9b7fe700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 0x7fba9c0fdd40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009150 tx=0x7fba8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.789+0000 7fba83fff700 1 -- 172.21.15.40:0/448262867 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8c00e040 con 0x7fba9c0fb220 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.789+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c0fcbb0 con 0x7fba9c0fb220 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.789+0000 7fba83fff700 1 -- 172.21.15.40:0/448262867 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba8c00bbf0 con 0x7fba9c0fb220 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.789+0000 7fba83fff700 1 -- 172.21.15.40:0/448262867 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8c025640 con 0x7fba9c0fb220 2024-02-16T19:24:33.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.789+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c0fd030 con 0x7fba9c0fb220 2024-02-16T19:24:33.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.790+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9c04bed0 con 0x7fba9c0fb220 2024-02-16T19:24:33.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.791+0000 7fba83fff700 1 -- 172.21.15.40:0/448262867 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fba8c00bd60 con 0x7fba9c0fb220 2024-02-16T19:24:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.791+0000 7fba83fff700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fba840692a0 0x7fba8406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.791+0000 7fba83fff700 1 -- 172.21.15.40:0/448262867 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fba8c088400 con 0x7fba9c0fb220 2024-02-16T19:24:33.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.793+0000 7fba9affd700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fba840692a0 0x7fba8406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:33.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.795+0000 7fba9affd700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fba840692a0 0x7fba8406b750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba90004990 tx=0x7fba90009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:33.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:33.799+0000 7fba83fff700 1 -- 172.21.15.40:0/448262867 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fba8c058150 con 0x7fba9c0fb220 2024-02-16T19:24:33.862 INFO:tasks.workunit.client.1.smithi163.stdout:4/711: sync 2024-02-16T19:24:33.862 INFO:tasks.workunit.client.1.smithi163.stdout:4/712: dread - d11/d18/d1f/d23/d3a/d4a/f6e zero size 2024-02-16T19:24:33.862 INFO:tasks.workunit.client.1.smithi163.stdout:4/713: creat d11/d2d/d4f/d5f/feb x:0 0 0 2024-02-16T19:24:33.862 INFO:tasks.workunit.client.1.smithi163.stdout:4/714: dread - d11/d18/d1f/d23/d3a/d4a/fd2 zero size 2024-02-16T19:24:33.862 INFO:tasks.workunit.client.1.smithi163.stdout:4/715: chown d11/d2d/d4f/d5f/d70/d74/fd4 336 1 2024-02-16T19:24:33.863 INFO:tasks.workunit.client.1.smithi163.stdout:4/716: read d11/d2d/d4f/d60/fc6 [240790,62821] 0 2024-02-16T19:24:33.863 INFO:tasks.workunit.client.1.smithi163.stdout:4/717: chown d11/d18/d1f/d29/d52/l88 0 1 2024-02-16T19:24:33.867 INFO:tasks.workunit.client.1.smithi163.stdout:4/718: creat d11/d2d/d4f/d5f/d70/d74/d75/fec x:0 0 0 2024-02-16T19:24:33.869 INFO:tasks.workunit.client.1.smithi163.stdout:4/719: mknod d11/d18/d1f/d23/d3a/ced 0 2024-02-16T19:24:33.869 INFO:tasks.workunit.client.1.smithi163.stdout:4/720: write d11/d3d/fbc [4650,76691] 0 2024-02-16T19:24:33.871 INFO:tasks.workunit.client.1.smithi163.stdout:4/721: creat d11/d18/d1f/d29/d52/dd6/fee x:0 0 0 2024-02-16T19:24:33.873 INFO:tasks.workunit.client.1.smithi163.stdout:4/722: rename d11/d18/d1f/d29/d52/d76/da2/ce2 to d11/d2d/d4f/d60/d79/dc9/cef 0 2024-02-16T19:24:33.876 INFO:tasks.workunit.client.1.smithi163.stdout:4/723: link d11/f91 d11/d2d/d4f/d5f/ff0 0 2024-02-16T19:24:33.876 INFO:tasks.workunit.client.1.smithi163.stdout:4/724: fsync d11/d18/d1f/d29/f36 0 2024-02-16T19:24:33.879 INFO:tasks.workunit.client.1.smithi163.stdout:4/725: getdents d11/d18/d1f/d29/d52/d57/d6c/d7c 0 2024-02-16T19:24:33.886 INFO:tasks.workunit.client.1.smithi163.stdout:4/726: dread d11/d18/d1f/d29/d52/d57/f84 [0,4194304] 0 2024-02-16T19:24:33.886 INFO:tasks.workunit.client.1.smithi163.stdout:4/727: creat d11/d2d/d4f/d60/d79/ff1 x:0 0 0 2024-02-16T19:24:33.900 INFO:tasks.workunit.client.1.smithi163.stdout:4/728: dwrite d11/d18/d1f/d29/d52/f5a [4194304,4194304] 0 2024-02-16T19:24:33.902 INFO:tasks.workunit.client.1.smithi163.stdout:4/729: unlink d11/f2f 0 2024-02-16T19:24:33.920 INFO:tasks.workunit.client.1.smithi163.stdout:4/730: dwrite d11/d18/fea [0,4194304] 0 2024-02-16T19:24:33.930 INFO:tasks.workunit.client.1.smithi163.stdout:4/731: write d11/d18/d1f/d29/d52/f6a [2023533,66561] 0 2024-02-16T19:24:33.931 INFO:tasks.workunit.client.1.smithi163.stdout:4/732: link d11/d2d/d4f/d60/c8b d11/d2d/d4f/d60/d79/cf2 0 2024-02-16T19:24:33.934 INFO:tasks.workunit.client.1.smithi163.stdout:4/733: unlink d11/d18/d1f/d29/d52/d57/d6c/d9d/cd9 0 2024-02-16T19:24:33.935 INFO:tasks.workunit.client.1.smithi163.stdout:4/734: symlink d11/d2d/d4f/d5f/d70/d74/d75/dca/dbf/lf3 0 2024-02-16T19:24:33.944 INFO:tasks.workunit.client.1.smithi163.stdout:4/735: dread d11/d2d/d4f/d60/d79/f90 [0,4194304] 0 2024-02-16T19:24:33.946 INFO:tasks.workunit.client.1.smithi163.stdout:4/736: dread f9 [0,4194304] 0 2024-02-16T19:24:33.948 INFO:tasks.workunit.client.1.smithi163.stdout:4/737: creat d11/d2d/d4f/d60/d79/dc9/ff4 x:0 0 0 2024-02-16T19:24:33.948 INFO:tasks.workunit.client.1.smithi163.stdout:4/738: write d11/d18/d66/f98 [1030977,128592] 0 2024-02-16T19:24:33.963 INFO:tasks.workunit.client.1.smithi163.stdout:4/739: dwrite d11/d18/d1f/d34/f51 [0,4194304] 0 2024-02-16T19:24:33.969 INFO:tasks.workunit.client.1.smithi163.stdout:4/740: dread d11/f14 [0,4194304] 0 2024-02-16T19:24:33.972 INFO:tasks.workunit.client.1.smithi163.stdout:4/741: mkdir d11/d18/d1f/d29/d52/d76/da2/df5 0 2024-02-16T19:24:33.972 INFO:tasks.workunit.client.1.smithi163.stdout:4/742: chown d11/d18/d1f/d29/d52/d76/fdd 5028 1 2024-02-16T19:24:33.981 INFO:tasks.workunit.client.1.smithi163.stdout:0/974: sync 2024-02-16T19:24:33.981 INFO:tasks.workunit.client.1.smithi163.stdout:0/975: fdatasync d11/f62 0 2024-02-16T19:24:33.984 INFO:tasks.workunit.client.1.smithi163.stdout:4/743: write d11/f4b [3214283,88439] 0 2024-02-16T19:24:34.015 INFO:tasks.workunit.client.1.smithi163.stdout:0/976: dwrite d11/d21/d7c/fce [0,4194304] 0 2024-02-16T19:24:34.015 INFO:tasks.workunit.client.1.smithi163.stdout:0/977: fdatasync d11/d22/d3e/f12d 0 2024-02-16T19:24:34.015 INFO:tasks.workunit.client.1.smithi163.stdout:4/744: dwrite d11/d18/d1f/d29/f3e [0,4194304] 0 2024-02-16T19:24:34.018 INFO:tasks.workunit.client.1.smithi163.stdout:0/978: symlink d11/d12/d67/l146 0 2024-02-16T19:24:34.021 INFO:tasks.workunit.client.1.smithi163.stdout:4/745: creat d11/d18/d1f/d29/d52/d57/d6c/d9c/ff6 x:0 0 0 2024-02-16T19:24:34.022 INFO:tasks.workunit.client.1.smithi163.stdout:4/746: creat d11/d18/d1f/d34/de5/ff7 x:0 0 0 2024-02-16T19:24:34.022 INFO:tasks.workunit.client.1.smithi163.stdout:4/747: truncate d11/d18/f43 5213404 0 2024-02-16T19:24:34.023 INFO:tasks.workunit.client.1.smithi163.stdout:4/748: mknod d11/d2d/d4f/d60/cf8 0 2024-02-16T19:24:34.025 INFO:tasks.workunit.client.1.smithi163.stdout:4/749: mknod d11/d2d/d4f/cf9 0 2024-02-16T19:24:34.026 INFO:tasks.workunit.client.1.smithi163.stdout:4/750: unlink d11/d2d/d4f/d60/d79/dc9/cef 0 2024-02-16T19:24:34.027 INFO:tasks.workunit.client.1.smithi163.stdout:4/751: mknod d11/d18/d1f/d29/d52/d57/d6c/d7c/dad/cfa 0 2024-02-16T19:24:34.029 INFO:tasks.workunit.client.1.smithi163.stdout:4/752: unlink d11/d18/d1f/d29/l9e 0 2024-02-16T19:24:34.029 INFO:tasks.workunit.client.1.smithi163.stdout:4/753: creat d11/d2d/d4f/d5f/d70/d74/d75/ffb x:0 0 0 2024-02-16T19:24:34.029 INFO:tasks.workunit.client.1.smithi163.stdout:4/754: creat d11/d2d/d4f/d5f/d70/d74/d75/dca/ffc x:0 0 0 2024-02-16T19:24:34.029 INFO:tasks.workunit.client.1.smithi163.stdout:4/755: fsync d11/d2d/d4f/d60/d79/fd3 0 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:3/799: sync 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:5/949: sync 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:1/838: sync 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:3/800: chown d0/dc/d94/d9e/le7 22584 1 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:5/950: readlink d0/d2/d5c/l9c 0 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:8/762: sync 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:6/727: sync 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:2/907: sync 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:2/908: fdatasync d2/f1c 0 2024-02-16T19:24:34.032 INFO:tasks.workunit.client.1.smithi163.stdout:7/746: sync 2024-02-16T19:24:34.033 INFO:tasks.workunit.client.1.smithi163.stdout:4/756: rename d11/d18/d1f/d29/d52/d76/da2/df5 to d11/d2d/d4f/dfd 0 2024-02-16T19:24:34.033 INFO:tasks.workunit.client.1.smithi163.stdout:4/757: chown d11/d2d/d4f/d5f/d70/d74/d75/dca/dbf 1844290 1 2024-02-16T19:24:34.033 INFO:tasks.workunit.client.1.smithi163.stdout:3/801: dread d0/dc/f6e [0,4194304] 0 2024-02-16T19:24:34.033 INFO:tasks.workunit.client.1.smithi163.stdout:5/951: rmdir d0/d1c/d5a/d121 0 2024-02-16T19:24:34.034 INFO:tasks.workunit.client.1.smithi163.stdout:1/839: truncate d7/dc/d2d/d3f/fad 227893 0 2024-02-16T19:24:34.034 INFO:tasks.workunit.client.1.smithi163.stdout:8/763: dread - d0/d7/d19/d1f/d30/d6a/db8/fcd zero size 2024-02-16T19:24:34.035 INFO:tasks.workunit.client.1.smithi163.stdout:6/728: getdents d0/d18/d94 0 2024-02-16T19:24:34.036 INFO:tasks.workunit.client.1.smithi163.stdout:2/909: mknod d2/d26/d97/d10f/c139 0 2024-02-16T19:24:34.039 INFO:tasks.workunit.client.1.smithi163.stdout:4/758: link d11/d18/d1f/d29/d52/ldf d11/d18/d1f/d29/d52/d57/d6c/dc3/lfe 0 2024-02-16T19:24:34.039 INFO:tasks.workunit.client.1.smithi163.stdout:4/759: creat d11/d2d/d4f/d5f/fff x:0 0 0 2024-02-16T19:24:34.039 INFO:tasks.workunit.client.1.smithi163.stdout:4/760: truncate d11/d18/d1f/d29/d52/d76/fb8 253879 0 2024-02-16T19:24:34.042 INFO:tasks.workunit.client.1.smithi163.stdout:3/802: creat d0/dc/d1c/d21/deb/f11c x:0 0 0 2024-02-16T19:24:34.042 INFO:tasks.workunit.client.1.smithi163.stdout:3/803: truncate d0/dc/d1c/d91/fc3 819922 0 2024-02-16T19:24:34.043 INFO:tasks.workunit.client.1.smithi163.stdout:5/952: symlink d0/d22/d28/l13e 0 2024-02-16T19:24:34.044 INFO:tasks.workunit.client.1.smithi163.stdout:6/729: write d0/d18/d77/fbf [3596761,120144] 0 2024-02-16T19:24:34.044 INFO:tasks.workunit.client.1.smithi163.stdout:6/730: fdatasync d0/d6/f31 0 2024-02-16T19:24:34.044 INFO:tasks.workunit.client.1.smithi163.stdout:1/840: getdents d7/d20/d93/dd2 0 2024-02-16T19:24:34.046 INFO:tasks.workunit.client.1.smithi163.stdout:2/910: symlink d2/d26/d97/d10f/l13a 0 2024-02-16T19:24:34.046 INFO:tasks.workunit.client.1.smithi163.stdout:7/747: write d11/f89 [1923154,63000] 0 2024-02-16T19:24:34.046 INFO:tasks.workunit.client.1.smithi163.stdout:2/911: write d2/d2a/d2d/d73/f113 [307764,131044] 0 2024-02-16T19:24:34.046 INFO:tasks.workunit.client.1.smithi163.stdout:2/912: write d2/d2a/d2d/d73/fa2 [810957,2356] 0 2024-02-16T19:24:34.047 INFO:tasks.workunit.client.1.smithi163.stdout:8/764: write d0/d7/d18/f37 [2858346,90381] 0 2024-02-16T19:24:34.049 INFO:tasks.workunit.client.1.smithi163.stdout:4/761: mkdir d11/d18/d100 0 2024-02-16T19:24:34.049 INFO:tasks.workunit.client.1.smithi163.stdout:4/762: dread - d11/d2d/d4f/d60/d79/dc9/ff4 zero size 2024-02-16T19:24:34.049 INFO:tasks.workunit.client.1.smithi163.stdout:4/763: readlink d11/d18/d1f/d29/l3c 0 2024-02-16T19:24:34.050 INFO:tasks.workunit.client.1.smithi163.stdout:3/804: symlink d0/d3e/l11d 0 2024-02-16T19:24:34.051 INFO:tasks.workunit.client.1.smithi163.stdout:8/765: write d0/dc/f2d [8309189,46231] 0 2024-02-16T19:24:34.053 INFO:tasks.workunit.client.1.smithi163.stdout:6/731: symlink d0/dd/d7f/d15/d1c/d65/led 0 2024-02-16T19:24:34.053 INFO:tasks.workunit.client.1.smithi163.stdout:1/841: dread d7/dc/d2d/d32/f46 [0,4194304] 0 2024-02-16T19:24:34.053 INFO:tasks.workunit.client.1.smithi163.stdout:6/732: creat d0/d6/df/d20/d69/fee x:0 0 0 2024-02-16T19:24:34.053 INFO:tasks.workunit.client.1.smithi163.stdout:6/733: read - d0/d6/df/d30/f5a zero size 2024-02-16T19:24:34.053 INFO:tasks.workunit.client.1.smithi163.stdout:7/748: rename d11/d24/d25/d4b/d66/l83 to d11/d50/d5d/d8b/d67/dd6/da6/l104 0 2024-02-16T19:24:34.054 INFO:tasks.workunit.client.1.smithi163.stdout:7/749: chown d11/d24/d25/d4b/ff2 99875 1 2024-02-16T19:24:34.054 INFO:tasks.workunit.client.1.smithi163.stdout:2/913: fdatasync d2/d26/f5d 0 2024-02-16T19:24:34.054 INFO:tasks.workunit.client.1.smithi163.stdout:2/914: fsync d2/fe5 0 2024-02-16T19:24:34.057 INFO:tasks.workunit.client.1.smithi163.stdout:4/764: creat d11/d18/d1f/d29/d52/f101 x:0 0 0 2024-02-16T19:24:34.057 INFO:tasks.workunit.client.1.smithi163.stdout:4/765: chown d11/d18/d1f/d23/l82 24 1 2024-02-16T19:24:34.061 INFO:tasks.workunit.client.1.smithi163.stdout:5/953: write d0/d22/d28/d36/d3c/f43 [4245874,74582] 0 2024-02-16T19:24:34.062 INFO:tasks.workunit.client.1.smithi163.stdout:1/842: mkdir d7/dc/d11d 0 2024-02-16T19:24:34.064 INFO:tasks.workunit.client.1.smithi163.stdout:6/734: truncate d0/d6/df/d20/d69/fcf 3848708 0 2024-02-16T19:24:34.065 INFO:tasks.workunit.client.1.smithi163.stdout:7/750: mkdir d11/d24/d49/d105 0 2024-02-16T19:24:34.065 INFO:tasks.workunit.client.1.smithi163.stdout:7/751: creat d11/d26/f106 x:0 0 0 2024-02-16T19:24:34.065 INFO:tasks.workunit.client.1.smithi163.stdout:7/752: write d11/d24/f78 [384280,22083] 0 2024-02-16T19:24:34.065 INFO:tasks.workunit.client.1.smithi163.stdout:5/954: dread d0/d2/fe4 [0,4194304] 0 2024-02-16T19:24:34.066 INFO:tasks.workunit.client.1.smithi163.stdout:5/955: truncate d0/d22/d28/f95 1022830 0 2024-02-16T19:24:34.067 INFO:tasks.workunit.client.1.smithi163.stdout:2/915: symlink d2/d26/d54/l13b 0 2024-02-16T19:24:34.067 INFO:tasks.workunit.client.1.smithi163.stdout:4/766: symlink d11/d18/d66/l102 0 2024-02-16T19:24:34.069 INFO:tasks.workunit.client.1.smithi163.stdout:1/843: mknod d7/d20/d21/d4e/c11e 0 2024-02-16T19:24:34.072 INFO:tasks.workunit.client.1.smithi163.stdout:7/753: unlink d11/d24/d25/d4b/d66/dc1/fcb 0 2024-02-16T19:24:34.072 INFO:tasks.workunit.client.1.smithi163.stdout:7/754: chown d11/d50/d5d/d8b/f90 14 1 2024-02-16T19:24:34.072 INFO:tasks.workunit.client.1.smithi163.stdout:7/755: write d11/d24/d25/fbf [239300,130975] 0 2024-02-16T19:24:34.073 INFO:tasks.workunit.client.1.smithi163.stdout:8/766: dwrite d0/d7/d18/d53/ded/fe6 [0,4194304] 0 2024-02-16T19:24:34.073 INFO:tasks.workunit.client.1.smithi163.stdout:8/767: creat d0/d6f/de8/ffa x:0 0 0 2024-02-16T19:24:34.073 INFO:tasks.workunit.client.1.smithi163.stdout:8/768: write d0/d7/d18/d53/fc5 [4605865,28699] 0 2024-02-16T19:24:34.074 INFO:tasks.workunit.client.1.smithi163.stdout:5/956: creat d0/d22/d3b/dc0/f13f x:0 0 0 2024-02-16T19:24:34.074 INFO:tasks.workunit.client.1.smithi163.stdout:2/916: creat d2/d2a/f13c x:0 0 0 2024-02-16T19:24:34.075 INFO:tasks.workunit.client.1.smithi163.stdout:2/917: read - d2/d26/d54/d58/d98/f11c zero size 2024-02-16T19:24:34.078 INFO:tasks.workunit.client.1.smithi163.stdout:1/844: mkdir d7/d5b/d11f 0 2024-02-16T19:24:34.078 INFO:tasks.workunit.client.1.smithi163.stdout:6/735: write d0/d6/df/d20/d40/d17/f4b [3352599,41796] 0 2024-02-16T19:24:34.082 INFO:tasks.workunit.client.1.smithi163.stdout:8/769: write d0/d7/d19/d1f/d30/d48/f65 [3396020,91510] 0 2024-02-16T19:24:34.082 INFO:tasks.workunit.client.1.smithi163.stdout:8/770: chown d0/d7/d18/d53/ded/d85 99516 1 2024-02-16T19:24:34.083 INFO:tasks.workunit.client.1.smithi163.stdout:8/771: readlink d0/d24/d3c/d3d/d41/lbc 0 2024-02-16T19:24:34.083 INFO:tasks.workunit.client.1.smithi163.stdout:5/957: mknod d0/d22/d28/d36/dc4/c140 0 2024-02-16T19:24:34.084 INFO:tasks.workunit.client.1.smithi163.stdout:2/918: creat d2/d2a/d39/d130/f13d x:0 0 0 2024-02-16T19:24:34.084 INFO:tasks.workunit.client.1.smithi163.stdout:4/767: getdents d11/d18/d1f/d34/de5 0 2024-02-16T19:24:34.088 INFO:tasks.workunit.client.1.smithi163.stdout:3/805: rmdir d0/d83/d104 0 2024-02-16T19:24:34.088 INFO:tasks.workunit.client.1.smithi163.stdout:3/806: fdatasync d0/f102 0 2024-02-16T19:24:34.088 INFO:tasks.workunit.client.1.smithi163.stdout:2/919: mkdir d2/dce/dee/d13e 0 2024-02-16T19:24:34.091 INFO:tasks.workunit.client.1.smithi163.stdout:3/807: link d0/dc/d1c/lcd d0/dc/d1d/d29/d95/l11e 0 2024-02-16T19:24:34.091 INFO:tasks.workunit.client.1.smithi163.stdout:3/808: chown d0/dc/d1c/d32/f36 908150 1 2024-02-16T19:24:34.092 INFO:tasks.workunit.client.1.smithi163.stdout:1/845: dread d7/dc/f33 [0,4194304] 0 2024-02-16T19:24:34.092 INFO:tasks.workunit.client.1.smithi163.stdout:3/809: dread d0/dc/f8c [0,4194304] 0 2024-02-16T19:24:34.095 INFO:tasks.workunit.client.1.smithi163.stdout:2/920: symlink d2/d26/d97/l13f 0 2024-02-16T19:24:34.097 INFO:tasks.workunit.client.1.smithi163.stdout:1/846: mknod d7/dc/d2d/d32/d87/c120 0 2024-02-16T19:24:34.097 INFO:tasks.workunit.client.1.smithi163.stdout:1/847: chown d7/d20/d21/d74/dab 17 1 2024-02-16T19:24:34.098 INFO:tasks.workunit.client.1.smithi163.stdout:1/848: chown d7/d5b 0 1 2024-02-16T19:24:34.098 INFO:tasks.workunit.client.1.smithi163.stdout:1/849: read d7/dc/d2d/d3f/f67 [133372,64519] 0 2024-02-16T19:24:34.099 INFO:tasks.workunit.client.1.smithi163.stdout:8/772: dread d0/d7/d18/fd9 [0,4194304] 0 2024-02-16T19:24:34.103 INFO:tasks.workunit.client.1.smithi163.stdout:2/921: dread d2/f5 [0,4194304] 0 2024-02-16T19:24:34.103 INFO:tasks.workunit.client.1.smithi163.stdout:2/922: write d2/d26/d54/f9b [338187,88481] 0 2024-02-16T19:24:34.105 INFO:tasks.workunit.client.1.smithi163.stdout:2/923: creat d2/d2a/d2d/d73/d84/dba/f140 x:0 0 0 2024-02-16T19:24:34.105 INFO:tasks.workunit.client.1.smithi163.stdout:2/924: chown d2/d44/f138 25924 1 2024-02-16T19:24:34.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.165+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fba9c0fd030 con 0x7fba9c0fb220 2024-02-16T19:24:34.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.166+0000 7fba83fff700 1 -- 172.21.15.40:0/448262867 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fba9c0fd030 con 0x7fba9c0fb220 2024-02-16T19:24:34.167 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:24:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:24:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:24:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:24:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:24:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:24:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 14 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:24:34.168 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:24:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fba840692a0 msgr2=0x7fba8406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fba840692a0 0x7fba8406b750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba90004990 tx=0x7fba90009ee0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 msgr2=0x7fba9c0fdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 0x7fba9c0fdd40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009150 tx=0x7fba8c009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 shutdown_connections 2024-02-16T19:24:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fba840692a0 0x7fba8406b750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fb220 0x7fba9c0fdd40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 --2- 172.21.15.40:0/448262867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba9c0fe2b0 0x7fba9c0fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 >> 172.21.15.40:0/448262867 conn(0x7fba9c0f6ca0 msgr2=0x7fba9c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 shutdown_connections 2024-02-16T19:24:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.168+0000 7fbaa1c77700 1 -- 172.21.15.40:0/448262867 wait complete. 2024-02-16T19:24:34.170 INFO:tasks.workunit.client.1.smithi163.stdout:7/756: dwrite d11/d30/f3e [0,4194304] 0 2024-02-16T19:24:34.171 INFO:tasks.workunit.client.1.smithi163.stdout:1/850: read d7/dc/d2d/d32/d87/d97/fe6 [178265,102482] 0 2024-02-16T19:24:34.171 INFO:tasks.workunit.client.1.smithi163.stdout:1/851: dread - d7/d20/d21/d4e/feb zero size 2024-02-16T19:24:34.171 INFO:tasks.workunit.client.1.smithi163.stdout:4/768: dwrite d11/d18/d1f/d29/d52/f101 [0,4194304] 0 2024-02-16T19:24:34.171 INFO:tasks.workunit.client.1.smithi163.stdout:4/769: write d11/d18/f43 [676905,106737] 0 2024-02-16T19:24:34.171 INFO:tasks.workunit.client.1.smithi163.stdout:4/770: write d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/fb6 [950792,49340] 0 2024-02-16T19:24:34.177 INFO:tasks.workunit.client.1.smithi163.stdout:1/852: rename d7/dc/l13 to d7/d20/df7/l121 0 2024-02-16T19:24:34.177 INFO:tasks.workunit.client.1.smithi163.stdout:8/773: dread d0/d7/d19/d1f/d32/f34 [0,4194304] 0 2024-02-16T19:24:34.178 INFO:tasks.workunit.client.1.smithi163.stdout:3/810: dwrite d0/dc/d96/ddc/fe8 [0,4194304] 0 2024-02-16T19:24:34.178 INFO:tasks.workunit.client.1.smithi163.stdout:3/811: creat d0/f11f x:0 0 0 2024-02-16T19:24:34.180 INFO:tasks.workunit.client.1.smithi163.stdout:4/771: symlink d11/d2d/l103 0 2024-02-16T19:24:34.180 INFO:tasks.workunit.client.1.smithi163.stdout:4/772: fdatasync d11/d18/d1f/f27 0 2024-02-16T19:24:34.182 INFO:tasks.workunit.client.1.smithi163.stdout:2/925: write d2/f1c [51588,55771] 0 2024-02-16T19:24:34.182 INFO:tasks.workunit.client.1.smithi163.stdout:7/757: getdents d11/d24/d25/d4b/d8c 0 2024-02-16T19:24:34.182 INFO:tasks.workunit.client.1.smithi163.stdout:7/758: write d11/d24/d49/fdb [255501,123984] 0 2024-02-16T19:24:34.182 INFO:tasks.workunit.client.1.smithi163.stdout:7/759: write d11/d24/d25/d4b/ff2 [392373,40720] 0 2024-02-16T19:24:34.183 INFO:tasks.workunit.client.1.smithi163.stdout:4/773: fsync d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/fb6 0 2024-02-16T19:24:34.183 INFO:tasks.workunit.client.1.smithi163.stdout:4/774: creat d11/d18/d1f/d29/d52/d76/d89/d94/f104 x:0 0 0 2024-02-16T19:24:34.185 INFO:tasks.workunit.client.1.smithi163.stdout:8/774: mknod d0/d24/d2c/ddf/de5/cfb 0 2024-02-16T19:24:34.186 INFO:tasks.workunit.client.1.smithi163.stdout:8/775: chown d0/d7/d19/d1f/cbe 15657 1 2024-02-16T19:24:34.193 INFO:tasks.workunit.client.1.smithi163.stdout:7/760: dread d11/f19 [0,4194304] 0 2024-02-16T19:24:34.193 INFO:tasks.workunit.client.1.smithi163.stdout:7/761: chown d11/d24/d25/f4a 14 1 2024-02-16T19:24:34.193 INFO:tasks.workunit.client.1.smithi163.stdout:1/853: dwrite d7/dc/d2d/d32/ff0 [0,4194304] 0 2024-02-16T19:24:34.201 INFO:tasks.workunit.client.1.smithi163.stdout:7/762: mknod d11/d50/c107 0 2024-02-16T19:24:34.206 INFO:tasks.workunit.client.1.smithi163.stdout:8/776: symlink d0/d7/d19/d1f/d30/d6a/d7b/lfc 0 2024-02-16T19:24:34.206 INFO:tasks.workunit.client.1.smithi163.stdout:2/926: rename d2/d2a/d39/d130 to d2/ddb/d7c/d9c/d141 0 2024-02-16T19:24:34.207 INFO:tasks.workunit.client.1.smithi163.stdout:7/763: rmdir d11/d24 39 2024-02-16T19:24:34.215 INFO:tasks.workunit.client.1.smithi163.stdout:1/854: rename d7/d20/d21/l109 to d7/d20/d96/l122 0 2024-02-16T19:24:34.216 INFO:tasks.workunit.client.1.smithi163.stdout:2/927: link d2/d2a/d2d/d73/db5/c110 d2/d2a/d2d/d73/d94/c142 0 2024-02-16T19:24:34.216 INFO:tasks.workunit.client.1.smithi163.stdout:2/928: rename d2/d26 to d2/d26/d54/dfb/d104/d143 22 2024-02-16T19:24:34.217 INFO:tasks.workunit.client.1.smithi163.stdout:7/764: creat d11/d24/d25/f108 x:0 0 0 2024-02-16T19:24:34.217 INFO:tasks.workunit.client.1.smithi163.stdout:7/765: creat d11/d50/d5d/dcd/f109 x:0 0 0 2024-02-16T19:24:34.220 INFO:tasks.workunit.client.1.smithi163.stdout:8/777: link d0/d24/d3c/d3d/d41/lf4 d0/d7/d19/d1f/d30/d6a/df3/df2/lfd 0 2024-02-16T19:24:34.222 INFO:tasks.workunit.client.1.smithi163.stdout:4/775: dwrite d11/f16 [4194304,4194304] 0 2024-02-16T19:24:34.226 INFO:tasks.workunit.client.1.smithi163.stdout:2/929: rmdir d2/d26/d97/d10f 39 2024-02-16T19:24:34.228 INFO:tasks.workunit.client.1.smithi163.stdout:2/930: dread d2/d2a/d39/fae [0,4194304] 0 2024-02-16T19:24:34.229 INFO:tasks.workunit.client.1.smithi163.stdout:7/766: truncate d11/d24/d25/f48 632207 0 2024-02-16T19:24:34.229 INFO:tasks.workunit.client.1.smithi163.stdout:7/767: readlink d11/d50/d5d/la0 0 2024-02-16T19:24:34.229 INFO:tasks.workunit.client.1.smithi163.stdout:7/768: write d11/d24/d49/f84 [275015,74521] 0 2024-02-16T19:24:34.230 INFO:tasks.workunit.client.1.smithi163.stdout:7/769: chown d11/d24/d34/l8f 1032743 1 2024-02-16T19:24:34.230 INFO:tasks.workunit.client.1.smithi163.stdout:2/931: mkdir d2/d26/d54/d58/d8c/dc5/d144 0 2024-02-16T19:24:34.231 INFO:tasks.workunit.client.1.smithi163.stdout:2/932: truncate d2/f1a 3692325 0 2024-02-16T19:24:34.231 INFO:tasks.workunit.client.1.smithi163.stdout:2/933: truncate d2/ddb/d7c/d9c/fea 752547 0 2024-02-16T19:24:34.232 INFO:tasks.workunit.client.1.smithi163.stdout:8/778: write d0/d6f/fd2 [2670675,16416] 0 2024-02-16T19:24:34.235 INFO:tasks.workunit.client.1.smithi163.stdout:7/770: dread d11/d50/d5d/d8b/d67/f85 [0,4194304] 0 2024-02-16T19:24:34.235 INFO:tasks.workunit.client.1.smithi163.stdout:8/779: truncate d0/d7/d19/d1f/d30/f52 6673607 0 2024-02-16T19:24:34.236 INFO:tasks.workunit.client.1.smithi163.stdout:2/934: dread d2/d26/f28 [0,4194304] 0 2024-02-16T19:24:34.236 INFO:tasks.workunit.client.1.smithi163.stdout:2/935: fdatasync d2/d44/ff4 0 2024-02-16T19:24:34.236 INFO:tasks.workunit.client.1.smithi163.stdout:2/936: read - d2/d44/f138 zero size 2024-02-16T19:24:34.239 INFO:tasks.workunit.client.1.smithi163.stdout:8/780: creat d0/d24/d2c/ddf/ffe x:0 0 0 2024-02-16T19:24:34.240 INFO:tasks.workunit.client.1.smithi163.stdout:2/937: link d2/d26/d54/d58/da6/lc3 d2/d26/d54/d58/d8c/l145 0 2024-02-16T19:24:34.240 INFO:tasks.workunit.client.1.smithi163.stdout:2/938: creat d2/d26/d54/f146 x:0 0 0 2024-02-16T19:24:34.240 INFO:tasks.workunit.client.1.smithi163.stdout:2/939: dread - d2/dce/dee/f102 zero size 2024-02-16T19:24:34.244 INFO:tasks.workunit.client.1.smithi163.stdout:8/781: unlink d0/d7/d19/d1f/d30/d48/ff8 0 2024-02-16T19:24:34.244 INFO:tasks.workunit.client.1.smithi163.stdout:8/782: chown d0/dc 0 1 2024-02-16T19:24:34.244 INFO:tasks.workunit.client.1.smithi163.stdout:8/783: link d0/dc/f7d d0/d7/d19/d1f/d30/fff 0 2024-02-16T19:24:34.244 INFO:tasks.workunit.client.1.smithi163.stdout:2/940: rename d2/d26/d54/dfb/d114/fd4 to d2/d2a/d39/f147 0 2024-02-16T19:24:34.244 INFO:tasks.workunit.client.1.smithi163.stdout:2/941: chown d2/d2a/d2d/d73/c88 257405 1 2024-02-16T19:24:34.245 INFO:tasks.workunit.client.1.smithi163.stdout:8/784: creat d0/d7/d18/d53/ded/f100 x:0 0 0 2024-02-16T19:24:34.245 INFO:tasks.workunit.client.1.smithi163.stdout:8/785: write d0/d7/d18/f3f [839011,93000] 0 2024-02-16T19:24:34.245 INFO:tasks.workunit.client.1.smithi163.stdout:4/776: dwrite d11/d2d/d4f/d5f/faa [0,4194304] 0 2024-02-16T19:24:34.246 INFO:tasks.workunit.client.1.smithi163.stdout:1/855: dwrite d7/d20/d2a/f40 [0,4194304] 0 2024-02-16T19:24:34.246 INFO:tasks.workunit.client.1.smithi163.stdout:1/856: read d7/dc/d2d/d32/fbe [775257,95787] 0 2024-02-16T19:24:34.246 INFO:tasks.workunit.client.1.smithi163.stdout:2/942: unlink d2/d26/d54/d58/d8c/faa 0 2024-02-16T19:24:34.249 INFO:tasks.workunit.client.1.smithi163.stdout:8/786: unlink d0/d7/d18/d2e/c61 0 2024-02-16T19:24:34.249 INFO:tasks.workunit.client.1.smithi163.stdout:8/787: chown d0/d7/d55/f80 298699 1 2024-02-16T19:24:34.255 INFO:tasks.workunit.client.1.smithi163.stdout:1/857: mknod d7/dc/d2d/d32/d87/da6/de5/c123 0 2024-02-16T19:24:34.257 INFO:tasks.workunit.client.1.smithi163.stdout:2/943: symlink d2/d44/l148 0 2024-02-16T19:24:34.257 INFO:tasks.workunit.client.1.smithi163.stdout:2/944: chown d2/ddb/d7c/d9c/d141/f13d 463164785 1 2024-02-16T19:24:34.260 INFO:tasks.workunit.client.1.smithi163.stdout:4/777: getdents d11/d2d/d4f/d5f/d70/d74/d75/dca/dbf 0 2024-02-16T19:24:34.262 INFO:tasks.workunit.client.1.smithi163.stdout:1/858: truncate d7/d20/f99 4284325 0 2024-02-16T19:24:34.262 INFO:tasks.workunit.client.1.smithi163.stdout:1/859: dread - d7/d20/f11a zero size 2024-02-16T19:24:34.263 INFO:tasks.workunit.client.1.smithi163.stdout:2/945: readlink d2/d2a/d39/l59 0 2024-02-16T19:24:34.263 INFO:tasks.workunit.client.1.smithi163.stdout:2/946: write d2/d26/d54/d58/da6/f136 [945328,72239] 0 2024-02-16T19:24:34.266 INFO:tasks.workunit.client.1.smithi163.stdout:8/788: rename d0/d24/d3c/d3d/l1d to d0/d7/d19/d1f/d30/d6a/db8/dcf/l101 0 2024-02-16T19:24:34.267 INFO:tasks.workunit.client.1.smithi163.stdout:1/860: mknod d7/d20/c124 0 2024-02-16T19:24:34.269 INFO:tasks.workunit.client.1.smithi163.stdout:2/947: creat d2/d26/d97/d10f/f149 x:0 0 0 2024-02-16T19:24:34.272 INFO:tasks.workunit.client.1.smithi163.stdout:7/771: dwrite d11/d24/d34/d8d/ffe [0,4194304] 0 2024-02-16T19:24:34.272 INFO:tasks.workunit.client.1.smithi163.stdout:1/861: dread d7/dc/d2d/d32/d3b/fdb [0,4194304] 0 2024-02-16T19:24:34.272 INFO:tasks.workunit.client.1.smithi163.stdout:4/778: rename d11/d18/d1f/d23/d3a/ced to d11/d2d/d4f/d5f/d70/c105 0 2024-02-16T19:24:34.272 INFO:tasks.workunit.client.1.smithi163.stdout:1/862: dread - d7/d5b/faa zero size 2024-02-16T19:24:34.276 INFO:tasks.workunit.client.1.smithi163.stdout:1/863: creat d7/dc/d2d/d32/d87/d97/f125 x:0 0 0 2024-02-16T19:24:34.278 INFO:tasks.workunit.client.1.smithi163.stdout:1/864: symlink d7/d5b/l126 0 2024-02-16T19:24:34.278 INFO:tasks.workunit.client.1.smithi163.stdout:1/865: chown d7/d20/lda 315 1 2024-02-16T19:24:34.278 INFO:tasks.workunit.client.1.smithi163.stdout:8/789: dread d0/d24/f60 [0,4194304] 0 2024-02-16T19:24:34.281 INFO:tasks.workunit.client.1.smithi163.stdout:1/866: creat d7/dc/d2d/d32/d87/de1/dfd/f127 x:0 0 0 2024-02-16T19:24:34.282 INFO:tasks.workunit.client.1.smithi163.stdout:1/867: truncate d7/dc/d2d/d32/ff0 4766906 0 2024-02-16T19:24:34.282 INFO:tasks.workunit.client.1.smithi163.stdout:1/868: rmdir d7/d5b 39 2024-02-16T19:24:34.282 INFO:tasks.workunit.client.1.smithi163.stdout:1/869: write d7/dc/d2d/d32/d87/d97/fe6 [1990180,52351] 0 2024-02-16T19:24:34.289 INFO:tasks.workunit.client.1.smithi163.stdout:8/790: dread d0/d24/d2c/f71 [0,4194304] 0 2024-02-16T19:24:34.289 INFO:tasks.workunit.client.1.smithi163.stdout:8/791: chown d0/d7/d18/d2e 21 1 2024-02-16T19:24:34.301 INFO:tasks.workunit.client.1.smithi163.stdout:8/792: write d0/d7/d19/d1f/d30/f92 [3861461,6310] 0 2024-02-16T19:24:34.303 INFO:tasks.workunit.client.1.smithi163.stdout:8/793: unlink d0/d24/fbb 0 2024-02-16T19:24:34.304 INFO:tasks.workunit.client.1.smithi163.stdout:8/794: symlink d0/d7/d19/d1f/d30/d48/dbd/l102 0 2024-02-16T19:24:34.304 INFO:tasks.workunit.client.1.smithi163.stdout:8/795: truncate d0/d7/d18/d53/ded/f100 525862 0 2024-02-16T19:24:34.304 INFO:tasks.workunit.client.1.smithi163.stdout:8/796: write d0/d7/d19/d1f/d30/d6a/d7b/fac [737580,50066] 0 2024-02-16T19:24:34.319 INFO:tasks.workunit.client.1.smithi163.stdout:8/797: write d0/d7/d19/f26 [3734015,66049] 0 2024-02-16T19:24:34.319 INFO:tasks.workunit.client.1.smithi163.stdout:4/779: dwrite d11/d18/d1f/d23/d3a/d4a/f6e [0,4194304] 0 2024-02-16T19:24:34.319 INFO:tasks.workunit.client.1.smithi163.stdout:8/798: chown d0/dc/cd4 15 1 2024-02-16T19:24:34.320 INFO:tasks.workunit.client.1.smithi163.stdout:2/948: dwrite d2/d26/d54/d58/d98/feb [0,4194304] 0 2024-02-16T19:24:34.321 INFO:tasks.workunit.client.1.smithi163.stdout:1/870: dwrite d7/f1d [0,4194304] 0 2024-02-16T19:24:34.321 INFO:tasks.workunit.client.1.smithi163.stdout:7/772: dwrite d11/d24/d25/d4b/d66/dc1/fd7 [0,4194304] 0 2024-02-16T19:24:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.321+0000 7f87f17b0700 1 -- 172.21.15.40:0/394232986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 msgr2=0x7f87ec0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.321+0000 7f87f17b0700 1 --2- 172.21.15.40:0/394232986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 0x7f87ec0fe550 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f87e0004940 tx=0x7f87e0009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.321+0000 7f87f17b0700 1 -- 172.21.15.40:0/394232986 shutdown_connections 2024-02-16T19:24:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.321+0000 7f87f17b0700 1 --2- 172.21.15.40:0/394232986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 0x7f87ec0fe550 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.321+0000 7f87f17b0700 1 --2- 172.21.15.40:0/394232986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0fcef0 0x7f87ec0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.321+0000 7f87f17b0700 1 -- 172.21.15.40:0/394232986 >> 172.21.15.40:0/394232986 conn(0x7f87ec0f8620 msgr2=0x7f87ec0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.321+0000 7f87f17b0700 1 -- 172.21.15.40:0/394232986 shutdown_connections 2024-02-16T19:24:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.321+0000 7f87f17b0700 1 -- 172.21.15.40:0/394232986 wait complete. 2024-02-16T19:24:34.323 INFO:tasks.workunit.client.1.smithi163.stdout:4/780: link d11/d18/d1f/d29/d52/d57/d6c/d9d/fa0 d11/d18/d100/f106 0 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.322+0000 7f87f17b0700 1 Processor -- start 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.322+0000 7f87f17b0700 1 -- start start 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.322+0000 7f87f17b0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0fcef0 0x7f87ec10d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.322+0000 7f87f17b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 0x7f87ec10de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.322+0000 7f87f17b0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec10e430 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.322+0000 7f87f17b0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec10ecd0 con 0x7f87ec0fcef0 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.323+0000 7f87ea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 0x7f87ec10de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.323+0000 7f87ea7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 0x7f87ec10de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37180/0 (socket says 172.21.15.40:37180) 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.323+0000 7f87ea7fc700 1 -- 172.21.15.40:0/1301358190 learned_addr learned my addr 172.21.15.40:0/1301358190 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:34.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.323+0000 7f87eaffd700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0fcef0 0x7f87ec10d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:34.325 INFO:tasks.workunit.client.1.smithi163.stdout:8/799: mkdir d0/d7/d103 0 2024-02-16T19:24:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.323+0000 7f87ea7fc700 1 -- 172.21.15.40:0/1301358190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0fcef0 msgr2=0x7f87ec10d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.323+0000 7f87ea7fc700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0fcef0 0x7f87ec10d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.323+0000 7f87ea7fc700 1 -- 172.21.15.40:0/1301358190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e0005040 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.323+0000 7f87ea7fc700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 0x7f87ec10de30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f87e0000c00 tx=0x7f87e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.324+0000 7f87d37fe700 1 -- 172.21.15.40:0/1301358190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e000e070 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.324+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec10ef50 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.324+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec10f430 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.324+0000 7f87d37fe700 1 -- 172.21.15.40:0/1301358190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87e0009430 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.324+0000 7f87d37fe700 1 -- 172.21.15.40:0/1301358190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e000b410 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.326+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d80052e0 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.326+0000 7f87d37fe700 1 -- 172.21.15.40:0/1301358190 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f87e0013af0 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.327+0000 7f87d37fe700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f87d40692b0 0x7f87d406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.327+0000 7f87eaffd700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f87d40692b0 0x7f87d406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.328+0000 7f87eaffd700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f87d40692b0 0x7f87d406b760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009b80 tx=0x7f87dc00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.330+0000 7f87d37fe700 1 -- 172.21.15.40:0/1301358190 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f87e0011e50 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.336+0000 7f87d37fe700 1 -- 172.21.15.40:0/1301358190 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f87e008a030 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.340 INFO:tasks.workunit.client.1.smithi163.stdout:7/773: symlink d11/d24/d25/d4b/d8c/dee/l10a 0 2024-02-16T19:24:34.340 INFO:tasks.workunit.client.1.smithi163.stdout:7/774: creat d11/d24/d25/d4b/d66/f10b x:0 0 0 2024-02-16T19:24:34.341 INFO:tasks.workunit.client.1.smithi163.stdout:4/781: symlink d11/d2d/d4f/d5f/d70/d74/d75/dca/dbf/l107 0 2024-02-16T19:24:34.343 INFO:tasks.workunit.client.1.smithi163.stdout:8/800: mkdir d0/d6f/d104 0 2024-02-16T19:24:34.345 INFO:tasks.workunit.client.1.smithi163.stdout:4/782: mknod d11/d18/c108 0 2024-02-16T19:24:34.347 INFO:tasks.workunit.client.1.smithi163.stdout:8/801: link d0/d24/d2c/d74/dcb/cd8 d0/dc/c105 0 2024-02-16T19:24:34.347 INFO:tasks.workunit.client.1.smithi163.stdout:8/802: fdatasync d0/d7/d18/d53/fc5 0 2024-02-16T19:24:34.347 INFO:tasks.workunit.client.1.smithi163.stdout:8/803: write d0/d24/d2c/f90 [4232058,68595] 0 2024-02-16T19:24:34.347 INFO:tasks.workunit.client.1.smithi163.stdout:1/871: dwrite d7/dc/d2d/d32/d87/d97/fb0 [0,4194304] 0 2024-02-16T19:24:34.348 INFO:tasks.workunit.client.1.smithi163.stdout:2/949: dwrite d2/d26/d54/dfb/d114/fe3 [0,4194304] 0 2024-02-16T19:24:34.348 INFO:tasks.workunit.client.1.smithi163.stdout:2/950: fdatasync d2/d26/f2f 0 2024-02-16T19:24:34.349 INFO:tasks.workunit.client.1.smithi163.stdout:4/783: rename d11/d2d/d4f/f55 to d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/f109 0 2024-02-16T19:24:34.351 INFO:tasks.workunit.client.1.smithi163.stdout:1/872: mknod d7/dc/d2d/d3f/d85/c128 0 2024-02-16T19:24:34.353 INFO:tasks.workunit.client.1.smithi163.stdout:7/775: write d11/d24/d49/dc4/fca [422364,87126] 0 2024-02-16T19:24:34.353 INFO:tasks.workunit.client.1.smithi163.stdout:7/776: readlink d11/d24/d25/d4b/lc7 0 2024-02-16T19:24:34.353 INFO:tasks.workunit.client.1.smithi163.stdout:7/777: readlink d11/d24/d25/d4b/d8c/dee/l10a 0 2024-02-16T19:24:34.355 INFO:tasks.workunit.client.1.smithi163.stdout:2/951: mknod d2/d2a/c14a 0 2024-02-16T19:24:34.355 INFO:tasks.workunit.client.1.smithi163.stdout:2/952: fdatasync d2/d2a/d39/d3a/f7b 0 2024-02-16T19:24:34.355 INFO:tasks.workunit.client.1.smithi163.stdout:2/953: write d2/d44/f138 [338191,87811] 0 2024-02-16T19:24:34.355 INFO:tasks.workunit.client.1.smithi163.stdout:2/954: write d2/d2a/d2d/d73/d84/f89 [2548669,113981] 0 2024-02-16T19:24:34.356 INFO:tasks.workunit.client.1.smithi163.stdout:4/784: symlink d11/d2d/l10a 0 2024-02-16T19:24:34.356 INFO:tasks.workunit.client.1.smithi163.stdout:4/785: chown d11/d18/d1f/d29/d52/d57/d6c/l87 10485521 1 2024-02-16T19:24:34.356 INFO:tasks.workunit.client.1.smithi163.stdout:4/786: fdatasync d11/f16 0 2024-02-16T19:24:34.356 INFO:tasks.workunit.client.1.smithi163.stdout:4/787: fdatasync d11/f41 0 2024-02-16T19:24:34.356 INFO:tasks.workunit.client.1.smithi163.stdout:8/804: dread d0/d7/d19/d1f/d30/d6a/df3/df2/f75 [0,4194304] 0 2024-02-16T19:24:34.357 INFO:tasks.workunit.client.1.smithi163.stdout:8/805: write d0/d7/d19/d1f/d30/d48/f65 [254254,121872] 0 2024-02-16T19:24:34.359 INFO:tasks.workunit.client.1.smithi163.stdout:1/873: creat d7/d20/d93/dd2/f129 x:0 0 0 2024-02-16T19:24:34.360 INFO:tasks.workunit.client.1.smithi163.stdout:0/979: sync 2024-02-16T19:24:34.363 INFO:tasks.workunit.client.1.smithi163.stdout:2/955: link d2/d2a/d39/d3a/l72 d2/d26/d54/d58/d8c/d128/l14b 0 2024-02-16T19:24:34.365 INFO:tasks.workunit.client.1.smithi163.stdout:8/806: mknod d0/d24/d2c/ddf/de5/c106 0 2024-02-16T19:24:34.369 INFO:tasks.workunit.client.1.smithi163.stdout:0/980: symlink d11/d22/d61/l147 0 2024-02-16T19:24:34.370 INFO:tasks.workunit.client.1.smithi163.stdout:8/807: mkdir d0/d24/d2c/ddf/de5/d107 0 2024-02-16T19:24:34.371 INFO:tasks.workunit.client.1.smithi163.stdout:8/808: fsync d0/d24/d2c/ddf/de5/ff9 0 2024-02-16T19:24:34.372 INFO:tasks.workunit.client.1.smithi163.stdout:1/874: symlink d7/l12a 0 2024-02-16T19:24:34.374 INFO:tasks.workunit.client.1.smithi163.stdout:8/809: rmdir d0/d7/d18/d53/ded 39 2024-02-16T19:24:34.374 INFO:tasks.workunit.client.1.smithi163.stdout:8/810: write d0/d7/d18/d53/fc5 [4657858,47512] 0 2024-02-16T19:24:34.375 INFO:tasks.workunit.client.1.smithi163.stdout:1/875: getdents d7/d20/d2a 0 2024-02-16T19:24:34.375 INFO:tasks.workunit.client.1.smithi163.stdout:1/876: dread - d7/d20/d93/ff5 zero size 2024-02-16T19:24:34.375 INFO:tasks.workunit.client.1.smithi163.stdout:2/956: dwrite d2/d2a/d9f/fb9 [0,4194304] 0 2024-02-16T19:24:34.379 INFO:tasks.workunit.client.1.smithi163.stdout:8/811: mkdir d0/d7/d18/d53/d108 0 2024-02-16T19:24:34.382 INFO:tasks.workunit.client.1.smithi163.stdout:1/877: unlink d7/c65 0 2024-02-16T19:24:34.387 INFO:tasks.workunit.client.1.smithi163.stdout:3/812: sync 2024-02-16T19:24:34.387 INFO:tasks.workunit.client.1.smithi163.stdout:5/958: sync 2024-02-16T19:24:34.388 INFO:tasks.workunit.client.1.smithi163.stdout:6/736: sync 2024-02-16T19:24:34.388 INFO:tasks.workunit.client.1.smithi163.stdout:6/737: write d0/d6/f8 [6201235,104218] 0 2024-02-16T19:24:34.388 INFO:tasks.workunit.client.1.smithi163.stdout:6/738: read - d0/dd/d7f/d81/daa/fde zero size 2024-02-16T19:24:34.390 INFO:tasks.workunit.client.1.smithi163.stdout:5/959: mknod d0/d1c/c141 0 2024-02-16T19:24:34.392 INFO:tasks.workunit.client.1.smithi163.stdout:5/960: symlink d0/d2/d134/l142 0 2024-02-16T19:24:34.392 INFO:tasks.workunit.client.1.smithi163.stdout:5/961: fsync d0/d1c/dbe/de3/fec 0 2024-02-16T19:24:34.397 INFO:tasks.workunit.client.1.smithi163.stdout:0/981: dwrite d11/d21/d7c/f81 [0,4194304] 0 2024-02-16T19:24:34.401 INFO:tasks.workunit.client.1.smithi163.stdout:0/982: rename d11/d22/d3e/l5d to d11/d21/d50/d58/d72/d9e/dd8/l148 0 2024-02-16T19:24:34.401 INFO:tasks.workunit.client.1.smithi163.stdout:0/983: fsync d11/d21/d50/d58/d74/f8f 0 2024-02-16T19:24:34.401 INFO:tasks.workunit.client.1.smithi163.stdout:0/984: chown d11/d12/d67/d6a/d6d 295 1 2024-02-16T19:24:34.401 INFO:tasks.workunit.client.1.smithi163.stdout:0/985: fsync d11/d21/d50/d58/d72/fa8 0 2024-02-16T19:24:34.402 INFO:tasks.workunit.client.1.smithi163.stdout:3/813: dread d0/dc/d1d/d29/d38/d46/fa6 [0,4194304] 0 2024-02-16T19:24:34.402 INFO:tasks.workunit.client.1.smithi163.stdout:3/814: creat d0/dc/d1c/d21/f120 x:0 0 0 2024-02-16T19:24:34.403 INFO:tasks.workunit.client.1.smithi163.stdout:0/986: creat d11/d12/d67/d6a/dae/dcc/f149 x:0 0 0 2024-02-16T19:24:34.404 INFO:tasks.workunit.client.1.smithi163.stdout:8/812: dwrite d0/d7/d19/d1f/d30/d6a/df3/df2/fae [0,4194304] 0 2024-02-16T19:24:34.406 INFO:tasks.workunit.client.1.smithi163.stdout:1/878: dwrite d7/d20/d21/d4e/dc6/fdf [4194304,4194304] 0 2024-02-16T19:24:34.406 INFO:tasks.workunit.client.1.smithi163.stdout:1/879: stat d7/dc/l8c 0 2024-02-16T19:24:34.406 INFO:tasks.workunit.client.1.smithi163.stdout:0/987: creat d11/d12/d67/d6a/dae/d118/f14a x:0 0 0 2024-02-16T19:24:34.406 INFO:tasks.workunit.client.1.smithi163.stdout:0/988: fdatasync d11/d21/d50/d58/f8b 0 2024-02-16T19:24:34.409 INFO:tasks.workunit.client.1.smithi163.stdout:0/989: mknod d11/d12/d11f/c14b 0 2024-02-16T19:24:34.409 INFO:tasks.workunit.client.1.smithi163.stdout:6/739: dread d0/d6/df/d20/d40/d17/f4b [0,4194304] 0 2024-02-16T19:24:34.412 INFO:tasks.workunit.client.1.smithi163.stdout:5/962: dwrite d0/f2c [4194304,4194304] 0 2024-02-16T19:24:34.416 INFO:tasks.workunit.client.1.smithi163.stdout:6/740: rmdir d0/dd/d7f/d2e 39 2024-02-16T19:24:34.416 INFO:tasks.workunit.client.1.smithi163.stdout:5/963: truncate d0/d22/d28/d111/da8/f10a 2001754 0 2024-02-16T19:24:34.420 INFO:tasks.workunit.client.1.smithi163.stdout:5/964: rename d0/d11c to d0/d22/d28/d36/d3c/d104/d143 0 2024-02-16T19:24:34.420 INFO:tasks.workunit.client.1.smithi163.stdout:5/965: creat d0/d1c/dbe/de3/f144 x:0 0 0 2024-02-16T19:24:34.427 INFO:tasks.workunit.client.1.smithi163.stdout:5/966: rename d0/d22/d28/d36/db7/fb8 to d0/d1c/dbe/de3/f145 0 2024-02-16T19:24:34.428 INFO:tasks.workunit.client.1.smithi163.stdout:6/741: write d0/d6/df/d20/d69/fcf [3630225,101700] 0 2024-02-16T19:24:34.430 INFO:tasks.workunit.client.1.smithi163.stdout:6/742: rename d0/fdd to d0/dd/d7f/d81/d85/da4/fef 0 2024-02-16T19:24:34.432 INFO:tasks.workunit.client.1.smithi163.stdout:6/743: link d0/d6/df/f83 d0/dd/d7f/d81/d85/ff0 0 2024-02-16T19:24:34.432 INFO:tasks.workunit.client.1.smithi163.stdout:6/744: dread - d0/dd/dbc/ddc/de5/f9f zero size 2024-02-16T19:24:34.437 INFO:tasks.workunit.client.1.smithi163.stdout:8/813: dwrite d0/d6f/fd2 [0,4194304] 0 2024-02-16T19:24:34.439 INFO:tasks.workunit.client.1.smithi163.stdout:0/990: dwrite d11/d21/d50/d58/d6b/fe9 [0,4194304] 0 2024-02-16T19:24:34.439 INFO:tasks.workunit.client.1.smithi163.stdout:3/815: dwrite d0/dc/d1d/f5a [0,4194304] 0 2024-02-16T19:24:34.441 INFO:tasks.workunit.client.1.smithi163.stdout:5/967: dread d0/d1c/d5a/da5/f137 [0,4194304] 0 2024-02-16T19:24:34.442 INFO:tasks.workunit.client.1.smithi163.stdout:0/991: creat d11/d21/d7c/f14c x:0 0 0 2024-02-16T19:24:34.442 INFO:tasks.workunit.client.1.smithi163.stdout:0/992: creat d11/d21/d50/d58/d6b/f14d x:0 0 0 2024-02-16T19:24:34.444 INFO:tasks.workunit.client.1.smithi163.stdout:3/816: mkdir d0/dc/d94/d9e/d121 0 2024-02-16T19:24:34.446 INFO:tasks.workunit.client.1.smithi163.stdout:5/968: unlink d0/d22/d28/d36/d3c/fff 0 2024-02-16T19:24:34.449 INFO:tasks.workunit.client.1.smithi163.stdout:8/814: read d0/d7/d18/f1e [3807348,74759] 0 2024-02-16T19:24:34.449 INFO:tasks.workunit.client.1.smithi163.stdout:0/993: symlink d11/d21/d50/d58/l14e 0 2024-02-16T19:24:34.449 INFO:tasks.workunit.client.1.smithi163.stdout:0/994: creat d11/d12/d67/d6a/dae/dcc/dd9/f14f x:0 0 0 2024-02-16T19:24:34.450 INFO:tasks.workunit.client.1.smithi163.stdout:0/995: creat d11/d12/d67/d6a/dae/dcc/dd9/f150 x:0 0 0 2024-02-16T19:24:34.451 INFO:tasks.workunit.client.1.smithi163.stdout:3/817: link d0/dc/d1c/d21/f119 d0/dc/d1c/d32/f122 0 2024-02-16T19:24:34.452 INFO:tasks.workunit.client.1.smithi163.stdout:5/969: mkdir d0/d22/d28/d36/db7/d146 0 2024-02-16T19:24:34.453 INFO:tasks.workunit.client.1.smithi163.stdout:8/815: getdents d0/d24/d3c/d3d/d41 0 2024-02-16T19:24:34.453 INFO:tasks.workunit.client.1.smithi163.stdout:8/816: fdatasync d0/d24/d2c/ddf/de5/ff9 0 2024-02-16T19:24:34.454 INFO:tasks.workunit.client.1.smithi163.stdout:6/745: dwrite d0/d6/df/f2a [4194304,4194304] 0 2024-02-16T19:24:34.456 INFO:tasks.workunit.client.1.smithi163.stdout:3/818: rmdir d0/dc/d1c/d32 39 2024-02-16T19:24:34.456 INFO:tasks.workunit.client.1.smithi163.stdout:8/817: truncate d0/d7/d18/d53/ded/fb9 2805618 0 2024-02-16T19:24:34.457 INFO:tasks.workunit.client.1.smithi163.stdout:6/746: truncate d0/d18/d77/dc9/f80 2680852 0 2024-02-16T19:24:34.459 INFO:tasks.workunit.client.1.smithi163.stdout:3/819: truncate d0/f8 3073389 0 2024-02-16T19:24:34.461 INFO:tasks.workunit.client.1.smithi163.stdout:8/818: unlink d0/d7/d18/f1e 0 2024-02-16T19:24:34.461 INFO:tasks.workunit.client.1.smithi163.stdout:8/819: stat d0/d7/d103 0 2024-02-16T19:24:34.462 INFO:tasks.workunit.client.1.smithi163.stdout:6/747: unlink d0/dd/d7f/d81/d85/ff0 0 2024-02-16T19:24:34.462 INFO:tasks.workunit.client.1.smithi163.stdout:5/970: dread d0/d2/d34/f4b [0,4194304] 0 2024-02-16T19:24:34.463 INFO:tasks.workunit.client.1.smithi163.stdout:3/820: mknod d0/dc/d96/c123 0 2024-02-16T19:24:34.463 INFO:tasks.workunit.client.1.smithi163.stdout:3/821: chown d0/dc/d96/ddc 986595 1 2024-02-16T19:24:34.464 INFO:tasks.workunit.client.1.smithi163.stdout:3/822: dread - d0/dc/d50/d66/da3/fd7 zero size 2024-02-16T19:24:34.465 INFO:tasks.workunit.client.1.smithi163.stdout:8/820: rmdir d0/d7/d18/d53 39 2024-02-16T19:24:34.466 INFO:tasks.workunit.client.1.smithi163.stdout:3/823: dread d0/dc/d1c/f2c [0,4194304] 0 2024-02-16T19:24:34.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/748: rename d0/d6/df/d20/d40/d17/dcb to d0/dd/df1 0 2024-02-16T19:24:34.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/749: write d0/d18/d94/f95 [326368,14228] 0 2024-02-16T19:24:34.469 INFO:tasks.workunit.client.1.smithi163.stdout:5/971: creat d0/d2/d34/f147 x:0 0 0 2024-02-16T19:24:34.469 INFO:tasks.workunit.client.1.smithi163.stdout:5/972: dread - d0/d2/d34/d44/d53/ff2 zero size 2024-02-16T19:24:34.469 INFO:tasks.workunit.client.1.smithi163.stdout:0/996: rmdir d11/d12/d67/d6a/dae/dcc/dd9 39 2024-02-16T19:24:34.471 INFO:tasks.workunit.client.1.smithi163.stdout:8/821: creat d0/d7/d19/d1f/d30/d6a/d7b/f109 x:0 0 0 2024-02-16T19:24:34.471 INFO:tasks.workunit.client.1.smithi163.stdout:8/822: chown d0/d7/d18/d53/ded 8486 1 2024-02-16T19:24:34.471 INFO:tasks.workunit.client.1.smithi163.stdout:8/823: creat d0/d7/d19/d1f/d32/f10a x:0 0 0 2024-02-16T19:24:34.474 INFO:tasks.workunit.client.1.smithi163.stdout:3/824: creat d0/dc/d50/d66/f124 x:0 0 0 2024-02-16T19:24:34.476 INFO:tasks.workunit.client.1.smithi163.stdout:6/750: mkdir d0/dd/d7f/d81/d85/da4/df2 0 2024-02-16T19:24:34.479 INFO:tasks.workunit.client.1.smithi163.stdout:5/973: unlink d0/d2/d34/d44/f51 0 2024-02-16T19:24:34.480 INFO:tasks.workunit.client.1.smithi163.stdout:8/824: creat d0/d7/d18/d53/ded/d85/de2/f10b x:0 0 0 2024-02-16T19:24:34.481 INFO:tasks.workunit.client.1.smithi163.stdout:6/751: unlink d0/dd/d7f/d2e/fd6 0 2024-02-16T19:24:34.483 INFO:tasks.workunit.client.1.smithi163.stdout:8/825: rename d0/d7/d19/d1f/d30/d6a/df3/df2/f75 to d0/d7/d19/d1f/d32/f10c 0 2024-02-16T19:24:34.483 INFO:tasks.workunit.client.1.smithi163.stdout:8/826: write d0/d7/d19/d1f/d32/d70/f8b [1448808,40383] 0 2024-02-16T19:24:34.483 INFO:tasks.workunit.client.1.smithi163.stdout:5/974: dread d0/d2/d34/d44/d53/f68 [0,4194304] 0 2024-02-16T19:24:34.483 INFO:tasks.workunit.client.1.smithi163.stdout:5/975: write d0/d22/d28/f38 [1724761,29511] 0 2024-02-16T19:24:34.484 INFO:tasks.workunit.client.1.smithi163.stdout:8/827: symlink d0/d24/d2c/ddf/de5/d107/l10d 0 2024-02-16T19:24:34.486 INFO:tasks.workunit.client.1.smithi163.stdout:8/828: rename d0/d7/d18/d53/ded/d85/cab to d0/d7/d19/d1f/d30/d6a/d7b/c10e 0 2024-02-16T19:24:34.486 INFO:tasks.workunit.client.1.smithi163.stdout:8/829: dread - d0/d7/fdb zero size 2024-02-16T19:24:34.486 INFO:tasks.workunit.client.1.smithi163.stdout:8/830: write d0/dc/f2d [6537318,28684] 0 2024-02-16T19:24:34.488 INFO:tasks.workunit.client.1.smithi163.stdout:8/831: link d0/d7/d18/d2e/dd0/lf1 d0/d7/d19/d1f/d32/d70/l10f 0 2024-02-16T19:24:34.490 INFO:tasks.workunit.client.1.smithi163.stdout:8/832: mkdir d0/d7/d19/d1f/d30/d48/d110 0 2024-02-16T19:24:34.492 INFO:tasks.workunit.client.1.smithi163.stdout:6/752: dread d0/dd/f6d [0,4194304] 0 2024-02-16T19:24:34.492 INFO:tasks.workunit.client.1.smithi163.stdout:6/753: readlink d0/dd/d7f/d81/le8 0 2024-02-16T19:24:34.492 INFO:tasks.workunit.client.1.smithi163.stdout:6/754: truncate d0/d18/d77/d70/fe7 963940 0 2024-02-16T19:24:34.492 INFO:tasks.workunit.client.1.smithi163.stdout:6/755: dread - d0/dd/d7f/d81/d85/da4/fef zero size 2024-02-16T19:24:34.493 INFO:tasks.workunit.client.1.smithi163.stdout:6/756: symlink d0/d18/d77/d97/lf3 0 2024-02-16T19:24:34.494 INFO:tasks.workunit.client.1.smithi163.stdout:6/757: dread d0/d6/df/d20/d40/f59 [0,4194304] 0 2024-02-16T19:24:34.496 INFO:tasks.workunit.client.1.smithi163.stdout:6/758: link d0/d6/f31 d0/d6/df/d8f/dcd/ff4 0 2024-02-16T19:24:34.497 INFO:tasks.workunit.client.1.smithi163.stdout:6/759: creat d0/d6/df/d20/d69/db4/ff5 x:0 0 0 2024-02-16T19:24:34.499 INFO:tasks.workunit.client.1.smithi163.stdout:8/833: dread d0/d24/d2c/d74/fd7 [0,4194304] 0 2024-02-16T19:24:34.499 INFO:tasks.workunit.client.1.smithi163.stdout:8/834: write d0/d7/d19/fa8 [854107,112593] 0 2024-02-16T19:24:34.499 INFO:tasks.workunit.client.1.smithi163.stdout:8/835: write d0/faa [3433061,69026] 0 2024-02-16T19:24:34.501 INFO:tasks.workunit.client.1.smithi163.stdout:3/825: dwrite d0/dc/d50/d66/da3/d71/f87 [0,4194304] 0 2024-02-16T19:24:34.501 INFO:tasks.workunit.client.1.smithi163.stdout:3/826: dread - d0/dc/d1c/d91/ff9 zero size 2024-02-16T19:24:34.501 INFO:tasks.workunit.client.1.smithi163.stdout:0/997: dwrite d11/d12/d67/f144 [0,4194304] 0 2024-02-16T19:24:34.503 INFO:tasks.workunit.client.1.smithi163.stdout:6/760: mkdir d0/dd/d7f/d81/d85/da4/df6 0 2024-02-16T19:24:34.505 INFO:tasks.workunit.client.1.smithi163.stdout:5/976: dwrite d0/d22/d28/d111/da8/f10a [0,4194304] 0 2024-02-16T19:24:34.508 INFO:tasks.workunit.client.1.smithi163.stdout:3/827: truncate d0/dc/d50/d66/fec 2326883 0 2024-02-16T19:24:34.508 INFO:tasks.workunit.client.1.smithi163.stdout:6/761: dread d0/d18/f58 [0,4194304] 0 2024-02-16T19:24:34.514 INFO:tasks.workunit.client.1.smithi163.stdout:0/998: unlink d11/d22/d3e/l4b 0 2024-02-16T19:24:34.514 INFO:tasks.workunit.client.1.smithi163.stdout:0/999: read d11/d12/d67/d6a/d6d/f7d [692721,86994] 0 2024-02-16T19:24:34.528 INFO:tasks.workunit.client.1.smithi163.stdout:3/828: symlink d0/dc/d1c/d32/db8/d99/l125 0 2024-02-16T19:24:34.529 INFO:tasks.workunit.client.1.smithi163.stdout:3/829: write d0/dc/d1c/d32/df0/ff1 [378138,61702] 0 2024-02-16T19:24:34.529 INFO:tasks.workunit.client.1.smithi163.stdout:3/830: write d0/dc/d1c/f101 [237433,80877] 0 2024-02-16T19:24:34.536 INFO:tasks.workunit.client.1.smithi163.stdout:6/762: getdents d0/dd/dbc 0 2024-02-16T19:24:34.539 INFO:tasks.workunit.client.1.smithi163.stdout:3/831: mkdir d0/dc/d94/d9e/d126 0 2024-02-16T19:24:34.539 INFO:tasks.workunit.client.1.smithi163.stdout:6/763: dread d0/dd/dbc/ddc/de5/f60 [0,4194304] 0 2024-02-16T19:24:34.541 INFO:tasks.workunit.client.1.smithi163.stdout:6/764: truncate d0/d6/f92 1754004 0 2024-02-16T19:24:34.542 INFO:tasks.workunit.client.1.smithi163.stdout:6/765: mkdir d0/dd/df7 0 2024-02-16T19:24:34.543 INFO:tasks.workunit.client.1.smithi163.stdout:6/766: rename d0/d6/df/d7b/d87/dd0 to d0/dd/d7f/d15/df8 0 2024-02-16T19:24:34.547 INFO:tasks.workunit.client.1.smithi163.stdout:6/767: write d0/dd/dbc/ddc/de5/f3d [617690,116809] 0 2024-02-16T19:24:34.547 INFO:tasks.workunit.client.1.smithi163.stdout:5/977: dwrite d0/d22/d28/f95 [0,4194304] 0 2024-02-16T19:24:34.548 INFO:tasks.workunit.client.1.smithi163.stdout:6/768: read d0/d6/df/f83 [52342,55617] 0 2024-02-16T19:24:34.552 INFO:tasks.workunit.client.1.smithi163.stdout:3/832: dread d0/dc/d50/fd3 [0,4194304] 0 2024-02-16T19:24:34.555 INFO:tasks.workunit.client.1.smithi163.stdout:6/769: rename d0/dd/dbc/ddc/de5/f3c to d0/d18/d77/d97/ff9 0 2024-02-16T19:24:34.555 INFO:tasks.workunit.client.1.smithi163.stdout:6/770: truncate d0/dd/d7f/d2e/fbd 4017 0 2024-02-16T19:24:34.558 INFO:tasks.workunit.client.1.smithi163.stdout:5/978: mknod d0/d2/d134/c148 0 2024-02-16T19:24:34.560 INFO:tasks.workunit.client.1.smithi163.stdout:6/771: creat d0/d6/df/d8f/dcd/ffa x:0 0 0 2024-02-16T19:24:34.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/772: dread d0/d18/d77/dc9/f72 [0,4194304] 0 2024-02-16T19:24:34.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/773: creat d0/dd/d7f/d81/daa/ffb x:0 0 0 2024-02-16T19:24:34.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/774: dread - d0/dd/dbc/fd4 zero size 2024-02-16T19:24:34.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/775: fdatasync d0/d6/df/d20/d40/d17/fa2 0 2024-02-16T19:24:34.567 INFO:tasks.workunit.client.1.smithi163.stdout:3/833: dwrite d0/dc/d50/f8b [4194304,4194304] 0 2024-02-16T19:24:34.567 INFO:tasks.workunit.client.1.smithi163.stdout:3/834: stat d0/dc/d50/d66/d80/f10d 0 2024-02-16T19:24:34.569 INFO:tasks.workunit.client.1.smithi163.stdout:5/979: write d0/f4d [563896,105218] 0 2024-02-16T19:24:34.571 INFO:tasks.workunit.client.1.smithi163.stdout:5/980: mknod d0/d2/c149 0 2024-02-16T19:24:34.571 INFO:tasks.workunit.client.1.smithi163.stdout:5/981: read d0/d1c/d5a/f96 [650571,7862] 0 2024-02-16T19:24:34.571 INFO:tasks.workunit.client.1.smithi163.stdout:5/982: truncate d0/d1c/d5a/fcf 4644313 0 2024-02-16T19:24:34.574 INFO:tasks.workunit.client.1.smithi163.stdout:3/835: read d0/dc/d1d/d49/f56 [3200842,102020] 0 2024-02-16T19:24:34.582 INFO:tasks.workunit.client.1.smithi163.stdout:3/836: dread d0/dc/d1c/d32/f57 [4194304,4194304] 0 2024-02-16T19:24:34.582 INFO:tasks.workunit.client.1.smithi163.stdout:3/837: fsync d0/dc/d1d/d49/d98/fe9 0 2024-02-16T19:24:34.586 INFO:tasks.workunit.client.1.smithi163.stdout:5/983: dread d0/d22/d28/fad [0,4194304] 0 2024-02-16T19:24:34.586 INFO:tasks.workunit.client.1.smithi163.stdout:6/776: dwrite d0/d6/df/d20/d69/db4/fb9 [0,4194304] 0 2024-02-16T19:24:34.586 INFO:tasks.workunit.client.1.smithi163.stdout:5/984: creat d0/d22/d28/d36/db7/f14a x:0 0 0 2024-02-16T19:24:34.586 INFO:tasks.workunit.client.1.smithi163.stdout:6/777: creat d0/d6/df/d8f/dcd/ffc x:0 0 0 2024-02-16T19:24:34.586 INFO:tasks.workunit.client.1.smithi163.stdout:3/838: rename d0/d3e to d0/d83/da5/d127 0 2024-02-16T19:24:34.613 INFO:tasks.workunit.client.1.smithi163.stdout:3/839: dwrite d0/d83/f108 [0,4194304] 0 2024-02-16T19:24:34.613 INFO:tasks.workunit.client.1.smithi163.stdout:3/840: fdatasync d0/dc/d1c/d32/df0/ff1 0 2024-02-16T19:24:34.613 INFO:tasks.workunit.client.1.smithi163.stdout:6/778: dwrite d0/dd/dbc/fd4 [0,4194304] 0 2024-02-16T19:24:34.613 INFO:tasks.workunit.client.1.smithi163.stdout:6/779: creat d0/d6/df/d20/ffd x:0 0 0 2024-02-16T19:24:34.614 INFO:tasks.workunit.client.1.smithi163.stdout:5/985: dwrite d0/d1c/f11f [0,4194304] 0 2024-02-16T19:24:34.616 INFO:tasks.workunit.client.1.smithi163.stdout:3/841: link d0/dc/d1d/d49/c70 d0/dc/d1c/d21/deb/c128 0 2024-02-16T19:24:34.625 INFO:tasks.workunit.client.1.smithi163.stdout:6/780: creat d0/d6/df/d8f/da9/ffe x:0 0 0 2024-02-16T19:24:34.625 INFO:tasks.workunit.client.1.smithi163.stdout:6/781: chown d0/dd/d7f/d81 1 1 2024-02-16T19:24:34.631 INFO:tasks.workunit.client.1.smithi163.stdout:5/986: rename d0/d22/d28/f38 to d0/d22/d3b/d8d/f14b 0 2024-02-16T19:24:34.631 INFO:tasks.workunit.client.1.smithi163.stdout:5/987: readlink d0/d1c/d5a/da5/lf6 0 2024-02-16T19:24:34.632 INFO:tasks.workunit.client.1.smithi163.stdout:3/842: creat d0/dc/d1c/d32/db8/f129 x:0 0 0 2024-02-16T19:24:34.633 INFO:tasks.workunit.client.1.smithi163.stdout:3/843: dread - d0/dc/d50/f11a zero size 2024-02-16T19:24:34.633 INFO:tasks.workunit.client.1.smithi163.stdout:3/844: fsync d0/d83/f108 0 2024-02-16T19:24:34.633 INFO:tasks.workunit.client.1.smithi163.stdout:5/988: readlink d0/d2/d5c/l76 0 2024-02-16T19:24:34.633 INFO:tasks.workunit.client.1.smithi163.stdout:5/989: creat d0/d22/d28/d36/d3c/d104/f14c x:0 0 0 2024-02-16T19:24:34.635 INFO:tasks.workunit.client.1.smithi163.stdout:3/845: mkdir d0/dc/d1d/d49/d12a 0 2024-02-16T19:24:34.635 INFO:tasks.workunit.client.1.smithi163.stdout:3/846: dread - d0/dc/d50/d66/d80/fb7 zero size 2024-02-16T19:24:34.639 INFO:tasks.workunit.client.1.smithi163.stdout:5/990: rmdir d0/d1c/d60 39 2024-02-16T19:24:34.639 INFO:tasks.workunit.client.1.smithi163.stdout:5/991: write d0/d1c/dbe/de3/fec [5138098,88383] 0 2024-02-16T19:24:34.641 INFO:tasks.workunit.client.1.smithi163.stdout:5/992: mknod d0/d2/d34/d44/dd4/c14d 0 2024-02-16T19:24:34.643 INFO:tasks.workunit.client.1.smithi163.stdout:5/993: symlink d0/d22/d28/d36/db7/l14e 0 2024-02-16T19:24:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:34 smithi040 ceph-mon[29201]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:34 smithi040 ceph-mon[29201]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:34 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/448262867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:24:34.644 INFO:tasks.workunit.client.1.smithi163.stdout:5/994: mknod d0/d22/d28/d36/d3c/d4f/c14f 0 2024-02-16T19:24:34.655 INFO:tasks.workunit.client.1.smithi163.stdout:6/782: dwrite d0/dd/dbc/ddc/de5/f5f [0,4194304] 0 2024-02-16T19:24:34.655 INFO:tasks.workunit.client.1.smithi163.stdout:3/847: dwrite d0/dc/d50/f11a [0,4194304] 0 2024-02-16T19:24:34.656 INFO:tasks.workunit.client.1.smithi163.stdout:6/783: mkdir d0/dd/d7f/d15/d1c/d65/d63/d86/dff 0 2024-02-16T19:24:34.658 INFO:tasks.workunit.client.1.smithi163.stdout:3/848: mknod d0/dc/d1d/d29/d38/c12b 0 2024-02-16T19:24:34.658 INFO:tasks.workunit.client.1.smithi163.stdout:3/849: write d0/f8 [3826922,43007] 0 2024-02-16T19:24:34.658 INFO:tasks.workunit.client.1.smithi163.stdout:3/850: readlink d0/dc/d1c/lcd 0 2024-02-16T19:24:34.658 INFO:tasks.workunit.client.1.smithi163.stdout:3/851: write d0/dc/d94/d9e/dac/fd0 [1637390,123604] 0 2024-02-16T19:24:34.658 INFO:tasks.workunit.client.1.smithi163.stdout:3/852: creat d0/dc/d1c/d91/f12c x:0 0 0 2024-02-16T19:24:34.675 INFO:tasks.workunit.client.1.smithi163.stdout:6/784: dwrite d0/d6/df/d20/f68 [0,4194304] 0 2024-02-16T19:24:34.676 INFO:tasks.workunit.client.1.smithi163.stdout:7/778: sync 2024-02-16T19:24:34.676 INFO:tasks.workunit.client.1.smithi163.stdout:4/788: sync 2024-02-16T19:24:34.677 INFO:tasks.workunit.client.1.smithi163.stdout:6/785: mknod d0/dd/d7f/d81/d85/da4/df2/c100 0 2024-02-16T19:24:34.678 INFO:tasks.workunit.client.1.smithi163.stdout:7/779: truncate d11/d24/d49/f69 1196548 0 2024-02-16T19:24:34.678 INFO:tasks.workunit.client.1.smithi163.stdout:7/780: chown d11/c80 285 1 2024-02-16T19:24:34.679 INFO:tasks.workunit.client.1.smithi163.stdout:7/781: symlink d11/d24/d49/d105/l10c 0 2024-02-16T19:24:34.679 INFO:tasks.workunit.client.1.smithi163.stdout:7/782: chown d11/d24/d49 599898 1 2024-02-16T19:24:34.681 INFO:tasks.workunit.client.1.smithi163.stdout:7/783: dread d11/d24/d25/d4b/d66/f70 [0,4194304] 0 2024-02-16T19:24:34.686 INFO:tasks.workunit.client.1.smithi163.stdout:4/789: dread d11/d18/d1f/d23/d3a/d4a/f6e [0,4194304] 0 2024-02-16T19:24:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.686+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f87d8005da0 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.686+0000 7f87d37fe700 1 -- 172.21.15.40:0/1301358190 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1854 (secure 0 0 0) 0x7f87e00584d0 con 0x7f87ec0fe0e0 2024-02-16T19:24:34.688 INFO:tasks.workunit.client.1.smithi163.stdout:4/790: link d11/d2d/d4f/d5f/fff d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/f10b 0 2024-02-16T19:24:34.688 INFO:tasks.workunit.client.1.smithi163.stdout:4/791: write d11/d2d/d4f/d5f/d70/d74/fd4 [4984215,94225] 0 2024-02-16T19:24:34.688 INFO:teuthology.orchestra.run.smithi040.stdout:e19 2024-02-16T19:24:34.688 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:24:34.688 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:24:34.688 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 19 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:24:09.512876+0000 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:24:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:24:34.690 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:24:34.691 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:24:34.691 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:24:34.691 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:24:34.691 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:24:34.691 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:24:34.691 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:24:34.692 INFO:tasks.workunit.client.1.smithi163.stdout:4/792: link d11/d18/d66/l6d d11/d2d/d4f/d5f/d70/l10c 0 2024-02-16T19:24:34.692 INFO:tasks.workunit.client.1.smithi163.stdout:4/793: write d11/d18/f1d [4871077,127925] 0 2024-02-16T19:24:34.692 INFO:tasks.workunit.client.1.smithi163.stdout:4/794: write d11/d3d/fbc [516013,101240] 0 2024-02-16T19:24:34.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f87d40692b0 msgr2=0x7f87d406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f87d40692b0 0x7f87d406b760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009b80 tx=0x7f87dc00b040 comp rx=0 tx=0).stop 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 msgr2=0x7f87ec10de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 0x7f87ec10de30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f87e0000c00 tx=0x7f87e0009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 shutdown_connections 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7f87d40692b0 0x7f87d406b760 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0fcef0 0x7f87ec10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 --2- 172.21.15.40:0/1301358190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec0fe0e0 0x7f87ec10de30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 >> 172.21.15.40:0/1301358190 conn(0x7f87ec0f8620 msgr2=0x7f87ec1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 shutdown_connections 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.689+0000 7f87f17b0700 1 -- 172.21.15.40:0/1301358190 wait complete. 2024-02-16T19:24:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 19 2024-02-16T19:24:34.694 INFO:tasks.workunit.client.1.smithi163.stdout:4/795: creat d11/d18/d1f/d23/f10d x:0 0 0 2024-02-16T19:24:34.694 INFO:tasks.workunit.client.1.smithi163.stdout:4/796: chown d11/d18/d1f/d29/fa1 416 1 2024-02-16T19:24:34.694 INFO:tasks.workunit.client.1.smithi163.stdout:4/797: dread - d11/f46 zero size 2024-02-16T19:24:34.694 INFO:tasks.workunit.client.1.smithi163.stdout:6/786: dwrite d0/dd/d7f/d2e/f4f [0,4194304] 0 2024-02-16T19:24:34.694 INFO:tasks.workunit.client.1.smithi163.stdout:4/798: link d11/d18/d1f/d29/d52/f6a d11/d2d/d4f/d5f/d70/d74/d75/f10e 0 2024-02-16T19:24:34.700 INFO:tasks.workunit.client.1.smithi163.stdout:4/799: unlink d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/fb5 0 2024-02-16T19:24:34.700 INFO:tasks.workunit.client.1.smithi163.stdout:4/800: creat d11/d2d/d4f/d5f/f10f x:0 0 0 2024-02-16T19:24:34.701 INFO:tasks.workunit.client.1.smithi163.stdout:4/801: write d11/d2d/d4f/d60/f8d [4802863,40819] 0 2024-02-16T19:24:34.709 INFO:tasks.workunit.client.1.smithi163.stdout:6/787: dwrite d0/d6/df/d20/d40/f89 [0,4194304] 0 2024-02-16T19:24:34.709 INFO:tasks.workunit.client.1.smithi163.stdout:6/788: write d0/d6/df/d8f/dcd/ffc [338068,59859] 0 2024-02-16T19:24:34.711 INFO:tasks.workunit.client.1.smithi163.stdout:4/802: write d11/d18/d1f/f27 [1881149,35200] 0 2024-02-16T19:24:34.713 INFO:tasks.workunit.client.1.smithi163.stdout:6/789: dread d0/d6/df/d20/d40/f71 [0,4194304] 0 2024-02-16T19:24:34.713 INFO:tasks.workunit.client.1.smithi163.stdout:6/790: write d0/d6/f45 [1085198,46317] 0 2024-02-16T19:24:34.714 INFO:tasks.workunit.client.1.smithi163.stdout:6/791: dread d0/d18/d77/d70/f8c [0,4194304] 0 2024-02-16T19:24:34.715 INFO:tasks.workunit.client.1.smithi163.stdout:6/792: mknod d0/d6/df/d20/d40/d1e/c101 0 2024-02-16T19:24:34.716 INFO:tasks.workunit.client.1.smithi163.stdout:6/793: symlink d0/dd/d7f/d81/d85/da4/l102 0 2024-02-16T19:24:34.718 INFO:tasks.workunit.client.1.smithi163.stdout:6/794: link d0/le0 d0/dd/dbc/ddc/l103 0 2024-02-16T19:24:34.718 INFO:tasks.workunit.client.1.smithi163.stdout:6/795: write d0/d6/df/f83 [269721,30561] 0 2024-02-16T19:24:34.718 INFO:tasks.workunit.client.1.smithi163.stdout:6/796: mknod d0/dd/d7f/d15/df8/c104 0 2024-02-16T19:24:34.718 INFO:tasks.workunit.client.1.smithi163.stdout:6/797: dread - d0/d6/df/d20/ffd zero size 2024-02-16T19:24:34.726 INFO:tasks.workunit.client.1.smithi163.stdout:4/803: dwrite d11/d2d/d4f/d60/d79/fd3 [0,4194304] 0 2024-02-16T19:24:34.727 INFO:tasks.workunit.client.1.smithi163.stdout:6/798: dread d0/dd/dbc/ddc/de5/f5f [0,4194304] 0 2024-02-16T19:24:34.728 INFO:tasks.workunit.client.1.smithi163.stdout:4/804: symlink d11/d18/d1f/d29/d52/l110 0 2024-02-16T19:24:34.729 INFO:tasks.workunit.client.1.smithi163.stdout:4/805: mkdir d11/d2d/d4f/d60/d111 0 2024-02-16T19:24:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:34 smithi163 ceph-mon[36452]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:34 smithi163 ceph-mon[36452]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:34 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/448262867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:24:34.741 INFO:tasks.workunit.client.1.smithi163.stdout:6/799: dwrite d0/d6/df/d20/d69/db4/ff5 [0,4194304] 0 2024-02-16T19:24:34.744 INFO:tasks.workunit.client.1.smithi163.stdout:6/800: link d0/d18/d77/fdb d0/d6/df/d20/d40/d1e/db2/dda/f105 0 2024-02-16T19:24:34.749 INFO:tasks.workunit.client.1.smithi163.stdout:6/801: mkdir d0/dd/d7f/d81/d85/da4/df6/d106 0 2024-02-16T19:24:34.759 INFO:tasks.workunit.client.1.smithi163.stdout:6/802: write d0/dd/d7f/f73 [3639811,47830] 0 2024-02-16T19:24:34.767 INFO:tasks.workunit.client.1.smithi163.stdout:6/803: rmdir d0 39 2024-02-16T19:24:34.767 INFO:tasks.workunit.client.1.smithi163.stdout:6/804: getdents d0/dd/d7f/d15 0 2024-02-16T19:24:34.769 INFO:tasks.workunit.client.1.smithi163.stdout:6/805: rename d0/d6/df/c4a to d0/d6/df/d20/d40/d1e/c107 0 2024-02-16T19:24:34.770 INFO:tasks.workunit.client.1.smithi163.stdout:6/806: write d0/dd/d7f/d15/d1c/f99 [185771,33454] 0 2024-02-16T19:24:34.770 INFO:tasks.workunit.client.1.smithi163.stdout:6/807: symlink d0/dd/d7f/d15/df8/l108 0 2024-02-16T19:24:34.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/808: mknod d0/d6/df/c109 0 2024-02-16T19:24:34.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/809: stat d0/dd/d7f/d15/d1c/d65/d63/d86/c96 0 2024-02-16T19:24:34.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/810: fsync d0/d6/df/d20/d40/d1e/db2/dda/f105 0 2024-02-16T19:24:34.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/811: fsync d0/dd/dbc/fd4 0 2024-02-16T19:24:34.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/812: stat d0/dd/d7f/d15/df8 0 2024-02-16T19:24:34.772 INFO:tasks.workunit.client.1.smithi163.stdout:6/813: dread d0/d6/df/d20/d40/d1e/db2/dda/f105 [0,4194304] 0 2024-02-16T19:24:34.773 INFO:tasks.workunit.client.1.smithi163.stdout:6/814: creat d0/dd/d7f/d15/d1c/dca/f10a x:0 0 0 2024-02-16T19:24:34.778 INFO:tasks.workunit.client.1.smithi163.stdout:6/815: write d0/dd/f6d [634165,30184] 0 2024-02-16T19:24:34.778 INFO:tasks.workunit.client.1.smithi163.stdout:6/816: chown d0/d18/d94/fac 1900 1 2024-02-16T19:24:34.778 INFO:tasks.workunit.client.1.smithi163.stdout:6/817: fsync d0/d6/df/d20/d69/f8d 0 2024-02-16T19:24:34.779 INFO:tasks.workunit.client.1.smithi163.stdout:6/818: creat d0/dd/d7f/d81/d85/da4/dad/f10b x:0 0 0 2024-02-16T19:24:34.780 INFO:tasks.workunit.client.1.smithi163.stdout:6/819: dread - d0/d6/df/d8f/f91 zero size 2024-02-16T19:24:34.780 INFO:tasks.workunit.client.1.smithi163.stdout:6/820: read d0/d6/f31 [927883,75055] 0 2024-02-16T19:24:34.781 INFO:tasks.workunit.client.1.smithi163.stdout:6/821: symlink d0/d6/df/d20/d40/d1e/l10c 0 2024-02-16T19:24:34.781 INFO:tasks.workunit.client.1.smithi163.stdout:6/822: getdents d0/dd/d7f/d15/d1c/d65/d63/d86/dff 0 2024-02-16T19:24:34.781 INFO:tasks.workunit.client.1.smithi163.stdout:6/823: fdatasync d0/d6/f8 0 2024-02-16T19:24:34.785 INFO:tasks.workunit.client.1.smithi163.stdout:8/836: sync 2024-02-16T19:24:34.785 INFO:tasks.workunit.client.1.smithi163.stdout:2/957: sync 2024-02-16T19:24:34.785 INFO:tasks.workunit.client.1.smithi163.stdout:1/880: sync 2024-02-16T19:24:34.786 INFO:tasks.workunit.client.1.smithi163.stdout:8/837: rename d0/d6f/fc8 to d0/d24/f111 0 2024-02-16T19:24:34.787 INFO:tasks.workunit.client.1.smithi163.stdout:1/881: link d7/d20/cf3 d7/dc/d2d/d32/d10f/c12b 0 2024-02-16T19:24:34.787 INFO:tasks.workunit.client.1.smithi163.stdout:1/882: fdatasync d7/d20/d21/d4e/f79 0 2024-02-16T19:24:34.787 INFO:tasks.workunit.client.1.smithi163.stdout:2/958: symlink d2/d26/d54/d58/d98/l14c 0 2024-02-16T19:24:34.789 INFO:tasks.workunit.client.1.smithi163.stdout:1/883: dread d7/dc/d2d/d32/d3b/fdb [0,4194304] 0 2024-02-16T19:24:34.789 INFO:tasks.workunit.client.1.smithi163.stdout:1/884: fdatasync d7/d20/d21/d74/dab/f102 0 2024-02-16T19:24:34.794 INFO:tasks.workunit.client.1.smithi163.stdout:8/838: rename d0/d24 to d0/d7/d18/d112 0 2024-02-16T19:24:34.795 INFO:tasks.workunit.client.1.smithi163.stdout:8/839: write d0/d7/d18/d53/ded/fe6 [5146078,27821] 0 2024-02-16T19:24:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.817+0000 7fbf34f72700 1 -- 172.21.15.40:0/4026292972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fbe80 msgr2=0x7fbf300fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.817+0000 7fbf34f72700 1 --2- 172.21.15.40:0/4026292972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fbe80 0x7fbf300fc290 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbf20004940 tx=0x7fbf20009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.817+0000 7fbf34f72700 1 -- 172.21.15.40:0/4026292972 shutdown_connections 2024-02-16T19:24:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.817+0000 7fbf34f72700 1 --2- 172.21.15.40:0/4026292972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300fd080 0x7fbf300fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.817+0000 7fbf34f72700 1 --2- 172.21.15.40:0/4026292972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fbe80 0x7fbf300fc290 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.817+0000 7fbf34f72700 1 -- 172.21.15.40:0/4026292972 >> 172.21.15.40:0/4026292972 conn(0x7fbf300f7590 msgr2=0x7fbf300f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.817+0000 7fbf34f72700 1 -- 172.21.15.40:0/4026292972 shutdown_connections 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.817+0000 7fbf34f72700 1 -- 172.21.15.40:0/4026292972 wait complete. 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf34f72700 1 Processor -- start 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf34f72700 1 -- start start 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf34f72700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300fbe80 0x7fbf3010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf34f72700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fd080 0x7fbf3010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf34f72700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3010e450 con 0x7fbf300fd080 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf34f72700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3010ecf0 con 0x7fbf300fbe80 2024-02-16T19:24:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf2dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fd080 0x7fbf3010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf2dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fd080 0x7fbf3010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37196/0 (socket says 172.21.15.40:37196) 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf2dd9b700 1 -- 172.21.15.40:0/54928473 learned_addr learned my addr 172.21.15.40:0/54928473 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf2e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300fbe80 0x7fbf3010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf2dd9b700 1 -- 172.21.15.40:0/54928473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300fbe80 msgr2=0x7fbf3010d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf2dd9b700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300fbe80 0x7fbf3010d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf2dd9b700 1 -- 172.21.15.40:0/54928473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf20005040 con 0x7fbf300fd080 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.818+0000 7fbf2e59c700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300fbe80 0x7fbf3010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.819+0000 7fbf2dd9b700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fd080 0x7fbf3010de50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400eb40 tx=0x7fbf2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.819+0000 7fbf1effd700 1 -- 172.21.15.40:0/54928473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf24007cc0 con 0x7fbf300fd080 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.819+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3010ef70 con 0x7fbf300fd080 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.819+0000 7fbf1effd700 1 -- 172.21.15.40:0/54928473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf2400f040 con 0x7fbf300fd080 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.819+0000 7fbf1effd700 1 -- 172.21.15.40:0/54928473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf2400ca10 con 0x7fbf300fd080 2024-02-16T19:24:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.819+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3010f4b0 con 0x7fbf300fd080 2024-02-16T19:24:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.820+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3004bed0 con 0x7fbf300fd080 2024-02-16T19:24:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.820+0000 7fbf1effd700 1 -- 172.21.15.40:0/54928473 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbf2400cc80 con 0x7fbf300fd080 2024-02-16T19:24:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.821+0000 7fbf1effd700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbf180692a0 0x7fbf1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.821+0000 7fbf1effd700 1 -- 172.21.15.40:0/54928473 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbf24086dd0 con 0x7fbf300fd080 2024-02-16T19:24:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.821+0000 7fbf2e59c700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbf180692a0 0x7fbf1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:34.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.821+0000 7fbf2e59c700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbf180692a0 0x7fbf1806b750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf20000c00 tx=0x7fbf20009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:34.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:34.825+0000 7fbf1effd700 1 -- 172.21.15.40:0/54928473 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbf2404dbd0 con 0x7fbf300fd080 2024-02-16T19:24:34.845 INFO:tasks.workunit.client.1.smithi163.stdout:2/959: dwrite d2/d2a/d9f/fb9 [0,4194304] 0 2024-02-16T19:24:34.852 INFO:tasks.workunit.client.1.smithi163.stdout:8/840: dwrite d0/d7/d19/d1f/d30/d48/f65 [0,4194304] 0 2024-02-16T19:24:34.854 INFO:tasks.workunit.client.1.smithi163.stdout:8/841: symlink d0/dc/de3/l113 0 2024-02-16T19:24:34.862 INFO:tasks.workunit.client.1.smithi163.stdout:8/842: fdatasync d0/dc/f2d 0 2024-02-16T19:24:34.862 INFO:tasks.workunit.client.1.smithi163.stdout:8/843: readlink d0/d7/d18/d112/d3c/d3d/l2b 0 2024-02-16T19:24:34.862 INFO:tasks.workunit.client.1.smithi163.stdout:2/960: dwrite d2/d44/ff4 [0,4194304] 0 2024-02-16T19:24:34.862 INFO:tasks.workunit.client.1.smithi163.stdout:8/844: creat d0/d7/d19/d1f/d30/d6a/d7b/deb/f114 x:0 0 0 2024-02-16T19:24:34.864 INFO:tasks.workunit.client.1.smithi163.stdout:8/845: rename d0/d7/d18/d112/d2c/d74/c8d to d0/d7/d19/d1f/d30/d6a/df3/df2/c115 0 2024-02-16T19:24:34.868 INFO:tasks.workunit.client.1.smithi163.stdout:8/846: dread d0/dc/f2d [4194304,4194304] 0 2024-02-16T19:24:34.871 INFO:tasks.workunit.client.1.smithi163.stdout:2/961: write d2/d26/f28 [1090426,56251] 0 2024-02-16T19:24:34.871 INFO:tasks.workunit.client.1.smithi163.stdout:2/962: creat d2/d26/d97/f14d x:0 0 0 2024-02-16T19:24:34.871 INFO:tasks.workunit.client.1.smithi163.stdout:2/963: chown d2/d26/f2f 0 1 2024-02-16T19:24:34.872 INFO:tasks.workunit.client.1.smithi163.stdout:2/964: readlink d2/d2a/d2d/d73/d84/l93 0 2024-02-16T19:24:34.872 INFO:tasks.workunit.client.1.smithi163.stdout:2/965: fsync d2/d26/d97/d10f/f149 0 2024-02-16T19:24:34.872 INFO:tasks.workunit.client.1.smithi163.stdout:2/966: chown d2/d26/l48 211885776 1 2024-02-16T19:24:34.875 INFO:tasks.workunit.client.1.smithi163.stdout:2/967: creat d2/d2a/d2d/d73/d94/f14e x:0 0 0 2024-02-16T19:24:34.875 INFO:tasks.workunit.client.1.smithi163.stdout:2/968: truncate d2/d26/d54/f116 1857013 0 2024-02-16T19:24:34.876 INFO:tasks.workunit.client.1.smithi163.stdout:2/969: write d2/d2a/d39/d3a/fc2 [1430096,10220] 0 2024-02-16T19:24:34.876 INFO:tasks.workunit.client.1.smithi163.stdout:2/970: symlink d2/d26/d54/dfb/d114/l14f 0 2024-02-16T19:24:34.877 INFO:tasks.workunit.client.1.smithi163.stdout:8/847: dread d0/d7/d18/d112/d2c/d74/dcb/fee [0,4194304] 0 2024-02-16T19:24:34.879 INFO:tasks.workunit.client.1.smithi163.stdout:2/971: rename d2/d44/f138 to d2/d2a/d39/f150 0 2024-02-16T19:24:34.880 INFO:tasks.workunit.client.1.smithi163.stdout:2/972: truncate d2/d2a/d2d/d73/d94/dc9/fcf 541645 0 2024-02-16T19:24:34.881 INFO:tasks.workunit.client.1.smithi163.stdout:2/973: truncate d2/d2a/d2d/d73/d84/dba/f140 369294 0 2024-02-16T19:24:34.881 INFO:tasks.workunit.client.1.smithi163.stdout:2/974: readlink d2/d26/d54/d58/da6/lf0 0 2024-02-16T19:24:34.881 INFO:tasks.workunit.client.1.smithi163.stdout:2/975: chown d2/d26/d54/d58/dff/lbe 7925 1 2024-02-16T19:24:34.881 INFO:tasks.workunit.client.1.smithi163.stdout:2/976: write d2/dce/dee/f11e [4932909,100077] 0 2024-02-16T19:24:34.882 INFO:tasks.workunit.client.1.smithi163.stdout:2/977: truncate d2/d26/d54/d58/d8c/ff1 3445254 0 2024-02-16T19:24:34.887 INFO:tasks.workunit.client.1.smithi163.stdout:2/978: mknod d2/d26/d54/d58/d7d/c151 0 2024-02-16T19:24:34.887 INFO:tasks.workunit.client.1.smithi163.stdout:8/848: dread d0/d7/d18/d2e/f7f [0,4194304] 0 2024-02-16T19:24:34.889 INFO:tasks.workunit.client.1.smithi163.stdout:2/979: truncate d2/d2a/d39/f75 672768 0 2024-02-16T19:24:34.889 INFO:tasks.workunit.client.1.smithi163.stdout:2/980: write d2/d2a/f13c [23480,34743] 0 2024-02-16T19:24:34.889 INFO:tasks.workunit.client.1.smithi163.stdout:2/981: write d2/ddb/d7c/f99 [1471891,19458] 0 2024-02-16T19:24:34.889 INFO:tasks.workunit.client.1.smithi163.stdout:5/995: sync 2024-02-16T19:24:34.898 INFO:tasks.workunit.client.1.smithi163.stdout:8/849: dread d0/d6f/f98 [0,4194304] 0 2024-02-16T19:24:34.898 INFO:tasks.workunit.client.1.smithi163.stdout:8/850: readlink d0/d7/d19/d1f/d32/d70/l10f 0 2024-02-16T19:24:34.898 INFO:tasks.workunit.client.1.smithi163.stdout:8/851: readlink d0/d7/d19/d1f/d30/l3a 0 2024-02-16T19:24:34.900 INFO:tasks.workunit.client.1.smithi163.stdout:8/852: mkdir d0/d7/d19/d1f/d30/d6a/db8/d116 0 2024-02-16T19:24:34.916 INFO:tasks.workunit.client.1.smithi163.stdout:5/996: dwrite d0/d2/d34/f100 [0,4194304] 0 2024-02-16T19:24:34.918 INFO:tasks.workunit.client.1.smithi163.stdout:8/853: dwrite d0/d7/d19/d1f/d30/d6a/df3/df2/f5a [0,4194304] 0 2024-02-16T19:24:34.918 INFO:tasks.workunit.client.1.smithi163.stdout:5/997: truncate d0/d1c/d5a/fcf 2725677 0 2024-02-16T19:24:34.919 INFO:tasks.workunit.client.1.smithi163.stdout:8/854: truncate d0/d7/d18/d2e/f5d 3366206 0 2024-02-16T19:24:34.922 INFO:tasks.workunit.client.1.smithi163.stdout:8/855: rename d0/d7/c11 to d0/d7/d18/d2e/dd0/ddd/c117 0 2024-02-16T19:24:34.924 INFO:tasks.workunit.client.1.smithi163.stdout:8/856: mknod d0/d7/d19/d1f/d30/d48/c118 0 2024-02-16T19:24:34.925 INFO:tasks.workunit.client.1.smithi163.stdout:8/857: truncate d0/d7/d19/d1f/d30/f92 761353 0 2024-02-16T19:24:34.925 INFO:tasks.workunit.client.1.smithi163.stdout:8/858: chown d0/d7/d18/d112/d3c/d3d/d41 0 1 2024-02-16T19:24:34.926 INFO:tasks.workunit.client.1.smithi163.stdout:8/859: mknod d0/d7/d18/d53/ded/c119 0 2024-02-16T19:24:34.929 INFO:tasks.workunit.client.1.smithi163.stdout:5/998: dread d0/d22/d28/d111/fa0 [0,4194304] 0 2024-02-16T19:24:34.933 INFO:tasks.workunit.client.1.smithi163.stdout:5/999: creat d0/d22/d28/d36/d3c/d4f/f150 x:0 0 0 2024-02-16T19:24:34.944 INFO:tasks.workunit.client.1.smithi163.stdout:8/860: dwrite d0/d7/d18/d112/d2c/f4d [0,4194304] 0 2024-02-16T19:24:34.945 INFO:tasks.workunit.client.1.smithi163.stdout:8/861: rmdir d0/d7/d18/d112 39 2024-02-16T19:24:34.945 INFO:tasks.workunit.client.1.smithi163.stdout:8/862: fdatasync d0/fea 0 2024-02-16T19:24:34.945 INFO:tasks.workunit.client.1.smithi163.stdout:8/863: truncate d0/d7/d19/fe4 110216 0 2024-02-16T19:24:34.961 INFO:tasks.workunit.client.1.smithi163.stdout:8/864: dwrite d0/d6f/fd2 [0,4194304] 0 2024-02-16T19:24:34.963 INFO:tasks.workunit.client.1.smithi163.stdout:8/865: mkdir d0/d7/d18/d112/d3c/d3d/d11a 0 2024-02-16T19:24:34.965 INFO:tasks.workunit.client.1.smithi163.stdout:8/866: dread d0/d7/d18/d112/d3c/d3d/fde [0,4194304] 0 2024-02-16T19:24:34.974 INFO:tasks.workunit.client.1.smithi163.stdout:8/867: write d0/d7/d18/d112/d2c/d74/fb0 [1357541,4809] 0 2024-02-16T19:24:34.974 INFO:tasks.workunit.client.1.smithi163.stdout:8/868: dread - d0/fea zero size 2024-02-16T19:24:34.976 INFO:tasks.workunit.client.1.smithi163.stdout:8/869: rename d0/d7/d19/d1f/d30/d6a/df3/df2/f4a to d0/d7/d18/d112/d3c/d3d/d41/f11b 0 2024-02-16T19:24:34.977 INFO:tasks.workunit.client.1.smithi163.stdout:8/870: symlink d0/d7/d19/d1f/d32/l11c 0 2024-02-16T19:24:34.977 INFO:tasks.workunit.client.1.smithi163.stdout:8/871: write d0/d7/d19/d1f/d30/d6a/d7b/f109 [582354,26540] 0 2024-02-16T19:24:34.978 INFO:tasks.workunit.client.1.smithi163.stdout:8/872: getdents d0/d7/d19/d1f/d30/d6a/d7b 0 2024-02-16T19:24:34.978 INFO:tasks.workunit.client.1.smithi163.stdout:8/873: dread - d0/d7/d18/d53/ded/d85/f86 zero size 2024-02-16T19:24:34.979 INFO:tasks.workunit.client.1.smithi163.stdout:8/874: symlink d0/d6f/d104/l11d 0 2024-02-16T19:24:34.988 INFO:tasks.workunit.client.1.smithi163.stdout:8/875: write d0/d7/d18/d53/fc5 [1364616,14950] 0 2024-02-16T19:24:34.996 INFO:tasks.workunit.client.1.smithi163.stdout:8/876: dread d0/d7/d18/fad [0,4194304] 0 2024-02-16T19:24:34.997 INFO:tasks.workunit.client.1.smithi163.stdout:8/877: symlink d0/d7/d19/d1f/d30/d6a/db8/d116/l11e 0 2024-02-16T19:24:34.998 INFO:tasks.workunit.client.1.smithi163.stdout:8/878: getdents d0/d7/d19/d1f/d30/d6a/df3/df2 0 2024-02-16T19:24:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.063+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 --> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf3010f900 con 0x7fbf180692a0 2024-02-16T19:24:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.064+0000 7fbf1effd700 1 -- 172.21.15.40:0/54928473 <== mgr.14212 v2:172.21.15.40:6800/1321006798 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7fbf3010f900 con 0x7fbf180692a0 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [], 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "", 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:24:35.066 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:24:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbf180692a0 msgr2=0x7fbf1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbf180692a0 0x7fbf1806b750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf20000c00 tx=0x7fbf20009e30 comp rx=0 tx=0).stop 2024-02-16T19:24:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fd080 msgr2=0x7fbf3010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fd080 0x7fbf3010de50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400eb40 tx=0x7fbf2400b040 comp rx=0 tx=0).stop 2024-02-16T19:24:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 shutdown_connections 2024-02-16T19:24:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fbf180692a0 0x7fbf1806b750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf300fbe80 0x7fbf3010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 --2- 172.21.15.40:0/54928473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf300fd080 0x7fbf3010de50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 >> 172.21.15.40:0/54928473 conn(0x7fbf300f7590 msgr2=0x7fbf30065f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 shutdown_connections 2024-02-16T19:24:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.067+0000 7fbf34f72700 1 -- 172.21.15.40:0/54928473 wait complete. 2024-02-16T19:24:35.132 INFO:tasks.workunit.client.1.smithi163.stdout:7/784: sync 2024-02-16T19:24:35.132 INFO:tasks.workunit.client.1.smithi163.stdout:3/853: sync 2024-02-16T19:24:35.133 INFO:tasks.workunit.client.1.smithi163.stdout:7/785: creat d11/d50/d5d/d8b/d67/dd6/f10d x:0 0 0 2024-02-16T19:24:35.135 INFO:tasks.workunit.client.1.smithi163.stdout:7/786: creat d11/d24/f10e x:0 0 0 2024-02-16T19:24:35.135 INFO:tasks.workunit.client.1.smithi163.stdout:7/787: chown d11/l23 208 1 2024-02-16T19:24:35.135 INFO:tasks.workunit.client.1.smithi163.stdout:7/788: chown d11/l14 23344444 1 2024-02-16T19:24:35.140 INFO:tasks.workunit.client.1.smithi163.stdout:3/854: write d0/d83/da5/d127/f78 [4137303,73931] 0 2024-02-16T19:24:35.143 INFO:tasks.workunit.client.1.smithi163.stdout:3/855: creat d0/dc/d50/d66/da3/f12d x:0 0 0 2024-02-16T19:24:35.159 INFO:tasks.workunit.client.1.smithi163.stdout:3/856: dwrite d0/dc/d50/f11a [0,4194304] 0 2024-02-16T19:24:35.159 INFO:tasks.workunit.client.1.smithi163.stdout:3/857: read d0/dc/d1c/d32/db8/fc7 [295776,85645] 0 2024-02-16T19:24:35.168 INFO:tasks.workunit.client.1.smithi163.stdout:3/858: dread d0/dc/d1c/d32/d7f/fbd [0,4194304] 0 2024-02-16T19:24:35.170 INFO:tasks.workunit.client.1.smithi163.stdout:3/859: symlink d0/dc/d94/d9e/dac/l12e 0 2024-02-16T19:24:35.171 INFO:tasks.workunit.client.1.smithi163.stdout:3/860: truncate d0/dc/d50/d66/da3/d71/f89 420291 0 2024-02-16T19:24:35.179 INFO:tasks.workunit.client.1.smithi163.stdout:3/861: dread d0/dc/d1d/d29/f33 [4194304,4194304] 0 2024-02-16T19:24:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.179+0000 7fee3cad2700 1 -- 172.21.15.40:0/12693058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 msgr2=0x7fee380fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.179+0000 7fee3cad2700 1 --2- 172.21.15.40:0/12693058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 0x7fee380fd420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee28009ce0 tx=0x7fee28004d10 comp rx=0 tx=0).stop 2024-02-16T19:24:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.179+0000 7fee3cad2700 1 -- 172.21.15.40:0/12693058 shutdown_connections 2024-02-16T19:24:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.179+0000 7fee3cad2700 1 --2- 172.21.15.40:0/12693058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 0x7fee380fd420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.179+0000 7fee3cad2700 1 --2- 172.21.15.40:0/12693058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee380fbe40 0x7fee380fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.179+0000 7fee3cad2700 1 -- 172.21.15.40:0/12693058 >> 172.21.15.40:0/12693058 conn(0x7fee380f74e0 msgr2=0x7fee380f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.179+0000 7fee3cad2700 1 -- 172.21.15.40:0/12693058 shutdown_connections 2024-02-16T19:24:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.179+0000 7fee3cad2700 1 -- 172.21.15.40:0/12693058 wait complete. 2024-02-16T19:24:35.181 INFO:tasks.workunit.client.1.smithi163.stdout:3/862: rename d0/dc/f3c to d0/dc/d50/d66/d93/f12f 0 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee3cad2700 1 Processor -- start 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee3cad2700 1 -- start start 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee3cad2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee380fbe40 0x7fee38110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee3cad2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 0x7fee38112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee3cad2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee38066480 con 0x7fee380fcfb0 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee3cad2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380665c0 con 0x7fee380fbe40 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee35d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 0x7fee38112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee35d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 0x7fee38112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37212/0 (socket says 172.21.15.40:37212) 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee35d9b700 1 -- 172.21.15.40:0/766353290 learned_addr learned my addr 172.21.15.40:0/766353290 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee3659c700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee380fbe40 0x7fee38110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee35d9b700 1 -- 172.21.15.40:0/766353290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee380fbe40 msgr2=0x7fee38110690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee35d9b700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee380fbe40 0x7fee38110690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.180+0000 7fee35d9b700 1 -- 172.21.15.40:0/766353290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee28009940 con 0x7fee380fcfb0 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.181+0000 7fee35d9b700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 0x7fee38112be0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee28009380 tx=0x7fee28004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.181+0000 7fee2effd700 1 -- 172.21.15.40:0/766353290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee2800e040 con 0x7fee380fcfb0 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.181+0000 7fee2effd700 1 -- 172.21.15.40:0/766353290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee2801cb20 con 0x7fee380fcfb0 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.181+0000 7fee2effd700 1 -- 172.21.15.40:0/766353290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee28012b50 con 0x7fee380fcfb0 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.181+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee381131e0 con 0x7fee380fcfb0 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.181+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee381136a0 con 0x7fee380fcfb0 2024-02-16T19:24:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.182+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3804bed0 con 0x7fee380fcfb0 2024-02-16T19:24:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.182+0000 7fee2effd700 1 -- 172.21.15.40:0/766353290 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fee2801c650 con 0x7fee380fcfb0 2024-02-16T19:24:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.183+0000 7fee2effd700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fee240692b0 0x7fee2406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:24:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.183+0000 7fee2effd700 1 -- 172.21.15.40:0/766353290 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fee280895a0 con 0x7fee380fcfb0 2024-02-16T19:24:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.183+0000 7fee3659c700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fee240692b0 0x7fee2406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:24:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.183+0000 7fee3659c700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fee240692b0 0x7fee2406b760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee38066100 tx=0x7fee3000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:24:35.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.188+0000 7fee2effd700 1 -- 172.21.15.40:0/766353290 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fee28055800 con 0x7fee380fcfb0 2024-02-16T19:24:35.194 INFO:tasks.workunit.client.1.smithi163.stdout:3/863: dwrite d0/dc/d1c/d32/ddb/f10b [0,4194304] 0 2024-02-16T19:24:35.194 INFO:tasks.workunit.client.1.smithi163.stdout:3/864: chown d0/dc/d1d/d49/fc9 47 1 2024-02-16T19:24:35.194 INFO:tasks.workunit.client.1.smithi163.stdout:3/865: chown d0/dc/d94/d9e/le7 229 1 2024-02-16T19:24:35.197 INFO:tasks.workunit.client.1.smithi163.stdout:3/866: mknod d0/dc/d50/d66/d80/c130 0 2024-02-16T19:24:35.197 INFO:tasks.workunit.client.1.smithi163.stdout:3/867: fdatasync d0/dc/d50/d66/da3/d71/f111 0 2024-02-16T19:24:35.197 INFO:tasks.workunit.client.1.smithi163.stdout:3/868: creat d0/dc/d50/d66/d93/f131 x:0 0 0 2024-02-16T19:24:35.200 INFO:tasks.workunit.client.1.smithi163.stdout:3/869: mknod d0/dc/d1c/d32/db8/c132 0 2024-02-16T19:24:35.200 INFO:tasks.workunit.client.1.smithi163.stdout:3/870: fsync d0/dc/d1d/d29/d38/d46/fa6 0 2024-02-16T19:24:35.341 INFO:tasks.workunit.client.1.smithi163.stdout:2/982: sync 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:4/806: sync 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:1/885: sync 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:4/807: creat d11/d18/d1f/d29/d52/d57/f112 x:0 0 0 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:1/886: creat d7/d20/d96/f12c x:0 0 0 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:4/808: write d11/d18/d1f/d29/fe1 [154684,27586] 0 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:6/824: sync 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:6/825: dread - d0/d6/df/fd3 zero size 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:8/879: sync 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:8/880: chown d0/d7/d19/d1f/d30/d6a/df3/l9f 622495323 1 2024-02-16T19:24:35.342 INFO:tasks.workunit.client.1.smithi163.stdout:8/881: chown d0/dc 7968586 1 2024-02-16T19:24:35.343 INFO:tasks.workunit.client.1.smithi163.stdout:4/809: mkdir d11/d18/d1f/d29/d52/d57/d6c/d9d/d113 0 2024-02-16T19:24:35.344 INFO:tasks.workunit.client.1.smithi163.stdout:6/826: creat d0/d18/d77/f10d x:0 0 0 2024-02-16T19:24:35.345 INFO:tasks.workunit.client.1.smithi163.stdout:8/882: link d0/d7/d18/d53/ded/fe6 d0/d7/d19/d1f/d30/d48/d110/f11f 0 2024-02-16T19:24:35.346 INFO:tasks.workunit.client.1.smithi163.stdout:6/827: mkdir d0/dd/d7f/d2e/d10e 0 2024-02-16T19:24:35.348 INFO:tasks.workunit.client.1.smithi163.stdout:6/828: mkdir d0/d6/df/d30/d10f 0 2024-02-16T19:24:35.350 INFO:tasks.workunit.client.1.smithi163.stdout:6/829: link d0/d6/df/d20/f90 d0/d18/d77/dc9/d74/f110 0 2024-02-16T19:24:35.352 INFO:tasks.workunit.client.1.smithi163.stdout:6/830: unlink d0/d6/df/d20/d69/fee 0 2024-02-16T19:24:35.353 INFO:tasks.workunit.client.1.smithi163.stdout:6/831: getdents d0/d18/d77/d70 0 2024-02-16T19:24:35.354 INFO:tasks.workunit.client.1.smithi163.stdout:8/883: dread d0/d7/d19/d1f/d32/f39 [0,4194304] 0 2024-02-16T19:24:35.354 INFO:tasks.workunit.client.1.smithi163.stdout:8/884: write d0/d7/d18/d53/f9d [1178337,67171] 0 2024-02-16T19:24:35.354 INFO:tasks.workunit.client.1.smithi163.stdout:8/885: fdatasync d0/d7/d19/d1f/d30/d6a/d7b/f89 0 2024-02-16T19:24:35.355 INFO:tasks.workunit.client.1.smithi163.stdout:6/832: link d0/d6/df/d20/d40/d1e/lea d0/d6/l111 0 2024-02-16T19:24:35.357 INFO:tasks.workunit.client.1.smithi163.stdout:8/886: mknod d0/d7/d18/d112/d2c/ddf/c120 0 2024-02-16T19:24:35.358 INFO:tasks.workunit.client.1.smithi163.stdout:6/833: creat d0/dd/df7/f112 x:0 0 0 2024-02-16T19:24:35.359 INFO:tasks.workunit.client.1.smithi163.stdout:8/887: creat d0/d7/d19/d1f/d32/d70/f121 x:0 0 0 2024-02-16T19:24:35.370 INFO:tasks.workunit.client.1.smithi163.stdout:6/834: dread d0/dd/d7f/d15/f4d [0,4194304] 0 2024-02-16T19:24:35.370 INFO:tasks.workunit.client.1.smithi163.stdout:1/887: dwrite d7/f7d [0,4194304] 0 2024-02-16T19:24:35.371 INFO:tasks.workunit.client.1.smithi163.stdout:6/835: truncate d0/d6/df/d20/f68 3921322 0 2024-02-16T19:24:35.373 INFO:tasks.workunit.client.1.smithi163.stdout:1/888: rename d7/dc/d2d/d90/l10a to d7/d5b/d11f/l12d 0 2024-02-16T19:24:35.374 INFO:tasks.workunit.client.1.smithi163.stdout:6/836: mkdir d0/d6/df/d20/d40/d1e/db2/d113 0 2024-02-16T19:24:35.375 INFO:tasks.workunit.client.1.smithi163.stdout:6/837: unlink d0/d6/df/d8f/da9/ffe 0 2024-02-16T19:24:35.378 INFO:tasks.workunit.client.1.smithi163.stdout:6/838: mkdir d0/d6/df/d20/d40/d1e/db2/d114 0 2024-02-16T19:24:35.378 INFO:tasks.workunit.client.1.smithi163.stdout:6/839: readlink d0/d18/d94/lbb 0 2024-02-16T19:24:35.378 INFO:tasks.workunit.client.1.smithi163.stdout:6/840: read d0/d6/df/d20/d69/fa6 [1760124,93429] 0 2024-02-16T19:24:35.378 INFO:tasks.workunit.client.1.smithi163.stdout:6/841: symlink d0/d18/d77/dc9/l115 0 2024-02-16T19:24:35.399 INFO:tasks.workunit.client.1.smithi163.stdout:8/888: dwrite d0/d7/d19/d1f/f4e [0,4194304] 0 2024-02-16T19:24:35.399 INFO:tasks.workunit.client.1.smithi163.stdout:1/889: dwrite d7/d20/d96/fce [0,4194304] 0 2024-02-16T19:24:35.400 INFO:tasks.workunit.client.1.smithi163.stdout:8/889: readlink d0/d7/d18/d112/d2c/l96 0 2024-02-16T19:24:35.400 INFO:tasks.workunit.client.1.smithi163.stdout:6/842: dwrite d0/dd/d7f/f2d [0,4194304] 0 2024-02-16T19:24:35.401 INFO:tasks.workunit.client.1.smithi163.stdout:1/890: dread d7/dc/d78/d7c/f81 [0,4194304] 0 2024-02-16T19:24:35.407 INFO:tasks.workunit.client.1.smithi163.stdout:6/843: truncate d0/d6/df/f2a 2273165 0 2024-02-16T19:24:35.407 INFO:tasks.workunit.client.1.smithi163.stdout:6/844: chown d0/dd/d7f/d2e/cd5 1 1 2024-02-16T19:24:35.407 INFO:tasks.workunit.client.1.smithi163.stdout:6/845: chown d0/c8b 1199 1 2024-02-16T19:24:35.409 INFO:tasks.workunit.client.1.smithi163.stdout:1/891: creat d7/dc/d2d/d32/d87/da6/de5/f12e x:0 0 0 2024-02-16T19:24:35.410 INFO:tasks.workunit.client.1.smithi163.stdout:6/846: rename d0/d6/df/d30/f5a to d0/d18/d77/dc9/d74/f116 0 2024-02-16T19:24:35.414 INFO:tasks.workunit.client.1.smithi163.stdout:6/847: getdents d0/dd/dbc/ddc 0 2024-02-16T19:24:35.415 INFO:tasks.workunit.client.1.smithi163.stdout:1/892: rmdir d7/d20/d21/d74/dab 39 2024-02-16T19:24:35.420 INFO:tasks.workunit.client.1.smithi163.stdout:8/890: dwrite d0/d7/d19/d1f/d32/d70/f121 [0,4194304] 0 2024-02-16T19:24:35.420 INFO:tasks.workunit.client.1.smithi163.stdout:8/891: chown d0/d6f/fd2 20 1 2024-02-16T19:24:35.424 INFO:tasks.workunit.client.1.smithi163.stdout:8/892: rename d0/d7/d19/d1f/d30/d6a/df3/df2/lfd to d0/d95/l122 0 2024-02-16T19:24:35.424 INFO:tasks.workunit.client.1.smithi163.stdout:8/893: write d0/d7/d19/d1f/f4e [5483231,125879] 0 2024-02-16T19:24:35.424 INFO:tasks.workunit.client.1.smithi163.stdout:1/893: write d7/dc/d2d/d32/d3b/f50 [3748868,33712] 0 2024-02-16T19:24:35.424 INFO:tasks.workunit.client.1.smithi163.stdout:8/894: truncate d0/d7/d18/d53/ded/d85/f86 313060 0 2024-02-16T19:24:35.433 INFO:tasks.workunit.client.1.smithi163.stdout:6/848: dwrite d0/d6/df/d20/d40/d1e/f52 [0,4194304] 0 2024-02-16T19:24:35.434 INFO:tasks.workunit.client.1.smithi163.stdout:6/849: write d0/d18/d77/d97/db8/fc7 [280257,23469] 0 2024-02-16T19:24:35.434 INFO:tasks.workunit.client.1.smithi163.stdout:6/850: mkdir d0/d6/df/d20/d40/d1e/db2/d117 0 2024-02-16T19:24:35.436 INFO:tasks.workunit.client.1.smithi163.stdout:6/851: creat d0/d6/df/d20/d40/d1e/db2/d114/f118 x:0 0 0 2024-02-16T19:24:35.436 INFO:tasks.workunit.client.1.smithi163.stdout:6/852: fsync d0/d18/d77/dc9/fec 0 2024-02-16T19:24:35.437 INFO:tasks.workunit.client.1.smithi163.stdout:1/894: dread d7/d20/d2a/f5c [0,4194304] 0 2024-02-16T19:24:35.439 INFO:tasks.workunit.client.1.smithi163.stdout:1/895: truncate d7/d20/d96/fd5 306933 0 2024-02-16T19:24:35.441 INFO:tasks.workunit.client.1.smithi163.stdout:1/896: truncate d7/d20/d21/d4e/f79 746763 0 2024-02-16T19:24:35.441 INFO:tasks.workunit.client.1.smithi163.stdout:1/897: readlink d7/d20/d21/d4e/dc6/ld8 0 2024-02-16T19:24:35.441 INFO:tasks.workunit.client.1.smithi163.stdout:1/898: chown d7/d20/d21/d74/dab/ff8 786369741 1 2024-02-16T19:24:35.441 INFO:tasks.workunit.client.1.smithi163.stdout:1/899: chown d7/d5b/d63/f7b 1717983 1 2024-02-16T19:24:35.442 INFO:tasks.workunit.client.1.smithi163.stdout:1/900: mknod d7/d20/d21/d74/de2/c12f 0 2024-02-16T19:24:35.442 INFO:tasks.workunit.client.1.smithi163.stdout:8/895: dwrite d0/fea [0,4194304] 0 2024-02-16T19:24:35.445 INFO:tasks.workunit.client.1.smithi163.stdout:1/901: symlink d7/dc/d2d/d32/d10f/l130 0 2024-02-16T19:24:35.445 INFO:tasks.workunit.client.1.smithi163.stdout:1/902: dread - d7/dc/d2d/d90/f116 zero size 2024-02-16T19:24:35.457 INFO:tasks.workunit.client.1.smithi163.stdout:1/903: write d7/dc/d2d/d32/d87/d97/fb0 [3495103,73753] 0 2024-02-16T19:24:35.459 INFO:tasks.workunit.client.1.smithi163.stdout:1/904: link d7/d20/d93/dd2/f129 d7/dc/d2d/d32/d87/da6/f131 0 2024-02-16T19:24:35.460 INFO:tasks.workunit.client.1.smithi163.stdout:1/905: creat d7/dc/d2d/d32/d87/f132 x:0 0 0 2024-02-16T19:24:35.460 INFO:tasks.workunit.client.1.smithi163.stdout:1/906: write d7/dc/dc1/f108 [164308,34398] 0 2024-02-16T19:24:35.460 INFO:tasks.workunit.client.1.smithi163.stdout:1/907: write d7/f22 [1600239,97769] 0 2024-02-16T19:24:35.461 INFO:tasks.workunit.client.1.smithi163.stdout:1/908: creat d7/d20/d21/d74/f133 x:0 0 0 2024-02-16T19:24:35.462 INFO:tasks.workunit.client.1.smithi163.stdout:8/896: dwrite d0/d7/d18/f37 [4194304,4194304] 0 2024-02-16T19:24:35.462 INFO:tasks.workunit.client.1.smithi163.stdout:1/909: rename d7/dc/d78/le7 to d7/dc/d78/dee/l134 0 2024-02-16T19:24:35.463 INFO:tasks.workunit.client.1.smithi163.stdout:8/897: mkdir d0/d7/d19/d1f/d123 0 2024-02-16T19:24:35.463 INFO:tasks.workunit.client.1.smithi163.stdout:8/898: read d0/dc/f7d [253377,26355] 0 2024-02-16T19:24:35.463 INFO:tasks.workunit.client.1.smithi163.stdout:1/910: mknod d7/dc/c135 0 2024-02-16T19:24:35.464 INFO:tasks.workunit.client.1.smithi163.stdout:8/899: write d0/d7/d55/f80 [1932895,99023] 0 2024-02-16T19:24:35.482 INFO:tasks.workunit.client.1.smithi163.stdout:7/789: sync 2024-02-16T19:24:35.483 INFO:tasks.workunit.client.1.smithi163.stdout:7/790: unlink d11/d50/d5d/dcd/cd3 0 2024-02-16T19:24:35.484 INFO:tasks.workunit.client.1.smithi163.stdout:7/791: mkdir d11/d24/d34/d8d/df9/d10f 0 2024-02-16T19:24:35.485 INFO:tasks.workunit.client.1.smithi163.stdout:7/792: truncate d11/d24/d25/d4b/d66/f70 450248 0 2024-02-16T19:24:35.485 INFO:tasks.workunit.client.1.smithi163.stdout:7/793: truncate d11/f15 474082 0 2024-02-16T19:24:35.489 INFO:tasks.workunit.client.1.smithi163.stdout:1/911: dwrite d7/dc/dc1/fde [0,4194304] 0 2024-02-16T19:24:35.489 INFO:tasks.workunit.client.1.smithi163.stdout:7/794: dread d11/d24/d49/fdb [0,4194304] 0 2024-02-16T19:24:35.489 INFO:tasks.workunit.client.1.smithi163.stdout:7/795: dread - d11/d50/d5d/d8b/d67/dd6/f10d zero size 2024-02-16T19:24:35.489 INFO:tasks.workunit.client.1.smithi163.stdout:7/796: getdents d11/d24/d34/d8d/df9/d10f 0 2024-02-16T19:24:35.489 INFO:tasks.workunit.client.1.smithi163.stdout:8/900: dwrite d0/d7/d18/d112/d3c/d3d/d41/f11b [4194304,4194304] 0 2024-02-16T19:24:35.489 INFO:tasks.workunit.client.1.smithi163.stdout:8/901: creat d0/d7/d19/d1f/d32/f124 x:0 0 0 2024-02-16T19:24:35.493 INFO:tasks.workunit.client.1.smithi163.stdout:1/912: mknod d7/dc/d2d/d32/d87/da6/c136 0 2024-02-16T19:24:35.493 INFO:tasks.workunit.client.1.smithi163.stdout:1/913: dread - d7/d5b/faa zero size 2024-02-16T19:24:35.496 INFO:tasks.workunit.client.1.smithi163.stdout:8/902: symlink d0/d7/d19/d1f/d30/d6a/db8/d116/l125 0 2024-02-16T19:24:35.498 INFO:tasks.workunit.client.1.smithi163.stdout:1/914: rename d7/d20/f66 to d7/d20/d21/d74/f137 0 2024-02-16T19:24:35.501 INFO:tasks.workunit.client.1.smithi163.stdout:7/797: dread d11/d24/d49/dc4/fca [0,4194304] 0 2024-02-16T19:24:35.508 INFO:tasks.workunit.client.1.smithi163.stdout:1/915: mkdir d7/d5b/d138 0 2024-02-16T19:24:35.509 INFO:tasks.workunit.client.1.smithi163.stdout:1/916: creat d7/d20/f139 x:0 0 0 2024-02-16T19:24:35.513 INFO:tasks.workunit.client.1.smithi163.stdout:8/903: rename d0/d7/d18/d112/d3c/d3d/c9a to d0/d7/d18/d53/d108/c126 0 2024-02-16T19:24:35.515 INFO:tasks.workunit.client.1.smithi163.stdout:1/917: creat d7/d20/d21/d74/de2/f13a x:0 0 0 2024-02-16T19:24:35.515 INFO:tasks.workunit.client.1.smithi163.stdout:1/918: write d7/dc/d2d/d32/d3b/fdd [1170216,96117] 0 2024-02-16T19:24:35.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/798: rmdir d11/d24 39 2024-02-16T19:24:35.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/799: dread - d11/d24/d49/fa7 zero size 2024-02-16T19:24:35.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/800: dread - d11/d24/d25/d4b/d66/f10b zero size 2024-02-16T19:24:35.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/801: fsync d11/d26/f2b 0 2024-02-16T19:24:35.523 INFO:tasks.workunit.client.1.smithi163.stdout:1/919: unlink d7/dc/d2d/d32/f11b 0 2024-02-16T19:24:35.523 INFO:tasks.workunit.client.1.smithi163.stdout:1/920: fdatasync d7/dc/d2d/d32/f46 0 2024-02-16T19:24:35.524 INFO:tasks.workunit.client.1.smithi163.stdout:7/802: creat d11/d24/d25/d4b/f110 x:0 0 0 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.523+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fee38113b00 con 0x7fee380fcfb0 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.524+0000 7fee2effd700 1 -- 172.21.15.40:0/766353290 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fee28058e20 con 0x7fee380fcfb0 2024-02-16T19:24:35.525 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:24:35.526 INFO:tasks.workunit.client.1.smithi163.stdout:8/904: rename d0/d7/d19/d1f/d30 to d0/d7/d19/d127 0 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fee240692b0 msgr2=0x7fee2406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fee240692b0 0x7fee2406b760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee38066100 tx=0x7fee3000b040 comp rx=0 tx=0).stop 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 msgr2=0x7fee38112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 0x7fee38112be0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee28009380 tx=0x7fee28004d10 comp rx=0 tx=0).stop 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 shutdown_connections 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:6800/1321006798,v1:172.21.15.40:6801/1321006798] conn(0x7fee240692b0 0x7fee2406b760 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee380fbe40 0x7fee38110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 --2- 172.21.15.40:0/766353290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee380fcfb0 0x7fee38112be0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:24:35.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 >> 172.21.15.40:0/766353290 conn(0x7fee380f74e0 msgr2=0x7fee38100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:24:35.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.526+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 shutdown_connections 2024-02-16T19:24:35.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:24:35.527+0000 7fee3cad2700 1 -- 172.21.15.40:0/766353290 wait complete. 2024-02-16T19:24:35.529 INFO:tasks.workunit.client.1.smithi163.stdout:8/905: truncate d0/d7/d19/d127/d48/d110/f11f 2348503 0 2024-02-16T19:24:35.529 INFO:tasks.workunit.client.1.smithi163.stdout:8/906: creat d0/d7/d19/d1f/d32/f128 x:0 0 0 2024-02-16T19:24:35.531 INFO:tasks.workunit.client.1.smithi163.stdout:1/921: dread d7/dc/d2d/d32/d87/d97/fb0 [0,4194304] 0 2024-02-16T19:24:35.531 INFO:tasks.workunit.client.1.smithi163.stdout:1/922: rename d7/d5b to d7/d5b/d11f/d13b 22 2024-02-16T19:24:35.531 INFO:tasks.workunit.client.1.smithi163.stdout:1/923: fdatasync d7/d5b/f91 0 2024-02-16T19:24:35.531 INFO:tasks.workunit.client.1.smithi163.stdout:1/924: fsync d7/d20/d21/fe3 0 2024-02-16T19:24:35.531 INFO:tasks.workunit.client.1.smithi163.stdout:1/925: creat d7/dc/d2d/d32/d87/da6/de5/f13c x:0 0 0 2024-02-16T19:24:35.533 INFO:tasks.workunit.client.1.smithi163.stdout:1/926: symlink d7/dc/d2d/d32/d87/d97/l13d 0 2024-02-16T19:24:35.533 INFO:tasks.workunit.client.1.smithi163.stdout:1/927: fdatasync d7/dc/d2d/d3f/f67 0 2024-02-16T19:24:35.533 INFO:tasks.workunit.client.1.smithi163.stdout:1/928: stat d7/dc/d2d/c62 0 2024-02-16T19:24:35.535 INFO:tasks.workunit.client.1.smithi163.stdout:1/929: dread d7/dc/d2d/d32/d3b/fdb [0,4194304] 0 2024-02-16T19:24:35.537 INFO:tasks.workunit.client.1.smithi163.stdout:1/930: truncate d7/dc/d2d/d32/d3b/f50 403546 0 2024-02-16T19:24:35.539 INFO:tasks.workunit.client.1.smithi163.stdout:1/931: link d7/d20/d21/d4e/c8b d7/d20/d96/c13e 0 2024-02-16T19:24:35.539 INFO:tasks.workunit.client.1.smithi163.stdout:1/932: chown d7/dc/d2d/d32/d10f/c12b 727406108 1 2024-02-16T19:24:35.540 INFO:tasks.workunit.client.1.smithi163.stdout:8/907: dread d0/d7/d18/d112/d2c/d74/dcb/fee [0,4194304] 0 2024-02-16T19:24:35.540 INFO:tasks.workunit.client.1.smithi163.stdout:8/908: fdatasync d0/d7/d19/d127/d6a/d7b/fac 0 2024-02-16T19:24:35.540 INFO:tasks.workunit.client.1.smithi163.stdout:8/909: read - d0/d6f/de8/ffa zero size 2024-02-16T19:24:35.540 INFO:tasks.workunit.client.1.smithi163.stdout:1/933: creat d7/dc/d2d/d3f/d85/f13f x:0 0 0 2024-02-16T19:24:35.541 INFO:tasks.workunit.client.1.smithi163.stdout:8/910: creat d0/d7/d18/d53/ded/f129 x:0 0 0 2024-02-16T19:24:35.541 INFO:tasks.workunit.client.1.smithi163.stdout:1/934: symlink d7/dc/d2d/d3f/l140 0 2024-02-16T19:24:35.542 INFO:tasks.workunit.client.1.smithi163.stdout:8/911: mknod d0/d7/d18/d112/d2c/ddf/c12a 0 2024-02-16T19:24:35.542 INFO:tasks.workunit.client.1.smithi163.stdout:8/912: fdatasync d0/d7/d18/d112/d2c/d74/fd7 0 2024-02-16T19:24:35.542 INFO:tasks.workunit.client.1.smithi163.stdout:8/913: readlink d0/d7/d18/d112/d2c/d74/la1 0 2024-02-16T19:24:35.542 INFO:tasks.workunit.client.1.smithi163.stdout:8/914: truncate d0/d7/d19/fe4 208768 0 2024-02-16T19:24:35.543 INFO:tasks.workunit.client.1.smithi163.stdout:8/915: truncate d0/d7/d19/d127/d6a/df3/df2/f5a 2007343 0 2024-02-16T19:24:35.543 INFO:tasks.workunit.client.1.smithi163.stdout:1/935: getdents d7/d20/d21/d74/de2 0 2024-02-16T19:24:35.543 INFO:tasks.workunit.client.1.smithi163.stdout:1/936: creat d7/d20/d21/d4e/dc6/f141 x:0 0 0 2024-02-16T19:24:35.544 INFO:tasks.workunit.client.1.smithi163.stdout:1/937: readlink d7/l8 0 2024-02-16T19:24:35.545 INFO:tasks.workunit.client.1.smithi163.stdout:7/803: dwrite d11/d26/fbb [0,4194304] 0 2024-02-16T19:24:35.545 INFO:tasks.workunit.client.1.smithi163.stdout:8/916: link d0/d7/d19/d127/d48/fa7 d0/d7/d18/d112/d2c/f12b 0 2024-02-16T19:24:35.549 INFO:tasks.workunit.client.1.smithi163.stdout:7/804: mknod d11/d24/d34/c111 0 2024-02-16T19:24:35.552 INFO:tasks.workunit.client.1.smithi163.stdout:8/917: mkdir d0/d7/d19/d1f/d32/d12c 0 2024-02-16T19:24:35.554 INFO:tasks.workunit.client.1.smithi163.stdout:8/918: truncate d0/d7/d18/d112/d2c/d74/dcb/fee 740751 0 2024-02-16T19:24:35.559 INFO:tasks.workunit.client.1.smithi163.stdout:8/919: truncate d0/d7/d18/d112/d3c/d3d/d41/f11b 6283565 0 2024-02-16T19:24:35.559 INFO:tasks.workunit.client.1.smithi163.stdout:8/920: fsync d0/d7/d18/d112/d2c/f3b 0 2024-02-16T19:24:35.563 INFO:tasks.workunit.client.1.smithi163.stdout:8/921: getdents d0/d6f/de8 0 2024-02-16T19:24:35.570 INFO:tasks.workunit.client.1.smithi163.stdout:1/938: dwrite d7/dc/d2d/d32/d87/da6/f103 [0,4194304] 0 2024-02-16T19:24:35.575 INFO:tasks.workunit.client.1.smithi163.stdout:8/922: dread d0/d7/d18/d2e/f7f [0,4194304] 0 2024-02-16T19:24:35.575 INFO:tasks.workunit.client.1.smithi163.stdout:8/923: chown d0/d7/d18/d112/d2c/l96 7 1 2024-02-16T19:24:35.576 INFO:tasks.workunit.client.1.smithi163.stdout:8/924: symlink d0/d7/d19/d127/d6a/d7b/l12d 0 2024-02-16T19:24:35.577 INFO:tasks.workunit.client.1.smithi163.stdout:8/925: read d0/d7/d19/d127/d48/f65 [4490521,111745] 0 2024-02-16T19:24:35.577 INFO:tasks.workunit.client.1.smithi163.stdout:8/926: readlink d0/d7/d18/d112/d3c/d3d/d41/lbc 0 2024-02-16T19:24:35.577 INFO:tasks.workunit.client.1.smithi163.stdout:8/927: mknod d0/d7/d19/d1f/d123/c12e 0 2024-02-16T19:24:35.578 INFO:tasks.workunit.client.1.smithi163.stdout:8/928: truncate d0/d7/d19/d1f/fc7 2113585 0 2024-02-16T19:24:35.579 INFO:tasks.workunit.client.1.smithi163.stdout:3/871: sync 2024-02-16T19:24:35.579 INFO:tasks.workunit.client.1.smithi163.stdout:3/872: truncate d0/dc/d50/d66/fec 3011554 0 2024-02-16T19:24:35.579 INFO:tasks.workunit.client.1.smithi163.stdout:8/929: mkdir d0/d7/d18/d53/d12f 0 2024-02-16T19:24:35.580 INFO:tasks.workunit.client.1.smithi163.stdout:7/805: dwrite d11/d24/d49/f51 [0,4194304] 0 2024-02-16T19:24:35.581 INFO:tasks.workunit.client.1.smithi163.stdout:8/930: mkdir d0/d7/d18/d112/d2c/d74/dcb/d130 0 2024-02-16T19:24:35.585 INFO:tasks.workunit.client.1.smithi163.stdout:8/931: link d0/d7/d18/d112/lf7 d0/d7/d18/d112/d2c/ddf/l131 0 2024-02-16T19:24:35.585 INFO:tasks.workunit.client.1.smithi163.stdout:8/932: getdents d0/d7/d18/d53/d12f 0 2024-02-16T19:24:35.585 INFO:tasks.workunit.client.1.smithi163.stdout:8/933: readlink d0/d7/d18/l77 0 2024-02-16T19:24:35.590 INFO:tasks.workunit.client.1.smithi163.stdout:8/934: write d0/d7/d19/d1f/d32/f10c [1556392,101792] 0 2024-02-16T19:24:35.592 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:35 smithi163 ceph-mon[36452]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:35.592 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:35 smithi163 ceph-mon[36452]: pgmap v256: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 266 MiB/s rd, 334 MiB/s wr, 441 op/s 2024-02-16T19:24:35.592 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:35 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1301358190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:24:35.592 INFO:tasks.workunit.client.1.smithi163.stdout:8/935: truncate d0/d7/d19/d127/d6a/df3/f64 202138 0 2024-02-16T19:24:35.594 INFO:tasks.workunit.client.1.smithi163.stdout:8/936: write d0/d7/d19/fc6 [682736,118135] 0 2024-02-16T19:24:35.595 INFO:tasks.workunit.client.1.smithi163.stdout:8/937: dread d0/d7/d18/d53/ded/d85/f86 [0,4194304] 0 2024-02-16T19:24:35.595 INFO:tasks.workunit.client.1.smithi163.stdout:8/938: getdents d0/d7/d103 0 2024-02-16T19:24:35.595 INFO:tasks.workunit.client.1.smithi163.stdout:8/939: truncate d0/d7/d18/d112/d2c/f83 871451 0 2024-02-16T19:24:35.595 INFO:tasks.workunit.client.1.smithi163.stdout:8/940: stat d0/d7/d18/d112/d2c/d74/fd7 0 2024-02-16T19:24:35.597 INFO:tasks.workunit.client.1.smithi163.stdout:8/941: link d0/d7/d18/d112/d3c/d3d/c38 d0/d7/d18/d53/ded/c132 0 2024-02-16T19:24:35.597 INFO:tasks.workunit.client.1.smithi163.stdout:8/942: write d0/d7/d19/d1f/d32/f10c [4457416,97929] 0 2024-02-16T19:24:35.598 INFO:tasks.workunit.client.1.smithi163.stdout:8/943: mknod d0/d7/d19/d127/d6a/d7b/deb/c133 0 2024-02-16T19:24:35.599 INFO:tasks.workunit.client.1.smithi163.stdout:8/944: symlink d0/d7/d19/d127/d48/dbd/l134 0 2024-02-16T19:24:35.602 INFO:tasks.workunit.client.1.smithi163.stdout:3/873: dwrite d0/dc/d1c/f23 [0,4194304] 0 2024-02-16T19:24:35.603 INFO:tasks.workunit.client.1.smithi163.stdout:7/806: dwrite d11/d24/d25/f39 [4194304,4194304] 0 2024-02-16T19:24:35.603 INFO:tasks.workunit.client.1.smithi163.stdout:7/807: chown d11/d24/d25/cae 15064636 1 2024-02-16T19:24:35.611 INFO:tasks.workunit.client.1.smithi163.stdout:7/808: creat d11/d24/d34/d8d/dff/f112 x:0 0 0 2024-02-16T19:24:35.612 INFO:tasks.workunit.client.1.smithi163.stdout:3/874: write d0/d83/da5/d127/f60 [2850521,57929] 0 2024-02-16T19:24:35.612 INFO:tasks.workunit.client.1.smithi163.stdout:8/945: rmdir d0/d7 39 2024-02-16T19:24:35.618 INFO:tasks.workunit.client.1.smithi163.stdout:7/809: rename d11/d30/f6a to d11/d26/f113 0 2024-02-16T19:24:35.619 INFO:tasks.workunit.client.1.smithi163.stdout:3/875: mknod d0/dc/d96/da7/c133 0 2024-02-16T19:24:35.620 INFO:tasks.workunit.client.1.smithi163.stdout:3/876: creat d0/dc/d1d/f134 x:0 0 0 2024-02-16T19:24:35.622 INFO:tasks.workunit.client.1.smithi163.stdout:7/810: symlink d11/d24/d49/dac/l114 0 2024-02-16T19:24:35.622 INFO:tasks.workunit.client.1.smithi163.stdout:7/811: read d11/d24/d49/dc4/fca [4057995,31604] 0 2024-02-16T19:24:35.624 INFO:tasks.workunit.client.1.smithi163.stdout:7/812: mknod d11/d24/d25/c115 0 2024-02-16T19:24:35.628 INFO:tasks.workunit.client.1.smithi163.stdout:8/946: dwrite d0/d7/d19/d127/d6a/df3/df2/f5a [0,4194304] 0 2024-02-16T19:24:35.628 INFO:tasks.workunit.client.1.smithi163.stdout:8/947: chown d0/d6f/de8 734780690 1 2024-02-16T19:24:35.634 INFO:tasks.workunit.client.1.smithi163.stdout:8/948: read d0/d7/d18/d2e/f7f [2935356,96189] 0 2024-02-16T19:24:35.634 INFO:tasks.workunit.client.1.smithi163.stdout:8/949: write d0/d7/d18/d53/ded/f100 [1069831,32679] 0 2024-02-16T19:24:35.635 INFO:tasks.workunit.client.1.smithi163.stdout:8/950: symlink d0/d7/d18/d112/d2c/l135 0 2024-02-16T19:24:35.635 INFO:tasks.workunit.client.1.smithi163.stdout:8/951: write d0/d7/d19/d1f/d32/f124 [57556,32084] 0 2024-02-16T19:24:35.636 INFO:tasks.workunit.client.1.smithi163.stdout:8/952: symlink d0/d7/d18/d112/d2c/ddf/l136 0 2024-02-16T19:24:35.636 INFO:tasks.workunit.client.1.smithi163.stdout:8/953: stat d0/d7/d19/d127/d6a/df3/df2/c88 0 2024-02-16T19:24:35.641 INFO:tasks.workunit.client.1.smithi163.stdout:8/954: dread d0/d7/d18/d53/f9d [0,4194304] 0 2024-02-16T19:24:35.641 INFO:tasks.workunit.client.1.smithi163.stdout:3/877: dwrite d0/dc/f17 [0,4194304] 0 2024-02-16T19:24:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:35 smithi040 ceph-mon[29201]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:35 smithi040 ceph-mon[29201]: pgmap v256: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 266 MiB/s rd, 334 MiB/s wr, 441 op/s 2024-02-16T19:24:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:35 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1301358190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:24:35.661 INFO:tasks.workunit.client.1.smithi163.stdout:3/878: dwrite d0/d83/f108 [4194304,4194304] 0 2024-02-16T19:24:35.663 INFO:tasks.workunit.client.1.smithi163.stdout:3/879: mkdir d0/dc/d1c/d21/deb/d135 0 2024-02-16T19:24:35.664 INFO:tasks.workunit.client.1.smithi163.stdout:3/880: mknod d0/dc/d1c/d116/c136 0 2024-02-16T19:24:35.665 INFO:tasks.workunit.client.1.smithi163.stdout:3/881: unlink d0/dc/d1c/d32/ddb/f10b 0 2024-02-16T19:24:35.667 INFO:tasks.workunit.client.1.smithi163.stdout:3/882: mknod d0/dc/d1c/d21/deb/d135/c137 0 2024-02-16T19:24:35.671 INFO:tasks.workunit.client.1.smithi163.stdout:3/883: link d0/d83/da5/d127/c47 d0/dc/d94/d9e/dac/c138 0 2024-02-16T19:24:35.672 INFO:tasks.workunit.client.1.smithi163.stdout:3/884: getdents d0/dc/d50 0 2024-02-16T19:24:35.678 INFO:tasks.workunit.client.1.smithi163.stdout:3/885: dread d0/dc/d1c/d32/f36 [0,4194304] 0 2024-02-16T19:24:35.679 INFO:tasks.workunit.client.1.smithi163.stdout:3/886: creat d0/dc/d1d/d29/d38/d46/f139 x:0 0 0 2024-02-16T19:24:35.680 INFO:tasks.workunit.client.1.smithi163.stdout:3/887: mknod d0/dc/d1c/d32/ddb/c13a 0 2024-02-16T19:24:35.680 INFO:tasks.workunit.client.1.smithi163.stdout:3/888: creat d0/dc/d50/d66/da3/d71/f13b x:0 0 0 2024-02-16T19:24:35.687 INFO:tasks.workunit.client.1.smithi163.stdout:3/889: dread d0/dc/d1c/f23 [0,4194304] 0 2024-02-16T19:24:35.694 INFO:tasks.workunit.client.1.smithi163.stdout:3/890: dread d0/f2 [0,4194304] 0 2024-02-16T19:24:35.695 INFO:tasks.workunit.client.1.smithi163.stdout:3/891: creat d0/dc/d96/da7/f13c x:0 0 0 2024-02-16T19:24:35.696 INFO:tasks.workunit.client.1.smithi163.stdout:3/892: truncate d0/dc/d1c/d32/f57 1019019 0 2024-02-16T19:24:35.697 INFO:tasks.workunit.client.1.smithi163.stdout:3/893: mkdir d0/dc/d94/d9e/d121/d13d 0 2024-02-16T19:24:35.698 INFO:tasks.workunit.client.1.smithi163.stdout:3/894: link d0/dc/d96/da7/f117 d0/dc/d1d/d29/f13e 0 2024-02-16T19:24:35.706 INFO:tasks.workunit.client.1.smithi163.stdout:4/810: sync 2024-02-16T19:24:35.706 INFO:tasks.workunit.client.1.smithi163.stdout:4/811: readlink d11/d18/d1f/d29/la4 0 2024-02-16T19:24:35.706 INFO:tasks.workunit.client.1.smithi163.stdout:4/812: write d11/d2d/d4f/d5f/feb [677043,30600] 0 2024-02-16T19:24:35.708 INFO:tasks.workunit.client.1.smithi163.stdout:2/983: sync 2024-02-16T19:24:35.708 INFO:tasks.workunit.client.1.smithi163.stdout:6/853: sync 2024-02-16T19:24:35.711 INFO:tasks.workunit.client.1.smithi163.stdout:2/984: rmdir d2/ddb/d7c/d132 0 2024-02-16T19:24:35.713 INFO:tasks.workunit.client.1.smithi163.stdout:2/985: write d2/d2a/d39/fae [114795,49743] 0 2024-02-16T19:24:35.713 INFO:tasks.workunit.client.1.smithi163.stdout:2/986: write d2/fe5 [2215688,112953] 0 2024-02-16T19:24:35.713 INFO:tasks.workunit.client.1.smithi163.stdout:2/987: dread - d2/d2a/d9f/df8/f129 zero size 2024-02-16T19:24:35.715 INFO:tasks.workunit.client.1.smithi163.stdout:3/895: dwrite d0/dc/d96/fbb [0,4194304] 0 2024-02-16T19:24:35.715 INFO:tasks.workunit.client.1.smithi163.stdout:3/896: truncate d0/dc/d50/d66/da3/d71/f13b 810766 0 2024-02-16T19:24:35.716 INFO:tasks.workunit.client.1.smithi163.stdout:2/988: rename d2/d2a/d2d/d73/d84/dba/f140 to d2/d2a/d2d/d73/d84/f152 0 2024-02-16T19:24:35.716 INFO:tasks.workunit.client.1.smithi163.stdout:2/989: read - d2/d2a/d2d/fc6 zero size 2024-02-16T19:24:35.719 INFO:tasks.workunit.client.1.smithi163.stdout:3/897: mknod d0/dc/d1c/d21/deb/c13f 0 2024-02-16T19:24:35.719 INFO:tasks.workunit.client.1.smithi163.stdout:3/898: truncate d0/dc/d50/d66/da3/fd7 954695 0 2024-02-16T19:24:35.719 INFO:tasks.workunit.client.1.smithi163.stdout:3/899: chown d0/dc/d1c/d116 501768 1 2024-02-16T19:24:35.721 INFO:tasks.workunit.client.1.smithi163.stdout:2/990: rename d2/d26/d54/d58/d8c/d128 to d2/d26/d54/d58/d8c/dc5/d12b/d153 0 2024-02-16T19:24:35.731 INFO:tasks.workunit.client.1.smithi163.stdout:6/854: dwrite d0/dd/d7f/d81/daa/fde [0,4194304] 0 2024-02-16T19:24:35.732 INFO:tasks.workunit.client.1.smithi163.stdout:3/900: write d0/dc/d1c/d21/f7a [5180070,119982] 0 2024-02-16T19:24:35.732 INFO:tasks.workunit.client.1.smithi163.stdout:4/813: dwrite d11/d18/d1f/d29/d52/f6a [0,4194304] 0 2024-02-16T19:24:35.732 INFO:tasks.workunit.client.1.smithi163.stdout:4/814: read - d11/d18/d1f/d34/de5/ff7 zero size 2024-02-16T19:24:35.735 INFO:tasks.workunit.client.1.smithi163.stdout:6/855: mkdir d0/dd/d7f/d2e/d10e/d119 0 2024-02-16T19:24:35.739 INFO:tasks.workunit.client.1.smithi163.stdout:3/901: mkdir d0/dc/d96/dff/d140 0 2024-02-16T19:24:35.745 INFO:tasks.workunit.client.1.smithi163.stdout:2/991: dwrite d2/f9 [0,4194304] 0 2024-02-16T19:24:35.746 INFO:tasks.workunit.client.1.smithi163.stdout:2/992: symlink d2/dce/dee/l154 0 2024-02-16T19:24:35.747 INFO:tasks.workunit.client.1.smithi163.stdout:2/993: creat d2/d2a/d2d/f155 x:0 0 0 2024-02-16T19:24:35.748 INFO:tasks.workunit.client.1.smithi163.stdout:6/856: dread d0/d6/df/d8f/dcd/ff4 [0,4194304] 0 2024-02-16T19:24:35.749 INFO:tasks.workunit.client.1.smithi163.stdout:6/857: fdatasync d0/d6/df/d20/d40/d17/fa2 0 2024-02-16T19:24:35.749 INFO:tasks.workunit.client.1.smithi163.stdout:2/994: mkdir d2/dce/dee/d156 0 2024-02-16T19:24:35.750 INFO:tasks.workunit.client.1.smithi163.stdout:3/902: dread d0/dc/d1d/d49/f59 [4194304,4194304] 0 2024-02-16T19:24:35.750 INFO:tasks.workunit.client.1.smithi163.stdout:3/903: fdatasync d0/dc/d1d/d29/f45 0 2024-02-16T19:24:35.750 INFO:tasks.workunit.client.1.smithi163.stdout:2/995: rename d2/d2a/cc0 to d2/d26/d54/c157 0 2024-02-16T19:24:35.752 INFO:tasks.workunit.client.1.smithi163.stdout:3/904: unlink d0/l5 0 2024-02-16T19:24:35.752 INFO:tasks.workunit.client.1.smithi163.stdout:4/815: dwrite d11/d18/f43 [0,4194304] 0 2024-02-16T19:24:35.754 INFO:tasks.workunit.client.1.smithi163.stdout:2/996: creat d2/d26/d54/d58/d7d/daf/d101/f158 x:0 0 0 2024-02-16T19:24:35.756 INFO:tasks.workunit.client.1.smithi163.stdout:3/905: unlink d0/dc/d1c/f23 0 2024-02-16T19:24:35.756 INFO:tasks.workunit.client.1.smithi163.stdout:3/906: write d0/dc/d50/d66/d93/f12f [4852759,3746] 0 2024-02-16T19:24:35.759 INFO:tasks.workunit.client.1.smithi163.stdout:3/907: unlink d0/dc/d1d/d29/f45 0 2024-02-16T19:24:35.760 INFO:tasks.workunit.client.1.smithi163.stdout:2/997: getdents d2/d2a/d2d/d73/d84/dba 0 2024-02-16T19:24:35.780 INFO:tasks.workunit.client.1.smithi163.stdout:6/858: dwrite d0/dd/d7f/d81/daa/fde [4194304,4194304] 0 2024-02-16T19:24:35.780 INFO:tasks.workunit.client.1.smithi163.stdout:6/859: readlink d0/d6/le 0 2024-02-16T19:24:35.780 INFO:tasks.workunit.client.1.smithi163.stdout:6/860: dread - d0/d6/df/d8f/dcd/ffa zero size 2024-02-16T19:24:35.780 INFO:tasks.workunit.client.1.smithi163.stdout:6/861: truncate d0/d18/f48 4445906 0 2024-02-16T19:24:35.781 INFO:tasks.workunit.client.1.smithi163.stdout:3/908: dwrite d0/dc/d1c/d21/fce [0,4194304] 0 2024-02-16T19:24:35.787 INFO:tasks.workunit.client.1.smithi163.stdout:3/909: unlink d0/dc/d50/d66/la9 0 2024-02-16T19:24:35.788 INFO:tasks.workunit.client.1.smithi163.stdout:3/910: creat d0/dc/d1c/d21/f141 x:0 0 0 2024-02-16T19:24:35.788 INFO:tasks.workunit.client.1.smithi163.stdout:3/911: read d0/dc/d1c/d32/db8/fc7 [297362,100452] 0 2024-02-16T19:24:35.791 INFO:tasks.workunit.client.1.smithi163.stdout:6/862: dread d0/d6/f8 [4194304,4194304] 0 2024-02-16T19:24:35.797 INFO:tasks.workunit.client.1.smithi163.stdout:6/863: creat d0/dd/d7f/d2e/d10e/f11a x:0 0 0 2024-02-16T19:24:35.798 INFO:tasks.workunit.client.1.smithi163.stdout:6/864: truncate d0/d18/d77/d70/fe7 388265 0 2024-02-16T19:24:35.799 INFO:tasks.workunit.client.1.smithi163.stdout:6/865: truncate d0/d6/df/d20/d40/d17/f4b 2545417 0 2024-02-16T19:24:35.801 INFO:tasks.workunit.client.1.smithi163.stdout:6/866: creat d0/d6/df/d20/d40/d17/f11b x:0 0 0 2024-02-16T19:24:35.801 INFO:tasks.workunit.client.1.smithi163.stdout:6/867: chown d0/dd/d7f/d15/d1c/dca 34 1 2024-02-16T19:24:35.801 INFO:tasks.workunit.client.1.smithi163.stdout:6/868: read - d0/d6/fb5 zero size 2024-02-16T19:24:35.801 INFO:tasks.workunit.client.1.smithi163.stdout:6/869: mkdir d0/dd/d7f/d15/d11c 0 2024-02-16T19:24:35.808 INFO:tasks.workunit.client.1.smithi163.stdout:3/912: dwrite d0/d83/da5/f105 [0,4194304] 0 2024-02-16T19:24:35.811 INFO:tasks.workunit.client.1.smithi163.stdout:3/913: rmdir d0/dc/d50/d66/da3 39 2024-02-16T19:24:35.816 INFO:tasks.workunit.client.1.smithi163.stdout:3/914: dread d0/dc/d50/d66/fec [0,4194304] 0 2024-02-16T19:24:35.816 INFO:tasks.workunit.client.1.smithi163.stdout:3/915: write d0/dc/d1c/d91/fc3 [1660184,40491] 0 2024-02-16T19:24:35.818 INFO:tasks.workunit.client.1.smithi163.stdout:3/916: rename d0/dc/d50/d66/d80/fa0 to d0/dc/d1c/d21/deb/f142 0 2024-02-16T19:24:35.819 INFO:tasks.workunit.client.1.smithi163.stdout:3/917: symlink d0/dc/d1c/d21/deb/d135/l143 0 2024-02-16T19:24:35.820 INFO:tasks.workunit.client.1.smithi163.stdout:3/918: mkdir d0/dc/d1c/d91/d144 0 2024-02-16T19:24:35.821 INFO:tasks.workunit.client.1.smithi163.stdout:3/919: write d0/dc/d50/d66/d93/f131 [324937,44584] 0 2024-02-16T19:24:35.828 INFO:tasks.workunit.client.1.smithi163.stdout:6/870: write d0/d6/df/d20/f90 [1776157,60667] 0 2024-02-16T19:24:35.828 INFO:tasks.workunit.client.1.smithi163.stdout:6/871: chown d0/d6/df/d20/d40 65 1 2024-02-16T19:24:35.830 INFO:tasks.workunit.client.1.smithi163.stdout:6/872: link d0/dd/dbc/ddc/de5/l75 d0/dd/dbc/l11d 0 2024-02-16T19:24:35.830 INFO:tasks.workunit.client.1.smithi163.stdout:6/873: truncate d0/d6/df/d8f/f91 735995 0 2024-02-16T19:24:35.830 INFO:tasks.workunit.client.1.smithi163.stdout:6/874: stat d0/d6/df/f2a 0 2024-02-16T19:24:35.830 INFO:tasks.workunit.client.1.smithi163.stdout:6/875: write d0/dd/d7f/f73 [243136,17573] 0 2024-02-16T19:24:35.831 INFO:tasks.workunit.client.1.smithi163.stdout:6/876: symlink d0/d18/d77/d97/db8/l11e 0 2024-02-16T19:24:35.833 INFO:tasks.workunit.client.1.smithi163.stdout:6/877: rename d0/d6/df/d20/d40/f89 to d0/dd/df7/f11f 0 2024-02-16T19:24:35.835 INFO:tasks.workunit.client.1.smithi163.stdout:6/878: creat d0/d18/d77/d97/f120 x:0 0 0 2024-02-16T19:24:35.837 INFO:tasks.workunit.client.1.smithi163.stdout:6/879: unlink d0/d6/df/d20/d40/f7e 0 2024-02-16T19:24:35.837 INFO:tasks.workunit.client.1.smithi163.stdout:3/920: write d0/dc/d1c/d21/f42 [1933626,62071] 0 2024-02-16T19:24:35.839 INFO:tasks.workunit.client.1.smithi163.stdout:6/880: symlink d0/dd/dbc/ddc/l121 0 2024-02-16T19:24:35.840 INFO:tasks.workunit.client.1.smithi163.stdout:3/921: write d0/dc/d1d/d49/d98/fcc [219217,66184] 0 2024-02-16T19:24:35.840 INFO:tasks.workunit.client.1.smithi163.stdout:6/881: truncate d0/d18/d77/f9e 2767603 0 2024-02-16T19:24:35.841 INFO:tasks.workunit.client.1.smithi163.stdout:6/882: rmdir d0/d18/d77/dc9 39 2024-02-16T19:24:35.845 INFO:tasks.workunit.client.1.smithi163.stdout:6/883: unlink d0/d6/fa7 0 2024-02-16T19:24:35.855 INFO:tasks.workunit.client.1.smithi163.stdout:6/884: dread d0/dd/d7f/f4c [0,4194304] 0 2024-02-16T19:24:35.858 INFO:tasks.workunit.client.1.smithi163.stdout:6/885: rmdir d0/dd/d7f/d81/d9a 0 2024-02-16T19:24:35.859 INFO:tasks.workunit.client.1.smithi163.stdout:6/886: mknod d0/d6/df/d8f/dcd/c122 0 2024-02-16T19:24:35.859 INFO:tasks.workunit.client.1.smithi163.stdout:6/887: chown d0/d6/df/d20/d40/d1e/c107 55 1 2024-02-16T19:24:35.859 INFO:tasks.workunit.client.1.smithi163.stdout:6/888: fsync d0/dd/d7f/d2e/fbd 0 2024-02-16T19:24:35.860 INFO:tasks.workunit.client.1.smithi163.stdout:6/889: link d0/dd/d7f/d81/d85/da4/l102 d0/d6/df/d20/d40/d1e/db2/dda/l123 0 2024-02-16T19:24:35.860 INFO:tasks.workunit.client.1.smithi163.stdout:6/890: chown d0/d6/df/d20/d69/db4/fb9 4 1 2024-02-16T19:24:35.862 INFO:tasks.workunit.client.1.smithi163.stdout:6/891: creat d0/dd/d7f/d15/d1c/d65/d63/d86/dff/f124 x:0 0 0 2024-02-16T19:24:35.862 INFO:tasks.workunit.client.1.smithi163.stdout:6/892: stat d0/dd/df7 0 2024-02-16T19:24:35.867 INFO:tasks.workunit.client.1.smithi163.stdout:3/922: dwrite d0/dc/f28 [0,4194304] 0 2024-02-16T19:24:35.868 INFO:tasks.workunit.client.1.smithi163.stdout:3/923: getdents d0/dc/d94/d9e/d121 0 2024-02-16T19:24:35.868 INFO:tasks.workunit.client.1.smithi163.stdout:3/924: creat d0/dc/d1c/d32/db8/f145 x:0 0 0 2024-02-16T19:24:35.870 INFO:tasks.workunit.client.1.smithi163.stdout:3/925: mknod d0/dc/d96/dff/d140/c146 0 2024-02-16T19:24:35.872 INFO:tasks.workunit.client.1.smithi163.stdout:3/926: read d0/dc/d1c/d32/df0/ff1 [245452,116421] 0 2024-02-16T19:24:35.872 INFO:tasks.workunit.client.1.smithi163.stdout:3/927: write d0/dc/d96/da7/fb1 [130053,65321] 0 2024-02-16T19:24:35.875 INFO:tasks.workunit.client.1.smithi163.stdout:3/928: mknod d0/dc/d96/dff/c147 0 2024-02-16T19:24:35.875 INFO:tasks.workunit.client.1.smithi163.stdout:3/929: chown d0/dc/d1c/d32/db8/fc7 1641 1 2024-02-16T19:24:35.879 INFO:tasks.workunit.client.1.smithi163.stdout:3/930: truncate d0/f2 2539336 0 2024-02-16T19:24:35.879 INFO:tasks.workunit.client.1.smithi163.stdout:3/931: fdatasync d0/dc/d94/d9e/dac/fd0 0 2024-02-16T19:24:35.879 INFO:tasks.workunit.client.1.smithi163.stdout:3/932: getdents d0/dc/d1c 0 2024-02-16T19:24:35.880 INFO:tasks.workunit.client.1.smithi163.stdout:3/933: symlink d0/dc/d1c/d21/deb/d135/l148 0 2024-02-16T19:24:35.880 INFO:tasks.workunit.client.1.smithi163.stdout:3/934: write d0/dc/d1c/d21/f120 [527367,75318] 0 2024-02-16T19:24:35.884 INFO:tasks.workunit.client.1.smithi163.stdout:3/935: dread d0/dc/d1c/f2c [0,4194304] 0 2024-02-16T19:24:35.884 INFO:tasks.workunit.client.1.smithi163.stdout:6/893: dwrite d0/dd/d7f/d15/d1c/d65/d63/d86/dff/f124 [0,4194304] 0 2024-02-16T19:24:35.890 INFO:tasks.workunit.client.1.smithi163.stdout:3/936: write d0/dc/d50/d66/da3/d71/f89 [371430,7887] 0 2024-02-16T19:24:35.890 INFO:tasks.workunit.client.1.smithi163.stdout:6/894: mknod d0/d6/df/d8f/c125 0 2024-02-16T19:24:35.890 INFO:tasks.workunit.client.1.smithi163.stdout:6/895: creat d0/dd/df7/f126 x:0 0 0 2024-02-16T19:24:35.891 INFO:tasks.workunit.client.1.smithi163.stdout:3/937: creat d0/dc/d94/d9e/d126/f149 x:0 0 0 2024-02-16T19:24:35.891 INFO:tasks.workunit.client.1.smithi163.stdout:3/938: truncate d0/dc/d1d/fd8 492426 0 2024-02-16T19:24:35.891 INFO:tasks.workunit.client.1.smithi163.stdout:3/939: chown d0/dc/d1d/d29/d38/cab 929 1 2024-02-16T19:24:35.894 INFO:tasks.workunit.client.1.smithi163.stdout:6/896: link d0/d18/d77/d97/laf d0/d6/df/d20/d40/l127 0 2024-02-16T19:24:35.896 INFO:tasks.workunit.client.1.smithi163.stdout:3/940: creat d0/dc/d94/d9e/d121/f14a x:0 0 0 2024-02-16T19:24:35.897 INFO:tasks.workunit.client.1.smithi163.stdout:6/897: symlink d0/dd/d7f/l128 0 2024-02-16T19:24:35.901 INFO:tasks.workunit.client.1.smithi163.stdout:3/941: unlink d0/dc/d1d/d29/d38/d46/l6b 0 2024-02-16T19:24:35.903 INFO:tasks.workunit.client.1.smithi163.stdout:3/942: mkdir d0/d83/da5/d14b 0 2024-02-16T19:24:35.904 INFO:tasks.workunit.client.1.smithi163.stdout:3/943: mkdir d0/d14c 0 2024-02-16T19:24:35.907 INFO:tasks.workunit.client.1.smithi163.stdout:3/944: mkdir d0/dc/d1d/d49/d98/d14d 0 2024-02-16T19:24:35.909 INFO:tasks.workunit.client.1.smithi163.stdout:3/945: creat d0/dc/d96/dff/f14e x:0 0 0 2024-02-16T19:24:35.910 INFO:tasks.workunit.client.1.smithi163.stdout:3/946: dread d0/dc/d1c/d32/db8/fc7 [0,4194304] 0 2024-02-16T19:24:35.919 INFO:tasks.workunit.client.1.smithi163.stdout:3/947: dread d0/dc/d50/d66/d93/f12f [0,4194304] 0 2024-02-16T19:24:35.920 INFO:tasks.workunit.client.1.smithi163.stdout:3/948: symlink d0/d83/da5/l14f 0 2024-02-16T19:24:35.932 INFO:tasks.workunit.client.1.smithi163.stdout:3/949: read d0/dc/d1c/d32/d7f/fbd [3581612,106681] 0 2024-02-16T19:24:35.949 INFO:tasks.workunit.client.1.smithi163.stdout:3/950: dwrite d0/dc/d1d/d49/d98/fe9 [0,4194304] 0 2024-02-16T19:24:35.969 INFO:tasks.workunit.client.1.smithi163.stdout:3/951: dwrite d0/dc/d1c/d32/f36 [4194304,4194304] 0 2024-02-16T19:24:35.974 INFO:tasks.workunit.client.1.smithi163.stdout:3/952: link d0/dc/d1d/d49/fc9 d0/d83/da5/d127/dea/f150 0 2024-02-16T19:24:35.980 INFO:tasks.workunit.client.1.smithi163.stdout:3/953: getdents d0/dc/d96/dff 0 2024-02-16T19:24:35.980 INFO:tasks.workunit.client.1.smithi163.stdout:3/954: chown d0/dc/d1c/d21/f42 2767939 1 2024-02-16T19:24:35.991 INFO:tasks.workunit.client.1.smithi163.stdout:3/955: write d0/dc/d50/f11a [994351,21456] 0 2024-02-16T19:24:35.994 INFO:tasks.workunit.client.1.smithi163.stdout:3/956: creat d0/dc/d1c/f151 x:0 0 0 2024-02-16T19:24:36.009 INFO:tasks.workunit.client.1.smithi163.stdout:3/957: dwrite d0/dc/d50/d66/da3/f81 [0,4194304] 0 2024-02-16T19:24:36.011 INFO:tasks.workunit.client.1.smithi163.stdout:3/958: rename d0/dc/d1c/d32/f36 to d0/dc/d1c/d116/f152 0 2024-02-16T19:24:36.012 INFO:tasks.workunit.client.1.smithi163.stdout:3/959: mknod d0/d83/da5/d14b/c153 0 2024-02-16T19:24:36.013 INFO:tasks.workunit.client.1.smithi163.stdout:3/960: mkdir d0/dc/d1c/d32/db8/d154 0 2024-02-16T19:24:36.013 INFO:tasks.workunit.client.1.smithi163.stdout:3/961: chown d0/dc/d1d/fd8 116203258 1 2024-02-16T19:24:36.013 INFO:tasks.workunit.client.1.smithi163.stdout:3/962: dread - d0/dc/d96/dff/f14e zero size 2024-02-16T19:24:36.014 INFO:tasks.workunit.client.1.smithi163.stdout:3/963: readlink d0/dc/d1c/d32/db8/d99/l125 0 2024-02-16T19:24:36.014 INFO:tasks.workunit.client.1.smithi163.stdout:3/964: stat d0/dc/d94/d9e/d121/d13d 0 2024-02-16T19:24:36.014 INFO:tasks.workunit.client.1.smithi163.stdout:3/965: dread - d0/dc/d1c/d21/f141 zero size 2024-02-16T19:24:36.015 INFO:tasks.workunit.client.1.smithi163.stdout:3/966: dread d0/dc/d94/d9e/fad [0,4194304] 0 2024-02-16T19:24:36.015 INFO:tasks.workunit.client.1.smithi163.stdout:3/967: chown d0/dc/d50/d66/da3/d71/f87 125393780 1 2024-02-16T19:24:36.016 INFO:tasks.workunit.client.1.smithi163.stdout:3/968: truncate d0/dc/d50/f11a 3479262 0 2024-02-16T19:24:36.018 INFO:tasks.workunit.client.1.smithi163.stdout:3/969: link d0/dc/d94/d9e/d121/f14a d0/d83/f155 0 2024-02-16T19:24:36.018 INFO:tasks.workunit.client.1.smithi163.stdout:3/970: fdatasync d0/dc/d1c/d32/db8/f129 0 2024-02-16T19:24:36.020 INFO:tasks.workunit.client.1.smithi163.stdout:3/971: rmdir d0/dc/d50/d66/d80 39 2024-02-16T19:24:36.023 INFO:tasks.workunit.client.1.smithi163.stdout:3/972: unlink d0/dc/d1c/d32/d7f/fbd 0 2024-02-16T19:24:36.023 INFO:tasks.workunit.client.1.smithi163.stdout:3/973: chown d0/dc/d1d/d49/d98/d14d 2664 1 2024-02-16T19:24:36.025 INFO:tasks.workunit.client.1.smithi163.stdout:3/974: mknod d0/dc/d1d/d49/d98/d14d/c156 0 2024-02-16T19:24:36.036 INFO:tasks.workunit.client.1.smithi163.stdout:3/975: read d0/dc/d96/fdd [3924103,123323] 0 2024-02-16T19:24:36.037 INFO:tasks.workunit.client.1.smithi163.stdout:3/976: unlink d0/c7 0 2024-02-16T19:24:36.052 INFO:tasks.workunit.client.1.smithi163.stdout:3/977: dwrite d0/dc/d1d/d29/f30 [0,4194304] 0 2024-02-16T19:24:36.052 INFO:tasks.workunit.client.1.smithi163.stdout:3/978: chown d0/dc/d1d/d49 11056 1 2024-02-16T19:24:36.066 INFO:tasks.workunit.client.1.smithi163.stdout:1/939: sync 2024-02-16T19:24:36.074 INFO:tasks.workunit.client.1.smithi163.stdout:1/940: dread d7/dc/d78/fbd [0,4194304] 0 2024-02-16T19:24:36.074 INFO:tasks.workunit.client.1.smithi163.stdout:1/941: dread - d7/dc/d2d/d90/fea zero size 2024-02-16T19:24:36.074 INFO:tasks.workunit.client.1.smithi163.stdout:1/942: creat d7/dc/dc1/f142 x:0 0 0 2024-02-16T19:24:36.079 INFO:tasks.workunit.client.1.smithi163.stdout:1/943: rename d7/dc/d2d/d32/d10f to d7/d20/d21/d4e/dc6/d143 0 2024-02-16T19:24:36.079 INFO:tasks.workunit.client.1.smithi163.stdout:1/944: chown d7/dc/d2d/d32/d87/da6 30599 1 2024-02-16T19:24:36.080 INFO:tasks.workunit.client.1.smithi163.stdout:1/945: stat d7/dc/d78/d7c/c7f 0 2024-02-16T19:24:36.080 INFO:tasks.workunit.client.1.smithi163.stdout:1/946: chown d7/d20/d21/d74/dab/lb4 16090 1 2024-02-16T19:24:36.082 INFO:tasks.workunit.client.1.smithi163.stdout:1/947: link d7/d20/d93/fb3 d7/dc/d2d/d32/d87/da6/de5/d110/f144 0 2024-02-16T19:24:36.082 INFO:tasks.workunit.client.1.smithi163.stdout:1/948: write d7/d20/d21/d74/de2/f13a [418218,34570] 0 2024-02-16T19:24:36.083 INFO:tasks.workunit.client.1.smithi163.stdout:1/949: creat d7/d5b/d138/f145 x:0 0 0 2024-02-16T19:24:36.099 INFO:tasks.workunit.client.1.smithi163.stdout:1/950: dwrite d7/d20/d96/fd5 [0,4194304] 0 2024-02-16T19:24:36.114 INFO:tasks.workunit.client.1.smithi163.stdout:1/951: dwrite d7/d20/d21/d4e/dc6/d143/f113 [0,4194304] 0 2024-02-16T19:24:36.114 INFO:tasks.workunit.client.1.smithi163.stdout:1/952: creat d7/d20/d93/dd2/f146 x:0 0 0 2024-02-16T19:24:36.116 INFO:tasks.workunit.client.1.smithi163.stdout:1/953: mkdir d7/d20/d96/d147 0 2024-02-16T19:24:36.117 INFO:tasks.workunit.client.1.smithi163.stdout:1/954: read d7/dc/d78/d7c/f81 [152700,933] 0 2024-02-16T19:24:36.117 INFO:tasks.workunit.client.1.smithi163.stdout:1/955: readlink d7/dc/d2d/l73 0 2024-02-16T19:24:36.117 INFO:tasks.workunit.client.1.smithi163.stdout:1/956: dread - d7/d20/f11a zero size 2024-02-16T19:24:36.117 INFO:tasks.workunit.client.1.smithi163.stdout:1/957: getdents d7/dc/d2d 0 2024-02-16T19:24:36.117 INFO:tasks.workunit.client.1.smithi163.stdout:1/958: dread - d7/d20/d21/d4e/dc6/f141 zero size 2024-02-16T19:24:36.118 INFO:tasks.workunit.client.1.smithi163.stdout:1/959: getdents d7/dc/d2d/d3f/d85 0 2024-02-16T19:24:36.119 INFO:tasks.workunit.client.1.smithi163.stdout:1/960: mkdir d7/d20/d21/d4e/dc6/d143/d148 0 2024-02-16T19:24:36.120 INFO:tasks.workunit.client.1.smithi163.stdout:1/961: symlink d7/dc/d2d/d3f/d85/l149 0 2024-02-16T19:24:36.121 INFO:tasks.workunit.client.1.smithi163.stdout:1/962: creat d7/d20/d96/d147/f14a x:0 0 0 2024-02-16T19:24:36.122 INFO:tasks.workunit.client.1.smithi163.stdout:1/963: rmdir d7/d20/d21/d74/de2/dfb 0 2024-02-16T19:24:36.131 INFO:tasks.workunit.client.1.smithi163.stdout:1/964: dread d7/dc/d2d/d32/f43 [4194304,4194304] 0 2024-02-16T19:24:36.132 INFO:tasks.workunit.client.1.smithi163.stdout:1/965: mknod d7/dc/d78/c14b 0 2024-02-16T19:24:36.184 INFO:tasks.workunit.client.1.smithi163.stdout:2/998: sync 2024-02-16T19:24:36.184 INFO:tasks.workunit.client.1.smithi163.stdout:4/816: sync 2024-02-16T19:24:36.184 INFO:tasks.workunit.client.1.smithi163.stdout:8/955: sync 2024-02-16T19:24:36.184 INFO:tasks.workunit.client.1.smithi163.stdout:7/813: sync 2024-02-16T19:24:36.184 INFO:tasks.workunit.client.1.smithi163.stdout:4/817: readlink d11/d2d/l103 0 2024-02-16T19:24:36.184 INFO:tasks.workunit.client.1.smithi163.stdout:4/818: chown d11/d2d/d4f/d60/d111 47602184 1 2024-02-16T19:24:36.184 INFO:tasks.workunit.client.1.smithi163.stdout:4/819: dread - d11/d18/d1f/d29/d52/d57/d6c/dc3/fb2 zero size 2024-02-16T19:24:36.185 INFO:tasks.workunit.client.1.smithi163.stdout:8/956: link d0/dc/de3/l113 d0/d7/d18/d53/ded/d85/l137 0 2024-02-16T19:24:36.185 INFO:tasks.workunit.client.1.smithi163.stdout:8/957: fdatasync d0/d7/d19/d1f/d32/f124 0 2024-02-16T19:24:36.187 INFO:tasks.workunit.client.1.smithi163.stdout:7/814: rename d11/d50/d5d/d8b/d67/dd6/da6/la9 to d11/d24/d49/l116 0 2024-02-16T19:24:36.188 INFO:tasks.workunit.client.1.smithi163.stdout:8/958: mknod d0/d7/d103/c138 0 2024-02-16T19:24:36.188 INFO:tasks.workunit.client.1.smithi163.stdout:8/959: fdatasync d0/d7/fdb 0 2024-02-16T19:24:36.189 INFO:tasks.workunit.client.1.smithi163.stdout:4/820: rename d11/d18/d1f/d29/d52/d57/c9b to d11/d18/d1f/d23/c114 0 2024-02-16T19:24:36.190 INFO:tasks.workunit.client.1.smithi163.stdout:7/815: symlink d11/d30/dda/l117 0 2024-02-16T19:24:36.190 INFO:tasks.workunit.client.1.smithi163.stdout:8/960: mknod d0/d7/d18/d112/d2c/ddf/c139 0 2024-02-16T19:24:36.190 INFO:tasks.workunit.client.1.smithi163.stdout:8/961: readlink d0/d7/d18/d112/d2c/l96 0 2024-02-16T19:24:36.190 INFO:tasks.workunit.client.1.smithi163.stdout:2/999: dread d2/f5 [0,4194304] 0 2024-02-16T19:24:36.191 INFO:tasks.workunit.client.1.smithi163.stdout:7/816: truncate d11/d24/d34/d8d/f92 1244065 0 2024-02-16T19:24:36.191 INFO:tasks.workunit.client.1.smithi163.stdout:7/817: fsync fb 0 2024-02-16T19:24:36.191 INFO:tasks.workunit.client.1.smithi163.stdout:8/962: truncate d0/d7/d18/d112/d2c/d74/fb0 264469 0 2024-02-16T19:24:36.193 INFO:tasks.workunit.client.1.smithi163.stdout:7/818: mknod d11/d24/d25/d4b/d8c/dee/c118 0 2024-02-16T19:24:36.193 INFO:tasks.workunit.client.1.smithi163.stdout:7/819: readlink d11/d50/d5d/d8b/l38 0 2024-02-16T19:24:36.195 INFO:tasks.workunit.client.1.smithi163.stdout:7/820: mknod d11/d50/d5d/c119 0 2024-02-16T19:24:36.195 INFO:tasks.workunit.client.1.smithi163.stdout:7/821: fdatasync d11/d24/d34/d8d/f92 0 2024-02-16T19:24:36.195 INFO:tasks.workunit.client.1.smithi163.stdout:7/822: dread - d11/d24/d49/dc4/fe6 zero size 2024-02-16T19:24:36.195 INFO:tasks.workunit.client.1.smithi163.stdout:7/823: readlink d11/d24/d25/d4b/d66/dc1/de7/l103 0 2024-02-16T19:24:36.195 INFO:tasks.workunit.client.1.smithi163.stdout:7/824: chown d11/d24/d34/d8d/lad 2 1 2024-02-16T19:24:36.196 INFO:tasks.workunit.client.1.smithi163.stdout:7/825: getdents d11/d24/d34 0 2024-02-16T19:24:36.202 INFO:tasks.workunit.client.1.smithi163.stdout:7/826: unlink d11/d24/d25/d4b/d66/dc1/de7/l103 0 2024-02-16T19:24:36.202 INFO:tasks.workunit.client.1.smithi163.stdout:7/827: write d11/d30/dda/fe1 [715745,16823] 0 2024-02-16T19:24:36.202 INFO:tasks.workunit.client.1.smithi163.stdout:7/828: fdatasync d11/d24/d25/f39 0 2024-02-16T19:24:36.205 INFO:tasks.workunit.client.1.smithi163.stdout:7/829: symlink d11/d24/d49/dc4/l11a 0 2024-02-16T19:24:36.215 INFO:tasks.workunit.client.1.smithi163.stdout:7/830: dread d11/d24/d34/d8d/ffe [0,4194304] 0 2024-02-16T19:24:36.216 INFO:tasks.workunit.client.1.smithi163.stdout:7/831: symlink d11/d50/d5d/d8b/d67/dd6/da6/l11b 0 2024-02-16T19:24:36.216 INFO:tasks.workunit.client.1.smithi163.stdout:8/963: dwrite d0/d7/d18/d2e/f8f [0,4194304] 0 2024-02-16T19:24:36.216 INFO:tasks.workunit.client.1.smithi163.stdout:8/964: write d0/d7/d18/f37 [2588657,13920] 0 2024-02-16T19:24:36.216 INFO:tasks.workunit.client.1.smithi163.stdout:4/821: dwrite d11/d18/d1f/d29/d52/d57/f112 [0,4194304] 0 2024-02-16T19:24:36.216 INFO:tasks.workunit.client.1.smithi163.stdout:4/822: creat d11/d2d/d4f/d60/d79/dc9/f115 x:0 0 0 2024-02-16T19:24:36.220 INFO:tasks.workunit.client.1.smithi163.stdout:4/823: mkdir d11/d18/d1f/d29/d116 0 2024-02-16T19:24:36.226 INFO:tasks.workunit.client.1.smithi163.stdout:8/965: dread d0/d7/d19/d1f/d32/f10c [0,4194304] 0 2024-02-16T19:24:36.226 INFO:tasks.workunit.client.1.smithi163.stdout:8/966: rmdir d0/d7/d19/d127/d6a/d7b/deb 39 2024-02-16T19:24:36.227 INFO:tasks.workunit.client.1.smithi163.stdout:8/967: write d0/d7/d18/d53/ded/fb9 [2522770,55805] 0 2024-02-16T19:24:36.240 INFO:tasks.workunit.client.1.smithi163.stdout:8/968: dwrite d0/d7/d19/d1f/d32/fe9 [4194304,4194304] 0 2024-02-16T19:24:36.241 INFO:tasks.workunit.client.1.smithi163.stdout:8/969: chown d0/d7/d19/d127/d6a/d7b/deb/cc2 116 1 2024-02-16T19:24:36.242 INFO:tasks.workunit.client.1.smithi163.stdout:8/970: mkdir d0/d7/d19/d1f/d32/d13a 0 2024-02-16T19:24:36.243 INFO:tasks.workunit.client.1.smithi163.stdout:8/971: creat d0/d7/d18/d53/d108/f13b x:0 0 0 2024-02-16T19:24:36.244 INFO:tasks.workunit.client.1.smithi163.stdout:8/972: creat d0/d7/d18/d112/d2c/ddf/de5/f13c x:0 0 0 2024-02-16T19:24:36.244 INFO:tasks.workunit.client.1.smithi163.stdout:8/973: mkdir d0/d7/d18/d53/d108/d13d 0 2024-02-16T19:24:36.246 INFO:tasks.workunit.client.1.smithi163.stdout:8/974: creat d0/d7/d18/d112/d2c/f13e x:0 0 0 2024-02-16T19:24:36.246 INFO:tasks.workunit.client.1.smithi163.stdout:8/975: chown d0/d7/d19/d127/d6a/db8/d116/l125 0 1 2024-02-16T19:24:36.246 INFO:tasks.workunit.client.1.smithi163.stdout:8/976: mkdir d0/d7/d18/d112/d3c/d3d/d11a/d13f 0 2024-02-16T19:24:36.254 INFO:tasks.workunit.client.1.smithi163.stdout:8/977: write d0/d7/d18/d112/d2c/f90 [3154786,62501] 0 2024-02-16T19:24:36.381 INFO:tasks.workunit.client.1.smithi163.stdout:6/898: sync 2024-02-16T19:24:36.381 INFO:tasks.workunit.client.1.smithi163.stdout:3/979: sync 2024-02-16T19:24:36.381 INFO:tasks.workunit.client.1.smithi163.stdout:6/899: fsync d0/dd/d7f/d81/fc8 0 2024-02-16T19:24:36.382 INFO:tasks.workunit.client.1.smithi163.stdout:3/980: link d0/dc/l35 d0/dc/d1c/d32/db8/d154/l157 0 2024-02-16T19:24:36.388 INFO:tasks.workunit.client.1.smithi163.stdout:3/981: dread d0/dc/d96/fdd [0,4194304] 0 2024-02-16T19:24:36.389 INFO:tasks.workunit.client.1.smithi163.stdout:3/982: mknod d0/d83/da5/d127/dea/c158 0 2024-02-16T19:24:36.389 INFO:tasks.workunit.client.1.smithi163.stdout:3/983: truncate d0/dc/d1d/d49/f10c 361150 0 2024-02-16T19:24:36.391 INFO:tasks.workunit.client.1.smithi163.stdout:3/984: creat d0/dc/d1c/d32/df0/f159 x:0 0 0 2024-02-16T19:24:36.391 INFO:tasks.workunit.client.1.smithi163.stdout:3/985: dread - d0/dc/d50/d66/da3/d71/f111 zero size 2024-02-16T19:24:36.391 INFO:tasks.workunit.client.1.smithi163.stdout:3/986: truncate d0/dc/d94/d9e/d126/f149 172486 0 2024-02-16T19:24:36.391 INFO:tasks.workunit.client.1.smithi163.stdout:3/987: truncate d0/dc/d94/fae 791047 0 2024-02-16T19:24:36.393 INFO:tasks.workunit.client.1.smithi163.stdout:3/988: unlink d0/dc/d50/d66/d80/c130 0 2024-02-16T19:24:36.408 INFO:tasks.workunit.client.1.smithi163.stdout:6/900: dwrite d0/d6/df/d20/d40/d17/f88 [0,4194304] 0 2024-02-16T19:24:36.408 INFO:tasks.workunit.client.1.smithi163.stdout:3/989: dwrite d0/dc/d1c/d32/ddb/fdf [0,4194304] 0 2024-02-16T19:24:36.410 INFO:tasks.workunit.client.1.smithi163.stdout:3/990: link d0/f102 d0/f15a 0 2024-02-16T19:24:36.410 INFO:tasks.workunit.client.1.smithi163.stdout:3/991: write d0/d83/da5/d127/dea/f150 [906419,95184] 0 2024-02-16T19:24:36.411 INFO:tasks.workunit.client.1.smithi163.stdout:3/992: symlink d0/dc/d94/d9e/d121/l15b 0 2024-02-16T19:24:36.417 INFO:tasks.workunit.client.1.smithi163.stdout:3/993: dread d0/dc/d94/d9e/d126/f149 [0,4194304] 0 2024-02-16T19:24:36.418 INFO:tasks.workunit.client.1.smithi163.stdout:3/994: link d0/dc/d1c/d32/d7f/lbe d0/dc/d1d/d29/d38/l15c 0 2024-02-16T19:24:36.418 INFO:tasks.workunit.client.1.smithi163.stdout:3/995: dread - d0/dc/d96/da7/f13c zero size 2024-02-16T19:24:36.419 INFO:tasks.workunit.client.1.smithi163.stdout:3/996: truncate d0/f15a 532722 0 2024-02-16T19:24:36.422 INFO:tasks.workunit.client.1.smithi163.stdout:3/997: write d0/dc/d1c/d91/fc3 [1530155,46431] 0 2024-02-16T19:24:36.423 INFO:tasks.workunit.client.1.smithi163.stdout:3/998: mknod d0/d83/da5/c15d 0 2024-02-16T19:24:36.427 INFO:tasks.workunit.client.1.smithi163.stdout:6/901: dwrite d0/d18/d94/fac [0,4194304] 0 2024-02-16T19:24:36.427 INFO:tasks.workunit.client.1.smithi163.stdout:6/902: write d0/dd/d7f/d81/fc8 [5124648,8742] 0 2024-02-16T19:24:36.431 INFO:tasks.workunit.client.1.smithi163.stdout:6/903: symlink d0/d18/d77/d97/l129 0 2024-02-16T19:24:36.433 INFO:tasks.workunit.client.1.smithi163.stdout:6/904: creat d0/f12a x:0 0 0 2024-02-16T19:24:36.454 INFO:tasks.workunit.client.1.smithi163.stdout:3/999: dwrite d0/dc/d1d/d29/d38/d46/f139 [0,4194304] 0 2024-02-16T19:24:36.454 INFO:tasks.workunit.client.1.smithi163.stdout:6/905: dwrite d0/dd/df7/f126 [0,4194304] 0 2024-02-16T19:24:36.454 INFO:tasks.workunit.client.1.smithi163.stdout:6/906: fdatasync d0/d18/d77/f9e 0 2024-02-16T19:24:36.463 INFO:tasks.workunit.client.1.smithi163.stdout:6/907: write d0/dd/d7f/f2d [2422921,58630] 0 2024-02-16T19:24:36.464 INFO:tasks.workunit.client.1.smithi163.stdout:6/908: symlink d0/dd/d7f/d15/d11c/l12b 0 2024-02-16T19:24:36.465 INFO:tasks.workunit.client.1.smithi163.stdout:6/909: rmdir d0/dd/d7f/d81/daa 39 2024-02-16T19:24:36.466 INFO:tasks.workunit.client.1.smithi163.stdout:6/910: symlink d0/d18/d77/d97/db8/dd7/l12c 0 2024-02-16T19:24:36.466 INFO:tasks.workunit.client.1.smithi163.stdout:6/911: write d0/dd/d7f/d81/d85/da4/dad/f10b [436475,40667] 0 2024-02-16T19:24:36.466 INFO:tasks.workunit.client.1.smithi163.stdout:6/912: write d0/d18/d77/d70/f8c [630224,88265] 0 2024-02-16T19:24:36.466 INFO:tasks.workunit.client.1.smithi163.stdout:6/913: dread - d0/d6/df/fd3 zero size 2024-02-16T19:24:36.466 INFO:tasks.workunit.client.1.smithi163.stdout:6/914: mkdir d0/d18/d77/d97/d12d 0 2024-02-16T19:24:36.467 INFO:tasks.workunit.client.1.smithi163.stdout:6/915: mkdir d0/dd/d7f/d2e/d12e 0 2024-02-16T19:24:36.468 INFO:tasks.workunit.client.1.smithi163.stdout:6/916: rename d0/d18/d77/d97/f120 to d0/dd/d7f/d15/d11c/f12f 0 2024-02-16T19:24:36.469 INFO:tasks.workunit.client.1.smithi163.stdout:6/917: rename d0/d6/l10 to d0/dd/l130 0 2024-02-16T19:24:36.477 INFO:tasks.workunit.client.1.smithi163.stdout:6/918: write d0/d6/df/d20/d40/d1e/f52 [2509093,79439] 0 2024-02-16T19:24:36.478 INFO:tasks.workunit.client.1.smithi163.stdout:6/919: creat d0/d6/df/d20/d40/d1e/db2/d117/f131 x:0 0 0 2024-02-16T19:24:36.478 INFO:tasks.workunit.client.1.smithi163.stdout:6/920: truncate d0/dd/dbc/ddc/de5/fb7 721797 0 2024-02-16T19:24:36.479 INFO:tasks.workunit.client.1.smithi163.stdout:6/921: mkdir d0/d6/df/d20/d40/d1e/db2/dda/d132 0 2024-02-16T19:24:36.479 INFO:tasks.workunit.client.1.smithi163.stdout:6/922: chown d0/dd/d7f/f73 22625912 1 2024-02-16T19:24:36.481 INFO:tasks.workunit.client.1.smithi163.stdout:6/923: dread d0/d18/d77/dc9/f72 [0,4194304] 0 2024-02-16T19:24:36.483 INFO:tasks.workunit.client.1.smithi163.stdout:6/924: fsync d0/dd/d7f/f2d 0 2024-02-16T19:24:36.484 INFO:tasks.workunit.client.1.smithi163.stdout:6/925: rmdir d0/d6/df/d30 39 2024-02-16T19:24:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/766353290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:24:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: Standby manager daemon smithi163.wjfclz restarted 2024-02-16T19:24:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:24:36.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:24:36.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:24:36.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:24:36.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:24:36.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:36.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:36 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:36.489 INFO:tasks.workunit.client.1.smithi163.stdout:6/926: symlink d0/dd/d7f/l133 0 2024-02-16T19:24:36.490 INFO:tasks.workunit.client.1.smithi163.stdout:6/927: truncate d0/dd/d7f/d15/f4d 1847048 0 2024-02-16T19:24:36.498 INFO:tasks.workunit.client.1.smithi163.stdout:6/928: dread d0/d6/df/d20/d69/db4/ff5 [0,4194304] 0 2024-02-16T19:24:36.500 INFO:tasks.workunit.client.1.smithi163.stdout:6/929: getdents d0 0 2024-02-16T19:24:36.504 INFO:tasks.workunit.client.1.smithi163.stdout:1/966: sync 2024-02-16T19:24:36.504 INFO:tasks.workunit.client.1.smithi163.stdout:7/832: sync 2024-02-16T19:24:36.504 INFO:tasks.workunit.client.1.smithi163.stdout:1/967: fsync d7/dc/d2d/d32/d3b/fdd 0 2024-02-16T19:24:36.504 INFO:tasks.workunit.client.1.smithi163.stdout:1/968: write d7/d20/d21/d74/dab/f102 [1012383,104903] 0 2024-02-16T19:24:36.505 INFO:tasks.workunit.client.1.smithi163.stdout:7/833: symlink d11/d24/d25/l11c 0 2024-02-16T19:24:36.505 INFO:tasks.workunit.client.1.smithi163.stdout:1/969: symlink d7/dc/d78/d7c/l14c 0 2024-02-16T19:24:36.506 INFO:tasks.workunit.client.1.smithi163.stdout:7/834: symlink d11/d24/d25/d4b/d66/l11d 0 2024-02-16T19:24:36.506 INFO:tasks.workunit.client.1.smithi163.stdout:7/835: chown d11/d24/d34/d8d/df9/d10f 8 1 2024-02-16T19:24:36.507 INFO:tasks.workunit.client.1.smithi163.stdout:1/970: creat d7/d20/d21/d4e/dc6/d143/d148/f14d x:0 0 0 2024-02-16T19:24:36.507 INFO:tasks.workunit.client.1.smithi163.stdout:1/971: creat d7/d20/d93/dd2/f14e x:0 0 0 2024-02-16T19:24:36.509 INFO:tasks.workunit.client.1.smithi163.stdout:1/972: mkdir d7/d20/d21/d4e/dc6/d14f 0 2024-02-16T19:24:36.511 INFO:tasks.workunit.client.1.smithi163.stdout:6/930: dread d0/d18/d94/fac [0,4194304] 0 2024-02-16T19:24:36.514 INFO:tasks.workunit.client.1.smithi163.stdout:6/931: truncate d0/d6/df/f32 311958 0 2024-02-16T19:24:36.514 INFO:tasks.workunit.client.1.smithi163.stdout:6/932: write d0/d6/df/d20/d69/f8d [619445,65264] 0 2024-02-16T19:24:36.515 INFO:tasks.workunit.client.1.smithi163.stdout:6/933: mkdir d0/dd/d7f/d81/d85/da4/dad/d134 0 2024-02-16T19:24:36.516 INFO:tasks.workunit.client.1.smithi163.stdout:7/836: dread d11/f20 [0,4194304] 0 2024-02-16T19:24:36.516 INFO:tasks.workunit.client.1.smithi163.stdout:7/837: chown d11/d50/l55 0 1 2024-02-16T19:24:36.517 INFO:tasks.workunit.client.1.smithi163.stdout:6/934: link d0/dd/d7f/d2e/cd5 d0/dd/d7f/d15/df8/c135 0 2024-02-16T19:24:36.517 INFO:tasks.workunit.client.1.smithi163.stdout:6/935: write d0/dd/d7f/d81/d85/fce [2640117,34534] 0 2024-02-16T19:24:36.518 INFO:tasks.workunit.client.1.smithi163.stdout:6/936: truncate d0/d6/df/d20/d40/d17/fa2 927003 0 2024-02-16T19:24:36.518 INFO:tasks.workunit.client.1.smithi163.stdout:6/937: fdatasync d0/d18/f48 0 2024-02-16T19:24:36.518 INFO:tasks.workunit.client.1.smithi163.stdout:6/938: chown d0/d6/f92 1239061 1 2024-02-16T19:24:36.520 INFO:tasks.workunit.client.1.smithi163.stdout:7/838: unlink d11/d24/d34/l8f 0 2024-02-16T19:24:36.520 INFO:tasks.workunit.client.1.smithi163.stdout:6/939: mknod d0/d6/c136 0 2024-02-16T19:24:36.520 INFO:tasks.workunit.client.1.smithi163.stdout:6/940: write d0/d6/df/d30/f5c [423363,117793] 0 2024-02-16T19:24:36.521 INFO:tasks.workunit.client.1.smithi163.stdout:7/839: rmdir d11/d50/d5d/d8b/dc8 39 2024-02-16T19:24:36.521 INFO:tasks.workunit.client.1.smithi163.stdout:7/840: write d11/d26/f106 [972992,100242] 0 2024-02-16T19:24:36.522 INFO:tasks.workunit.client.1.smithi163.stdout:7/841: mknod d11/d24/d34/d8d/c11e 0 2024-02-16T19:24:36.523 INFO:tasks.workunit.client.1.smithi163.stdout:7/842: write d11/d50/f88 [467156,93920] 0 2024-02-16T19:24:36.523 INFO:tasks.workunit.client.1.smithi163.stdout:7/843: chown d11/d24/d49/l116 0 1 2024-02-16T19:24:36.525 INFO:tasks.workunit.client.1.smithi163.stdout:7/844: rename d11/f20 to d11/d24/d34/f11f 0 2024-02-16T19:24:36.530 INFO:tasks.workunit.client.1.smithi163.stdout:1/973: dwrite d7/d20/d21/d4e/f9e [0,4194304] 0 2024-02-16T19:24:36.530 INFO:tasks.workunit.client.1.smithi163.stdout:1/974: dread - d7/dc/d2d/d32/d87/de1/dfd/f127 zero size 2024-02-16T19:24:36.530 INFO:tasks.workunit.client.1.smithi163.stdout:1/975: write d7/d20/f11a [991323,78842] 0 2024-02-16T19:24:36.533 INFO:tasks.workunit.client.1.smithi163.stdout:1/976: rmdir d7/dc/d2d/d90 39 2024-02-16T19:24:36.554 INFO:tasks.workunit.client.1.smithi163.stdout:7/845: dwrite d11/d50/d5d/dcd/fe0 [0,4194304] 0 2024-02-16T19:24:36.554 INFO:tasks.workunit.client.1.smithi163.stdout:7/846: chown d11/d24/d25/d4b/d66/f9f 4 1 2024-02-16T19:24:36.555 INFO:tasks.workunit.client.1.smithi163.stdout:6/941: dwrite d0/d6/f31 [0,4194304] 0 2024-02-16T19:24:36.555 INFO:tasks.workunit.client.1.smithi163.stdout:6/942: fsync d0/d6/df/f83 0 2024-02-16T19:24:36.559 INFO:tasks.workunit.client.1.smithi163.stdout:7/847: truncate d11/d26/f2b 4992818 0 2024-02-16T19:24:36.562 INFO:tasks.workunit.client.1.smithi163.stdout:6/943: rmdir d0/d6/df/d20/d40/d1e/db2/d113 0 2024-02-16T19:24:36.563 INFO:tasks.workunit.client.1.smithi163.stdout:6/944: link d0/d6/df/f32 d0/d18/d77/f137 0 2024-02-16T19:24:36.570 INFO:tasks.workunit.client.1.smithi163.stdout:7/848: write f5 [2775122,125691] 0 2024-02-16T19:24:36.571 INFO:tasks.workunit.client.1.smithi163.stdout:7/849: creat d11/d24/d25/d4b/d66/dc1/f120 x:0 0 0 2024-02-16T19:24:36.571 INFO:tasks.workunit.client.1.smithi163.stdout:7/850: chown d11/d24/d49/d105 48847 1 2024-02-16T19:24:36.574 INFO:tasks.workunit.client.1.smithi163.stdout:7/851: dread d11/d26/f106 [0,4194304] 0 2024-02-16T19:24:36.575 INFO:tasks.workunit.client.1.smithi163.stdout:7/852: rename d11/d24/d49/d105/l10c to d11/d24/d34/d8d/l121 0 2024-02-16T19:24:36.577 INFO:tasks.workunit.client.1.smithi163.stdout:7/853: unlink d11/d50/d5d/d8b/c72 0 2024-02-16T19:24:36.577 INFO:tasks.workunit.client.1.smithi163.stdout:7/854: mknod d11/d50/d5d/c122 0 2024-02-16T19:24:36.577 INFO:tasks.workunit.client.1.smithi163.stdout:7/855: chown d11/d50/d5d/d8b/l41 2029915 1 2024-02-16T19:24:36.578 INFO:tasks.workunit.client.1.smithi163.stdout:7/856: mkdir d11/d50/d5d/dcd/d123 0 2024-02-16T19:24:36.579 INFO:tasks.workunit.client.1.smithi163.stdout:7/857: write d11/d24/d25/d4b/d66/f6f [44509,89404] 0 2024-02-16T19:24:36.579 INFO:tasks.workunit.client.1.smithi163.stdout:7/858: symlink d11/d50/d5d/d8b/d67/dd6/l124 0 2024-02-16T19:24:36.581 INFO:tasks.workunit.client.1.smithi163.stdout:6/945: dwrite d0/d18/d77/dc9/fec [0,4194304] 0 2024-02-16T19:24:36.583 INFO:tasks.workunit.client.1.smithi163.stdout:6/946: mknod d0/d18/d77/d97/c138 0 2024-02-16T19:24:36.583 INFO:tasks.workunit.client.1.smithi163.stdout:6/947: fdatasync d0/d6/f45 0 2024-02-16T19:24:36.583 INFO:tasks.workunit.client.1.smithi163.stdout:6/948: truncate d0/d6/df/d20/d69/f8d 1596982 0 2024-02-16T19:24:36.591 INFO:tasks.workunit.client.1.smithi163.stdout:7/859: write d11/d24/d25/d4b/d66/f9f [4152982,71782] 0 2024-02-16T19:24:36.591 INFO:tasks.workunit.client.1.smithi163.stdout:7/860: truncate d11/d24/d25/d4b/d66/f10b 969717 0 2024-02-16T19:24:36.595 INFO:tasks.workunit.client.1.smithi163.stdout:7/861: dread d11/d24/d49/f69 [0,4194304] 0 2024-02-16T19:24:36.595 INFO:tasks.workunit.client.1.smithi163.stdout:7/862: readlink d11/d24/d25/d4b/d8c/dee/l10a 0 2024-02-16T19:24:36.595 INFO:tasks.workunit.client.1.smithi163.stdout:6/949: dread d0/dd/f6d [0,4194304] 0 2024-02-16T19:24:36.596 INFO:tasks.workunit.client.1.smithi163.stdout:7/863: creat d11/d24/d25/d4b/d66/f125 x:0 0 0 2024-02-16T19:24:36.596 INFO:tasks.workunit.client.1.smithi163.stdout:7/864: chown d11/d50/d5d/d8b/c99 5 1 2024-02-16T19:24:36.597 INFO:tasks.workunit.client.1.smithi163.stdout:6/950: mknod d0/dd/d7f/d2e/c139 0 2024-02-16T19:24:36.597 INFO:tasks.workunit.client.1.smithi163.stdout:7/865: truncate d11/d50/d5d/d8b/f90 2051748 0 2024-02-16T19:24:36.597 INFO:tasks.workunit.client.1.smithi163.stdout:7/866: stat d11/d50/d5d/lbc 0 2024-02-16T19:24:36.599 INFO:tasks.workunit.client.1.smithi163.stdout:7/867: unlink fa 0 2024-02-16T19:24:36.607 INFO:tasks.workunit.client.1.smithi163.stdout:6/951: read d0/d6/df/d20/f68 [3420046,51447] 0 2024-02-16T19:24:36.608 INFO:tasks.workunit.client.1.smithi163.stdout:6/952: stat d0/dd/d7f/d15/d1c/dca/f10a 0 2024-02-16T19:24:36.609 INFO:tasks.workunit.client.1.smithi163.stdout:6/953: creat d0/dd/d7f/d81/daa/f13a x:0 0 0 2024-02-16T19:24:36.609 INFO:tasks.workunit.client.1.smithi163.stdout:6/954: write d0/d18/d77/f9e [3688322,86639] 0 2024-02-16T19:24:36.611 INFO:tasks.workunit.client.1.smithi163.stdout:6/955: getdents d0/d6/df/d7b/d87 0 2024-02-16T19:24:36.616 INFO:tasks.workunit.client.1.smithi163.stdout:7/868: dwrite d11/d24/d49/f69 [0,4194304] 0 2024-02-16T19:24:36.626 INFO:tasks.workunit.client.1.smithi163.stdout:7/869: dread d11/ff4 [0,4194304] 0 2024-02-16T19:24:36.627 INFO:tasks.workunit.client.1.smithi163.stdout:7/870: rename d11/d24/d34/d8d/dd9/le8 to d11/d24/d49/dac/l126 0 2024-02-16T19:24:36.639 INFO:tasks.workunit.client.1.smithi163.stdout:6/956: dwrite d0/d18/d94/fac [0,4194304] 0 2024-02-16T19:24:36.639 INFO:tasks.workunit.client.1.smithi163.stdout:7/871: dread d11/d30/f32 [4194304,4194304] 0 2024-02-16T19:24:36.640 INFO:tasks.workunit.client.1.smithi163.stdout:7/872: mkdir d11/d24/d34/d8d/dff/d127 0 2024-02-16T19:24:36.641 INFO:tasks.workunit.client.1.smithi163.stdout:6/957: creat d0/d18/d77/d97/d12d/f13b x:0 0 0 2024-02-16T19:24:36.641 INFO:tasks.workunit.client.1.smithi163.stdout:7/873: mknod d11/d30/dda/c128 0 2024-02-16T19:24:36.641 INFO:tasks.workunit.client.1.smithi163.stdout:7/874: readlink d11/d50/d5d/d8b/d67/dd6/da6/l11b 0 2024-02-16T19:24:36.642 INFO:tasks.workunit.client.1.smithi163.stdout:7/875: dread - d11/d24/d25/d4b/d8c/fcf zero size 2024-02-16T19:24:36.643 INFO:tasks.workunit.client.1.smithi163.stdout:6/958: rename d0/d6/df/d7b/d87/fba to d0/dd/f13c 0 2024-02-16T19:24:36.643 INFO:tasks.workunit.client.1.smithi163.stdout:6/959: chown d0/dd/dbc/ddc/de5/f3d 116813 1 2024-02-16T19:24:36.643 INFO:tasks.workunit.client.1.smithi163.stdout:6/960: fdatasync d0/d6/f45 0 2024-02-16T19:24:36.643 INFO:tasks.workunit.client.1.smithi163.stdout:6/961: dread - d0/dd/d7f/d81/daa/f13a zero size 2024-02-16T19:24:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:24:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/766353290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:24:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: Standby manager daemon smithi163.wjfclz restarted 2024-02-16T19:24:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:24:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:24:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:24:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:24:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:24:36.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:36.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:36 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:36.645 INFO:tasks.workunit.client.1.smithi163.stdout:6/962: dread d0/d6/df/d20/d40/f59 [0,4194304] 0 2024-02-16T19:24:36.645 INFO:tasks.workunit.client.1.smithi163.stdout:6/963: chown d0/dd/d7f/d15/d1c/f56 412006 1 2024-02-16T19:24:36.646 INFO:tasks.workunit.client.1.smithi163.stdout:6/964: creat d0/d18/d77/d70/f13d x:0 0 0 2024-02-16T19:24:36.649 INFO:tasks.workunit.client.1.smithi163.stdout:7/876: dread d11/d30/dda/fe1 [0,4194304] 0 2024-02-16T19:24:36.649 INFO:tasks.workunit.client.1.smithi163.stdout:7/877: readlink d11/d24/d49/ld1 0 2024-02-16T19:24:36.650 INFO:tasks.workunit.client.1.smithi163.stdout:6/965: dread d0/d18/d77/d97/db8/fc7 [0,4194304] 0 2024-02-16T19:24:36.683 INFO:tasks.workunit.client.1.smithi163.stdout:7/878: dwrite d11/d24/d25/d4b/fdc [0,4194304] 0 2024-02-16T19:24:36.683 INFO:tasks.workunit.client.1.smithi163.stdout:7/879: write d11/d26/f46 [1090295,121366] 0 2024-02-16T19:24:36.683 INFO:tasks.workunit.client.1.smithi163.stdout:7/880: dread - d11/d24/d49/fa7 zero size 2024-02-16T19:24:36.684 INFO:tasks.workunit.client.1.smithi163.stdout:6/966: dwrite d0/dd/dbc/ddc/de5/f38 [0,4194304] 0 2024-02-16T19:24:36.685 INFO:tasks.workunit.client.1.smithi163.stdout:7/881: rename d11/d50/c59 to d11/d24/d49/dac/c129 0 2024-02-16T19:24:36.695 INFO:tasks.workunit.client.1.smithi163.stdout:8/978: sync 2024-02-16T19:24:36.696 INFO:tasks.workunit.client.1.smithi163.stdout:4/824: sync 2024-02-16T19:24:36.696 INFO:tasks.workunit.client.1.smithi163.stdout:8/979: readlink d0/d7/d18/d112/d3c/d3d/d41/lf4 0 2024-02-16T19:24:36.696 INFO:tasks.workunit.client.1.smithi163.stdout:8/980: fdatasync d0/d7/d18/d53/ded/fb9 0 2024-02-16T19:24:36.696 INFO:tasks.workunit.client.1.smithi163.stdout:8/981: chown d0/d7/d18/d112/d2c/d74/l9b 57 1 2024-02-16T19:24:36.696 INFO:tasks.workunit.client.1.smithi163.stdout:8/982: truncate d0/d7/d18/d112/d3c/d3d/fde 1769127 0 2024-02-16T19:24:36.698 INFO:tasks.workunit.client.1.smithi163.stdout:4/825: mknod d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/c117 0 2024-02-16T19:24:36.700 INFO:tasks.workunit.client.1.smithi163.stdout:8/983: dread d0/d7/d18/d2e/f7f [0,4194304] 0 2024-02-16T19:24:36.703 INFO:tasks.workunit.client.1.smithi163.stdout:8/984: unlink d0/l1c 0 2024-02-16T19:24:36.708 INFO:tasks.workunit.client.1.smithi163.stdout:4/826: write d11/d18/d1f/d29/fa1 [6864985,43195] 0 2024-02-16T19:24:36.708 INFO:tasks.workunit.client.1.smithi163.stdout:7/882: dwrite d11/d26/f2b [4194304,4194304] 0 2024-02-16T19:24:36.708 INFO:tasks.workunit.client.1.smithi163.stdout:7/883: dread - d11/d24/d25/d4b/d8c/fcf zero size 2024-02-16T19:24:36.710 INFO:tasks.workunit.client.1.smithi163.stdout:4/827: rename d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/cda to d11/d18/d1f/d29/d52/d57/d6c/d9c/c118 0 2024-02-16T19:24:36.710 INFO:tasks.workunit.client.1.smithi163.stdout:4/828: creat d11/d2d/d4f/d5f/d70/d74/f119 x:0 0 0 2024-02-16T19:24:36.718 INFO:tasks.workunit.client.1.smithi163.stdout:8/985: dwrite d0/d7/d19/d127/d6a/df3/fa4 [0,4194304] 0 2024-02-16T19:24:36.718 INFO:tasks.workunit.client.1.smithi163.stdout:8/986: creat d0/d6f/f140 x:0 0 0 2024-02-16T19:24:36.720 INFO:tasks.workunit.client.1.smithi163.stdout:8/987: rmdir d0/d7/d19/d127/d6a/df3/df2 39 2024-02-16T19:24:36.740 INFO:tasks.workunit.client.1.smithi163.stdout:7/884: dwrite d11/d24/d34/f58 [0,4194304] 0 2024-02-16T19:24:36.741 INFO:tasks.workunit.client.1.smithi163.stdout:7/885: stat d11/d24/d25/d4b/d8c/fcf 0 2024-02-16T19:24:36.741 INFO:tasks.workunit.client.1.smithi163.stdout:8/988: dwrite d0/dc/f7d [0,4194304] 0 2024-02-16T19:24:36.741 INFO:tasks.workunit.client.1.smithi163.stdout:4/829: dwrite d11/d2d/d4f/d5f/d70/d74/d75/fec [0,4194304] 0 2024-02-16T19:24:36.741 INFO:tasks.workunit.client.1.smithi163.stdout:4/830: stat f2 0 2024-02-16T19:24:36.743 INFO:tasks.workunit.client.1.smithi163.stdout:7/886: rmdir d11/d24/d25/d4b/d8c 39 2024-02-16T19:24:36.743 INFO:tasks.workunit.client.1.smithi163.stdout:7/887: stat d11/d24/d25/d4b 0 2024-02-16T19:24:36.743 INFO:tasks.workunit.client.1.smithi163.stdout:7/888: stat d11/d50/d5d/dcd/f109 0 2024-02-16T19:24:36.746 INFO:tasks.workunit.client.1.smithi163.stdout:4/831: mknod d11/d18/d1f/d29/d52/d57/d6c/d7c/dad/c11a 0 2024-02-16T19:24:36.748 INFO:tasks.workunit.client.1.smithi163.stdout:7/889: link d11/f1f d11/d24/d49/dc4/f12a 0 2024-02-16T19:24:36.748 INFO:tasks.workunit.client.1.smithi163.stdout:4/832: unlink d11/d18/d1f/d23/l2c 0 2024-02-16T19:24:36.751 INFO:tasks.workunit.client.1.smithi163.stdout:7/890: dread d11/d24/d34/d8d/ffe [0,4194304] 0 2024-02-16T19:24:36.762 INFO:tasks.workunit.client.1.smithi163.stdout:8/989: dwrite d0/d7/d19/d127/d6a/df3/fa4 [0,4194304] 0 2024-02-16T19:24:36.766 INFO:tasks.workunit.client.1.smithi163.stdout:4/833: write d11/f8a [4058544,14181] 0 2024-02-16T19:24:36.766 INFO:tasks.workunit.client.1.smithi163.stdout:4/834: readlink d11/d3d/l93 0 2024-02-16T19:24:36.767 INFO:tasks.workunit.client.1.smithi163.stdout:7/891: dread d11/d24/d49/f69 [0,4194304] 0 2024-02-16T19:24:36.767 INFO:tasks.workunit.client.1.smithi163.stdout:7/892: chown d11/d50/d5d/dcd/ce4 50902 1 2024-02-16T19:24:36.767 INFO:tasks.workunit.client.1.smithi163.stdout:4/835: mkdir d11/d2d/d4f/d5f/d70/d74/d75/dca/d11b 0 2024-02-16T19:24:36.768 INFO:tasks.workunit.client.1.smithi163.stdout:7/893: symlink d11/d24/d34/d8d/df9/d10f/l12b 0 2024-02-16T19:24:36.770 INFO:tasks.workunit.client.1.smithi163.stdout:7/894: rename d11/d24/d34/d8d/c11e to d11/d24/d25/d4b/d8c/c12c 0 2024-02-16T19:24:36.770 INFO:tasks.workunit.client.1.smithi163.stdout:7/895: creat d11/d24/d25/d4b/f12d x:0 0 0 2024-02-16T19:24:36.771 INFO:tasks.workunit.client.1.smithi163.stdout:8/990: write d0/d7/d18/d53/ded/fe6 [653118,103660] 0 2024-02-16T19:24:36.771 INFO:tasks.workunit.client.1.smithi163.stdout:8/991: dread - d0/d7/d18/d112/d2c/d74/dcb/fec zero size 2024-02-16T19:24:36.774 INFO:tasks.workunit.client.1.smithi163.stdout:8/992: rename d0/d7/d18/d53/d108/c126 to d0/d7/d18/d53/ded/d85/de2/c141 0 2024-02-16T19:24:36.776 INFO:tasks.workunit.client.1.smithi163.stdout:8/993: mkdir d0/d7/d19/d1f/d32/d70/d142 0 2024-02-16T19:24:36.777 INFO:tasks.workunit.client.1.smithi163.stdout:8/994: creat d0/d7/d18/d112/d3c/d3d/d41/f143 x:0 0 0 2024-02-16T19:24:36.778 INFO:tasks.workunit.client.1.smithi163.stdout:8/995: mkdir d0/d7/d18/d112/d2c/d74/d144 0 2024-02-16T19:24:36.779 INFO:tasks.workunit.client.1.smithi163.stdout:7/896: dread d11/d50/d5d/d8b/d67/f9b [0,4194304] 0 2024-02-16T19:24:36.779 INFO:tasks.workunit.client.1.smithi163.stdout:7/897: creat d11/d50/d5d/dcd/f12e x:0 0 0 2024-02-16T19:24:36.780 INFO:tasks.workunit.client.1.smithi163.stdout:7/898: symlink d11/d50/d5d/dc9/l12f 0 2024-02-16T19:24:36.781 INFO:tasks.workunit.client.1.smithi163.stdout:7/899: mknod d11/d50/d5d/dcd/d123/c130 0 2024-02-16T19:24:36.783 INFO:tasks.workunit.client.1.smithi163.stdout:8/996: dread d0/d7/d18/d112/d2c/d74/fb0 [0,4194304] 0 2024-02-16T19:24:36.785 INFO:tasks.workunit.client.1.smithi163.stdout:8/997: unlink d0/d7/d19/d127/d6a/df3/df2/f5a 0 2024-02-16T19:24:36.785 INFO:tasks.workunit.client.1.smithi163.stdout:8/998: fdatasync d0/d7/d18/d112/d2c/ddf/ffe 0 2024-02-16T19:24:36.786 INFO:tasks.workunit.client.1.smithi163.stdout:4/836: dwrite d11/d18/d1f/d29/d52/d57/d6c/d7c/fce [0,4194304] 0 2024-02-16T19:24:36.786 INFO:tasks.workunit.client.1.smithi163.stdout:8/999: creat d0/d7/d18/d112/f145 x:0 0 0 2024-02-16T19:24:36.794 INFO:tasks.workunit.client.1.smithi163.stdout:7/900: dread d11/d24/d34/f58 [0,4194304] 0 2024-02-16T19:24:36.807 INFO:tasks.workunit.client.1.smithi163.stdout:4/837: dwrite d11/d2d/d4f/d60/d79/ff1 [0,4194304] 0 2024-02-16T19:24:36.808 INFO:tasks.workunit.client.1.smithi163.stdout:7/901: dread d11/d24/d49/fa5 [0,4194304] 0 2024-02-16T19:24:36.812 INFO:tasks.workunit.client.1.smithi163.stdout:4/838: chown d11/d2d/d4f/dfd 4 1 2024-02-16T19:24:36.813 INFO:tasks.workunit.client.1.smithi163.stdout:4/839: truncate d11/d18/d1f/d23/d3a/f58 327102 0 2024-02-16T19:24:36.815 INFO:tasks.workunit.client.1.smithi163.stdout:1/977: sync 2024-02-16T19:24:36.815 INFO:tasks.workunit.client.1.smithi163.stdout:4/840: mknod d11/d2d/d4f/d60/d79/dc9/c11c 0 2024-02-16T19:24:36.815 INFO:tasks.workunit.client.1.smithi163.stdout:6/967: sync 2024-02-16T19:24:36.818 INFO:tasks.workunit.client.1.smithi163.stdout:4/841: rename d11/d18/d1f/d29/d52/d57/d6c/d9d/ca3 to d11/d18/d1f/d29/d52/dd6/c11d 0 2024-02-16T19:24:36.818 INFO:tasks.workunit.client.1.smithi163.stdout:4/842: stat d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f 0 2024-02-16T19:24:36.818 INFO:tasks.workunit.client.1.smithi163.stdout:4/843: dread - d11/d18/d1f/d29/d52/d57/d6c/dc3/fe6 zero size 2024-02-16T19:24:36.822 INFO:tasks.workunit.client.1.smithi163.stdout:6/968: unlink d0/d6/l16 0 2024-02-16T19:24:36.825 INFO:tasks.workunit.client.1.smithi163.stdout:6/969: rename d0/d18/d77/dc9/f72 to d0/dd/dbc/ddc/deb/f13e 0 2024-02-16T19:24:36.827 INFO:tasks.workunit.client.1.smithi163.stdout:4/844: getdents d11/d2d/d81 0 2024-02-16T19:24:36.829 INFO:tasks.workunit.client.1.smithi163.stdout:4/845: truncate d11/d2d/d4f/f63 2625497 0 2024-02-16T19:24:36.829 INFO:tasks.workunit.client.1.smithi163.stdout:4/846: fsync d11/f16 0 2024-02-16T19:24:36.833 INFO:tasks.workunit.client.1.smithi163.stdout:4/847: creat d11/d18/d1f/d34/de5/f11e x:0 0 0 2024-02-16T19:24:36.843 INFO:tasks.workunit.client.1.smithi163.stdout:7/902: dwrite d11/d26/f33 [0,4194304] 0 2024-02-16T19:24:36.849 INFO:tasks.workunit.client.1.smithi163.stdout:6/970: dwrite d0/d6/df/d20/d40/d1e/db2/d117/f131 [0,4194304] 0 2024-02-16T19:24:36.851 INFO:tasks.workunit.client.1.smithi163.stdout:6/971: truncate d0/dd/d7f/d81/d85/fce 3233338 0 2024-02-16T19:24:36.852 INFO:tasks.workunit.client.1.smithi163.stdout:6/972: mknod d0/d6/df/d20/d69/c13f 0 2024-02-16T19:24:36.856 INFO:tasks.workunit.client.1.smithi163.stdout:4/848: dwrite d11/d18/d1f/d29/d52/d57/f84 [0,4194304] 0 2024-02-16T19:24:36.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/849: stat d11/d18/d1f/d23/f10d 0 2024-02-16T19:24:36.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/850: fdatasync d11/d18/d1f/d34/de5/f11e 0 2024-02-16T19:24:36.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/851: write d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/f109 [6156,43186] 0 2024-02-16T19:24:36.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/852: chown d11/d18/d1f/d29/d52/d76/d89/d94/fd5 5094304 1 2024-02-16T19:24:36.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/853: truncate d11/d2d/d4f/d5f/d70/d74/d75/fac 1963 0 2024-02-16T19:24:36.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/854: truncate d11/d18/d1f/d29/d52/d57/d6c/d9d/fa0 131180 0 2024-02-16T19:24:36.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/855: fdatasync d11/d18/d1f/d29/d52/d57/d6c/d7c/f92 0 2024-02-16T19:24:36.859 INFO:tasks.workunit.client.1.smithi163.stdout:4/856: mknod d11/d18/d1f/d29/d52/d57/dc5/c11f 0 2024-02-16T19:24:36.861 INFO:tasks.workunit.client.1.smithi163.stdout:6/973: dread d0/d18/d77/dc9/d74/f110 [0,4194304] 0 2024-02-16T19:24:36.861 INFO:tasks.workunit.client.1.smithi163.stdout:6/974: write d0/d6/df/d20/d40/d1e/db2/dda/f105 [1358610,82434] 0 2024-02-16T19:24:36.861 INFO:tasks.workunit.client.1.smithi163.stdout:6/975: read d0/d6/f8 [4391042,76706] 0 2024-02-16T19:24:36.863 INFO:tasks.workunit.client.1.smithi163.stdout:4/857: rmdir d11/d18/d1f/d23 39 2024-02-16T19:24:36.865 INFO:tasks.workunit.client.1.smithi163.stdout:6/976: mknod d0/dd/d7f/d2e/c140 0 2024-02-16T19:24:36.865 INFO:tasks.workunit.client.1.smithi163.stdout:4/858: creat d11/d2d/d4f/d60/d111/f120 x:0 0 0 2024-02-16T19:24:36.865 INFO:tasks.workunit.client.1.smithi163.stdout:4/859: creat d11/d2d/d4f/d60/f121 x:0 0 0 2024-02-16T19:24:36.867 INFO:tasks.workunit.client.1.smithi163.stdout:6/977: dread d0/d6/df/d30/f5c [0,4194304] 0 2024-02-16T19:24:36.867 INFO:tasks.workunit.client.1.smithi163.stdout:6/978: fsync d0/d18/d77/fbf 0 2024-02-16T19:24:36.867 INFO:tasks.workunit.client.1.smithi163.stdout:6/979: readlink d0/d6/df/d20/d40/d1e/db2/dda/l123 0 2024-02-16T19:24:36.869 INFO:tasks.workunit.client.1.smithi163.stdout:6/980: symlink d0/dd/d7f/d81/d85/da4/dad/l141 0 2024-02-16T19:24:36.869 INFO:tasks.workunit.client.1.smithi163.stdout:6/981: chown d0/dd/d7f/l133 218 1 2024-02-16T19:24:36.869 INFO:tasks.workunit.client.1.smithi163.stdout:6/982: write d0/d18/d77/fdb [1497610,30585] 0 2024-02-16T19:24:36.869 INFO:tasks.workunit.client.1.smithi163.stdout:6/983: write d0/dd/d7f/d81/daa/ffb [855769,109777] 0 2024-02-16T19:24:36.870 INFO:tasks.workunit.client.1.smithi163.stdout:6/984: symlink d0/d6/df/d7b/d87/l142 0 2024-02-16T19:24:36.879 INFO:tasks.workunit.client.1.smithi163.stdout:4/860: dread d11/d18/d1f/d34/f51 [0,4194304] 0 2024-02-16T19:24:36.879 INFO:tasks.workunit.client.1.smithi163.stdout:4/861: fdatasync d11/d2d/d4f/d5f/d70/d74/d75/dca/fa8 0 2024-02-16T19:24:36.891 INFO:tasks.workunit.client.1.smithi163.stdout:4/862: read d11/d2d/d4f/d5f/ff0 [3284983,13801] 0 2024-02-16T19:24:36.891 INFO:tasks.workunit.client.1.smithi163.stdout:4/863: stat d11/d18/d1f/d29/l4c 0 2024-02-16T19:24:36.892 INFO:tasks.workunit.client.1.smithi163.stdout:6/985: dwrite d0/dd/d7f/d2e/f4f [0,4194304] 0 2024-02-16T19:24:36.896 INFO:tasks.workunit.client.1.smithi163.stdout:6/986: link d0/dd/d7f/d15/l3e d0/dd/d7f/d15/d1c/d65/d63/d86/l143 0 2024-02-16T19:24:36.896 INFO:tasks.workunit.client.1.smithi163.stdout:6/987: chown d0/d6/df/d7b/lb3 1031597 1 2024-02-16T19:24:36.902 INFO:tasks.workunit.client.1.smithi163.stdout:4/864: write d11/d2d/d4f/d5f/d70/d74/fd4 [307418,88573] 0 2024-02-16T19:24:36.903 INFO:tasks.workunit.client.1.smithi163.stdout:4/865: chown d11/d18/d1f/d34/de5/ff7 15020 1 2024-02-16T19:24:36.915 INFO:tasks.workunit.client.1.smithi163.stdout:6/988: dwrite d0/dd/dbc/ddc/de5/fb7 [0,4194304] 0 2024-02-16T19:24:36.918 INFO:tasks.workunit.client.1.smithi163.stdout:6/989: rmdir d0/d18/d77/dc9/d74 39 2024-02-16T19:24:36.919 INFO:tasks.workunit.client.1.smithi163.stdout:6/990: rmdir d0/dd/dbc/ddc 39 2024-02-16T19:24:36.919 INFO:tasks.workunit.client.1.smithi163.stdout:6/991: stat d0/dd/d7f 0 2024-02-16T19:24:36.921 INFO:tasks.workunit.client.1.smithi163.stdout:6/992: mkdir d0/d6/df/d144 0 2024-02-16T19:24:36.921 INFO:tasks.workunit.client.1.smithi163.stdout:6/993: chown d0/dd/df7/f126 599060 1 2024-02-16T19:24:36.923 INFO:tasks.workunit.client.1.smithi163.stdout:6/994: dread d0/d18/d77/d70/f8c [0,4194304] 0 2024-02-16T19:24:36.924 INFO:tasks.workunit.client.1.smithi163.stdout:6/995: rmdir d0/d18/d94 39 2024-02-16T19:24:36.927 INFO:tasks.workunit.client.1.smithi163.stdout:6/996: creat d0/dd/d7f/d2e/f145 x:0 0 0 2024-02-16T19:24:37.106 INFO:tasks.workunit.client.1.smithi163.stdout:1/978: sync 2024-02-16T19:24:37.107 INFO:tasks.workunit.client.1.smithi163.stdout:1/979: mknod d7/dc/d78/dee/c150 0 2024-02-16T19:24:37.107 INFO:tasks.workunit.client.1.smithi163.stdout:1/980: stat d7/dc/d2d/d32/f46 0 2024-02-16T19:24:37.107 INFO:tasks.workunit.client.1.smithi163.stdout:1/981: write d7/d20/d21/d4e/f79 [1126344,103659] 0 2024-02-16T19:24:37.108 INFO:tasks.workunit.client.1.smithi163.stdout:1/982: mkdir d7/d20/d21/d4e/dc6/d143/d148/d151 0 2024-02-16T19:24:37.109 INFO:tasks.workunit.client.1.smithi163.stdout:1/983: truncate d7/d20/d21/d74/dab/ff8 252743 0 2024-02-16T19:24:37.109 INFO:tasks.workunit.client.1.smithi163.stdout:1/984: read d7/dc/dc1/f108 [192772,36934] 0 2024-02-16T19:24:37.113 INFO:tasks.workunit.client.1.smithi163.stdout:1/985: rename d7/d20/d96/d147 to d7/dc/d2d/d32/d87/d152 0 2024-02-16T19:24:37.113 INFO:tasks.workunit.client.1.smithi163.stdout:1/986: truncate d7/d20/d21/d74/de2/f13a 1392366 0 2024-02-16T19:24:37.115 INFO:tasks.workunit.client.1.smithi163.stdout:1/987: dread d7/d20/f35 [4194304,4194304] 0 2024-02-16T19:24:37.117 INFO:tasks.workunit.client.1.smithi163.stdout:1/988: mkdir d7/dc/d2d/d32/d87/da6/de5/d110/d153 0 2024-02-16T19:24:37.119 INFO:tasks.workunit.client.1.smithi163.stdout:1/989: creat d7/d5b/d63/f154 x:0 0 0 2024-02-16T19:24:37.120 INFO:tasks.workunit.client.1.smithi163.stdout:1/990: dread d7/dc/d2d/fc7 [0,4194304] 0 2024-02-16T19:24:37.120 INFO:tasks.workunit.client.1.smithi163.stdout:1/991: fdatasync d7/d20/f24 0 2024-02-16T19:24:37.121 INFO:tasks.workunit.client.1.smithi163.stdout:1/992: symlink d7/dc/d78/d7c/l155 0 2024-02-16T19:24:37.121 INFO:tasks.workunit.client.1.smithi163.stdout:1/993: chown d7/d20/c6e 26 1 2024-02-16T19:24:37.126 INFO:tasks.workunit.client.1.smithi163.stdout:1/994: rmdir d7/dc/d78/d7c 39 2024-02-16T19:24:37.126 INFO:tasks.workunit.client.1.smithi163.stdout:1/995: fsync d7/d20/d21/f55 0 2024-02-16T19:24:37.127 INFO:tasks.workunit.client.1.smithi163.stdout:1/996: fdatasync d7/d20/f99 0 2024-02-16T19:24:37.127 INFO:tasks.workunit.client.1.smithi163.stdout:1/997: symlink d7/dc/d2d/d3f/l156 0 2024-02-16T19:24:37.129 INFO:tasks.workunit.client.1.smithi163.stdout:1/998: mkdir d7/dc/d2d/d3f/d85/d157 0 2024-02-16T19:24:37.130 INFO:tasks.workunit.client.1.smithi163.stdout:1/999: creat d7/dc/d11d/f158 x:0 0 0 2024-02-16T19:24:37.145 INFO:tasks.workunit.client.1.smithi163.stdout:6/997: sync 2024-02-16T19:24:37.145 INFO:tasks.workunit.client.1.smithi163.stdout:7/903: sync 2024-02-16T19:24:37.145 INFO:tasks.workunit.client.1.smithi163.stdout:4/866: sync 2024-02-16T19:24:37.148 INFO:tasks.workunit.client.1.smithi163.stdout:6/998: unlink d0/d6/df/fd3 0 2024-02-16T19:24:37.151 INFO:tasks.workunit.client.1.smithi163.stdout:4/867: rename d11/d18/d1f/d29/d52/d57/d6c/d7c/dad/cfa to d11/d18/d1f/d29/d52/d57/d6c/d7c/c122 0 2024-02-16T19:24:37.152 INFO:tasks.workunit.client.1.smithi163.stdout:4/868: dread - d11/d2d/d4f/d5f/d70/d74/d75/dca/fa8 zero size 2024-02-16T19:24:37.161 INFO:tasks.workunit.client.1.smithi163.stdout:4/869: dread d11/f4b [0,4194304] 0 2024-02-16T19:24:37.164 INFO:tasks.workunit.client.1.smithi163.stdout:4/870: dread d11/f41 [0,4194304] 0 2024-02-16T19:24:37.164 INFO:tasks.workunit.client.1.smithi163.stdout:4/871: creat d11/d18/d1f/d34/de5/f123 x:0 0 0 2024-02-16T19:24:37.167 INFO:tasks.workunit.client.1.smithi163.stdout:4/872: rename d11/d18/d1f/d23/d3a to d11/d18/d1f/d29/d124 0 2024-02-16T19:24:37.176 INFO:tasks.workunit.client.1.smithi163.stdout:6/999: dwrite d0/dd/d7f/d2e/f145 [0,4194304] 0 2024-02-16T19:24:37.176 INFO:tasks.workunit.client.1.smithi163.stdout:7/904: dwrite d11/d24/d25/fbf [0,4194304] 0 2024-02-16T19:24:37.179 INFO:tasks.workunit.client.1.smithi163.stdout:7/905: mkdir d11/d50/d5d/dcd/d131 0 2024-02-16T19:24:37.181 INFO:tasks.workunit.client.1.smithi163.stdout:7/906: symlink d11/d24/d34/d8d/l132 0 2024-02-16T19:24:37.181 INFO:tasks.workunit.client.1.smithi163.stdout:7/907: dread - d11/d24/f10e zero size 2024-02-16T19:24:37.181 INFO:tasks.workunit.client.1.smithi163.stdout:7/908: creat d11/d26/f133 x:0 0 0 2024-02-16T19:24:37.181 INFO:tasks.workunit.client.1.smithi163.stdout:7/909: creat d11/d24/d25/d4b/f134 x:0 0 0 2024-02-16T19:24:37.181 INFO:tasks.workunit.client.1.smithi163.stdout:7/910: chown d11/d24/d25/d4b/d66/f10b 0 1 2024-02-16T19:24:37.181 INFO:tasks.workunit.client.1.smithi163.stdout:7/911: readlink d11/d50/d5d/d8b/l38 0 2024-02-16T19:24:37.184 INFO:tasks.workunit.client.1.smithi163.stdout:4/873: dwrite f2 [0,4194304] 0 2024-02-16T19:24:37.199 INFO:tasks.workunit.client.1.smithi163.stdout:7/912: dwrite d11/d24/d25/f2d [0,4194304] 0 2024-02-16T19:24:37.201 INFO:tasks.workunit.client.1.smithi163.stdout:7/913: rename d11/f89 to d11/d50/d5d/dcd/f135 0 2024-02-16T19:24:37.201 INFO:tasks.workunit.client.1.smithi163.stdout:7/914: chown d11/d24/d49/dac/fe9 874 1 2024-02-16T19:24:37.201 INFO:tasks.workunit.client.1.smithi163.stdout:7/915: fdatasync d11/d24/d34/d8d/ffe 0 2024-02-16T19:24:37.202 INFO:tasks.workunit.client.1.smithi163.stdout:7/916: symlink d11/d50/d5d/d8b/d67/db2/l136 0 2024-02-16T19:24:37.218 INFO:tasks.workunit.client.1.smithi163.stdout:7/917: dwrite d11/d24/d49/dc4/fca [0,4194304] 0 2024-02-16T19:24:37.220 INFO:tasks.workunit.client.1.smithi163.stdout:7/918: rename d11/d50/d5d/dcd/d131 to d11/d24/d25/d4b/d8c/d137 0 2024-02-16T19:24:37.221 INFO:tasks.workunit.client.1.smithi163.stdout:7/919: rename d11/d24/d25/d4b/d66/fb9 to d11/d50/d5d/dcd/f138 0 2024-02-16T19:24:37.222 INFO:tasks.workunit.client.1.smithi163.stdout:7/920: read d11/d24/d25/d4b/d66/f6f [49140,128961] 0 2024-02-16T19:24:37.222 INFO:tasks.workunit.client.1.smithi163.stdout:7/921: chown d11/d24/d49/fdd 182361129 1 2024-02-16T19:24:37.223 INFO:tasks.workunit.client.1.smithi163.stdout:7/922: symlink d11/d50/d5d/d8b/l139 0 2024-02-16T19:24:37.231 INFO:tasks.workunit.client.1.smithi163.stdout:7/923: dread fb [0,4194304] 0 2024-02-16T19:24:37.232 INFO:tasks.workunit.client.1.smithi163.stdout:7/924: creat d11/d24/d25/d4b/f13a x:0 0 0 2024-02-16T19:24:37.233 INFO:tasks.workunit.client.1.smithi163.stdout:7/925: mkdir d11/d24/d13b 0 2024-02-16T19:24:37.234 INFO:tasks.workunit.client.1.smithi163.stdout:7/926: unlink d11/d24/d49/ld1 0 2024-02-16T19:24:37.235 INFO:tasks.workunit.client.1.smithi163.stdout:7/927: mkdir d11/d50/d5d/dc9/d13c 0 2024-02-16T19:24:37.235 INFO:tasks.workunit.client.1.smithi163.stdout:7/928: chown d11/d50/c107 464890581 1 2024-02-16T19:24:37.236 INFO:tasks.workunit.client.1.smithi163.stdout:7/929: symlink d11/d50/d5d/dcd/d123/l13d 0 2024-02-16T19:24:37.239 INFO:tasks.workunit.client.1.smithi163.stdout:7/930: dread d11/ff4 [0,4194304] 0 2024-02-16T19:24:37.375 INFO:tasks.workunit.client.1.smithi163.stdout:4/874: sync 2024-02-16T19:24:37.376 INFO:tasks.workunit.client.1.smithi163.stdout:7/931: sync 2024-02-16T19:24:37.376 INFO:tasks.workunit.client.1.smithi163.stdout:4/875: read d11/f8a [493415,15325] 0 2024-02-16T19:24:37.376 INFO:tasks.workunit.client.1.smithi163.stdout:4/876: truncate d11/f46 936520 0 2024-02-16T19:24:37.376 INFO:tasks.workunit.client.1.smithi163.stdout:7/932: mknod d11/d50/d5d/d8b/dc8/c13e 0 2024-02-16T19:24:37.377 INFO:tasks.workunit.client.1.smithi163.stdout:4/877: truncate d11/d18/d66/f98 726799 0 2024-02-16T19:24:37.377 INFO:tasks.workunit.client.1.smithi163.stdout:4/878: symlink d11/d18/d1f/d29/d52/d57/d6c/d7c/dad/l125 0 2024-02-16T19:24:37.380 INFO:tasks.workunit.client.1.smithi163.stdout:7/933: read d11/d24/d49/f69 [3475733,73331] 0 2024-02-16T19:24:37.380 INFO:tasks.workunit.client.1.smithi163.stdout:7/934: symlink d11/l13f 0 2024-02-16T19:24:37.380 INFO:tasks.workunit.client.1.smithi163.stdout:7/935: write d11/d26/f106 [1456369,23683] 0 2024-02-16T19:24:37.381 INFO:tasks.workunit.client.1.smithi163.stdout:7/936: mknod d11/d24/d34/d8d/df9/c140 0 2024-02-16T19:24:37.381 INFO:tasks.workunit.client.1.smithi163.stdout:7/937: write d11/d24/d25/f9a [426614,37535] 0 2024-02-16T19:24:37.382 INFO:tasks.workunit.client.1.smithi163.stdout:7/938: unlink d11/d30/f56 0 2024-02-16T19:24:37.384 INFO:tasks.workunit.client.1.smithi163.stdout:7/939: rmdir d11/d26 39 2024-02-16T19:24:37.384 INFO:tasks.workunit.client.1.smithi163.stdout:7/940: truncate d11/d24/d49/fa7 803832 0 2024-02-16T19:24:37.384 INFO:tasks.workunit.client.1.smithi163.stdout:7/941: chown d11/d50/d5d/d8b/d67/c8a 992805 1 2024-02-16T19:24:37.384 INFO:tasks.workunit.client.1.smithi163.stdout:7/942: symlink d11/d24/d25/d4b/d66/dc1/l141 0 2024-02-16T19:24:37.385 INFO:tasks.workunit.client.1.smithi163.stdout:7/943: creat d11/d24/f142 x:0 0 0 2024-02-16T19:24:37.387 INFO:tasks.workunit.client.1.smithi163.stdout:4/879: write d11/d2d/d4f/d60/d79/fd3 [4190493,41724] 0 2024-02-16T19:24:37.387 INFO:tasks.workunit.client.1.smithi163.stdout:7/944: symlink d11/d24/d34/d8d/dff/l143 0 2024-02-16T19:24:37.389 INFO:tasks.workunit.client.1.smithi163.stdout:4/880: getdents d11/d18/d1f/d34 0 2024-02-16T19:24:37.392 INFO:tasks.workunit.client.1.smithi163.stdout:7/945: dread d11/d24/d34/f58 [0,4194304] 0 2024-02-16T19:24:37.393 INFO:tasks.workunit.client.1.smithi163.stdout:7/946: creat d11/f144 x:0 0 0 2024-02-16T19:24:37.394 INFO:tasks.workunit.client.1.smithi163.stdout:7/947: mknod d11/d24/d25/c145 0 2024-02-16T19:24:37.396 INFO:tasks.workunit.client.1.smithi163.stdout:7/948: rename d11/d24/d25/f7f to d11/d24/d25/d4b/d8c/f146 0 2024-02-16T19:24:37.397 INFO:tasks.workunit.client.1.smithi163.stdout:7/949: creat d11/d24/d25/d4b/d66/f147 x:0 0 0 2024-02-16T19:24:37.406 INFO:tasks.workunit.client.1.smithi163.stdout:4/881: dwrite d11/d18/d1f/d29/d52/f80 [4194304,4194304] 0 2024-02-16T19:24:37.416 INFO:tasks.workunit.client.1.smithi163.stdout:4/882: dread d11/d18/d1f/d29/d52/d57/f7e [0,4194304] 0 2024-02-16T19:24:37.417 INFO:tasks.workunit.client.1.smithi163.stdout:4/883: stat d11/d2d/l3b 0 2024-02-16T19:24:37.417 INFO:tasks.workunit.client.1.smithi163.stdout:4/884: creat d11/d18/d1f/d29/d52/dd6/f126 x:0 0 0 2024-02-16T19:24:37.417 INFO:tasks.workunit.client.1.smithi163.stdout:4/885: dread - d11/d2d/d4f/d5f/d70/d74/d75/dca/fd8 zero size 2024-02-16T19:24:37.417 INFO:tasks.workunit.client.1.smithi163.stdout:4/886: creat d11/d18/d1f/d29/d52/dd6/f127 x:0 0 0 2024-02-16T19:24:37.418 INFO:tasks.workunit.client.1.smithi163.stdout:4/887: dread - d11/d18/d1f/d29/d52/d57/d6c/d9c/ff6 zero size 2024-02-16T19:24:37.418 INFO:tasks.workunit.client.1.smithi163.stdout:4/888: creat d11/d3d/f128 x:0 0 0 2024-02-16T19:24:37.418 INFO:tasks.workunit.client.1.smithi163.stdout:4/889: dread - d11/d2d/d4f/d60/d111/f120 zero size 2024-02-16T19:24:37.418 INFO:tasks.workunit.client.1.smithi163.stdout:7/950: dwrite d11/d30/f5b [0,4194304] 0 2024-02-16T19:24:37.424 INFO:tasks.workunit.client.1.smithi163.stdout:4/890: unlink d11/d2d/d4f/d5f/ff0 0 2024-02-16T19:24:37.424 INFO:tasks.workunit.client.1.smithi163.stdout:4/891: stat d11/d2d/d4f/d60/d79/f90 0 2024-02-16T19:24:37.425 INFO:tasks.workunit.client.1.smithi163.stdout:4/892: mkdir d11/d2d/d4f/d5f/d70/d74/d75/dca/d11b/d129 0 2024-02-16T19:24:37.426 INFO:tasks.workunit.client.1.smithi163.stdout:4/893: symlink d11/d18/d1f/d29/d52/d57/d6c/l12a 0 2024-02-16T19:24:37.426 INFO:tasks.workunit.client.1.smithi163.stdout:4/894: write d11/d18/d1f/f27 [4422460,78068] 0 2024-02-16T19:24:37.431 INFO:tasks.workunit.client.1.smithi163.stdout:7/951: dwrite d11/d24/f10e [0,4194304] 0 2024-02-16T19:24:37.433 INFO:tasks.workunit.client.1.smithi163.stdout:7/952: getdents d11/d24/d34/d8d/df9/d10f 0 2024-02-16T19:24:37.434 INFO:tasks.workunit.client.1.smithi163.stdout:7/953: creat d11/d24/d34/d8d/df9/f148 x:0 0 0 2024-02-16T19:24:37.435 INFO:tasks.workunit.client.1.smithi163.stdout:7/954: mkdir d11/d24/d25/d4b/d66/d149 0 2024-02-16T19:24:37.437 INFO:tasks.workunit.client.1.smithi163.stdout:7/955: creat d11/d24/d25/d4b/d66/dc1/de7/f14a x:0 0 0 2024-02-16T19:24:37.437 INFO:tasks.workunit.client.1.smithi163.stdout:7/956: creat d11/d24/d25/d4b/d66/f14b x:0 0 0 2024-02-16T19:24:37.441 INFO:tasks.workunit.client.1.smithi163.stdout:4/895: dwrite d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/f109 [0,4194304] 0 2024-02-16T19:24:37.441 INFO:tasks.workunit.client.1.smithi163.stdout:4/896: truncate d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/fb6 2030749 0 2024-02-16T19:24:37.447 INFO:tasks.workunit.client.1.smithi163.stdout:4/897: link d11/d18/d1f/d29/d52/d76/da2/fcb d11/d18/d1f/d29/d52/d57/d6c/d7c/f12b 0 2024-02-16T19:24:37.456 INFO:tasks.workunit.client.1.smithi163.stdout:4/898: write d11/d18/d1f/d29/d52/d57/f112 [322004,52097] 0 2024-02-16T19:24:37.456 INFO:tasks.workunit.client.1.smithi163.stdout:4/899: chown d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/f10b 113 1 2024-02-16T19:24:37.457 INFO:tasks.workunit.client.1.smithi163.stdout:7/957: rmdir d11 39 2024-02-16T19:24:37.460 INFO:tasks.workunit.client.1.smithi163.stdout:4/900: rename d11/d18/d1f/d29/d52/d57/dc5/c11f to d11/d2d/d4f/d60/d79/dc9/c12c 0 2024-02-16T19:24:37.461 INFO:tasks.workunit.client.1.smithi163.stdout:4/901: unlink d11/d18/d1f/d29/d52/ldf 0 2024-02-16T19:24:37.461 INFO:tasks.workunit.client.1.smithi163.stdout:4/902: truncate d11/d2d/d4f/d5f/f10f 993819 0 2024-02-16T19:24:37.463 INFO:tasks.workunit.client.1.smithi163.stdout:4/903: mkdir d11/d12d 0 2024-02-16T19:24:37.464 INFO:tasks.workunit.client.1.smithi163.stdout:4/904: mkdir d11/d18/d100/d12e 0 2024-02-16T19:24:37.465 INFO:tasks.workunit.client.1.smithi163.stdout:4/905: link d11/d2d/d4f/d5f/d70/d74/d75/dca/f78 d11/d2d/d4f/d60/d79/f12f 0 2024-02-16T19:24:37.467 INFO:tasks.workunit.client.1.smithi163.stdout:4/906: mkdir d11/d2d/d4f/d5f/d70/d74/d75/dca/d130 0 2024-02-16T19:24:37.467 INFO:tasks.workunit.client.1.smithi163.stdout:4/907: creat d11/d18/d66/f131 x:0 0 0 2024-02-16T19:24:37.468 INFO:tasks.workunit.client.1.smithi163.stdout:4/908: link d11/d3d/f62 d11/d18/d1f/d29/d52/d57/d6c/d9c/f132 0 2024-02-16T19:24:37.468 INFO:tasks.workunit.client.1.smithi163.stdout:4/909: fdatasync d11/d2d/d4f/d5f/d70/d74/fd4 0 2024-02-16T19:24:37.469 INFO:tasks.workunit.client.1.smithi163.stdout:4/910: mkdir d11/d2d/d4f/d5f/d70/d74/d75/dca/d133 0 2024-02-16T19:24:37.473 INFO:tasks.workunit.client.1.smithi163.stdout:7/958: dwrite d11/d24/f78 [0,4194304] 0 2024-02-16T19:24:37.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:37 smithi163 ceph-mon[36452]: pgmap v257: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 263 MiB/s rd, 323 MiB/s wr, 454 op/s 2024-02-16T19:24:37.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:37 smithi163 ceph-mon[36452]: mgrmap e20: smithi040.jzygzo(active, since 7m), standbys: smithi163.wjfclz 2024-02-16T19:24:37.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:37 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:37.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:37 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:37.491 INFO:tasks.workunit.client.1.smithi163.stdout:7/959: dwrite d11/d30/dda/fea [0,4194304] 0 2024-02-16T19:24:37.491 INFO:tasks.workunit.client.1.smithi163.stdout:7/960: readlink d11/d50/d5d/d8b/l41 0 2024-02-16T19:24:37.500 INFO:tasks.workunit.client.1.smithi163.stdout:7/961: dread d11/d50/d5d/d8b/d67/db2/fba [0,4194304] 0 2024-02-16T19:24:37.500 INFO:tasks.workunit.client.1.smithi163.stdout:7/962: write d11/d50/d5d/d8b/f4e [624756,40826] 0 2024-02-16T19:24:37.501 INFO:tasks.workunit.client.1.smithi163.stdout:7/963: rmdir d11/d24/d25/d4b 39 2024-02-16T19:24:37.501 INFO:tasks.workunit.client.1.smithi163.stdout:7/964: dread - d11/d24/d49/dc4/fe6 zero size 2024-02-16T19:24:37.504 INFO:tasks.workunit.client.1.smithi163.stdout:7/965: write d11/d50/d5d/d8b/f90 [218890,18599] 0 2024-02-16T19:24:37.504 INFO:tasks.workunit.client.1.smithi163.stdout:7/966: write d11/d50/f7c [3357098,38954] 0 2024-02-16T19:24:37.505 INFO:tasks.workunit.client.1.smithi163.stdout:7/967: rename d11/d30/c7d to d11/d50/d5d/c14c 0 2024-02-16T19:24:37.506 INFO:tasks.workunit.client.1.smithi163.stdout:7/968: dread - d11/d24/d25/d4b/d66/f14b zero size 2024-02-16T19:24:37.506 INFO:tasks.workunit.client.1.smithi163.stdout:7/969: unlink d11/d24/lbd 0 2024-02-16T19:24:37.508 INFO:tasks.workunit.client.1.smithi163.stdout:7/970: unlink d11/d50/d5d/dcd/fe0 0 2024-02-16T19:24:37.508 INFO:tasks.workunit.client.1.smithi163.stdout:7/971: mknod d11/d26/dd4/c14d 0 2024-02-16T19:24:37.509 INFO:tasks.workunit.client.1.smithi163.stdout:7/972: creat d11/d24/d25/d4b/d66/dc1/f14e x:0 0 0 2024-02-16T19:24:37.511 INFO:tasks.workunit.client.1.smithi163.stdout:7/973: rename d11/d24/d49/f81 to d11/d24/d49/f14f 0 2024-02-16T19:24:37.511 INFO:tasks.workunit.client.1.smithi163.stdout:7/974: chown d11/d50/d5d/d8b/l139 2744389 1 2024-02-16T19:24:37.511 INFO:tasks.workunit.client.1.smithi163.stdout:7/975: dread - d11/d50/d5d/dcd/f109 zero size 2024-02-16T19:24:37.513 INFO:tasks.workunit.client.1.smithi163.stdout:7/976: dread d11/d50/d5d/d8b/d67/f85 [0,4194304] 0 2024-02-16T19:24:37.519 INFO:tasks.workunit.client.1.smithi163.stdout:7/977: rmdir d11/d50/d5d/dcd 39 2024-02-16T19:24:37.519 INFO:tasks.workunit.client.1.smithi163.stdout:7/978: truncate d11/d24/d25/d4b/d66/f125 981759 0 2024-02-16T19:24:37.539 INFO:tasks.workunit.client.1.smithi163.stdout:7/979: dwrite d11/d24/d49/dc4/f12a [0,4194304] 0 2024-02-16T19:24:37.543 INFO:tasks.workunit.client.1.smithi163.stdout:7/980: read d11/d24/d34/d8d/ffe [4032868,20466] 0 2024-02-16T19:24:37.543 INFO:tasks.workunit.client.1.smithi163.stdout:7/981: chown d11/d26/f4c 84645702 1 2024-02-16T19:24:37.544 INFO:tasks.workunit.client.1.smithi163.stdout:7/982: creat d11/d50/d5d/d8b/d67/db2/f150 x:0 0 0 2024-02-16T19:24:37.560 INFO:tasks.workunit.client.1.smithi163.stdout:7/983: dwrite f3 [0,4194304] 0 2024-02-16T19:24:37.561 INFO:tasks.workunit.client.1.smithi163.stdout:7/984: mkdir d11/d30/dda/d151 0 2024-02-16T19:24:37.562 INFO:tasks.workunit.client.1.smithi163.stdout:7/985: mknod d11/d50/d5d/dcd/d123/c152 0 2024-02-16T19:24:37.579 INFO:tasks.workunit.client.1.smithi163.stdout:7/986: dwrite d11/d24/d25/d4b/d66/dc1/f14e [0,4194304] 0 2024-02-16T19:24:37.579 INFO:tasks.workunit.client.1.smithi163.stdout:7/987: readlink d11/d24/d34/d8d/l132 0 2024-02-16T19:24:37.582 INFO:tasks.workunit.client.1.smithi163.stdout:7/988: symlink d11/d24/d25/d4b/d66/dc1/l153 0 2024-02-16T19:24:37.583 INFO:tasks.workunit.client.1.smithi163.stdout:7/989: write d11/d24/d49/dc4/ffc [62282,15245] 0 2024-02-16T19:24:37.591 INFO:tasks.workunit.client.1.smithi163.stdout:7/990: write d11/f7e [3535161,56945] 0 2024-02-16T19:24:37.607 INFO:tasks.workunit.client.1.smithi163.stdout:7/991: dwrite d11/d30/f5b [4194304,4194304] 0 2024-02-16T19:24:37.626 INFO:tasks.workunit.client.1.smithi163.stdout:7/992: dwrite d11/d24/d49/dc4/ffc [0,4194304] 0 2024-02-16T19:24:37.627 INFO:tasks.workunit.client.1.smithi163.stdout:7/993: write d11/d24/d25/f9a [373273,26707] 0 2024-02-16T19:24:37.627 INFO:tasks.workunit.client.1.smithi163.stdout:7/994: chown d11/d50/d5d/d8b/d67/c8a 2648340 1 2024-02-16T19:24:37.635 INFO:tasks.workunit.client.1.smithi163.stdout:7/995: write d11/d50/d5d/d8b/d67/dd6/ffb [1320235,93173] 0 2024-02-16T19:24:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:37 smithi040 ceph-mon[29201]: pgmap v257: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 263 MiB/s rd, 323 MiB/s wr, 454 op/s 2024-02-16T19:24:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:37 smithi040 ceph-mon[29201]: mgrmap e20: smithi040.jzygzo(active, since 7m), standbys: smithi163.wjfclz 2024-02-16T19:24:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:37 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:37 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:37.644 INFO:tasks.workunit.client.1.smithi163.stdout:7/996: dread d11/d26/f2b [0,4194304] 0 2024-02-16T19:24:37.647 INFO:tasks.workunit.client.1.smithi163.stdout:7/997: dread d11/d24/d34/d8d/ffe [0,4194304] 0 2024-02-16T19:24:37.647 INFO:tasks.workunit.client.1.smithi163.stdout:7/998: write d11/d24/d25/d4b/d8c/f146 [918833,110543] 0 2024-02-16T19:24:37.647 INFO:tasks.workunit.client.1.smithi163.stdout:7/999: chown d11/d50/d5d/d8b 8163 1 2024-02-16T19:24:37.780 INFO:tasks.workunit.client.1.smithi163.stdout:4/911: sync 2024-02-16T19:24:37.781 INFO:tasks.workunit.client.1.smithi163.stdout:4/912: stat d11/d18/d1f/d29/cde 0 2024-02-16T19:24:37.781 INFO:tasks.workunit.client.1.smithi163.stdout:4/913: chown d11/d18/d1f/d29/lbb 125 1 2024-02-16T19:24:37.781 INFO:tasks.workunit.client.1.smithi163.stdout:4/914: creat d11/d3d/f134 x:0 0 0 2024-02-16T19:24:37.781 INFO:tasks.workunit.client.1.smithi163.stdout:4/915: chown d11/d18/d1f/d29/d52/d57/d6c/d9d/fe4 6108 1 2024-02-16T19:24:37.799 INFO:tasks.workunit.client.1.smithi163.stdout:4/916: dwrite d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/f109 [0,4194304] 0 2024-02-16T19:24:37.801 INFO:tasks.workunit.client.1.smithi163.stdout:4/917: getdents d11/d18/d1f/d29/d52/d76 0 2024-02-16T19:24:37.809 INFO:tasks.workunit.client.1.smithi163.stdout:4/918: dread d11/d18/d1f/f20 [0,4194304] 0 2024-02-16T19:24:37.834 INFO:tasks.workunit.client.1.smithi163.stdout:4/919: dwrite d11/d18/d1f/d29/d52/dd6/f126 [0,4194304] 0 2024-02-16T19:24:37.846 INFO:tasks.workunit.client.1.smithi163.stdout:4/920: dread f2 [0,4194304] 0 2024-02-16T19:24:37.855 INFO:tasks.workunit.client.1.smithi163.stdout:4/921: dread d11/d2d/d4f/f63 [0,4194304] 0 2024-02-16T19:24:37.855 INFO:tasks.workunit.client.1.smithi163.stdout:4/922: stat d11/f14 0 2024-02-16T19:24:37.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/923: symlink d11/d18/d1f/d29/d52/d57/dc5/l135 0 2024-02-16T19:24:37.857 INFO:tasks.workunit.client.1.smithi163.stdout:4/924: chown d11/d2d/d4f/cf9 41433865 1 2024-02-16T19:24:37.868 INFO:tasks.workunit.client.1.smithi163.stdout:4/925: dread d11/d2d/d4f/d5f/d70/d74/d75/dca/d8f/f109 [0,4194304] 0 2024-02-16T19:24:37.870 INFO:tasks.workunit.client.1.smithi163.stdout:4/926: mkdir d11/d2d/d4f/d60/d79/dc9/d136 0 2024-02-16T19:24:37.870 INFO:tasks.workunit.client.1.smithi163.stdout:4/927: chown d11/d18/d1f/d29/fa1 1932798768 1 2024-02-16T19:24:37.875 INFO:tasks.workunit.client.1.smithi163.stdout:4/928: getdents d11/d2d/d4f/d60/d79/dc9 0 2024-02-16T19:24:37.878 INFO:tasks.workunit.client.1.smithi163.stdout:4/929: mkdir d11/d2d/d4f/d60/d79/dc9/de9/d137 0 2024-02-16T19:24:37.879 INFO:tasks.workunit.client.1.smithi163.stdout:4/930: symlink d11/d2d/d4f/d5f/d70/d74/d75/dca/d130/l138 0 2024-02-16T19:24:37.899 INFO:tasks.workunit.client.1.smithi163.stdout:4/931: dwrite d11/d18/d1f/d29/d52/d57/d6c/dc3/fe6 [0,4194304] 0 2024-02-16T19:24:37.900 INFO:tasks.workunit.client.1.smithi163.stdout:4/932: write d11/d18/d1f/d29/d52/d57/d6c/d7c/f92 [4858602,13376] 0 2024-02-16T19:24:37.902 INFO:tasks.workunit.client.1.smithi163.stdout:4/933: creat d11/d18/d1f/d29/d52/d57/d6c/d9d/d113/f139 x:0 0 0 2024-02-16T19:24:37.907 INFO:tasks.workunit.client.1.smithi163.stdout:4/934: write d11/d18/d1f/d29/d52/d57/f73 [701801,36846] 0 2024-02-16T19:24:37.911 INFO:tasks.workunit.client.1.smithi163.stdout:4/935: rmdir d11/d18/d1f/d29/d52/d57/d6c/d9c 39 2024-02-16T19:24:37.916 INFO:tasks.workunit.client.1.smithi163.stdout:4/936: rename d11/d2d/d81 to d11/d12d/d13a 0 2024-02-16T19:24:37.917 INFO:tasks.workunit.client.1.smithi163.stdout:4/937: creat d11/d2d/d4f/d5f/d70/d74/d75/dca/dbf/f13b x:0 0 0 2024-02-16T19:24:37.919 INFO:tasks.workunit.client.1.smithi163.stdout:4/938: creat d11/d2d/d4f/dfd/f13c x:0 0 0 2024-02-16T19:24:37.941 INFO:tasks.workunit.client.1.smithi163.stdout:4/939: dwrite d11/d18/d1f/d29/f85 [0,4194304] 0 2024-02-16T19:24:37.960 INFO:tasks.workunit.client.1.smithi163.stdout:4/940: dwrite d11/d18/fea [0,4194304] 0 2024-02-16T19:24:37.962 INFO:tasks.workunit.client.1.smithi163.stdout:4/941: creat d11/d18/d1f/da9/f13d x:0 0 0 2024-02-16T19:24:37.962 INFO:tasks.workunit.client.1.smithi163.stdout:4/942: fsync d11/d2d/d4f/d5f/d70/d74/d75/ffb 0 2024-02-16T19:24:37.963 INFO:tasks.workunit.client.1.smithi163.stdout:4/943: creat d11/d2d/d4f/dfd/f13e x:0 0 0 2024-02-16T19:24:37.964 INFO:tasks.workunit.client.1.smithi163.stdout:4/944: mknod d11/d18/d1f/d29/d52/c13f 0 2024-02-16T19:24:37.964 INFO:tasks.workunit.client.1.smithi163.stdout:4/945: read - d11/d18/d1f/d29/d52/dd6/f127 zero size 2024-02-16T19:24:37.981 INFO:tasks.workunit.client.1.smithi163.stdout:4/946: dwrite d11/d18/d1f/d29/d52/d76/fb8 [0,4194304] 0 2024-02-16T19:24:37.985 INFO:tasks.workunit.client.1.smithi163.stdout:4/947: rename d11/f26 to d11/d18/d1f/d29/d52/d57/dc5/f140 0 2024-02-16T19:24:37.985 INFO:tasks.workunit.client.1.smithi163.stdout:4/948: fsync d11/d18/d1f/d29/d52/d57/f112 0 2024-02-16T19:24:37.987 INFO:tasks.workunit.client.1.smithi163.stdout:4/949: truncate d11/d2d/d4f/d60/d79/ff1 331543 0 2024-02-16T19:24:37.999 INFO:tasks.workunit.client.1.smithi163.stdout:4/950: dread d11/d2d/d4f/f64 [0,4194304] 0 2024-02-16T19:24:38.001 INFO:tasks.workunit.client.1.smithi163.stdout:4/951: rmdir d11/d2d/d4f/d60 39 2024-02-16T19:24:38.001 INFO:tasks.workunit.client.1.smithi163.stdout:4/952: readlink d11/d18/d66/l102 0 2024-02-16T19:24:38.002 INFO:tasks.workunit.client.1.smithi163.stdout:4/953: getdents d11/d2d/d4f/d5f/d70 0 2024-02-16T19:24:38.004 INFO:tasks.workunit.client.1.smithi163.stdout:4/954: rename d11/d12d to d11/d18/d141 0 2024-02-16T19:24:38.004 INFO:tasks.workunit.client.1.smithi163.stdout:4/955: dread - d11/d2d/d4f/d5f/d70/d74/d75/dca/fc1 zero size 2024-02-16T19:24:38.006 INFO:tasks.workunit.client.1.smithi163.stdout:4/956: rmdir d11/d18/d100/d12e 0 2024-02-16T19:24:38.007 INFO:tasks.workunit.client.1.smithi163.stdout:4/957: truncate d11/d18/d100/f106 125801 0 2024-02-16T19:24:38.024 INFO:tasks.workunit.client.1.smithi163.stdout:4/958: dread d11/d18/d1f/d29/d52/d57/d6c/d7c/f92 [0,4194304] 0 2024-02-16T19:24:38.024 INFO:tasks.workunit.client.1.smithi163.stdout:4/959: write d11/d18/d1f/d29/d52/d57/dc5/fe0 [758815,98520] 0 2024-02-16T19:24:38.047 INFO:tasks.workunit.client.1.smithi163.stdout:4/960: dwrite d11/d18/d1f/d29/d52/d57/d6c/dc3/fe6 [0,4194304] 0 2024-02-16T19:24:38.049 INFO:tasks.workunit.client.1.smithi163.stdout:4/961: mknod d11/d2d/c142 0 2024-02-16T19:24:38.050 INFO:tasks.workunit.client.1.smithi163.stdout:4/962: truncate d11/d2d/d4f/d60/d79/ff1 1037337 0 2024-02-16T19:24:38.052 INFO:tasks.workunit.client.1.smithi163.stdout:4/963: rename d11/d18/l25 to d11/d2d/d4f/d5f/d70/d74/d75/dca/d11b/d129/l143 0 2024-02-16T19:24:38.055 INFO:tasks.workunit.client.1.smithi163.stdout:4/964: rename d11/d2d/d4f/d60/d79 to d11/d18/d1f/d23/d144 0 2024-02-16T19:24:38.056 INFO:tasks.workunit.client.1.smithi163.stdout:4/965: mkdir d11/d18/d1f/d23/d144/dc9/d145 0 2024-02-16T19:24:38.056 INFO:tasks.workunit.client.1.smithi163.stdout:4/966: fdatasync d11/d3d/f128 0 2024-02-16T19:24:38.057 INFO:tasks.workunit.client.1.smithi163.stdout:4/967: read d11/d18/d1f/d29/d52/f80 [3052148,123174] 0 2024-02-16T19:24:38.076 INFO:tasks.workunit.client.1.smithi163.stdout:4/968: dwrite d11/d18/d1f/f38 [4194304,4194304] 0 2024-02-16T19:24:38.090 INFO:tasks.workunit.client.1.smithi163.stdout:4/969: dwrite d11/d2d/d4f/d5f/d70/d74/d75/dca/fc1 [0,4194304] 0 2024-02-16T19:24:38.090 INFO:tasks.workunit.client.1.smithi163.stdout:4/970: getdents d11/d18/d1f/d23/d144/dc9/de9/d137 0 2024-02-16T19:24:38.092 INFO:tasks.workunit.client.1.smithi163.stdout:4/971: truncate d11/d18/d1f/d29/d52/d57/f112 2055623 0 2024-02-16T19:24:38.093 INFO:tasks.workunit.client.1.smithi163.stdout:4/972: mkdir d11/d2d/d4f/d60/d111/d146 0 2024-02-16T19:24:38.093 INFO:tasks.workunit.client.1.smithi163.stdout:4/973: write d11/d3d/f128 [616328,111510] 0 2024-02-16T19:24:38.120 INFO:tasks.workunit.client.1.smithi163.stdout:4/974: dwrite d11/d18/d1f/d29/d52/dd6/f126 [0,4194304] 0 2024-02-16T19:24:38.235 INFO:tasks.workunit.client.1.smithi163.stdout:4/975: creat d11/d2d/d4f/d5f/d70/d74/d75/dca/d11b/f147 x:0 0 0 2024-02-16T19:24:38.265 INFO:tasks.workunit.client.1.smithi163.stdout:4/976: chown d11/d2d/l10a 20507 1 2024-02-16T19:24:38.265 INFO:tasks.workunit.client.1.smithi163.stdout:4/977: dread - d11/d18/d1f/d29/d52/d57/d6c/d7c/f12b zero size 2024-02-16T19:24:38.265 INFO:tasks.workunit.client.1.smithi163.stdout:4/978: fsync d11/d18/d1f/d34/de5/f11e 0 2024-02-16T19:24:38.277 INFO:tasks.workunit.client.1.smithi163.stdout:4/979: dread d11/d18/d1f/d29/f31 [4194304,4194304] 0 2024-02-16T19:24:38.301 INFO:tasks.workunit.client.1.smithi163.stdout:4/980: dwrite d11/d2d/d4f/d5f/d70/d74/d75/dca/fd8 [0,4194304] 0 2024-02-16T19:24:38.303 INFO:tasks.workunit.client.1.smithi163.stdout:4/981: truncate d11/d2d/d4f/d5f/d70/d74/d75/fec 76952 0 2024-02-16T19:24:38.313 INFO:tasks.workunit.client.1.smithi163.stdout:4/982: dread d11/d2d/d4f/d5f/faa [0,4194304] 0 2024-02-16T19:24:38.314 INFO:tasks.workunit.client.1.smithi163.stdout:4/983: mkdir d11/d2d/d4f/d148 0 2024-02-16T19:24:38.314 INFO:tasks.workunit.client.1.smithi163.stdout:4/984: write f2 [816917,118163] 0 2024-02-16T19:24:38.318 INFO:tasks.workunit.client.1.smithi163.stdout:4/985: dread d11/f91 [0,4194304] 0 2024-02-16T19:24:38.319 INFO:tasks.workunit.client.1.smithi163.stdout:4/986: fsync d11/d18/d1f/d29/d52/d76/fb8 0 2024-02-16T19:24:38.320 INFO:tasks.workunit.client.1.smithi163.stdout:4/987: rename d11/d2d/c35 to d11/d18/d1f/d29/d52/d57/d6c/d9c/c149 0 2024-02-16T19:24:38.328 INFO:tasks.workunit.client.1.smithi163.stdout:4/988: write d11/d18/d1f/d29/f49 [791893,47857] 0 2024-02-16T19:24:38.337 INFO:tasks.workunit.client.1.smithi163.stdout:4/989: dread d11/f30 [0,4194304] 0 2024-02-16T19:24:38.358 INFO:tasks.workunit.client.1.smithi163.stdout:4/990: dwrite d11/d2d/d4f/d5f/d70/d74/d75/dca/fa8 [0,4194304] 0 2024-02-16T19:24:38.359 INFO:tasks.workunit.client.1.smithi163.stdout:4/991: mknod d11/d18/c14a 0 2024-02-16T19:24:38.360 INFO:tasks.workunit.client.1.smithi163.stdout:4/992: symlink d11/d18/d141/d13a/l14b 0 2024-02-16T19:24:38.382 INFO:tasks.workunit.client.1.smithi163.stdout:4/993: dwrite d11/d2d/d4f/d60/d111/f120 [0,4194304] 0 2024-02-16T19:24:38.384 INFO:tasks.workunit.client.1.smithi163.stdout:4/994: mkdir d11/d18/d1f/d34/de5/d14c 0 2024-02-16T19:24:38.388 INFO:tasks.workunit.client.1.smithi163.stdout:4/995: dread d11/d18/d1f/f20 [0,4194304] 0 2024-02-16T19:24:38.401 INFO:tasks.workunit.client.1.smithi163.stdout:4/996: dwrite d11/d2d/d4f/d5f/d70/fc2 [4194304,4194304] 0 2024-02-16T19:24:38.401 INFO:tasks.workunit.client.1.smithi163.stdout:4/997: dread - d11/d2d/d4f/d5f/fff zero size 2024-02-16T19:24:38.402 INFO:tasks.workunit.client.1.smithi163.stdout:4/998: mknod d11/d18/d1f/d34/de5/c14d 0 2024-02-16T19:24:38.402 INFO:tasks.workunit.client.1.smithi163.stdout:4/999: dread - d11/d2d/d4f/d5f/d70/d74/f119 zero size 2024-02-16T19:24:38.405 INFO:tasks.workunit.client.1.smithi163.stderr:+ rm -rf -- ./tmp.R3cIKKTS6p 2024-02-16T19:24:39.581 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:39 smithi163 ceph-mon[36452]: pgmap v258: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 222 MiB/s rd, 259 MiB/s wr, 439 op/s 2024-02-16T19:24:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:39 smithi040 ceph-mon[29201]: pgmap v258: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 222 MiB/s rd, 259 MiB/s wr, 439 op/s 2024-02-16T19:24:40.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:40 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:40.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:40 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:41.553 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:41 smithi163 ceph-mon[36452]: pgmap v259: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 166 MiB/s rd, 199 MiB/s wr, 309 op/s 2024-02-16T19:24:41.598 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:41 smithi040 ceph-mon[29201]: pgmap v259: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 166 MiB/s rd, 199 MiB/s wr, 309 op/s 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr fail", "who": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "mgr fail", "who": "smithi040.jzygzo"}]': finished 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: mgrmap e21: smithi163.wjfclz(active, starting, since 0.00843412s) 2024-02-16T19:24:42.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mgr metadata", "who": "smithi163.wjfclz", "id": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:24:42.646 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: Manager daemon smithi163.wjfclz is now available 2024-02-16T19:24:42.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:42.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:42.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:42.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:42.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi163.wjfclz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:24:42.647 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi163.wjfclz/trash_purge_schedule"}]: dispatch 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr fail", "who": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: osdmap e47: 6 total, 6 up, 6 in 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.14212 172.21.15.40:0/3680641481' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "mgr fail", "who": "smithi040.jzygzo"}]': finished 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: mgrmap e21: smithi163.wjfclz(active, starting, since 0.00843412s) 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:24:42.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mgr metadata", "who": "smithi163.wjfclz", "id": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: Manager daemon smithi163.wjfclz is now available 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:42.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:42.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:24:42.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:42.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi163.wjfclz/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:24:42.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi163.wjfclz/trash_purge_schedule"}]: dispatch 2024-02-16T19:24:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:43 smithi040 ceph-mon[29201]: Deploying cephadm binary to smithi040 2024-02-16T19:24:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:43 smithi040 ceph-mon[29201]: Deploying cephadm binary to smithi163 2024-02-16T19:24:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:43 smithi040 ceph-mon[29201]: mgrmap e22: smithi163.wjfclz(active, since 1.00847s) 2024-02-16T19:24:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:43 smithi040 ceph-mon[29201]: pgmap v3: 65 pgs: 65 active+clean; 2.2 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail 2024-02-16T19:24:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:43 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:43.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:43 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:43 smithi163 ceph-mon[36452]: Deploying cephadm binary to smithi040 2024-02-16T19:24:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:43 smithi163 ceph-mon[36452]: Deploying cephadm binary to smithi163 2024-02-16T19:24:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:43 smithi163 ceph-mon[36452]: mgrmap e22: smithi163.wjfclz(active, since 1.00847s) 2024-02-16T19:24:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:43 smithi163 ceph-mon[36452]: pgmap v3: 65 pgs: 65 active+clean; 2.2 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail 2024-02-16T19:24:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:43 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:43.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:43 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:44.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:44 smithi040 ceph-mon[29201]: pgmap v4: 65 pgs: 65 active+clean; 2.2 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail 2024-02-16T19:24:44.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:44 smithi163 ceph-mon[36452]: pgmap v4: 65 pgs: 65 active+clean; 2.2 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail 2024-02-16T19:24:45.679 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-16T19:24:45.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-16T19:24:45.881 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:45 smithi040 ceph-mon[29201]: [16/Feb/2024:19:24:43] ENGINE Bus STARTING 2024-02-16T19:24:45.881 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:45 smithi040 ceph-mon[29201]: [16/Feb/2024:19:24:44] ENGINE Serving on http://172.21.15.163:8765 2024-02-16T19:24:45.881 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:45 smithi040 ceph-mon[29201]: [16/Feb/2024:19:24:44] ENGINE Serving on https://172.21.15.163:7150 2024-02-16T19:24:45.881 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:45 smithi040 ceph-mon[29201]: [16/Feb/2024:19:24:44] ENGINE Bus STARTED 2024-02-16T19:24:45.881 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:45 smithi040 ceph-mon[29201]: mgrmap e23: smithi163.wjfclz(active, since 3s) 2024-02-16T19:24:45.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:45 smithi163 ceph-mon[36452]: [16/Feb/2024:19:24:43] ENGINE Bus STARTING 2024-02-16T19:24:45.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:45 smithi163 ceph-mon[36452]: [16/Feb/2024:19:24:44] ENGINE Serving on http://172.21.15.163:8765 2024-02-16T19:24:45.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:45 smithi163 ceph-mon[36452]: [16/Feb/2024:19:24:44] ENGINE Serving on https://172.21.15.163:7150 2024-02-16T19:24:45.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:45 smithi163 ceph-mon[36452]: [16/Feb/2024:19:24:44] ENGINE Bus STARTED 2024-02-16T19:24:45.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:45 smithi163 ceph-mon[36452]: mgrmap e23: smithi163.wjfclz(active, since 3s) 2024-02-16T19:24:46.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:46 smithi040 ceph-mon[29201]: pgmap v5: 65 pgs: 65 active+clean; 2.2 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail 2024-02-16T19:24:46.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:46 smithi163 ceph-mon[36452]: pgmap v5: 65 pgs: 65 active+clean; 2.2 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail 2024-02-16T19:24:48.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:48 smithi040 ceph-mon[29201]: pgmap v6: 65 pgs: 65 active+clean; 1.6 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 39 KiB/s rd, 244 KiB/s wr, 80 op/s 2024-02-16T19:24:48.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:48 smithi163 ceph-mon[36452]: pgmap v6: 65 pgs: 65 active+clean; 1.6 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 39 KiB/s rd, 244 KiB/s wr, 80 op/s 2024-02-16T19:24:50.825 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-16T19:24:50.825 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-16T19:24:50.860 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:50 smithi040 ceph-mon[29201]: pgmap v7: 65 pgs: 65 active+clean; 949 MiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 57 KiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-02-16T19:24:50.860 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:50 smithi040 ceph-mon[29201]: Standby manager daemon smithi040.jzygzo started 2024-02-16T19:24:50.860 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:50 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.40:0/1264313420' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.jzygzo/crt"}]: dispatch 2024-02-16T19:24:50.860 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:50 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.40:0/1264313420' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:24:50.861 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:50 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.40:0/1264313420' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.jzygzo/key"}]: dispatch 2024-02-16T19:24:50.861 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:50 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.40:0/1264313420' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:24:50.889 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:50 smithi163 ceph-mon[36452]: pgmap v7: 65 pgs: 65 active+clean; 949 MiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 57 KiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-02-16T19:24:50.890 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:50 smithi163 ceph-mon[36452]: Standby manager daemon smithi040.jzygzo started 2024-02-16T19:24:50.890 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:50 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.40:0/1264313420' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.jzygzo/crt"}]: dispatch 2024-02-16T19:24:50.890 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:50 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.40:0/1264313420' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:24:50.890 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:50 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.40:0/1264313420' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.jzygzo/key"}]: dispatch 2024-02-16T19:24:50.890 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:50 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.40:0/1264313420' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:24:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:51 smithi163 ceph-mon[36452]: mgrmap e24: smithi163.wjfclz(active, since 9s), standbys: smithi040.jzygzo 2024-02-16T19:24:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:51 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:24:51.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:51 smithi040 ceph-mon[29201]: mgrmap e24: smithi163.wjfclz(active, since 9s), standbys: smithi040.jzygzo 2024-02-16T19:24:51.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:51 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:24:52.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:52 smithi040 ceph-mon[29201]: pgmap v8: 65 pgs: 65 active+clean; 949 MiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-02-16T19:24:52.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:52 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:52.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:52 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:52.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:52 smithi163 ceph-mon[36452]: pgmap v8: 65 pgs: 65 active+clean; 949 MiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-02-16T19:24:52.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:52 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:52.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:52 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:54.693 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:54 smithi163 ceph-mon[36452]: pgmap v9: 65 pgs: 65 active+clean; 313 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 58 KiB/s rd, 1.8 MiB/s wr, 308 op/s 2024-02-16T19:24:54.693 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:54 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:54.693 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:54 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:54.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:54 smithi040 ceph-mon[29201]: pgmap v9: 65 pgs: 65 active+clean; 313 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 58 KiB/s rd, 1.8 MiB/s wr, 308 op/s 2024-02-16T19:24:54.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:54 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:54.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:54 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:56.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:56 smithi163 ceph-mon[36452]: pgmap v10: 65 pgs: 65 active+clean; 313 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-02-16T19:24:56.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:56 smithi040 ceph-mon[29201]: pgmap v10: 65 pgs: 65 active+clean; 313 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 53 KiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-02-16T19:24:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:57 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:57.658 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:57 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:24:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:58 smithi040 ceph-mon[29201]: pgmap v11: 65 pgs: 65 active+clean; 314 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 78 KiB/s rd, 2.0 MiB/s wr, 398 op/s 2024-02-16T19:24:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:58 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:58 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:58 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:58 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:58 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:58 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:58.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:58 smithi163 ceph-mon[36452]: pgmap v11: 65 pgs: 65 active+clean; 314 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 78 KiB/s rd, 2.0 MiB/s wr, 398 op/s 2024-02-16T19:24:58.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:58 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:58.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:58 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:58.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:58 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:58.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:58 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:58.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:58 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:24:58.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:58 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:24:59.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:24:59 smithi040 ceph-mon[29201]: Adjusting osd_memory_target on smithi163 to 2581M 2024-02-16T19:24:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:24:59 smithi163 ceph-mon[36452]: Adjusting osd_memory_target on smithi163 to 2581M 2024-02-16T19:25:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:00 smithi040 ceph-mon[29201]: pgmap v12: 65 pgs: 65 active+clean; 320 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 61 KiB/s rd, 2.7 MiB/s wr, 494 op/s 2024-02-16T19:25:00.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:00 smithi163 ceph-mon[36452]: pgmap v12: 65 pgs: 65 active+clean; 320 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 61 KiB/s rd, 2.7 MiB/s wr, 494 op/s 2024-02-16T19:25:01.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:01 smithi040 ceph-mon[29201]: pgmap v13: 65 pgs: 65 active+clean; 320 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 409 op/s 2024-02-16T19:25:01.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:01 smithi163 ceph-mon[36452]: pgmap v13: 65 pgs: 65 active+clean; 320 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 409 op/s 2024-02-16T19:25:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:03 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:03 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:03 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:03 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:04.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: pgmap v14: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 55 KiB/s rd, 3.7 MiB/s wr, 532 op/s 2024-02-16T19:25:04.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:04.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:04.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:04.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:04.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:04.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:04.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:25:04.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:04 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:25:04.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: pgmap v14: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 55 KiB/s rd, 3.7 MiB/s wr, 532 op/s 2024-02-16T19:25:04.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:04.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:04.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:04.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:04.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-16T19:25:04.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:04.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:25:04.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:04 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:25:05.614 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Adjusting osd_memory_target on smithi040 to 1559M 2024-02-16T19:25:05.614 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:25:05.614 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:25:05.614 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:25:05.614 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:25:05.614 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:05 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.638 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Adjusting osd_memory_target on smithi040 to 1559M 2024-02-16T19:25:05.638 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:25:05.638 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:25:05.638 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:25:05.638 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:05 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:05.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.677+0000 7fda80d95700 1 -- 172.21.15.40:0/1413739598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 msgr2=0x7fda7c0fc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:05.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.677+0000 7fda80d95700 1 --2- 172.21.15.40:0/1413739598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 0x7fda7c0fc630 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda64004940 tx=0x7fda64009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:05.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.677+0000 7fda80d95700 1 -- 172.21.15.40:0/1413739598 shutdown_connections 2024-02-16T19:25:05.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.677+0000 7fda80d95700 1 --2- 172.21.15.40:0/1413739598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 0x7fda7c0fc630 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.677+0000 7fda80d95700 1 --2- 172.21.15.40:0/1413739598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda7c0fb870 0x7fda7c0fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.677+0000 7fda80d95700 1 -- 172.21.15.40:0/1413739598 >> 172.21.15.40:0/1413739598 conn(0x7fda7c0f74e0 msgr2=0x7fda7c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.677+0000 7fda80d95700 1 -- 172.21.15.40:0/1413739598 shutdown_connections 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.677+0000 7fda80d95700 1 -- 172.21.15.40:0/1413739598 wait complete. 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda80d95700 1 Processor -- start 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda80d95700 1 -- start start 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda80d95700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda7c0fb870 0x7fda7c10fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda80d95700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 0x7fda7c110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda80d95700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c110660 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda80d95700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c110f00 con 0x7fda7c0fb870 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda79d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 0x7fda7c110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda79d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 0x7fda7c110060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52726/0 (socket says 172.21.15.40:52726) 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda79d9b700 1 -- 172.21.15.40:0/3535228071 learned_addr learned my addr 172.21.15.40:0/3535228071 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda7a59c700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda7c0fb870 0x7fda7c10fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda79d9b700 1 -- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda7c0fb870 msgr2=0x7fda7c10fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda79d9b700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda7c0fb870 0x7fda7c10fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.678+0000 7fda79d9b700 1 -- 172.21.15.40:0/3535228071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64005040 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.679+0000 7fda79d9b700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 0x7fda7c110060 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fda64000c00 tx=0x7fda64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.679+0000 7fda72ffd700 1 -- 172.21.15.40:0/3535228071 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda64021070 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.679+0000 7fda72ffd700 1 -- 172.21.15.40:0/3535228071 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda6401c400 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.679+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c111180 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.679+0000 7fda72ffd700 1 -- 172.21.15.40:0/3535228071 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda6400e070 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.679+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c111660 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.680+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c04bed0 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.681+0000 7fda72ffd700 1 -- 172.21.15.40:0/3535228071 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7fda6401ca50 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.681+0000 7fda72ffd700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fda6806b2e0 0x7fda6806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:05.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.681+0000 7fda72ffd700 1 -- 172.21.15.40:0/3535228071 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fda6408b9f0 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.681+0000 7fda7a59c700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fda6806b2e0 0x7fda6806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:05.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.682+0000 7fda7a59c700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fda6806b2e0 0x7fda6806d790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009b80 tx=0x7fda6c00b040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:05.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.686+0000 7fda72ffd700 1 -- 172.21.15.40:0/3535228071 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fda640595f0 con 0x7fda7c0fc1c0 2024-02-16T19:25:05.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.972+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda7c111b80 con 0x7fda6806b2e0 2024-02-16T19:25:05.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.974+0000 7fda72ffd700 1 -- 172.21.15.40:0/3535228071 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fda7c111b80 con 0x7fda6806b2e0 2024-02-16T19:25:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fda6806b2e0 msgr2=0x7fda6806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fda6806b2e0 0x7fda6806d790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009b80 tx=0x7fda6c00b040 comp rx=0 tx=0).stop 2024-02-16T19:25:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 msgr2=0x7fda7c110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 0x7fda7c110060 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fda64000c00 tx=0x7fda64009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 shutdown_connections 2024-02-16T19:25:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fda6806b2e0 0x7fda6806d790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda7c0fb870 0x7fda7c10fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 --2- 172.21.15.40:0/3535228071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fc1c0 0x7fda7c110060 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 >> 172.21.15.40:0/3535228071 conn(0x7fda7c0f74e0 msgr2=0x7fda7c103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 shutdown_connections 2024-02-16T19:25:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:05.976+0000 7fda80d95700 1 -- 172.21.15.40:0/3535228071 wait complete. 2024-02-16T19:25:05.990 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:25:06.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.104+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/317315058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fcef0 msgr2=0x7f2ec00fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.104+0000 7f2ec7ea5700 1 --2- 172.21.15.40:0/317315058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fcef0 0x7f2ec00fd320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8004940 tx=0x7f2eb8009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.104+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/317315058 shutdown_connections 2024-02-16T19:25:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.104+0000 7f2ec7ea5700 1 --2- 172.21.15.40:0/317315058 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ec00fe0e0 0x7f2ec00fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.104+0000 7f2ec7ea5700 1 --2- 172.21.15.40:0/317315058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fcef0 0x7f2ec00fd320 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.104+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/317315058 >> 172.21.15.40:0/317315058 conn(0x7f2ec00f8620 msgr2=0x7f2ec00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.104+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/317315058 shutdown_connections 2024-02-16T19:25:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.104+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/317315058 wait complete. 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec7ea5700 1 Processor -- start 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec7ea5700 1 -- start start 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec7ea5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ec00fcef0 0x7f2ec010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec7ea5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fe0e0 0x7f2ec010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec7ea5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec010e4c0 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec7ea5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec010ed60 con 0x7f2ec00fcef0 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec5440700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fe0e0 0x7f2ec010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec5440700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fe0e0 0x7f2ec010dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52744/0 (socket says 172.21.15.40:52744) 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec5440700 1 -- 172.21.15.40:0/222895331 learned_addr learned my addr 172.21.15.40:0/222895331 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.105+0000 7f2ec5c41700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ec00fcef0 0x7f2ec010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ec5440700 1 -- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ec00fcef0 msgr2=0x7f2ec010d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ec5440700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ec00fcef0 0x7f2ec010d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ec5440700 1 -- 172.21.15.40:0/222895331 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb8005040 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ec5440700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fe0e0 0x7f2ec010dec0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4001d00 tx=0x7f2eb4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ebe7fc700 1 -- 172.21.15.40:0/222895331 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb4001e60 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ebe7fc700 1 -- 172.21.15.40:0/222895331 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eb400f040 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec010f040 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ebe7fc700 1 -- 172.21.15.40:0/222895331 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb4009310 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.106+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec010f580 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.107+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec004bed0 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.107+0000 7f2ebe7fc700 1 -- 172.21.15.40:0/222895331 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f2eb4009470 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.108+0000 7f2ebe7fc700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f2ea806b2e0 0x7f2ea806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.108+0000 7f2ebe7fc700 1 -- 172.21.15.40:0/222895331 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f2eb4089680 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.108+0000 7f2ec5c41700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f2ea806b2e0 0x7f2ea806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.108+0000 7f2ec5c41700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f2ea806b2e0 0x7f2ea806d790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8004890 tx=0x7f2eb8009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.115+0000 7f2ebe7fc700 1 -- 172.21.15.40:0/222895331 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2eb404e620 con 0x7f2ec00fe0e0 2024-02-16T19:25:06.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.465+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ec010f9e0 con 0x7f2ea806b2e0 2024-02-16T19:25:06.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.466+0000 7f2ebe7fc700 1 -- 172.21.15.40:0/222895331 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f2ec010f9e0 con 0x7f2ea806b2e0 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f2ea806b2e0 msgr2=0x7f2ea806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f2ea806b2e0 0x7f2ea806d790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8004890 tx=0x7f2eb8009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fe0e0 msgr2=0x7f2ec010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fe0e0 0x7f2ec010dec0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4001d00 tx=0x7f2eb4009f30 comp rx=0 tx=0).stop 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 shutdown_connections 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f2ea806b2e0 0x7f2ea806d790 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ec00fcef0 0x7f2ec010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 --2- 172.21.15.40:0/222895331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec00fe0e0 0x7f2ec010dec0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 >> 172.21.15.40:0/222895331 conn(0x7f2ec00f8620 msgr2=0x7f2ec01013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 shutdown_connections 2024-02-16T19:25:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.468+0000 7f2ec7ea5700 1 -- 172.21.15.40:0/222895331 wait complete. 2024-02-16T19:25:06.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.604+0000 7fad5c7ab700 1 -- 172.21.15.40:0/3875412858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 msgr2=0x7fad54070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:06.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.604+0000 7fad5c7ab700 1 --2- 172.21.15.40:0/3875412858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 0x7fad54070380 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fad50004940 tx=0x7fad50009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:06.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.604+0000 7fad5c7ab700 1 -- 172.21.15.40:0/3875412858 shutdown_connections 2024-02-16T19:25:06.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.604+0000 7fad5c7ab700 1 --2- 172.21.15.40:0/3875412858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad540708c0 0x7fad54070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.604+0000 7fad5c7ab700 1 --2- 172.21.15.40:0/3875412858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 0x7fad54070380 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.604+0000 7fad5c7ab700 1 -- 172.21.15.40:0/3875412858 >> 172.21.15.40:0/3875412858 conn(0x7fad540f8620 msgr2=0x7fad540faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:06.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.604+0000 7fad5c7ab700 1 -- 172.21.15.40:0/3875412858 shutdown_connections 2024-02-16T19:25:06.606 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:06 smithi040 ceph-mon[29201]: Reconfiguring node-exporter.smithi040 (dependencies changed)... 2024-02-16T19:25:06.606 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:06 smithi040 ceph-mon[29201]: Reconfiguring daemon node-exporter.smithi040 on smithi040 2024-02-16T19:25:06.607 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:06 smithi040 ceph-mon[29201]: pgmap v15: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 2.8 MiB/s wr, 368 op/s 2024-02-16T19:25:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.604+0000 7fad5c7ab700 1 -- 172.21.15.40:0/3875412858 wait complete. 2024-02-16T19:25:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad5c7ab700 1 Processor -- start 2024-02-16T19:25:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad5c7ab700 1 -- start start 2024-02-16T19:25:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad5c7ab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad540708c0 0x7fad5410fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad5c7ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 0x7fad54110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad5c7ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54110690 con 0x7fad54071f10 2024-02-16T19:25:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad5c7ab700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54110f30 con 0x7fad540708c0 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad59d46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 0x7fad54110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad59d46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 0x7fad54110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52758/0 (socket says 172.21.15.40:52758) 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad59d46700 1 -- 172.21.15.40:0/253788333 learned_addr learned my addr 172.21.15.40:0/253788333 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.605+0000 7fad5a547700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad540708c0 0x7fad5410fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad59d46700 1 -- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad540708c0 msgr2=0x7fad5410fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad59d46700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad540708c0 0x7fad5410fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad59d46700 1 -- 172.21.15.40:0/253788333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad50005040 con 0x7fad54071f10 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad59d46700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 0x7fad54110090 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fad4400eb40 tx=0x7fad4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad4affd700 1 -- 172.21.15.40:0/253788333 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad44007cc0 con 0x7fad54071f10 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad4affd700 1 -- 172.21.15.40:0/253788333 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad4400f040 con 0x7fad54071f10 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad541111b0 con 0x7fad54071f10 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad541116f0 con 0x7fad54071f10 2024-02-16T19:25:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.606+0000 7fad4affd700 1 -- 172.21.15.40:0/253788333 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad4400ca10 con 0x7fad54071f10 2024-02-16T19:25:06.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.608+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5404bed0 con 0x7fad54071f10 2024-02-16T19:25:06.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.608+0000 7fad4affd700 1 -- 172.21.15.40:0/253788333 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7fad4400cc80 con 0x7fad54071f10 2024-02-16T19:25:06.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.608+0000 7fad4affd700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fad4006b2e0 0x7fad4006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:06.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.608+0000 7fad4affd700 1 -- 172.21.15.40:0/253788333 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fad44089a00 con 0x7fad54071f10 2024-02-16T19:25:06.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.609+0000 7fad5a547700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fad4006b2e0 0x7fad4006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:06.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.609+0000 7fad5a547700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fad4006b2e0 0x7fad4006d790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fad5000b0d0 tx=0x7fad50009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:06.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.613+0000 7fad4affd700 1 -- 172.21.15.40:0/253788333 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fad4404f620 con 0x7fad54071f10 2024-02-16T19:25:06.646 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:06 smithi163 ceph-mon[36452]: Reconfiguring node-exporter.smithi040 (dependencies changed)... 2024-02-16T19:25:06.646 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:06 smithi163 ceph-mon[36452]: Reconfiguring daemon node-exporter.smithi040 on smithi040 2024-02-16T19:25:06.646 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:06 smithi163 ceph-mon[36452]: pgmap v15: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 2.8 MiB/s wr, 368 op/s 2024-02-16T19:25:06.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.881+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fad54111b50 con 0x7fad4006b2e0 2024-02-16T19:25:06.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.891+0000 7fad4affd700 1 -- 172.21.15.40:0/253788333 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fad54111b50 con 0x7fad4006b2e0 2024-02-16T19:25:06.892 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:25:06.892 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (7m) 4s ago 8m 25.1M - 0.25.0 c8568f914cd2 aadccae1ccec 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (8m) 4s ago 8m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (7m) 14s ago 7m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (7m) 4s ago 8m 83.1M - 9.4.7 954c08fa6188 8447b5580554 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (2m) 4s ago 2m 164M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (3m) 4s ago 3m 11.7M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (2m) 14s ago 2m 15.6M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (3m) 14s ago 3m 162M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:9283 running (10m) 4s ago 10m 411M - 17.2.7-513-gb88fae66 870a0944ccb8 4e9e9bb4389a 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283 running (42s) 14s ago 7m 488M - 19.0.0-1450-gf78a58c0 f081f847070a 294828ee3e60 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (10m) 4s ago 10m 49.1M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (7m) 14s ago 7m 38.7M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (8m) 4s ago 8m 14.3M - 1.5.0 0da6a335fe13 61d16f3f00d4 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (7m) 14s ago 7m 16.5M - 1.5.0 0da6a335fe13 c45ebf41f1b8 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (6m) 4s ago 6m 206M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:25:06.893 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (6m) 4s ago 6m 228M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:25:06.894 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (5m) 4s ago 5m 189M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:25:06.894 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (5m) 14s ago 5m 234M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:25:06.894 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (5m) 14s ago 5m 204M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:25:06.894 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (4m) 14s ago 4m 238M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:25:06.894 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (6m) 4s ago 8m 55.8M - 2.43.0 a07b618ecd1d 557bd885dbc5 2024-02-16T19:25:06.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fad4006b2e0 msgr2=0x7fad4006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:06.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fad4006b2e0 0x7fad4006d790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fad5000b0d0 tx=0x7fad50009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 msgr2=0x7fad54110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 0x7fad54110090 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fad4400eb40 tx=0x7fad4400b040 comp rx=0 tx=0).stop 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 shutdown_connections 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fad4006b2e0 0x7fad4006d790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad540708c0 0x7fad5410fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 --2- 172.21.15.40:0/253788333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad54071f10 0x7fad54110090 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 >> 172.21.15.40:0/253788333 conn(0x7fad540f8620 msgr2=0x7fad54102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 shutdown_connections 2024-02-16T19:25:06.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.894+0000 7fad5c7ab700 1 -- 172.21.15.40:0/253788333 wait complete. 2024-02-16T19:25:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.998+0000 7f63a41e5700 1 -- 172.21.15.40:0/3496096447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 msgr2=0x7f639c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.998+0000 7f63a41e5700 1 --2- 172.21.15.40:0/3496096447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 0x7f639c102780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f638c004940 tx=0x7f638c009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.998+0000 7f63a41e5700 1 -- 172.21.15.40:0/3496096447 shutdown_connections 2024-02-16T19:25:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.998+0000 7f63a41e5700 1 --2- 172.21.15.40:0/3496096447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 0x7f639c102780 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.998+0000 7f63a41e5700 1 --2- 172.21.15.40:0/3496096447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f639c0fda80 0x7f639c0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.998+0000 7f63a41e5700 1 -- 172.21.15.40:0/3496096447 >> 172.21.15.40:0/3496096447 conn(0x7f639c0f74e0 msgr2=0x7f639c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.999+0000 7f63a41e5700 1 -- 172.21.15.40:0/3496096447 shutdown_connections 2024-02-16T19:25:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.999+0000 7f63a41e5700 1 -- 172.21.15.40:0/3496096447 wait complete. 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.999+0000 7f63a41e5700 1 Processor -- start 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:06.999+0000 7f63a41e5700 1 -- start start 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a41e5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f639c0fda80 0x7f639c10d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a41e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 0x7f639c10de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a41e5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c10e410 con 0x7f639c1003a0 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a41e5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c10ecb0 con 0x7f639c0fda80 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1780700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 0x7f639c10de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1780700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 0x7f639c10de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52770/0 (socket says 172.21.15.40:52770) 2024-02-16T19:25:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1780700 1 -- 172.21.15.40:0/1608755727 learned_addr learned my addr 172.21.15.40:0/1608755727 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1f81700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f639c0fda80 0x7f639c10d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1780700 1 -- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f639c0fda80 msgr2=0x7f639c10d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1780700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f639c0fda80 0x7f639c10d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1780700 1 -- 172.21.15.40:0/1608755727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c005040 con 0x7f639c1003a0 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1f81700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f639c0fda80 0x7f639c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.000+0000 7f63a1780700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 0x7f639c10de10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f638c000c00 tx=0x7f638c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.001+0000 7f63927fc700 1 -- 172.21.15.40:0/1608755727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f638c021070 con 0x7f639c1003a0 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.001+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c10ef30 con 0x7f639c1003a0 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.001+0000 7f63927fc700 1 -- 172.21.15.40:0/1608755727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f638c01c400 con 0x7f639c1003a0 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.001+0000 7f63927fc700 1 -- 172.21.15.40:0/1608755727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f638c00e070 con 0x7f639c1003a0 2024-02-16T19:25:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.001+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c10f410 con 0x7f639c1003a0 2024-02-16T19:25:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.002+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c04bed0 con 0x7f639c1003a0 2024-02-16T19:25:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.002+0000 7f63927fc700 1 -- 172.21.15.40:0/1608755727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f638c01ca50 con 0x7f639c1003a0 2024-02-16T19:25:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.003+0000 7f63927fc700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f638806b2e0 0x7f638806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.003+0000 7f63927fc700 1 -- 172.21.15.40:0/1608755727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f638c08aef0 con 0x7f639c1003a0 2024-02-16T19:25:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.003+0000 7f63a1f81700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f638806b2e0 0x7f638806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:07.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.004+0000 7f63a1f81700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f638806b2e0 0x7f638806d790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6398004fd0 tx=0x7f639800b040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:07.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.008+0000 7f63927fc700 1 -- 172.21.15.40:0/1608755727 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f638c058af0 con 0x7f639c1003a0 2024-02-16T19:25:07.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.445+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f639c105a30 con 0x7f639c1003a0 2024-02-16T19:25:07.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.446+0000 7f63927fc700 1 -- 172.21.15.40:0/1608755727 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f638c058680 con 0x7f639c1003a0 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:25:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 13, 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:25:07.449 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:25:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f638806b2e0 msgr2=0x7f638806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f638806b2e0 0x7f638806d790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6398004fd0 tx=0x7f639800b040 comp rx=0 tx=0).stop 2024-02-16T19:25:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 msgr2=0x7f639c10de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 0x7f639c10de10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f638c000c00 tx=0x7f638c009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 shutdown_connections 2024-02-16T19:25:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f638806b2e0 0x7f638806d790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f639c0fda80 0x7f639c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 --2- 172.21.15.40:0/1608755727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639c1003a0 0x7f639c10de10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 >> 172.21.15.40:0/1608755727 conn(0x7f639c0f74e0 msgr2=0x7f639c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 shutdown_connections 2024-02-16T19:25:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.448+0000 7f63a41e5700 1 -- 172.21.15.40:0/1608755727 wait complete. 2024-02-16T19:25:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.560+0000 7feb19b94700 1 -- 172.21.15.40:0/3228335061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb140fda80 msgr2=0x7feb140ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.560+0000 7feb19b94700 1 --2- 172.21.15.40:0/3228335061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb140fda80 0x7feb140ffe60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feb04004940 tx=0x7feb04009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.560+0000 7feb19b94700 1 -- 172.21.15.40:0/3228335061 shutdown_connections 2024-02-16T19:25:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.560+0000 7feb19b94700 1 --2- 172.21.15.40:0/3228335061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb141003a0 0x7feb14102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.560+0000 7feb19b94700 1 --2- 172.21.15.40:0/3228335061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb140fda80 0x7feb140ffe60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.560+0000 7feb19b94700 1 -- 172.21.15.40:0/3228335061 >> 172.21.15.40:0/3228335061 conn(0x7feb140f7520 msgr2=0x7feb140f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.560+0000 7feb19b94700 1 -- 172.21.15.40:0/3228335061 shutdown_connections 2024-02-16T19:25:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.560+0000 7feb19b94700 1 -- 172.21.15.40:0/3228335061 wait complete. 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.561+0000 7feb19b94700 1 Processor -- start 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.561+0000 7feb19b94700 1 -- start start 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.561+0000 7feb19b94700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb140fda80 0x7feb14113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.561+0000 7feb19b94700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb141003a0 0x7feb141143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.561+0000 7feb19b94700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb141149d0 con 0x7feb141003a0 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.561+0000 7feb19b94700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb14115270 con 0x7feb140fda80 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.561+0000 7feb12ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb141003a0 0x7feb141143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb137fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb140fda80 0x7feb14113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb12ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb141003a0 0x7feb141143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41886/0 (socket says 172.21.15.40:41886) 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb12ffd700 1 -- 172.21.15.40:0/1355470201 learned_addr learned my addr 172.21.15.40:0/1355470201 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb12ffd700 1 -- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb140fda80 msgr2=0x7feb14113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb12ffd700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb140fda80 0x7feb14113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb12ffd700 1 -- 172.21.15.40:0/1355470201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04005040 con 0x7feb141003a0 2024-02-16T19:25:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb12ffd700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb141003a0 0x7feb141143d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feb08002a80 tx=0x7feb080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feafbfff700 1 -- 172.21.15.40:0/1355470201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb08007cc0 con 0x7feb141003a0 2024-02-16T19:25:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb14115550 con 0x7feb141003a0 2024-02-16T19:25:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feafbfff700 1 -- 172.21.15.40:0/1355470201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb0800f040 con 0x7feb141003a0 2024-02-16T19:25:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feafbfff700 1 -- 172.21.15.40:0/1355470201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0800dac0 con 0x7feb141003a0 2024-02-16T19:25:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.562+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb14115a90 con 0x7feb141003a0 2024-02-16T19:25:07.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.563+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1404bed0 con 0x7feb141003a0 2024-02-16T19:25:07.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.565+0000 7feafbfff700 1 -- 172.21.15.40:0/1355470201 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7feb0800dc20 con 0x7feb141003a0 2024-02-16T19:25:07.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.565+0000 7feafbfff700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7feafc06b2e0 0x7feafc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:07.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.565+0000 7feafbfff700 1 -- 172.21.15.40:0/1355470201 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7feb0808a020 con 0x7feb141003a0 2024-02-16T19:25:07.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.565+0000 7feb137fe700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7feafc06b2e0 0x7feafc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:07.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.566+0000 7feb137fe700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7feafc06b2e0 0x7feafc06d790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb0400b3b0 tx=0x7feb04009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:07.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.570+0000 7feafbfff700 1 -- 172.21.15.40:0/1355470201 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7feb08057ae0 con 0x7feb141003a0 2024-02-16T19:25:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:07 smithi040 ceph-mon[29201]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:07 smithi040 ceph-mon[29201]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:07 smithi163 ceph-mon[36452]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:07 smithi163 ceph-mon[36452]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:07.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.934+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feb140fc870 con 0x7feb141003a0 2024-02-16T19:25:07.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.934+0000 7feafbfff700 1 -- 172.21.15.40:0/1355470201 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1854 (secure 0 0 0) 0x7feb08057670 con 0x7feb141003a0 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:e19 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 19 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:24:09.512876+0000 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:25:07.936 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:25:07.937 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:25:07.938 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:25:07.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7feafc06b2e0 msgr2=0x7feafc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:07.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7feafc06b2e0 0x7feafc06d790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb0400b3b0 tx=0x7feb04009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:07.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb141003a0 msgr2=0x7feb141143d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:07.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb141003a0 0x7feb141143d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feb08002a80 tx=0x7feb080046b0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 shutdown_connections 2024-02-16T19:25:07.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7feafc06b2e0 0x7feafc06d790 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb140fda80 0x7feb14113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 --2- 172.21.15.40:0/1355470201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb141003a0 0x7feb141143d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:07.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 >> 172.21.15.40:0/1355470201 conn(0x7feb140f7520 msgr2=0x7feb140f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:07.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 shutdown_connections 2024-02-16T19:25:07.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:07.937+0000 7feb19b94700 1 -- 172.21.15.40:0/1355470201 wait complete. 2024-02-16T19:25:07.940 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 19 2024-02-16T19:25:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.043+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/1576004991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe01000c0 msgr2=0x7f7fe0102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.043+0000 7f7fe6c6f700 1 --2- 172.21.15.40:0/1576004991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe01000c0 0x7f7fe0102520 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc004940 tx=0x7f7fcc009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.043+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/1576004991 shutdown_connections 2024-02-16T19:25:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.043+0000 7f7fe6c6f700 1 --2- 172.21.15.40:0/1576004991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe01000c0 0x7f7fe0102520 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.043+0000 7f7fe6c6f700 1 --2- 172.21.15.40:0/1576004991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe0066c70 0x7f7fe00ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.043+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/1576004991 >> 172.21.15.40:0/1576004991 conn(0x7f7fe00f7520 msgr2=0x7f7fe00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.043+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/1576004991 shutdown_connections 2024-02-16T19:25:08.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.043+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/1576004991 wait complete. 2024-02-16T19:25:08.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe6c6f700 1 Processor -- start 2024-02-16T19:25:08.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe6c6f700 1 -- start start 2024-02-16T19:25:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe6c6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe0066c70 0x7f7fe00fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe6c6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe01000c0 0x7f7fe00fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe6c6f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00fbff0 con 0x7f7fe0066c70 2024-02-16T19:25:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe6c6f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00fc160 con 0x7f7fe01000c0 2024-02-16T19:25:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe4a0b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe0066c70 0x7f7fe00fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe4a0b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe0066c70 0x7f7fe00fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41896/0 (socket says 172.21.15.40:41896) 2024-02-16T19:25:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.044+0000 7f7fe4a0b700 1 -- 172.21.15.40:0/2160920865 learned_addr learned my addr 172.21.15.40:0/2160920865 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fdffff700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe01000c0 0x7f7fe00fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fe4a0b700 1 -- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe01000c0 msgr2=0x7f7fe00fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fe4a0b700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe01000c0 0x7f7fe00fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fe4a0b700 1 -- 172.21.15.40:0/2160920865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc005040 con 0x7f7fe0066c70 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fe4a0b700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe0066c70 0x7f7fe00fd570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400eb40 tx=0x7f7fd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fdd7fa700 1 -- 172.21.15.40:0/2160920865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd4007cc0 con 0x7f7fe0066c70 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe00fc3e0 con 0x7f7fe0066c70 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fdd7fa700 1 -- 172.21.15.40:0/2160920865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fd400f040 con 0x7f7fe0066c70 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fdd7fa700 1 -- 172.21.15.40:0/2160920865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd400ca10 con 0x7f7fe0066c70 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.045+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe0115900 con 0x7f7fe0066c70 2024-02-16T19:25:08.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.046+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe004bed0 con 0x7f7fe0066c70 2024-02-16T19:25:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.047+0000 7f7fdd7fa700 1 -- 172.21.15.40:0/2160920865 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f7fd400cc80 con 0x7f7fe0066c70 2024-02-16T19:25:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.047+0000 7f7fdd7fa700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f7fd006b2e0 0x7f7fd006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.047+0000 7f7fdd7fa700 1 -- 172.21.15.40:0/2160920865 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f7fd4089a00 con 0x7f7fe0066c70 2024-02-16T19:25:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.047+0000 7f7fdffff700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f7fd006b2e0 0x7f7fd006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.048+0000 7f7fdffff700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f7fd006b2e0 0x7f7fd006d790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc000c00 tx=0x7f7fcc009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:08.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.052+0000 7f7fdd7fa700 1 -- 172.21.15.40:0/2160920865 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f7fd404f620 con 0x7f7fe0066c70 2024-02-16T19:25:08.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.337+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fe0100d60 con 0x7f7fd006b2e0 2024-02-16T19:25:08.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.339+0000 7f7fdd7fa700 1 -- 172.21.15.40:0/2160920865 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f7fe0100d60 con 0x7f7fd006b2e0 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [], 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "1/21 daemons upgraded", 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:25:08.341 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:25:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.343+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f7fd006b2e0 msgr2=0x7f7fd006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.343+0000 7f7fe6c6f700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f7fd006b2e0 0x7f7fd006d790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc000c00 tx=0x7f7fcc009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:08.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.343+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe0066c70 msgr2=0x7f7fe00fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.343+0000 7f7fe6c6f700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe0066c70 0x7f7fe00fd570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400eb40 tx=0x7f7fd400b040 comp rx=0 tx=0).stop 2024-02-16T19:25:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.343+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 shutdown_connections 2024-02-16T19:25:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.343+0000 7f7fe6c6f700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f7fd006b2e0 0x7f7fd006d790 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.344+0000 7f7fe6c6f700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe0066c70 0x7f7fe00fd570 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.344+0000 7f7fe6c6f700 1 --2- 172.21.15.40:0/2160920865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe01000c0 0x7f7fe00fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.344+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 >> 172.21.15.40:0/2160920865 conn(0x7f7fe00f7520 msgr2=0x7f7fe00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.344+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 shutdown_connections 2024-02-16T19:25:08.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.344+0000 7f7fe6c6f700 1 -- 172.21.15.40:0/2160920865 wait complete. 2024-02-16T19:25:08.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.512+0000 7ff334e4f700 1 -- 172.21.15.40:0/3129811997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3300fda80 msgr2=0x7ff3300ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.512+0000 7ff334e4f700 1 --2- 172.21.15.40:0/3129811997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3300fda80 0x7ff3300ffe60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff320009ce0 tx=0x7ff320004d10 comp rx=0 tx=0).stop 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.512+0000 7ff334e4f700 1 -- 172.21.15.40:0/3129811997 shutdown_connections 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.512+0000 7ff334e4f700 1 --2- 172.21.15.40:0/3129811997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3301003a0 0x7ff330102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.512+0000 7ff334e4f700 1 --2- 172.21.15.40:0/3129811997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3300fda80 0x7ff3300ffe60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.512+0000 7ff334e4f700 1 -- 172.21.15.40:0/3129811997 >> 172.21.15.40:0/3129811997 conn(0x7ff3300f74e0 msgr2=0x7ff3300f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff334e4f700 1 -- 172.21.15.40:0/3129811997 shutdown_connections 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff334e4f700 1 -- 172.21.15.40:0/3129811997 wait complete. 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff334e4f700 1 Processor -- start 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff334e4f700 1 -- start start 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff334e4f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3300fda80 0x7ff33010dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff334e4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3301003a0 0x7ff33010dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff334e4f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33010e5f0 con 0x7ff3301003a0 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff334e4f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33010ee90 con 0x7ff3300fda80 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.513+0000 7ff32dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3301003a0 0x7ff33010dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3300fda80 0x7ff33010dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3300fda80 0x7ff33010dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.40:35814/0 (socket says 172.21.15.40:35814) 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32e59c700 1 -- 172.21.15.40:0/2916761242 learned_addr learned my addr 172.21.15.40:0/2916761242 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3301003a0 0x7ff33010dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41922/0 (socket says 172.21.15.40:41922) 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32e59c700 1 -- 172.21.15.40:0/2916761242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3301003a0 msgr2=0x7ff33010dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32e59c700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3301003a0 0x7ff33010dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32e59c700 1 -- 172.21.15.40:0/2916761242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff320009940 con 0x7ff3300fda80 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32dd9b700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3301003a0 0x7ff33010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff32e59c700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3300fda80 0x7ff33010dab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff320015040 tx=0x7ff320004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff326ffd700 1 -- 172.21.15.40:0/2916761242 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32000e040 con 0x7ff3300fda80 2024-02-16T19:25:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff326ffd700 1 -- 172.21.15.40:0/2916761242 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff320008e20 con 0x7ff3300fda80 2024-02-16T19:25:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33010f110 con 0x7ff3300fda80 2024-02-16T19:25:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff326ffd700 1 -- 172.21.15.40:0/2916761242 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32001c710 con 0x7ff3300fda80 2024-02-16T19:25:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.514+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33010f650 con 0x7ff3300fda80 2024-02-16T19:25:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.515+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3100052e0 con 0x7ff3300fda80 2024-02-16T19:25:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.516+0000 7ff326ffd700 1 -- 172.21.15.40:0/2916761242 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7ff320012b00 con 0x7ff3300fda80 2024-02-16T19:25:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.516+0000 7ff326ffd700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ff31c06b2e0 0x7ff31c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.516+0000 7ff326ffd700 1 -- 172.21.15.40:0/2916761242 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff32008ba60 con 0x7ff3300fda80 2024-02-16T19:25:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.516+0000 7ff32dd9b700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ff31c06b2e0 0x7ff31c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.517+0000 7ff32dd9b700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ff31c06b2e0 0x7ff31c06d790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff31800f7c0 tx=0x7ff318011040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.521+0000 7ff326ffd700 1 -- 172.21.15.40:0/2916761242 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff3200594e0 con 0x7ff3300fda80 2024-02-16T19:25:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:08 smithi040 ceph-mon[29201]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:08 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1608755727' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:25:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:08 smithi040 ceph-mon[29201]: pgmap v16: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 55 KiB/s rd, 3.3 MiB/s wr, 447 op/s 2024-02-16T19:25:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:08 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1355470201' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:25:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:08 smithi163 ceph-mon[36452]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:08 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1608755727' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:25:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:08 smithi163 ceph-mon[36452]: pgmap v16: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 55 KiB/s rd, 3.3 MiB/s wr, 447 op/s 2024-02-16T19:25:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:08 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1355470201' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:25:08.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.978+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff310005860 con 0x7ff3300fda80 2024-02-16T19:25:08.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.979+0000 7ff326ffd700 1 -- 172.21.15.40:0/2916761242 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff320004e30 con 0x7ff3300fda80 2024-02-16T19:25:08.980 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:25:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ff31c06b2e0 msgr2=0x7ff31c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ff31c06b2e0 0x7ff31c06d790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff31800f7c0 tx=0x7ff318011040 comp rx=0 tx=0).stop 2024-02-16T19:25:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3300fda80 msgr2=0x7ff33010dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3300fda80 0x7ff33010dab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff320015040 tx=0x7ff320004d10 comp rx=0 tx=0).stop 2024-02-16T19:25:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 shutdown_connections 2024-02-16T19:25:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ff31c06b2e0 0x7ff31c06d790 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3300fda80 0x7ff33010dab0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 --2- 172.21.15.40:0/2916761242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3301003a0 0x7ff33010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:08.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 >> 172.21.15.40:0/2916761242 conn(0x7ff3300f74e0 msgr2=0x7ff3300f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:08.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 shutdown_connections 2024-02-16T19:25:08.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:08.981+0000 7ff334e4f700 1 -- 172.21.15.40:0/2916761242 wait complete. 2024-02-16T19:25:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:09 smithi040 ceph-mon[29201]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:09 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/2916761242' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:25:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:09 smithi163 ceph-mon[36452]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:09 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/2916761242' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:25:10.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:10 smithi040 ceph-mon[29201]: pgmap v17: 65 pgs: 65 active+clean; 316 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 4.1 MiB/s wr, 419 op/s 2024-02-16T19:25:10.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:10 smithi163 ceph-mon[36452]: pgmap v17: 65 pgs: 65 active+clean; 316 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 4.1 MiB/s wr, 419 op/s 2024-02-16T19:25:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:12 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:12 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:12 smithi040 ceph-mon[29201]: Reconfiguring alertmanager.smithi040 (dependencies changed)... 2024-02-16T19:25:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:12 smithi040 ceph-mon[29201]: Reconfiguring daemon alertmanager.smithi040 on smithi040 2024-02-16T19:25:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:12 smithi040 ceph-mon[29201]: pgmap v18: 65 pgs: 65 active+clean; 316 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 3.3 MiB/s wr, 290 op/s 2024-02-16T19:25:12.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:12 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:12 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:12 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:12 smithi163 ceph-mon[36452]: Reconfiguring alertmanager.smithi040 (dependencies changed)... 2024-02-16T19:25:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:12 smithi163 ceph-mon[36452]: Reconfiguring daemon alertmanager.smithi040 on smithi040 2024-02-16T19:25:12.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:12 smithi163 ceph-mon[36452]: pgmap v18: 65 pgs: 65 active+clean; 316 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 3.3 MiB/s wr, 290 op/s 2024-02-16T19:25:12.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:12 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:13.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:13 smithi040 ceph-mon[29201]: pgmap v19: 65 pgs: 65 active+clean; 318 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 60 KiB/s rd, 4.4 MiB/s wr, 592 op/s 2024-02-16T19:25:13.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:13 smithi163 ceph-mon[36452]: pgmap v19: 65 pgs: 65 active+clean; 318 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 60 KiB/s rd, 4.4 MiB/s wr, 592 op/s 2024-02-16T19:25:16.552 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:16 smithi040 ceph-mon[29201]: pgmap v20: 65 pgs: 65 active+clean; 318 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.7 MiB/s wr, 469 op/s 2024-02-16T19:25:16.692 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:16 smithi163 ceph-mon[36452]: pgmap v20: 65 pgs: 65 active+clean; 318 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.7 MiB/s wr, 469 op/s 2024-02-16T19:25:17.847 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:17 smithi040 ceph-mon[29201]: pgmap v21: 65 pgs: 65 active+clean; 324 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 72 KiB/s rd, 4.0 MiB/s wr, 611 op/s 2024-02-16T19:25:17.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:17 smithi163 ceph-mon[36452]: pgmap v21: 65 pgs: 65 active+clean; 324 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 72 KiB/s rd, 4.0 MiB/s wr, 611 op/s 2024-02-16T19:25:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:19 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:19 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:19 smithi040 ceph-mon[29201]: Reconfiguring grafana.smithi040 (dependencies changed)... 2024-02-16T19:25:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:19 smithi040 ceph-mon[29201]: Reconfiguring daemon grafana.smithi040 on smithi040 2024-02-16T19:25:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:19 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:19 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:19 smithi163 ceph-mon[36452]: Reconfiguring grafana.smithi040 (dependencies changed)... 2024-02-16T19:25:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:19 smithi163 ceph-mon[36452]: Reconfiguring daemon grafana.smithi040 on smithi040 2024-02-16T19:25:20.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:20 smithi040 ceph-mon[29201]: pgmap v22: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 53 KiB/s rd, 3.7 MiB/s wr, 657 op/s 2024-02-16T19:25:20.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:20 smithi163 ceph-mon[36452]: pgmap v22: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 53 KiB/s rd, 3.7 MiB/s wr, 657 op/s 2024-02-16T19:25:21.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:21 smithi040 ceph-mon[29201]: pgmap v23: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 53 KiB/s rd, 2.5 MiB/s wr, 570 op/s 2024-02-16T19:25:21.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:21 smithi163 ceph-mon[36452]: pgmap v23: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 53 KiB/s rd, 2.5 MiB/s wr, 570 op/s 2024-02-16T19:25:23.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:23 smithi040 ceph-mon[29201]: pgmap v24: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 79 KiB/s rd, 3.7 MiB/s wr, 846 op/s 2024-02-16T19:25:23.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:23 smithi163 ceph-mon[36452]: pgmap v24: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 79 KiB/s rd, 3.7 MiB/s wr, 846 op/s 2024-02-16T19:25:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:26 smithi040 ceph-mon[29201]: pgmap v25: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 2.6 MiB/s wr, 544 op/s 2024-02-16T19:25:26.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:26 smithi163 ceph-mon[36452]: pgmap v25: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 2.6 MiB/s wr, 544 op/s 2024-02-16T19:25:27.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:27 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:27 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:28 smithi040 ceph-mon[29201]: pgmap v26: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 72 KiB/s rd, 2.9 MiB/s wr, 697 op/s 2024-02-16T19:25:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:28 smithi163 ceph-mon[36452]: pgmap v26: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 72 KiB/s rd, 2.9 MiB/s wr, 697 op/s 2024-02-16T19:25:29.080 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-16T19:25:29.081 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-16T19:25:29.514 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:29.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:29 smithi040 ceph-mon[29201]: pgmap v27: 65 pgs: 65 active+clean; 315 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 656 op/s 2024-02-16T19:25:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:29 smithi163 ceph-mon[36452]: pgmap v27: 65 pgs: 65 active+clean; 315 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 656 op/s 2024-02-16T19:25:31.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:31 smithi040 ceph-mon[29201]: pgmap v28: 65 pgs: 65 active+clean; 315 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.2 MiB/s wr, 530 op/s 2024-02-16T19:25:31.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:31 smithi163 ceph-mon[36452]: pgmap v28: 65 pgs: 65 active+clean; 315 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.2 MiB/s wr, 530 op/s 2024-02-16T19:25:33.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:33 smithi040 ceph-mon[29201]: pgmap v29: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 66 KiB/s rd, 3.0 MiB/s wr, 712 op/s 2024-02-16T19:25:33.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:33 smithi163 ceph-mon[36452]: pgmap v29: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 66 KiB/s rd, 3.0 MiB/s wr, 712 op/s 2024-02-16T19:25:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:36 smithi040 ceph-mon[29201]: pgmap v30: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 436 op/s 2024-02-16T19:25:36.690 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:36 smithi163 ceph-mon[36452]: pgmap v30: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 436 op/s 2024-02-16T19:25:37.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:37 smithi040 ceph-mon[29201]: pgmap v31: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 2.2 MiB/s wr, 532 op/s 2024-02-16T19:25:37.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:37 smithi163 ceph-mon[36452]: pgmap v31: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 2.2 MiB/s wr, 532 op/s 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.125+0000 7fa0e9420700 1 -- 172.21.15.40:0/4233614625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4071eb0 msgr2=0x7fa0e4070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.125+0000 7fa0e9420700 1 --2- 172.21.15.40:0/4233614625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4071eb0 0x7fa0e4070320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4004940 tx=0x7fa0d4009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.125+0000 7fa0e9420700 1 -- 172.21.15.40:0/4233614625 shutdown_connections 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.125+0000 7fa0e9420700 1 --2- 172.21.15.40:0/4233614625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e4070860 0x7fa0e4070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.125+0000 7fa0e9420700 1 --2- 172.21.15.40:0/4233614625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4071eb0 0x7fa0e4070320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.125+0000 7fa0e9420700 1 -- 172.21.15.40:0/4233614625 >> 172.21.15.40:0/4233614625 conn(0x7fa0e40f8620 msgr2=0x7fa0e40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.125+0000 7fa0e9420700 1 -- 172.21.15.40:0/4233614625 shutdown_connections 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.125+0000 7fa0e9420700 1 -- 172.21.15.40:0/4233614625 wait complete. 2024-02-16T19:25:39.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.126+0000 7fa0e9420700 1 Processor -- start 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.126+0000 7fa0e9420700 1 -- start start 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.126+0000 7fa0e9420700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4070860 0x7fa0e410fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.126+0000 7fa0e9420700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e4071eb0 0x7fa0e4110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.126+0000 7fa0e9420700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e4110680 con 0x7fa0e4070860 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.126+0000 7fa0e9420700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e4110f20 con 0x7fa0e4071eb0 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4070860 0x7fa0e410fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4070860 0x7fa0e410fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34420/0 (socket says 172.21.15.40:34420) 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e2ffd700 1 -- 172.21.15.40:0/2509478863 learned_addr learned my addr 172.21.15.40:0/2509478863 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e27fc700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e4071eb0 0x7fa0e4110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e2ffd700 1 -- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e4071eb0 msgr2=0x7fa0e4110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e2ffd700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e4071eb0 0x7fa0e4110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e2ffd700 1 -- 172.21.15.40:0/2509478863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d4005040 con 0x7fa0e4070860 2024-02-16T19:25:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e2ffd700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4070860 0x7fa0e410fb40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40093b0 tx=0x7fa0d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0cb7fe700 1 -- 172.21.15.40:0/2509478863 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0d400e040 con 0x7fa0e4070860 2024-02-16T19:25:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0cb7fe700 1 -- 172.21.15.40:0/2509478863 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0d400bcc0 con 0x7fa0e4070860 2024-02-16T19:25:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.127+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e41111a0 con 0x7fa0e4070860 2024-02-16T19:25:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.128+0000 7fa0cb7fe700 1 -- 172.21.15.40:0/2509478863 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0d401c630 con 0x7fa0e4070860 2024-02-16T19:25:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.128+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e41116e0 con 0x7fa0e4070860 2024-02-16T19:25:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.129+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e404bed0 con 0x7fa0e4070860 2024-02-16T19:25:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.130+0000 7fa0cb7fe700 1 -- 172.21.15.40:0/2509478863 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7fa0d4013980 con 0x7fa0e4070860 2024-02-16T19:25:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.130+0000 7fa0cb7fe700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fa0cc06b2e0 0x7fa0cc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.130+0000 7fa0cb7fe700 1 -- 172.21.15.40:0/2509478863 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa0d408b180 con 0x7fa0e4070860 2024-02-16T19:25:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.130+0000 7fa0e27fc700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fa0cc06b2e0 0x7fa0cc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.131+0000 7fa0e27fc700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fa0cc06b2e0 0x7fa0cc06d790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8004f40 tx=0x7fa0d800b040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.135+0000 7fa0cb7fe700 1 -- 172.21.15.40:0/2509478863 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa0d4058cc0 con 0x7fa0e4070860 2024-02-16T19:25:39.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.407+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0e4104550 con 0x7fa0cc06b2e0 2024-02-16T19:25:39.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.409+0000 7fa0cb7fe700 1 -- 172.21.15.40:0/2509478863 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fa0e4104550 con 0x7fa0cc06b2e0 2024-02-16T19:25:39.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fa0cc06b2e0 msgr2=0x7fa0cc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fa0cc06b2e0 0x7fa0cc06d790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8004f40 tx=0x7fa0d800b040 comp rx=0 tx=0).stop 2024-02-16T19:25:39.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4070860 msgr2=0x7fa0e410fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4070860 0x7fa0e410fb40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40093b0 tx=0x7fa0d4009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:39.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 shutdown_connections 2024-02-16T19:25:39.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fa0cc06b2e0 0x7fa0cc06d790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e4070860 0x7fa0e410fb40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 --2- 172.21.15.40:0/2509478863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e4071eb0 0x7fa0e4110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 >> 172.21.15.40:0/2509478863 conn(0x7fa0e40f8620 msgr2=0x7fa0e4102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 shutdown_connections 2024-02-16T19:25:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.412+0000 7fa0e9420700 1 -- 172.21.15.40:0/2509478863 wait complete. 2024-02-16T19:25:39.424 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:25:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.537+0000 7fb63fcf9700 1 -- 172.21.15.40:0/3431958888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 msgr2=0x7fb6380fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.537+0000 7fb63fcf9700 1 --2- 172.21.15.40:0/3431958888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 0x7fb6380fe700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb628004940 tx=0x7fb628009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.537+0000 7fb63fcf9700 1 -- 172.21.15.40:0/3431958888 shutdown_connections 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.537+0000 7fb63fcf9700 1 --2- 172.21.15.40:0/3431958888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 0x7fb6380fe700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.537+0000 7fb63fcf9700 1 --2- 172.21.15.40:0/3431958888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6380fb220 0x7fb6380fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.537+0000 7fb63fcf9700 1 -- 172.21.15.40:0/3431958888 >> 172.21.15.40:0/3431958888 conn(0x7fb6380f6ca0 msgr2=0x7fb6380f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.537+0000 7fb63fcf9700 1 -- 172.21.15.40:0/3431958888 shutdown_connections 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.537+0000 7fb63fcf9700 1 -- 172.21.15.40:0/3431958888 wait complete. 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63fcf9700 1 Processor -- start 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63fcf9700 1 -- start start 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63fcf9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6380fb220 0x7fb63810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63fcf9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 0x7fb63810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63fcf9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63810e490 con 0x7fb6380fe2b0 2024-02-16T19:25:39.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63fcf9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63810ed30 con 0x7fb6380fb220 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63d294700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 0x7fb63810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63d294700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 0x7fb63810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34436/0 (socket says 172.21.15.40:34436) 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63d294700 1 -- 172.21.15.40:0/1704492189 learned_addr learned my addr 172.21.15.40:0/1704492189 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63da95700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6380fb220 0x7fb63810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63d294700 1 -- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6380fb220 msgr2=0x7fb63810d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63d294700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6380fb220 0x7fb63810d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.538+0000 7fb63d294700 1 -- 172.21.15.40:0/1704492189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628005040 con 0x7fb6380fe2b0 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.539+0000 7fb63d294700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 0x7fb63810de90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb628009b30 tx=0x7fb628009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.539+0000 7fb62e7fc700 1 -- 172.21.15.40:0/1704492189 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb628021070 con 0x7fb6380fe2b0 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.539+0000 7fb62e7fc700 1 -- 172.21.15.40:0/1704492189 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb62801c400 con 0x7fb6380fe2b0 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.539+0000 7fb62e7fc700 1 -- 172.21.15.40:0/1704492189 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb62800e070 con 0x7fb6380fe2b0 2024-02-16T19:25:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.539+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63810efb0 con 0x7fb6380fe2b0 2024-02-16T19:25:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.539+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63810f490 con 0x7fb6380fe2b0 2024-02-16T19:25:39.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.540+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63804bed0 con 0x7fb6380fe2b0 2024-02-16T19:25:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.541+0000 7fb62e7fc700 1 -- 172.21.15.40:0/1704492189 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7fb62801ca00 con 0x7fb6380fe2b0 2024-02-16T19:25:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.541+0000 7fb62e7fc700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fb62406b2e0 0x7fb62406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.541+0000 7fb62e7fc700 1 -- 172.21.15.40:0/1704492189 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb62808af70 con 0x7fb6380fe2b0 2024-02-16T19:25:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.541+0000 7fb63da95700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fb62406b2e0 0x7fb62406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.542+0000 7fb63da95700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fb62406b2e0 0x7fb62406d790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb634009b80 tx=0x7fb63400b040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.546+0000 7fb62e7fc700 1 -- 172.21.15.40:0/1704492189 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb6280be620 con 0x7fb6380fe2b0 2024-02-16T19:25:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.813+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb63810f8e0 con 0x7fb62406b2e0 2024-02-16T19:25:39.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:39 smithi040 ceph-mon[29201]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:39.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:39 smithi040 ceph-mon[29201]: pgmap v32: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.9 MiB/s wr, 409 op/s 2024-02-16T19:25:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.814+0000 7fb62e7fc700 1 -- 172.21.15.40:0/1704492189 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fb63810f8e0 con 0x7fb62406b2e0 2024-02-16T19:25:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fb62406b2e0 msgr2=0x7fb62406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fb62406b2e0 0x7fb62406d790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb634009b80 tx=0x7fb63400b040 comp rx=0 tx=0).stop 2024-02-16T19:25:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 msgr2=0x7fb63810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 0x7fb63810de90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb628009b30 tx=0x7fb628009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 shutdown_connections 2024-02-16T19:25:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fb62406b2e0 0x7fb62406d790 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6380fb220 0x7fb63810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 --2- 172.21.15.40:0/1704492189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6380fe2b0 0x7fb63810de90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 >> 172.21.15.40:0/1704492189 conn(0x7fb6380f6ca0 msgr2=0x7fb6381014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 shutdown_connections 2024-02-16T19:25:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.816+0000 7fb63fcf9700 1 -- 172.21.15.40:0/1704492189 wait complete. 2024-02-16T19:25:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.943+0000 7f6e3627d700 1 -- 172.21.15.40:0/3267731661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 msgr2=0x7f6e3006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.943+0000 7f6e3627d700 1 --2- 172.21.15.40:0/3267731661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 0x7f6e3006ebf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009ce0 tx=0x7f6e28004d10 comp rx=0 tx=0).stop 2024-02-16T19:25:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.944+0000 7f6e3627d700 1 -- 172.21.15.40:0/3267731661 shutdown_connections 2024-02-16T19:25:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.944+0000 7f6e3627d700 1 --2- 172.21.15.40:0/3267731661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e3006f130 0x7f6e3010b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.945+0000 7f6e3627d700 1 --2- 172.21.15.40:0/3267731661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 0x7f6e3006ebf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.945+0000 7f6e3627d700 1 -- 172.21.15.40:0/3267731661 >> 172.21.15.40:0/3267731661 conn(0x7f6e3006a2c0 msgr2=0x7f6e3006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.945+0000 7f6e3627d700 1 -- 172.21.15.40:0/3267731661 shutdown_connections 2024-02-16T19:25:39.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.945+0000 7f6e3627d700 1 -- 172.21.15.40:0/3267731661 wait complete. 2024-02-16T19:25:39.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.946+0000 7f6e3627d700 1 Processor -- start 2024-02-16T19:25:39.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.946+0000 7f6e3627d700 1 -- start start 2024-02-16T19:25:39.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.947+0000 7f6e3627d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 0x7f6e301161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.947+0000 7f6e3627d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e3006f130 0x7f6e301166e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.947+0000 7f6e3627d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30116ce0 con 0x7f6e3006e7e0 2024-02-16T19:25:39.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.947+0000 7f6e3627d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30117580 con 0x7f6e3006f130 2024-02-16T19:25:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.949+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 0x7f6e301161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.949+0000 7f6e2f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e3006f130 0x7f6e301166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.949+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 0x7f6e301161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34454/0 (socket says 172.21.15.40:34454) 2024-02-16T19:25:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.949+0000 7f6e2ffff700 1 -- 172.21.15.40:0/49787616 learned_addr learned my addr 172.21.15.40:0/49787616 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.949+0000 7f6e2ffff700 1 -- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e3006f130 msgr2=0x7f6e301166e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.949+0000 7f6e2ffff700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e3006f130 0x7f6e301166e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.949+0000 7f6e2ffff700 1 -- 172.21.15.40:0/49787616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e28009940 con 0x7f6e3006e7e0 2024-02-16T19:25:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.949+0000 7f6e2ffff700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 0x7f6e301161a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009380 tx=0x7f6e28004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.950+0000 7f6e2cff9700 1 -- 172.21.15.40:0/49787616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e2800e040 con 0x7f6e3006e7e0 2024-02-16T19:25:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.950+0000 7f6e2cff9700 1 -- 172.21.15.40:0/49787616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e28004610 con 0x7f6e3006e7e0 2024-02-16T19:25:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.950+0000 7f6e2cff9700 1 -- 172.21.15.40:0/49787616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e28012b80 con 0x7f6e3006e7e0 2024-02-16T19:25:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.950+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e30117800 con 0x7f6e3006e7e0 2024-02-16T19:25:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.950+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30117d40 con 0x7f6e3006e7e0 2024-02-16T19:25:39.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.951+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e30110240 con 0x7f6e3006e7e0 2024-02-16T19:25:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.952+0000 7f6e2cff9700 1 -- 172.21.15.40:0/49787616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f6e2801c650 con 0x7f6e3006e7e0 2024-02-16T19:25:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.953+0000 7f6e2cff9700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f6e1806b400 0x7f6e1806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:39.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.953+0000 7f6e2cff9700 1 -- 172.21.15.40:0/49787616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6e2808c0a0 con 0x7f6e3006e7e0 2024-02-16T19:25:39.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.954+0000 7f6e2f7fe700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f6e1806b400 0x7f6e1806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:39.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.954+0000 7f6e2f7fe700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f6e1806b400 0x7f6e1806d8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009b80 tx=0x7f6e2400b040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:39.961+0000 7f6e2cff9700 1 -- 172.21.15.40:0/49787616 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f6e28059b60 con 0x7f6e3006e7e0 2024-02-16T19:25:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:39 smithi163 ceph-mon[36452]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:39 smithi163 ceph-mon[36452]: pgmap v32: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.9 MiB/s wr, 409 op/s 2024-02-16T19:25:40.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.200+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e301180e0 con 0x7f6e1806b400 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.211+0000 7f6e2cff9700 1 -- 172.21.15.40:0/49787616 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f6e301180e0 con 0x7f6e1806b400 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 starting - - - - 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (9m) 37s ago 9m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (8m) 47s ago 8m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (7m) 37s ago 8m 83.1M - 9.4.7 954c08fa6188 8447b5580554 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (3m) 37s ago 3m 164M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (3m) 37s ago 3m 11.7M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (3m) 47s ago 3m 15.6M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (3m) 47s ago 3m 162M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:9283 running (10m) 37s ago 10m 411M - 17.2.7-513-gb88fae66 870a0944ccb8 4e9e9bb4389a 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283 running (75s) 47s ago 8m 488M - 19.0.0-1450-gf78a58c0 f081f847070a 294828ee3e60 2024-02-16T19:25:40.212 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (10m) 37s ago 10m 49.1M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (8m) 47s ago 8m 38.7M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 starting - - - - 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (8m) 47s ago 8m 16.5M - 1.5.0 0da6a335fe13 c45ebf41f1b8 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (7m) 37s ago 7m 206M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (6m) 37s ago 6m 228M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (6m) 37s ago 6m 189M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (6m) 47s ago 6m 234M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (5m) 47s ago 5m 204M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (5m) 47s ago 5m 238M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:25:40.213 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (7m) 37s ago 8m 55.8M - 2.43.0 a07b618ecd1d 557bd885dbc5 2024-02-16T19:25:40.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f6e1806b400 msgr2=0x7f6e1806d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:40.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f6e1806b400 0x7f6e1806d8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009b80 tx=0x7f6e2400b040 comp rx=0 tx=0).stop 2024-02-16T19:25:40.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 msgr2=0x7f6e301161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:40.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 0x7f6e301161a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009380 tx=0x7f6e28004d10 comp rx=0 tx=0).stop 2024-02-16T19:25:40.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 shutdown_connections 2024-02-16T19:25:40.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f6e1806b400 0x7f6e1806d8b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e3006e7e0 0x7f6e301161a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 --2- 172.21.15.40:0/49787616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e3006f130 0x7f6e301166e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 >> 172.21.15.40:0/49787616 conn(0x7f6e3006a2c0 msgr2=0x7f6e30109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:40.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.213+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 shutdown_connections 2024-02-16T19:25:40.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.214+0000 7f6e3627d700 1 -- 172.21.15.40:0/49787616 wait complete. 2024-02-16T19:25:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.356+0000 7f9d56c60700 1 -- 172.21.15.40:0/889978253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 msgr2=0x7f9d500fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.356+0000 7f9d56c60700 1 --2- 172.21.15.40:0/889978253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 0x7f9d500fd320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9d44004940 tx=0x7f9d44009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.356+0000 7f9d56c60700 1 -- 172.21.15.40:0/889978253 shutdown_connections 2024-02-16T19:25:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.356+0000 7f9d56c60700 1 --2- 172.21.15.40:0/889978253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d500fe0e0 0x7f9d500fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.356+0000 7f9d56c60700 1 --2- 172.21.15.40:0/889978253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 0x7f9d500fd320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.356+0000 7f9d56c60700 1 -- 172.21.15.40:0/889978253 >> 172.21.15.40:0/889978253 conn(0x7f9d500f8620 msgr2=0x7f9d500faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:40.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.356+0000 7f9d56c60700 1 -- 172.21.15.40:0/889978253 shutdown_connections 2024-02-16T19:25:40.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.356+0000 7f9d56c60700 1 -- 172.21.15.40:0/889978253 wait complete. 2024-02-16T19:25:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d56c60700 1 Processor -- start 2024-02-16T19:25:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d56c60700 1 -- start start 2024-02-16T19:25:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d56c60700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 0x7f9d5010d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d56c60700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d500fe0e0 0x7f9d5010de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d56c60700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5010e420 con 0x7f9d500fcef0 2024-02-16T19:25:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d56c60700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5010ecc0 con 0x7f9d500fe0e0 2024-02-16T19:25:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d549fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 0x7f9d5010d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d549fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 0x7f9d5010d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34476/0 (socket says 172.21.15.40:34476) 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d549fc700 1 -- 172.21.15.40:0/725466858 learned_addr learned my addr 172.21.15.40:0/725466858 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.357+0000 7f9d4ffff700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d500fe0e0 0x7f9d5010de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d549fc700 1 -- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d500fe0e0 msgr2=0x7f9d5010de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d549fc700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d500fe0e0 0x7f9d5010de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d549fc700 1 -- 172.21.15.40:0/725466858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d44005040 con 0x7f9d500fcef0 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d549fc700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 0x7f9d5010d8e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000c00 tx=0x7f9d44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d4d7fa700 1 -- 172.21.15.40:0/725466858 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4400e070 con 0x7f9d500fcef0 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d4d7fa700 1 -- 172.21.15.40:0/725466858 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d4401c400 con 0x7f9d500fcef0 2024-02-16T19:25:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5010ef40 con 0x7f9d500fcef0 2024-02-16T19:25:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d4d7fa700 1 -- 172.21.15.40:0/725466858 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4400b670 con 0x7f9d500fcef0 2024-02-16T19:25:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.358+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5010f420 con 0x7f9d500fcef0 2024-02-16T19:25:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.359+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5004bed0 con 0x7f9d500fcef0 2024-02-16T19:25:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.360+0000 7f9d4d7fa700 1 -- 172.21.15.40:0/725466858 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f9d4401c9e0 con 0x7f9d500fcef0 2024-02-16T19:25:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.360+0000 7f9d4d7fa700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9d4006b2e0 0x7f9d4006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.360+0000 7f9d4d7fa700 1 -- 172.21.15.40:0/725466858 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9d4408b480 con 0x7f9d500fcef0 2024-02-16T19:25:40.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.360+0000 7f9d4ffff700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9d4006b2e0 0x7f9d4006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:40.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.361+0000 7f9d4ffff700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9d4006b2e0 0x7f9d4006d790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009b80 tx=0x7f9d3c00b040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:40.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.368+0000 7f9d4d7fa700 1 -- 172.21.15.40:0/725466858 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f9d44058fc0 con 0x7f9d500fcef0 2024-02-16T19:25:40.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.679+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d5010f640 con 0x7f9d500fcef0 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.680+0000 7f9d4d7fa700 1 -- 172.21.15.40:0/725466858 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f9d44058b50 con 0x7f9d500fcef0 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:25:40.681 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 13, 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:25:40.682 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:25:40.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9d4006b2e0 msgr2=0x7f9d4006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:40.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9d4006b2e0 0x7f9d4006d790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009b80 tx=0x7f9d3c00b040 comp rx=0 tx=0).stop 2024-02-16T19:25:40.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 msgr2=0x7f9d5010d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 0x7f9d5010d8e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000c00 tx=0x7f9d44009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 shutdown_connections 2024-02-16T19:25:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9d4006b2e0 0x7f9d4006d790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d500fcef0 0x7f9d5010d8e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 --2- 172.21.15.40:0/725466858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d500fe0e0 0x7f9d5010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 >> 172.21.15.40:0/725466858 conn(0x7f9d500f8620 msgr2=0x7f9d501013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 shutdown_connections 2024-02-16T19:25:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.682+0000 7f9d56c60700 1 -- 172.21.15.40:0/725466858 wait complete. 2024-02-16T19:25:40.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.825+0000 7fdb08b9e700 1 -- 172.21.15.40:0/791605056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb040fd410 msgr2=0x7fdb040fd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:40.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.825+0000 7fdb08b9e700 1 --2- 172.21.15.40:0/791605056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb040fd410 0x7fdb040fd820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4004940 tx=0x7fdaf4009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:40.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.826+0000 7fdb08b9e700 1 -- 172.21.15.40:0/791605056 shutdown_connections 2024-02-16T19:25:40.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.826+0000 7fdb08b9e700 1 --2- 172.21.15.40:0/791605056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb040fddf0 0x7fdb040fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.826+0000 7fdb08b9e700 1 --2- 172.21.15.40:0/791605056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb040fd410 0x7fdb040fd820 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.826+0000 7fdb08b9e700 1 -- 172.21.15.40:0/791605056 >> 172.21.15.40:0/791605056 conn(0x7fdb040f6c80 msgr2=0x7fdb040f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:40.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.826+0000 7fdb08b9e700 1 -- 172.21.15.40:0/791605056 shutdown_connections 2024-02-16T19:25:40.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.826+0000 7fdb08b9e700 1 -- 172.21.15.40:0/791605056 wait complete. 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.826+0000 7fdb08b9e700 1 Processor -- start 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.826+0000 7fdb08b9e700 1 -- start start 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb08b9e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb040fd410 0x7fdb0410fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb08b9e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb04110080 0x7fdb04110f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb08b9e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04110540 con 0x7fdb04110080 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb08b9e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04110680 con 0x7fdb040fd410 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb01d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb04110080 0x7fdb04110f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb01d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb04110080 0x7fdb04110f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34484/0 (socket says 172.21.15.40:34484) 2024-02-16T19:25:40.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb01d9b700 1 -- 172.21.15.40:0/4034878424 learned_addr learned my addr 172.21.15.40:0/4034878424 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb0259c700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb040fd410 0x7fdb0410fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb01d9b700 1 -- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb040fd410 msgr2=0x7fdb0410fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb01d9b700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb040fd410 0x7fdb0410fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb01d9b700 1 -- 172.21.15.40:0/4034878424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf4005040 con 0x7fdb04110080 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb0259c700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb040fd410 0x7fdb0410fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.827+0000 7fdb01d9b700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb04110080 0x7fdb04110f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800eb40 tx=0x7fdaf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.828+0000 7fdaf2ffd700 1 -- 172.21.15.40:0/4034878424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf8007cc0 con 0x7fdb04110080 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.828+0000 7fdaf2ffd700 1 -- 172.21.15.40:0/4034878424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaf800f040 con 0x7fdb04110080 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.828+0000 7fdaf2ffd700 1 -- 172.21.15.40:0/4034878424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf800c9d0 con 0x7fdb04110080 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.828+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb04111520 con 0x7fdb04110080 2024-02-16T19:25:40.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.828+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb04111a30 con 0x7fdb04110080 2024-02-16T19:25:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.829+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0404bed0 con 0x7fdb04110080 2024-02-16T19:25:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.830+0000 7fdaf2ffd700 1 -- 172.21.15.40:0/4034878424 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7fdaf800cc40 con 0x7fdb04110080 2024-02-16T19:25:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.830+0000 7fdaf2ffd700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fdaec06b2e0 0x7fdaec06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:40.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.830+0000 7fdaf2ffd700 1 -- 172.21.15.40:0/4034878424 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fdaf8089840 con 0x7fdb04110080 2024-02-16T19:25:40.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.830+0000 7fdb0259c700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fdaec06b2e0 0x7fdaec06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:40.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.831+0000 7fdb0259c700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fdaec06b2e0 0x7fdaec06d790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009b30 tx=0x7fdaf4009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:40.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:40.835+0000 7fdaf2ffd700 1 -- 172.21.15.40:0/4034878424 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fdaf804f620 con 0x7fdb04110080 2024-02-16T19:25:41.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.181+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdb04107c40 con 0x7fdb04110080 2024-02-16T19:25:41.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.183+0000 7fdaf2ffd700 1 -- 172.21.15.40:0/4034878424 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1854 (secure 0 0 0) 0x7fdaf80573b0 con 0x7fdb04110080 2024-02-16T19:25:41.184 INFO:teuthology.orchestra.run.smithi040.stdout:e19 2024-02-16T19:25:41.184 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:25:41.184 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:25:41.184 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:25:41.184 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 19 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:24:09.512876+0000 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:25:41.185 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:25:41.186 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:25:41.187 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:25:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.187+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fdaec06b2e0 msgr2=0x7fdaec06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.187+0000 7fdb08b9e700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fdaec06b2e0 0x7fdaec06d790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009b30 tx=0x7fdaf4009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:41.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.187+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb04110080 msgr2=0x7fdb04110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.187+0000 7fdb08b9e700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb04110080 0x7fdb04110f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800eb40 tx=0x7fdaf800b040 comp rx=0 tx=0).stop 2024-02-16T19:25:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.188+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 shutdown_connections 2024-02-16T19:25:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.188+0000 7fdb08b9e700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fdaec06b2e0 0x7fdaec06d790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.188+0000 7fdb08b9e700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb040fd410 0x7fdb0410fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.188+0000 7fdb08b9e700 1 --2- 172.21.15.40:0/4034878424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb04110080 0x7fdb04110f20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.188+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 >> 172.21.15.40:0/4034878424 conn(0x7fdb040f6c80 msgr2=0x7fdb041046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.189+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 shutdown_connections 2024-02-16T19:25:41.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.190+0000 7fdb08b9e700 1 -- 172.21.15.40:0/4034878424 wait complete. 2024-02-16T19:25:41.193 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 19 2024-02-16T19:25:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.354+0000 7fe54a738700 1 -- 172.21.15.40:0/1832115048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 msgr2=0x7fe5440fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.354+0000 7fe54a738700 1 --2- 172.21.15.40:0/1832115048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 0x7fe5440fc250 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe534004940 tx=0x7fe534009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.354+0000 7fe54a738700 1 -- 172.21.15.40:0/1832115048 shutdown_connections 2024-02-16T19:25:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.354+0000 7fe54a738700 1 --2- 172.21.15.40:0/1832115048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5440fcfb0 0x7fe5440fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.354+0000 7fe54a738700 1 --2- 172.21.15.40:0/1832115048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 0x7fe5440fc250 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.354+0000 7fe54a738700 1 -- 172.21.15.40:0/1832115048 >> 172.21.15.40:0/1832115048 conn(0x7fe5440f7520 msgr2=0x7fe5440f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.354+0000 7fe54a738700 1 -- 172.21.15.40:0/1832115048 shutdown_connections 2024-02-16T19:25:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.354+0000 7fe54a738700 1 -- 172.21.15.40:0/1832115048 wait complete. 2024-02-16T19:25:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe54a738700 1 Processor -- start 2024-02-16T19:25:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe54a738700 1 -- start start 2024-02-16T19:25:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe54a738700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 0x7fe54410c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe54a738700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5440fcfb0 0x7fe54410cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe54a738700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54410d380 con 0x7fe5440fbe40 2024-02-16T19:25:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe54a738700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54410dc20 con 0x7fe5440fcfb0 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe543fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 0x7fe54410c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe543fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 0x7fe54410c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34506/0 (socket says 172.21.15.40:34506) 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe543fff700 1 -- 172.21.15.40:0/830654084 learned_addr learned my addr 172.21.15.40:0/830654084 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.355+0000 7fe5437fe700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5440fcfb0 0x7fe54410cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe543fff700 1 -- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5440fcfb0 msgr2=0x7fe54410cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe543fff700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5440fcfb0 0x7fe54410cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe543fff700 1 -- 172.21.15.40:0/830654084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe534005040 con 0x7fe5440fbe40 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe543fff700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 0x7fe54410c840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe534007df0 tx=0x7fe534009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe540ff9700 1 -- 172.21.15.40:0/830654084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe53400e040 con 0x7fe5440fbe40 2024-02-16T19:25:41.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe540ff9700 1 -- 172.21.15.40:0/830654084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe53400be00 con 0x7fe5440fbe40 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54410dea0 con 0x7fe5440fbe40 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe540ff9700 1 -- 172.21.15.40:0/830654084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe53401a700 con 0x7fe5440fbe40 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.356+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54410e3e0 con 0x7fe5440fbe40 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.357+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54404bed0 con 0x7fe5440fbe40 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.357+0000 7fe540ff9700 1 -- 172.21.15.40:0/830654084 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7fe534020020 con 0x7fe5440fbe40 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.358+0000 7fe540ff9700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fe53006b2e0 0x7fe53006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.358+0000 7fe540ff9700 1 -- 172.21.15.40:0/830654084 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe53408cc40 con 0x7fe5440fbe40 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.358+0000 7fe5437fe700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fe53006b2e0 0x7fe53006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.359+0000 7fe5437fe700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fe53006b2e0 0x7fe53006d790 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe52c004f40 tx=0x7fe52c00b040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:41.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.362+0000 7fe540ff9700 1 -- 172.21.15.40:0/830654084 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe534092020 con 0x7fe5440fbe40 2024-02-16T19:25:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.597+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe54410ea50 con 0x7fe53006b2e0 2024-02-16T19:25:41.599 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:41 smithi040 ceph-mon[29201]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:41.599 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:41 smithi040 ceph-mon[29201]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:41.599 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:41 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/725466858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:25:41.599 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:41 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/4034878424' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:25:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.599+0000 7fe540ff9700 1 -- 172.21.15.40:0/830654084 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fe54410ea50 con 0x7fe53006b2e0 2024-02-16T19:25:41.600 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:25:41.600 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:25:41.600 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:25:41.600 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:25:41.600 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [], 2024-02-16T19:25:41.601 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "1/21 daemons upgraded", 2024-02-16T19:25:41.601 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-02-16T19:25:41.601 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:25:41.601 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:25:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fe53006b2e0 msgr2=0x7fe53006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fe53006b2e0 0x7fe53006d790 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe52c004f40 tx=0x7fe52c00b040 comp rx=0 tx=0).stop 2024-02-16T19:25:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 msgr2=0x7fe54410c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 0x7fe54410c840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe534007df0 tx=0x7fe534009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 shutdown_connections 2024-02-16T19:25:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7fe53006b2e0 0x7fe53006d790 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5440fbe40 0x7fe54410c840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 --2- 172.21.15.40:0/830654084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5440fcfb0 0x7fe54410cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 >> 172.21.15.40:0/830654084 conn(0x7fe5440f7520 msgr2=0x7fe544100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 shutdown_connections 2024-02-16T19:25:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.601+0000 7fe54a738700 1 -- 172.21.15.40:0/830654084 wait complete. 2024-02-16T19:25:41.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.715+0000 7f4035aff700 1 -- 172.21.15.40:0/441041737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 msgr2=0x7f40300fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.715+0000 7f4035aff700 1 --2- 172.21.15.40:0/441041737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 0x7f40300fd430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4018004940 tx=0x7f4018009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:41.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.715+0000 7f4035aff700 1 -- 172.21.15.40:0/441041737 shutdown_connections 2024-02-16T19:25:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.715+0000 7f4035aff700 1 --2- 172.21.15.40:0/441041737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 0x7f40300fd430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.715+0000 7f4035aff700 1 --2- 172.21.15.40:0/441041737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40300fbe50 0x7f40300fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.715+0000 7f4035aff700 1 -- 172.21.15.40:0/441041737 >> 172.21.15.40:0/441041737 conn(0x7f40300f7510 msgr2=0x7f40300f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.715+0000 7f4035aff700 1 -- 172.21.15.40:0/441041737 shutdown_connections 2024-02-16T19:25:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.716+0000 7f4035aff700 1 -- 172.21.15.40:0/441041737 wait complete. 2024-02-16T19:25:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.716+0000 7f4035aff700 1 Processor -- start 2024-02-16T19:25:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.716+0000 7f4035aff700 1 -- start start 2024-02-16T19:25:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f4035aff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40300fbe50 0x7f40301105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f4035aff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 0x7f4030112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f4035aff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4030066d10 con 0x7f40300fcfc0 2024-02-16T19:25:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f4035aff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4030066e50 con 0x7f40300fbe50 2024-02-16T19:25:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f4026ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 0x7f4030112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f4026ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 0x7f4030112b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34528/0 (socket says 172.21.15.40:34528) 2024-02-16T19:25:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40300fbe50 0x7f40301105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f4026ffd700 1 -- 172.21.15.40:0/4266822835 learned_addr learned my addr 172.21.15.40:0/4266822835 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f402f7fe700 1 -- 172.21.15.40:0/4266822835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 msgr2=0x7f4030112b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f402f7fe700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 0x7f4030112b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f402f7fe700 1 -- 172.21.15.40:0/4266822835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018005040 con 0x7f40300fbe50 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f4026ffd700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 0x7f4030112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.717+0000 7f402f7fe700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40300fbe50 0x7f40301105b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4020009fd0 tx=0x7f402000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.718+0000 7f402cff9700 1 -- 172.21.15.40:0/4266822835 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4020004e90 con 0x7f40300fbe50 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.718+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4030113100 con 0x7f40300fbe50 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.718+0000 7f402cff9700 1 -- 172.21.15.40:0/4266822835 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f402000f040 con 0x7f40300fbe50 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.718+0000 7f402cff9700 1 -- 172.21.15.40:0/4266822835 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f402000ca50 con 0x7f40300fbe50 2024-02-16T19:25:41.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.718+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40301135f0 con 0x7f40300fbe50 2024-02-16T19:25:41.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.719+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403004bed0 con 0x7f40300fbe50 2024-02-16T19:25:41.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.719+0000 7f402cff9700 1 -- 172.21.15.40:0/4266822835 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f4020007cc0 con 0x7f40300fbe50 2024-02-16T19:25:41.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.720+0000 7f402cff9700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f401c06b2e0 0x7f401c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:25:41.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.720+0000 7f402cff9700 1 -- 172.21.15.40:0/4266822835 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f4020089880 con 0x7f40300fbe50 2024-02-16T19:25:41.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.720+0000 7f4026ffd700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f401c06b2e0 0x7f401c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:25:41.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.720+0000 7f4026ffd700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f401c06b2e0 0x7f401c06d790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f401800b0d0 tx=0x7f4018009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:25:41.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:41.724+0000 7f402cff9700 1 -- 172.21.15.40:0/4266822835 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f402004f620 con 0x7f40300fbe50 2024-02-16T19:25:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:41 smithi163 ceph-mon[36452]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:41 smithi163 ceph-mon[36452]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:41 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/725466858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:25:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:41 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/4034878424' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:25:41.867 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-16T19:25:41.867 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-16T19:25:42.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.123+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4030113ac0 con 0x7f40300fbe50 2024-02-16T19:25:42.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.124+0000 7f402cff9700 1 -- 172.21.15.40:0/4266822835 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4020053880 con 0x7f40300fbe50 2024-02-16T19:25:42.125 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f401c06b2e0 msgr2=0x7f401c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f401c06b2e0 0x7f401c06d790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f401800b0d0 tx=0x7f4018009e30 comp rx=0 tx=0).stop 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40300fbe50 msgr2=0x7f40301105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40300fbe50 0x7f40301105b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4020009fd0 tx=0x7f402000b040 comp rx=0 tx=0).stop 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 shutdown_connections 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f401c06b2e0 0x7f401c06d790 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40300fbe50 0x7f40301105b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 --2- 172.21.15.40:0/4266822835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40300fcfc0 0x7f4030112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 >> 172.21.15.40:0/4266822835 conn(0x7f40300f7510 msgr2=0x7f4030100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:25:42.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 shutdown_connections 2024-02-16T19:25:42.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:25:42.126+0000 7f4035aff700 1 -- 172.21.15.40:0/4266822835 wait complete. 2024-02-16T19:25:42.252 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:42.253 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-16T19:25:42.287 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-16T19:25:42.287 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-16T19:25:42.326 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-16T19:25:42.326 DEBUG:teuthology.parallel:result is None 2024-02-16T19:25:42.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:42 smithi040 ceph-mon[29201]: pgmap v33: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 308 op/s 2024-02-16T19:25:42.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:42 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:42.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:42 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/4266822835' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:25:42.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:42 smithi163 ceph-mon[36452]: pgmap v33: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 308 op/s 2024-02-16T19:25:42.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:42 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:42.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:42 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/4266822835' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:25:43.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:43 smithi040 ceph-mon[29201]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:43.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:43 smithi163 ceph-mon[36452]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:25:44.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:44 smithi040 ceph-mon[29201]: pgmap v34: 65 pgs: 65 active+clean; 308 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 430 op/s 2024-02-16T19:25:44.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:44 smithi163 ceph-mon[36452]: pgmap v34: 65 pgs: 65 active+clean; 308 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 430 op/s 2024-02-16T19:25:46.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:46 smithi040 ceph-mon[29201]: pgmap v35: 65 pgs: 65 active+clean; 308 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 248 op/s 2024-02-16T19:25:46.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:46 smithi163 ceph-mon[36452]: pgmap v35: 65 pgs: 65 active+clean; 308 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 248 op/s 2024-02-16T19:25:47.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:47 smithi040 ceph-mon[29201]: pgmap v36: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.0 MiB/s wr, 252 op/s 2024-02-16T19:25:47.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:47 smithi163 ceph-mon[36452]: pgmap v36: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.0 MiB/s wr, 252 op/s 2024-02-16T19:25:49.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:49 smithi040 ceph-mon[29201]: pgmap v37: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 604 KiB/s wr, 158 op/s 2024-02-16T19:25:49.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:49 smithi163 ceph-mon[36452]: pgmap v37: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 604 KiB/s wr, 158 op/s 2024-02-16T19:25:51.759 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:51 smithi040 ceph-mon[29201]: pgmap v38: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 507 KiB/s wr, 127 op/s 2024-02-16T19:25:51.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:51 smithi163 ceph-mon[36452]: pgmap v38: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 507 KiB/s wr, 127 op/s 2024-02-16T19:25:54.370 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:54 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:54.371 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:54 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:54.371 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:54 smithi040 ceph-mon[29201]: Reconfiguring prometheus.smithi040 (dependencies changed)... 2024-02-16T19:25:54.371 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:54 smithi040 ceph-mon[29201]: Reconfiguring daemon prometheus.smithi040 on smithi040 2024-02-16T19:25:54.371 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:54 smithi040 ceph-mon[29201]: pgmap v39: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 510 KiB/s wr, 128 op/s 2024-02-16T19:25:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:54 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:54 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:54 smithi163 ceph-mon[36452]: Reconfiguring prometheus.smithi040 (dependencies changed)... 2024-02-16T19:25:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:54 smithi163 ceph-mon[36452]: Reconfiguring daemon prometheus.smithi040 on smithi040 2024-02-16T19:25:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:54 smithi163 ceph-mon[36452]: pgmap v39: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 510 KiB/s wr, 128 op/s 2024-02-16T19:25:56.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:56 smithi040 ceph-mon[29201]: pgmap v40: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 4.5 KiB/s wr, 5 op/s 2024-02-16T19:25:56.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:56 smithi163 ceph-mon[36452]: pgmap v40: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 4.5 KiB/s wr, 5 op/s 2024-02-16T19:25:57.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:57 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:57 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:25:58.532 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:58 smithi040 ceph-mon[29201]: pgmap v41: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 4.5 KiB/s wr, 5 op/s 2024-02-16T19:25:58.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:58 smithi163 ceph-mon[36452]: pgmap v41: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 4.5 KiB/s wr, 5 op/s 2024-02-16T19:25:59.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:59 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:59.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:59 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:59.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:59 smithi163 ceph-mon[36452]: Reconfiguring node-exporter.smithi163 (dependencies changed)... 2024-02-16T19:25:59.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:59 smithi163 ceph-mon[36452]: Reconfiguring daemon node-exporter.smithi163 on smithi163 2024-02-16T19:25:59.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:25:59 smithi163 ceph-mon[36452]: pgmap v42: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 3.5 KiB/s wr, 1 op/s 2024-02-16T19:25:59.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:59 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:59.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:59 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:25:59.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:59 smithi040 ceph-mon[29201]: Reconfiguring node-exporter.smithi163 (dependencies changed)... 2024-02-16T19:25:59.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:59 smithi040 ceph-mon[29201]: Reconfiguring daemon node-exporter.smithi163 on smithi163 2024-02-16T19:25:59.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:25:59 smithi040 ceph-mon[29201]: pgmap v42: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 3.5 KiB/s wr, 1 op/s 2024-02-16T19:26:02.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:02.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:02.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: Upgrade: Need to upgrade myself (mgr.smithi163.wjfclz) 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: Upgrade: Need to upgrade myself (mgr.smithi163.wjfclz) 2024-02-16T19:26:02.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:01 smithi040 ceph-mon[29201]: pgmap v43: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 3.4 KiB/s wr, 0 op/s 2024-02-16T19:26:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:26:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:26:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:26:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:26:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:26:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:26:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: Upgrade: Need to upgrade myself (mgr.smithi163.wjfclz) 2024-02-16T19:26:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: Upgrade: Need to upgrade myself (mgr.smithi163.wjfclz) 2024-02-16T19:26:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:01 smithi163 ceph-mon[36452]: pgmap v43: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 3.4 KiB/s wr, 0 op/s 2024-02-16T19:26:03.862 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:03 smithi040 ceph-mon[29201]: Upgrade: Updating mgr.smithi040.jzygzo 2024-02-16T19:26:03.862 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:03 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:03.862 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:03 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.jzygzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:26:03.862 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:03 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:26:03.863 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:03 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:03.863 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:03 smithi040 ceph-mon[29201]: Deploying daemon mgr.smithi040.jzygzo on smithi040 2024-02-16T19:26:03.863 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:03 smithi040 ceph-mon[29201]: pgmap v44: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 3.6 KiB/s wr, 0 op/s 2024-02-16T19:26:03.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:03 smithi163 ceph-mon[36452]: Upgrade: Updating mgr.smithi040.jzygzo 2024-02-16T19:26:03.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:03 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:03.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:03 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.jzygzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:26:03.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:03 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:26:03.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:03 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:03.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:03 smithi163 ceph-mon[36452]: Deploying daemon mgr.smithi040.jzygzo on smithi040 2024-02-16T19:26:03.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:03 smithi163 ceph-mon[36452]: pgmap v44: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 3.6 KiB/s wr, 0 op/s 2024-02-16T19:26:06.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:06 smithi040 ceph-mon[29201]: pgmap v45: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 170 B/s wr, 0 op/s 2024-02-16T19:26:06.691 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:06 smithi163 ceph-mon[36452]: pgmap v45: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 170 B/s wr, 0 op/s 2024-02-16T19:26:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:07 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:07 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:07.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:07 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:07.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:07 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:07.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:07 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:07.869 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:07 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:07.869 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:07 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:07.869 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:07 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:07.869 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:07 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:07.869 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:07 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:08 smithi163 ceph-mon[36452]: pgmap v46: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:26:08.787 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:08 smithi040 ceph-mon[29201]: pgmap v46: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:26:09.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:09 smithi040 ceph-mon[29201]: pgmap v47: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:26:09.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:09 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:09.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:09 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:09 smithi163 ceph-mon[36452]: pgmap v47: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:26:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:09 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:09 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:11.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:11 smithi040 ceph-mon[29201]: pgmap v48: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:26:11.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:11 smithi163 ceph-mon[36452]: pgmap v48: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-02-16T19:26:12.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.235+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1149776252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b14102430 msgr2=0x7f9b14104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.235+0000 7f9b1a84a700 1 --2- 172.21.15.40:0/1149776252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b14102430 0x7f9b14104810 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9b08004940 tx=0x7f9b08009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:12.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.235+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1149776252 shutdown_connections 2024-02-16T19:26:12.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.235+0000 7f9b1a84a700 1 --2- 172.21.15.40:0/1149776252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b14102430 0x7f9b14104810 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.235+0000 7f9b1a84a700 1 --2- 172.21.15.40:0/1149776252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b140663e0 0x7f9b14101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.235+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1149776252 >> 172.21.15.40:0/1149776252 conn(0x7f9b140f74e0 msgr2=0x7f9b140f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:12.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.235+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1149776252 shutdown_connections 2024-02-16T19:26:12.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.235+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1149776252 wait complete. 2024-02-16T19:26:12.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b1a84a700 1 Processor -- start 2024-02-16T19:26:12.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b1a84a700 1 -- start start 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b1a84a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b140663e0 0x7f9b1410fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b1a84a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14102430 0x7f9b141100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b1a84a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b141106b0 con 0x7f9b140663e0 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b1a84a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b14110f50 con 0x7f9b14102430 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b13fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b140663e0 0x7f9b1410fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b13fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b140663e0 0x7f9b1410fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35236/0 (socket says 172.21.15.40:35236) 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b13fff700 1 -- 172.21.15.40:0/1168681697 learned_addr learned my addr 172.21.15.40:0/1168681697 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.236+0000 7f9b137fe700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14102430 0x7f9b141100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b13fff700 1 -- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14102430 msgr2=0x7f9b141100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b13fff700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14102430 0x7f9b141100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b13fff700 1 -- 172.21.15.40:0/1168681697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b08005040 con 0x7f9b140663e0 2024-02-16T19:26:12.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b13fff700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b140663e0 0x7f9b1410fb70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c830 tx=0x7f9b04004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:12.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b10ff9700 1 -- 172.21.15.40:0/1168681697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b0400a760 con 0x7f9b140663e0 2024-02-16T19:26:12.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b14111230 con 0x7f9b140663e0 2024-02-16T19:26:12.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b10ff9700 1 -- 172.21.15.40:0/1168681697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b0400ad90 con 0x7f9b140663e0 2024-02-16T19:26:12.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b14111770 con 0x7f9b140663e0 2024-02-16T19:26:12.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.237+0000 7f9b10ff9700 1 -- 172.21.15.40:0/1168681697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b040113a0 con 0x7f9b140663e0 2024-02-16T19:26:12.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.238+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1404bed0 con 0x7f9b140663e0 2024-02-16T19:26:12.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.239+0000 7f9b10ff9700 1 -- 172.21.15.40:0/1168681697 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f9b04011600 con 0x7f9b140663e0 2024-02-16T19:26:12.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.239+0000 7f9b10ff9700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9afc06b2e0 0x7f9afc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.239+0000 7f9b10ff9700 1 -- 172.21.15.40:0/1168681697 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9b04089cf0 con 0x7f9b140663e0 2024-02-16T19:26:12.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.240+0000 7f9b137fe700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9afc06b2e0 0x7f9afc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.240+0000 7f9b137fe700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9afc06b2e0 0x7f9afc06d790 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b3b0 tx=0x7f9b08009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:12.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.244+0000 7f9b10ff9700 1 -- 172.21.15.40:0/1168681697 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f9b04057940 con 0x7f9b140663e0 2024-02-16T19:26:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.470+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b14111c40 con 0x7f9afc06b2e0 2024-02-16T19:26:12.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.472+0000 7f9b10ff9700 1 -- 172.21.15.40:0/1168681697 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9b14111c40 con 0x7f9afc06b2e0 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9afc06b2e0 msgr2=0x7f9afc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9afc06b2e0 0x7f9afc06d790 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b3b0 tx=0x7f9b08009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b140663e0 msgr2=0x7f9b1410fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b140663e0 0x7f9b1410fb70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c830 tx=0x7f9b04004470 comp rx=0 tx=0).stop 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 shutdown_connections 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f9afc06b2e0 0x7f9afc06d790 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b140663e0 0x7f9b1410fb70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 --2- 172.21.15.40:0/1168681697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14102430 0x7f9b141100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 >> 172.21.15.40:0/1168681697 conn(0x7f9b140f74e0 msgr2=0x7f9b140f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 shutdown_connections 2024-02-16T19:26:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.474+0000 7f9b1a84a700 1 -- 172.21.15.40:0/1168681697 wait complete. 2024-02-16T19:26:12.487 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:26:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.588+0000 7f71d0b43700 1 -- 172.21.15.40:0/2679511165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 msgr2=0x7f71cc0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.588+0000 7f71d0b43700 1 --2- 172.21.15.40:0/2679511165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 0x7f71cc0fd420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f71c0004940 tx=0x7f71c0009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.588+0000 7f71d0b43700 1 -- 172.21.15.40:0/2679511165 shutdown_connections 2024-02-16T19:26:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.588+0000 7f71d0b43700 1 --2- 172.21.15.40:0/2679511165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 0x7f71cc0fd420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.588+0000 7f71d0b43700 1 --2- 172.21.15.40:0/2679511165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71cc0fbe40 0x7f71cc0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.588+0000 7f71d0b43700 1 -- 172.21.15.40:0/2679511165 >> 172.21.15.40:0/2679511165 conn(0x7f71cc0f74e0 msgr2=0x7f71cc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.588+0000 7f71d0b43700 1 -- 172.21.15.40:0/2679511165 shutdown_connections 2024-02-16T19:26:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.588+0000 7f71d0b43700 1 -- 172.21.15.40:0/2679511165 wait complete. 2024-02-16T19:26:12.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.589+0000 7f71d0b43700 1 Processor -- start 2024-02-16T19:26:12.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.589+0000 7f71d0b43700 1 -- start start 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.589+0000 7f71d0b43700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71cc0fbe40 0x7f71cc10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.589+0000 7f71d0b43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 0x7f71cc10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.589+0000 7f71d0b43700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc10e500 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.589+0000 7f71d0b43700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc10ed40 con 0x7f71cc0fbe40 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.589+0000 7f71c9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 0x7f71cc10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71c9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 0x7f71cc10de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35248/0 (socket says 172.21.15.40:35248) 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71c9d9b700 1 -- 172.21.15.40:0/1811091717 learned_addr learned my addr 172.21.15.40:0/1811091717 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71ca59c700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71cc0fbe40 0x7f71cc10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71c9d9b700 1 -- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71cc0fbe40 msgr2=0x7f71cc10d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71c9d9b700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71cc0fbe40 0x7f71cc10d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71c9d9b700 1 -- 172.21.15.40:0/1811091717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c0005040 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71c9d9b700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 0x7f71cc10de70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f71c00093b0 tx=0x7f71c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:12.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71baffd700 1 -- 172.21.15.40:0/1811091717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71c000e040 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc10efc0 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71baffd700 1 -- 172.21.15.40:0/1811091717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71c000bcc0 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71baffd700 1 -- 172.21.15.40:0/1811091717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71c001c630 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.590+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc10f500 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.592+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc04bed0 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.592+0000 7f71baffd700 1 -- 172.21.15.40:0/1811091717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f71c0013980 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.593+0000 7f71baffd700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f71b406b2e0 0x7f71b406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.593+0000 7f71baffd700 1 -- 172.21.15.40:0/1811091717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f71c008b200 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.593+0000 7f71ca59c700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f71b406b2e0 0x7f71b406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.593+0000 7f71ca59c700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f71b406b2e0 0x7f71b406d790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009bf0 tx=0x7f71bc0091a0 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:12.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.601+0000 7f71baffd700 1 -- 172.21.15.40:0/1811091717 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f71c0058cc0 con 0x7f71cc0fcfb0 2024-02-16T19:26:12.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:12 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:12.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:12 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:12.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:12 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:12.800 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:12 smithi040 ceph-mon[29201]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:12.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.836+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71cc10fa00 con 0x7f71b406b2e0 2024-02-16T19:26:12.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.838+0000 7f71baffd700 1 -- 172.21.15.40:0/1811091717 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f71cc10fa00 con 0x7f71b406b2e0 2024-02-16T19:26:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f71b406b2e0 msgr2=0x7f71b406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f71b406b2e0 0x7f71b406d790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009bf0 tx=0x7f71bc0091a0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 msgr2=0x7f71cc10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 0x7f71cc10de70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f71c00093b0 tx=0x7f71c0009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:12.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 shutdown_connections 2024-02-16T19:26:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f71b406b2e0 0x7f71b406d790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71cc0fbe40 0x7f71cc10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 --2- 172.21.15.40:0/1811091717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71cc0fcfb0 0x7f71cc10de70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 >> 172.21.15.40:0/1811091717 conn(0x7f71cc0f74e0 msgr2=0x7f71cc0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 shutdown_connections 2024-02-16T19:26:12.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.840+0000 7f71d0b43700 1 -- 172.21.15.40:0/1811091717 wait complete. 2024-02-16T19:26:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.955+0000 7f775833b700 1 -- 172.21.15.40:0/35179296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 msgr2=0x7f7750070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.955+0000 7f775833b700 1 --2- 172.21.15.40:0/35179296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 0x7f7750070cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7740004940 tx=0x7f7740009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 -- 172.21.15.40:0/35179296 shutdown_connections 2024-02-16T19:26:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 --2- 172.21.15.40:0/35179296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 0x7f7750070cb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 --2- 172.21.15.40:0/35179296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7750071e90 0x7f7750070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 -- 172.21.15.40:0/35179296 >> 172.21.15.40:0/35179296 conn(0x7f77500f8620 msgr2=0x7f77500faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 -- 172.21.15.40:0/35179296 shutdown_connections 2024-02-16T19:26:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 -- 172.21.15.40:0/35179296 wait complete. 2024-02-16T19:26:12.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 Processor -- start 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 -- start start 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 0x7f775010fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7750071e90 0x7f77501100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77501106f0 con 0x7f7750070840 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.956+0000 7f775833b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750110f90 con 0x7f7750071e90 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f77560d7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 0x7f775010fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f77560d7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 0x7f775010fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35276/0 (socket says 172.21.15.40:35276) 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f77560d7700 1 -- 172.21.15.40:0/1573292096 learned_addr learned my addr 172.21.15.40:0/1573292096 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f77558d6700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7750071e90 0x7f77501100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f77560d7700 1 -- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7750071e90 msgr2=0x7f77501100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f77560d7700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7750071e90 0x7f77501100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f77560d7700 1 -- 172.21.15.40:0/1573292096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7740005040 con 0x7f7750070840 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f77560d7700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 0x7f775010fbb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f774c002a50 tx=0x7f774c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f7746ffd700 1 -- 172.21.15.40:0/1573292096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f774c007cc0 con 0x7f7750070840 2024-02-16T19:26:12.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7750111270 con 0x7f7750070840 2024-02-16T19:26:12.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f7746ffd700 1 -- 172.21.15.40:0/1573292096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f774c00f040 con 0x7f7750070840 2024-02-16T19:26:12.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f7746ffd700 1 -- 172.21.15.40:0/1573292096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f774c00da30 con 0x7f7750070840 2024-02-16T19:26:12.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.957+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77501117b0 con 0x7f7750070840 2024-02-16T19:26:12.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.959+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775004bed0 con 0x7f7750070840 2024-02-16T19:26:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.960+0000 7f7746ffd700 1 -- 172.21.15.40:0/1573292096 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f774c00db90 con 0x7f7750070840 2024-02-16T19:26:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.960+0000 7f7746ffd700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f773c06b2e0 0x7f773c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.960+0000 7f7746ffd700 1 -- 172.21.15.40:0/1573292096 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f774c089bc0 con 0x7f7750070840 2024-02-16T19:26:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.960+0000 7f77558d6700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f773c06b2e0 0x7f773c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.961+0000 7f77558d6700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f773c06b2e0 0x7f773c06d790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7740004f90 tx=0x7f7740009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:12.969+0000 7f7746ffd700 1 -- 172.21.15.40:0/1573292096 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f774c057680 con 0x7f7750070840 2024-02-16T19:26:12.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:12 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:12.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:12 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:12.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:12 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:12.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:12 smithi163 ceph-mon[36452]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.184+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7750104570 con 0x7f773c06b2e0 2024-02-16T19:26:13.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.196+0000 7f7746ffd700 1 -- 172.21.15.40:0/1573292096 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f7750104570 con 0x7f773c06b2e0 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (55s) 1s ago 9m 16.2M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (10m) 1s ago 10m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (8m) 3s ago 8m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (20s) 1s ago 9m 75.7M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (4m) 1s ago 4m 127M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (4m) 1s ago 4m 12.1M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (4m) 3s ago 4m 16.0M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (4m) 3s ago 4m 122M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (7s) 1s ago 11m 90.5M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283 running (108s) 3s ago 8m 498M - 19.0.0-1450-gf78a58c0 f081f847070a 294828ee3e60 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (11m) 1s ago 11m 50.6M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:26:13.198 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (8m) 3s ago 8m 39.1M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (62s) 1s ago 9m 8066k - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (12s) 3s ago 8m 3573k - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (7m) 1s ago 7m 227M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (7m) 1s ago 7m 240M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (6m) 1s ago 6m 195M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (6m) 3s ago 6m 268M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (6m) 3s ago 6m 231M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (5m) 3s ago 5m 233M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:26:13.199 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (15s) 1s ago 9m 36.0M - 2.43.0 a07b618ecd1d 22b77b52198b 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f773c06b2e0 msgr2=0x7f773c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f773c06b2e0 0x7f773c06d790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7740004f90 tx=0x7f7740009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 msgr2=0x7f775010fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 0x7f775010fbb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f774c002a50 tx=0x7f774c0046b0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 shutdown_connections 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f773c06b2e0 0x7f773c06d790 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7750070840 0x7f775010fbb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 --2- 172.21.15.40:0/1573292096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7750071e90 0x7f77501100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.198+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 >> 172.21.15.40:0/1573292096 conn(0x7f77500f8620 msgr2=0x7f7750102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.199+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 shutdown_connections 2024-02-16T19:26:13.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.199+0000 7f775833b700 1 -- 172.21.15.40:0/1573292096 wait complete. 2024-02-16T19:26:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.306+0000 7f4632efb700 1 -- 172.21.15.40:0/1480076115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 msgr2=0x7f462c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.306+0000 7f4632efb700 1 --2- 172.21.15.40:0/1480076115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 0x7f462c0fe550 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4618004940 tx=0x7f4618009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.306+0000 7f4632efb700 1 -- 172.21.15.40:0/1480076115 shutdown_connections 2024-02-16T19:26:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.306+0000 7f4632efb700 1 --2- 172.21.15.40:0/1480076115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 0x7f462c0fe550 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.306+0000 7f4632efb700 1 --2- 172.21.15.40:0/1480076115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f462c0fcef0 0x7f462c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.306+0000 7f4632efb700 1 -- 172.21.15.40:0/1480076115 >> 172.21.15.40:0/1480076115 conn(0x7f462c0f8620 msgr2=0x7f462c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.307+0000 7f4632efb700 1 -- 172.21.15.40:0/1480076115 shutdown_connections 2024-02-16T19:26:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.307+0000 7f4632efb700 1 -- 172.21.15.40:0/1480076115 wait complete. 2024-02-16T19:26:13.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.307+0000 7f4632efb700 1 Processor -- start 2024-02-16T19:26:13.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.307+0000 7f4632efb700 1 -- start start 2024-02-16T19:26:13.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f4632efb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f462c0fcef0 0x7f462c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f4632efb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 0x7f462c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f4632efb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0708e0 con 0x7f462c0fe0e0 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f4632efb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c070a50 con 0x7f462c0fcef0 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f462bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 0x7f462c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f462bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 0x7f462c0702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35288/0 (socket says 172.21.15.40:35288) 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f462bfff700 1 -- 172.21.15.40:0/3083978739 learned_addr learned my addr 172.21.15.40:0/3083978739 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f4630c97700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f462c0fcef0 0x7f462c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f462bfff700 1 -- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f462c0fcef0 msgr2=0x7f462c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f462bfff700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f462c0fcef0 0x7f462c071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f462bfff700 1 -- 172.21.15.40:0/3083978739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618005040 con 0x7f462c0fe0e0 2024-02-16T19:26:13.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.308+0000 7f462bfff700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 0x7f462c0702e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4618009fd0 tx=0x7f4618009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:13.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.309+0000 7f46297fa700 1 -- 172.21.15.40:0/3083978739 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461800e040 con 0x7f462c0fe0e0 2024-02-16T19:26:13.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.309+0000 7f46297fa700 1 -- 172.21.15.40:0/3083978739 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f461800bcc0 con 0x7f462c0fe0e0 2024-02-16T19:26:13.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.309+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c070cd0 con 0x7f462c0fe0e0 2024-02-16T19:26:13.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.309+0000 7f46297fa700 1 -- 172.21.15.40:0/3083978739 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461801c630 con 0x7f462c0fe0e0 2024-02-16T19:26:13.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.309+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c1159a0 con 0x7f462c0fe0e0 2024-02-16T19:26:13.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.310+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c04bed0 con 0x7f462c0fe0e0 2024-02-16T19:26:13.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.310+0000 7f46297fa700 1 -- 172.21.15.40:0/3083978739 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89018+0+0 (secure 0 0 0) 0x7f4618013420 con 0x7f462c0fe0e0 2024-02-16T19:26:13.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.311+0000 7f46297fa700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f461c06b2e0 0x7f461c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.311+0000 7f46297fa700 1 -- 172.21.15.40:0/3083978739 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f461808b2a0 con 0x7f462c0fe0e0 2024-02-16T19:26:13.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.311+0000 7f4630c97700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f461c06b2e0 0x7f461c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.312+0000 7f4630c97700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f461c06b2e0 0x7f461c06d790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f462c071bf0 tx=0x7f4620009040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:13.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.316+0000 7f46297fa700 1 -- 172.21.15.40:0/3083978739 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f4618058da0 con 0x7f462c0fe0e0 2024-02-16T19:26:13.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.627+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f462c105a30 con 0x7f462c0fe0e0 2024-02-16T19:26:13.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.627+0000 7f46297fa700 1 -- 172.21.15.40:0/3083978739 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f4618058930 con 0x7f462c0fe0e0 2024-02-16T19:26:13.628 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:26:13.628 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 13, 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:26:13.629 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:26:13.630 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.630+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f461c06b2e0 msgr2=0x7f461c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.630+0000 7f4632efb700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f461c06b2e0 0x7f461c06d790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f462c071bf0 tx=0x7f4620009040 comp rx=0 tx=0).stop 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.630+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 msgr2=0x7f462c0702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.630+0000 7f4632efb700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 0x7f462c0702e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4618009fd0 tx=0x7f4618009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.631+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 shutdown_connections 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.631+0000 7f4632efb700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f461c06b2e0 0x7f461c06d790 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.631+0000 7f4632efb700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f462c0fcef0 0x7f462c071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.631+0000 7f4632efb700 1 --2- 172.21.15.40:0/3083978739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f462c0fe0e0 0x7f462c0702e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.631+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 >> 172.21.15.40:0/3083978739 conn(0x7f462c0f8620 msgr2=0x7f462c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.631+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 shutdown_connections 2024-02-16T19:26:13.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.631+0000 7f4632efb700 1 -- 172.21.15.40:0/3083978739 wait complete. 2024-02-16T19:26:13.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 -- 172.21.15.40:0/1705378235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 msgr2=0x7f0fd80fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 --2- 172.21.15.40:0/1705378235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 0x7f0fd80fc250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8004940 tx=0x7f0fc8009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:13.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 -- 172.21.15.40:0/1705378235 shutdown_connections 2024-02-16T19:26:13.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 --2- 172.21.15.40:0/1705378235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fd80fcfb0 0x7f0fd80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 --2- 172.21.15.40:0/1705378235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 0x7f0fd80fc250 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 -- 172.21.15.40:0/1705378235 >> 172.21.15.40:0/1705378235 conn(0x7f0fd80f7500 msgr2=0x7f0fd80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:13.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 -- 172.21.15.40:0/1705378235 shutdown_connections 2024-02-16T19:26:13.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 -- 172.21.15.40:0/1705378235 wait complete. 2024-02-16T19:26:13.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 Processor -- start 2024-02-16T19:26:13.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 -- start start 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 0x7f0fd80fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fd80fcfb0 0x7f0fd80fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80fea10 con 0x7f0fd80fbe40 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.762+0000 7f0fdd126700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80feb80 con 0x7f0fd80fcfb0 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fd6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 0x7f0fd80fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fd6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 0x7f0fd80fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35292/0 (socket says 172.21.15.40:35292) 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fd6d9d700 1 -- 172.21.15.40:0/1416181024 learned_addr learned my addr 172.21.15.40:0/1416181024 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fcffff700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fd80fcfb0 0x7f0fd80fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fd6d9d700 1 -- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fd80fcfb0 msgr2=0x7f0fd80fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fd6d9d700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fd80fcfb0 0x7f0fd80fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fd6d9d700 1 -- 172.21.15.40:0/1416181024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8005040 con 0x7f0fd80fbe40 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fcffff700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fd80fcfb0 0x7f0fd80fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:26:13.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fd6d9d700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 0x7f0fd80fff00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800b3b0 tx=0x7f0fc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fcf7fe700 1 -- 172.21.15.40:0/1416181024 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc800e040 con 0x7f0fd80fbe40 2024-02-16T19:26:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fcf7fe700 1 -- 172.21.15.40:0/1416181024 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fc800bd90 con 0x7f0fd80fbe40 2024-02-16T19:26:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd80fee00 con 0x7f0fd80fbe40 2024-02-16T19:26:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fcf7fe700 1 -- 172.21.15.40:0/1416181024 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc801c630 con 0x7f0fd80fbe40 2024-02-16T19:26:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.763+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd8115970 con 0x7f0fd80fbe40 2024-02-16T19:26:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.764+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd8107a00 con 0x7f0fd80fbe40 2024-02-16T19:26:13.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.765+0000 7f0fcf7fe700 1 -- 172.21.15.40:0/1416181024 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) v1 ==== 88916+0+0 (secure 0 0 0) 0x7f0fc80139f0 con 0x7f0fd80fbe40 2024-02-16T19:26:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.765+0000 7f0fcf7fe700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f0fc406b2a0 0x7f0fc406d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.766+0000 7f0fcf7fe700 1 -- 172.21.15.40:0/1416181024 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0fc808b070 con 0x7f0fd80fbe40 2024-02-16T19:26:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.766+0000 7f0fcffff700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f0fc406b2a0 0x7f0fc406d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:13.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.766+0000 7f0fcffff700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f0fc406b2a0 0x7f0fc406d750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009b10 tx=0x7f0fc0008040 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:13.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:13.771+0000 7f0fcf7fe700 1 -- 172.21.15.40:0/1416181024 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f0fc8058dd0 con 0x7f0fd80fbe40 2024-02-16T19:26:14.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.025+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0fd81049b0 con 0x7f0fd80fbe40 2024-02-16T19:26:14.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.026+0000 7f0fcf7fe700 1 -- 172.21.15.40:0/1416181024 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1854 (secure 0 0 0) 0x7f0fc8058960 con 0x7f0fd80fbe40 2024-02-16T19:26:14.027 INFO:teuthology.orchestra.run.smithi040.stdout:e19 2024-02-16T19:26:14.027 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:26:14.027 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:14.027 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:26:14.027 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 19 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:24:09.512876+0000 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:26:14.030 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:26:14.030 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:26:14.030 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:14.030 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:14.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f0fc406b2a0 msgr2=0x7f0fc406d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f0fc406b2a0 0x7f0fc406d750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009b10 tx=0x7f0fc0008040 comp rx=0 tx=0).stop 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 msgr2=0x7f0fd80fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 0x7f0fd80fff00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800b3b0 tx=0x7f0fc8009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 shutdown_connections 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f0fc406b2a0 0x7f0fc406d750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80fbe40 0x7f0fd80fff00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 --2- 172.21.15.40:0/1416181024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fd80fcfb0 0x7f0fd80fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 >> 172.21.15.40:0/1416181024 conn(0x7f0fd80f7500 msgr2=0x7f0fd80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 shutdown_connections 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.028+0000 7f0fdd126700 1 -- 172.21.15.40:0/1416181024 wait complete. 2024-02-16T19:26:14.031 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 19 2024-02-16T19:26:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.135+0000 7f21f379e700 1 -- 172.21.15.40:0/1661272904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec066c50 msgr2=0x7f21ec0ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.135+0000 7f21f379e700 1 --2- 172.21.15.40:0/1661272904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec066c50 0x7f21ec0ffb10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f21e8004940 tx=0x7f21e8009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.135+0000 7f21f379e700 1 -- 172.21.15.40:0/1661272904 shutdown_connections 2024-02-16T19:26:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.135+0000 7f21f379e700 1 --2- 172.21.15.40:0/1661272904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec1000e0 0x7f21ec102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.135+0000 7f21f379e700 1 --2- 172.21.15.40:0/1661272904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec066c50 0x7f21ec0ffb10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.135+0000 7f21f379e700 1 -- 172.21.15.40:0/1661272904 >> 172.21.15.40:0/1661272904 conn(0x7f21ec0f7520 msgr2=0x7f21ec0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.135+0000 7f21f379e700 1 -- 172.21.15.40:0/1661272904 shutdown_connections 2024-02-16T19:26:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.135+0000 7f21f379e700 1 -- 172.21.15.40:0/1661272904 wait complete. 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f379e700 1 Processor -- start 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f379e700 1 -- start start 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f379e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec066c50 0x7f21ec0fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f379e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec1000e0 0x7f21ec0fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f379e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec0fc010 con 0x7f21ec1000e0 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f379e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec0fc180 con 0x7f21ec066c50 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f0d39700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec1000e0 0x7f21ec0fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f0d39700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec1000e0 0x7f21ec0fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35310/0 (socket says 172.21.15.40:35310) 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f0d39700 1 -- 172.21.15.40:0/2941167534 learned_addr learned my addr 172.21.15.40:0/2941167534 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f153a700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec066c50 0x7f21ec0fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f0d39700 1 -- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec066c50 msgr2=0x7f21ec0fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f0d39700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec066c50 0x7f21ec0fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.136+0000 7f21f0d39700 1 -- 172.21.15.40:0/2941167534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8005040 con 0x7f21ec1000e0 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.137+0000 7f21f0d39700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec1000e0 0x7f21ec0fba10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00c890 tx=0x7f21dc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.137+0000 7f21e1ffb700 1 -- 172.21.15.40:0/2941167534 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21dc00a700 con 0x7f21ec1000e0 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.137+0000 7f21e1ffb700 1 -- 172.21.15.40:0/2941167534 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21dc00ad30 con 0x7f21ec1000e0 2024-02-16T19:26:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.137+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ec0fc460 con 0x7f21ec1000e0 2024-02-16T19:26:14.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.137+0000 7f21e1ffb700 1 -- 172.21.15.40:0/2941167534 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21dc0113a0 con 0x7f21ec1000e0 2024-02-16T19:26:14.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.137+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ec115970 con 0x7f21ec1000e0 2024-02-16T19:26:14.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.138+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ec107a70 con 0x7f21ec1000e0 2024-02-16T19:26:14.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.138+0000 7f21e1ffb700 1 -- 172.21.15.40:0/2941167534 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) v1 ==== 88916+0+0 (secure 0 0 0) 0x7f21dc012400 con 0x7f21ec1000e0 2024-02-16T19:26:14.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.139+0000 7f21e1ffb700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f21d806b1c0 0x7f21d806d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:14.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.139+0000 7f21e1ffb700 1 -- 172.21.15.40:0/2941167534 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f21dc089c40 con 0x7f21ec1000e0 2024-02-16T19:26:14.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.139+0000 7f21f153a700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f21d806b1c0 0x7f21d806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:14.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.139+0000 7f21f153a700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f21d806b1c0 0x7f21d806d670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21e8004890 tx=0x7f21e8009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:14.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.143+0000 7f21e1ffb700 1 -- 172.21.15.40:0/2941167534 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f21dc08f020 con 0x7f21ec1000e0 2024-02-16T19:26:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.366+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 --> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21ec05e3d0 con 0x7f21d806b1c0 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: pgmap v49: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 853 B/s wr, 0 op/s 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3083978739' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: mgrmap e25: smithi163.wjfclz(active, since 92s), standbys: smithi040.jzygzo 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: Standby manager daemon smithi040.jzygzo restarted 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: Standby manager daemon smithi040.jzygzo started 2024-02-16T19:26:14.368 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.jzygzo/crt"}]: dispatch 2024-02-16T19:26:14.369 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:26:14.369 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.jzygzo/key"}]: dispatch 2024-02-16T19:26:14.369 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:26:14.369 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:14 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1416181024' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:26:14.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.368+0000 7f21e1ffb700 1 -- 172.21.15.40:0/2941167534 <== mgr.24415 v2:172.21.15.163:6828/3793859838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f21ec05e3d0 con 0x7f21d806b1c0 2024-02-16T19:26:14.369 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:26:14.369 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "2/21 daemons upgraded", 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:26:14.370 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:26:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f21d806b1c0 msgr2=0x7f21d806d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f21d806b1c0 0x7f21d806d670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21e8004890 tx=0x7f21e8009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec1000e0 msgr2=0x7f21ec0fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec1000e0 0x7f21ec0fba10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00c890 tx=0x7f21dc0044c0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 shutdown_connections 2024-02-16T19:26:14.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7f21d806b1c0 0x7f21d806d670 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec066c50 0x7f21ec0fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 --2- 172.21.15.40:0/2941167534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21ec1000e0 0x7f21ec0fba10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 >> 172.21.15.40:0/2941167534 conn(0x7f21ec0f7520 msgr2=0x7f21ec0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:14.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 shutdown_connections 2024-02-16T19:26:14.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.370+0000 7f21f379e700 1 -- 172.21.15.40:0/2941167534 wait complete. 2024-02-16T19:26:14.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.473+0000 7ffad7feb700 1 -- 172.21.15.40:0/1822057747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fcfc0 msgr2=0x7ffad00fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.473+0000 7ffad7feb700 1 --2- 172.21.15.40:0/1822057747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fcfc0 0x7ffad00fd430 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffacc004940 tx=0x7ffacc009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.473+0000 7ffad7feb700 1 -- 172.21.15.40:0/1822057747 shutdown_connections 2024-02-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.473+0000 7ffad7feb700 1 --2- 172.21.15.40:0/1822057747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fcfc0 0x7ffad00fd430 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.473+0000 7ffad7feb700 1 --2- 172.21.15.40:0/1822057747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffad00fbe50 0x7ffad00fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.473+0000 7ffad7feb700 1 -- 172.21.15.40:0/1822057747 >> 172.21.15.40:0/1822057747 conn(0x7ffad00f7510 msgr2=0x7ffad00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.473+0000 7ffad7feb700 1 -- 172.21.15.40:0/1822057747 shutdown_connections 2024-02-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.473+0000 7ffad7feb700 1 -- 172.21.15.40:0/1822057747 wait complete. 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad7feb700 1 Processor -- start 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad7feb700 1 -- start start 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad7feb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fbe50 0x7ffad00fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad7feb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffad00fcfc0 0x7ffad00fe390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad7feb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00fe990 con 0x7ffad00fbe50 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad7feb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00feb00 con 0x7ffad00fcfc0 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad5d87700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fbe50 0x7ffad00fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad5d87700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fbe50 0x7ffad00fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35326/0 (socket says 172.21.15.40:35326) 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad5d87700 1 -- 172.21.15.40:0/1311672220 learned_addr learned my addr 172.21.15.40:0/1311672220 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad5586700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffad00fcfc0 0x7ffad00fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad5d87700 1 -- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffad00fcfc0 msgr2=0x7ffad00fe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad5d87700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffad00fcfc0 0x7ffad00fe390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.474+0000 7ffad5d87700 1 -- 172.21.15.40:0/1311672220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc005040 con 0x7ffad00fbe50 2024-02-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.475+0000 7ffad5d87700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fbe50 0x7ffad00fff10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffac0002a50 tx=0x7ffac00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:14.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.475+0000 7ffac67fc700 1 -- 172.21.15.40:0/1311672220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac0007cc0 con 0x7ffad00fbe50 2024-02-16T19:26:14.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.475+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad00fede0 con 0x7ffad00fbe50 2024-02-16T19:26:14.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.475+0000 7ffac67fc700 1 -- 172.21.15.40:0/1311672220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffac000f040 con 0x7ffad00fbe50 2024-02-16T19:26:14.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.475+0000 7ffac67fc700 1 -- 172.21.15.40:0/1311672220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac0009640 con 0x7ffad00fbe50 2024-02-16T19:26:14.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.475+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad0115970 con 0x7ffad00fbe50 2024-02-16T19:26:14.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.476+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad004bed0 con 0x7ffad00fbe50 2024-02-16T19:26:14.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.476+0000 7ffac67fc700 1 -- 172.21.15.40:0/1311672220 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) v1 ==== 88916+0+0 (secure 0 0 0) 0x7ffac00097a0 con 0x7ffad00fbe50 2024-02-16T19:26:14.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.477+0000 7ffac67fc700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ffabc06b1c0 0x7ffabc06d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:14.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.477+0000 7ffac67fc700 1 -- 172.21.15.40:0/1311672220 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ffac00899c0 con 0x7ffad00fbe50 2024-02-16T19:26:14.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.477+0000 7ffad5586700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ffabc06b1c0 0x7ffabc06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:14.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.477+0000 7ffad5586700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ffabc06b1c0 0x7ffabc06d670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffacc004f90 tx=0x7ffacc009e30 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.481+0000 7ffac67fc700 1 -- 172.21.15.40:0/1311672220 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ffac00576a0 con 0x7ffad00fbe50 2024-02-16T19:26:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: pgmap v49: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 853 B/s wr, 0 op/s 2024-02-16T19:26:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3083978739' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:26:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: mgrmap e25: smithi163.wjfclz(active, since 92s), standbys: smithi040.jzygzo 2024-02-16T19:26:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: Standby manager daemon smithi040.jzygzo restarted 2024-02-16T19:26:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: Standby manager daemon smithi040.jzygzo started 2024-02-16T19:26:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.jzygzo/crt"}]: dispatch 2024-02-16T19:26:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:26:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.jzygzo/key"}]: dispatch 2024-02-16T19:26:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:26:14.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:14 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1416181024' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:26:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.650+0000 7ffac67fc700 1 -- 172.21.15.40:0/1311672220 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 26) v1 ==== 91209+0+0 (secure 0 0 0) 0x7ffac0053f40 con 0x7ffad00fbe50 2024-02-16T19:26:14.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.770+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffad0115d70 con 0x7ffad00fbe50 2024-02-16T19:26:14.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.770+0000 7ffac67fc700 1 -- 172.21.15.40:0/1311672220 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ffac00615d0 con 0x7ffad00fbe50 2024-02-16T19:26:14.772 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:26:14.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ffabc06b1c0 msgr2=0x7ffabc06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ffabc06b1c0 0x7ffabc06d670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffacc004f90 tx=0x7ffacc009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:14.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fbe50 msgr2=0x7ffad00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:14.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fbe50 0x7ffad00fff10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffac0002a50 tx=0x7ffac00046b0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 shutdown_connections 2024-02-16T19:26:14.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:6828/3793859838,v1:172.21.15.163:6829/3793859838] conn(0x7ffabc06b1c0 0x7ffabc06d670 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffad00fbe50 0x7ffad00fff10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 --2- 172.21.15.40:0/1311672220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffad00fcfc0 0x7ffad00fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:14.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 >> 172.21.15.40:0/1311672220 conn(0x7ffad00f7510 msgr2=0x7ffad0065fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:14.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 shutdown_connections 2024-02-16T19:26:14.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:14.772+0000 7ffad7feb700 1 -- 172.21.15.40:0/1311672220 wait complete. 2024-02-16T19:26:15.574 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:15 smithi040 ceph-mon[29201]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:15.575 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:15 smithi040 ceph-mon[29201]: mgrmap e26: smithi163.wjfclz(active, since 93s), standbys: smithi040.jzygzo 2024-02-16T19:26:15.575 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:15 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/1311672220' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:26:15.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:15 smithi163 ceph-mon[36452]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:15.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:15 smithi163 ceph-mon[36452]: mgrmap e26: smithi163.wjfclz(active, since 93s), standbys: smithi040.jzygzo 2024-02-16T19:26:15.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:15 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/1311672220' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:26:16.555 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:16 smithi040 ceph-mon[29201]: pgmap v50: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T19:26:16.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:16 smithi163 ceph-mon[36452]: pgmap v50: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-16T19:26:18.343 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:18.343 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:18.343 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:18.343 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:18.343 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:18.343 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:18.343 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mgr fail", "who": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd='[{"prefix": "mgr fail", "who": "smithi163.wjfclz"}]': finished 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: mgrmap e27: smithi040.jzygzo(active, starting, since 0.00630025s) 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:26:18.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: Manager daemon smithi040.jzygzo is now available 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:26:18.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:18 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/trash_purge_schedule"}]: dispatch 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "mgr fail", "who": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.24415 172.21.15.163:0/839206212' entity='mgr.smithi163.wjfclz' cmd='[{"prefix": "mgr fail", "who": "smithi163.wjfclz"}]': finished 2024-02-16T19:26:18.367 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: mgrmap e27: smithi040.jzygzo(active, starting, since 0.00630025s) 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:26:18.368 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:26:18.369 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:26:18.369 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:26:18.369 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: Manager daemon smithi040.jzygzo is now available 2024-02-16T19:26:18.369 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:18.369 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:18.369 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:18.369 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:26:18.369 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:18 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/trash_purge_schedule"}]: dispatch 2024-02-16T19:26:19.467 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:19 smithi163 ceph-mon[36452]: [16/Feb/2024:19:26:17] ENGINE Bus STARTING 2024-02-16T19:26:19.467 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:19 smithi163 ceph-mon[36452]: [16/Feb/2024:19:26:17] ENGINE Serving on https://172.21.15.40:7150 2024-02-16T19:26:19.467 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:19 smithi163 ceph-mon[36452]: [16/Feb/2024:19:26:17] ENGINE Serving on http://172.21.15.40:8765 2024-02-16T19:26:19.467 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:19 smithi163 ceph-mon[36452]: [16/Feb/2024:19:26:17] ENGINE Bus STARTED 2024-02-16T19:26:19.467 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:19 smithi163 ceph-mon[36452]: mgrmap e28: smithi040.jzygzo(active, since 1.00796s) 2024-02-16T19:26:19.467 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:19 smithi163 ceph-mon[36452]: pgmap v3: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:19 smithi040 ceph-mon[29201]: [16/Feb/2024:19:26:17] ENGINE Bus STARTING 2024-02-16T19:26:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:19 smithi040 ceph-mon[29201]: [16/Feb/2024:19:26:17] ENGINE Serving on https://172.21.15.40:7150 2024-02-16T19:26:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:19 smithi040 ceph-mon[29201]: [16/Feb/2024:19:26:17] ENGINE Serving on http://172.21.15.40:8765 2024-02-16T19:26:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:19 smithi040 ceph-mon[29201]: [16/Feb/2024:19:26:17] ENGINE Bus STARTED 2024-02-16T19:26:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:19 smithi040 ceph-mon[29201]: mgrmap e28: smithi040.jzygzo(active, since 1.00796s) 2024-02-16T19:26:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:19 smithi040 ceph-mon[29201]: pgmap v3: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:20.460 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:20 smithi040 ceph-mon[29201]: pgmap v4: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:20.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:20 smithi163 ceph-mon[36452]: pgmap v4: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:21 smithi163 ceph-mon[36452]: mgrmap e29: smithi040.jzygzo(active, since 3s) 2024-02-16T19:26:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:21 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:21 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:21.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:21 smithi040 ceph-mon[29201]: mgrmap e29: smithi040.jzygzo(active, since 3s) 2024-02-16T19:26:21.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:21 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:21.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:21 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:22.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:22 smithi163 ceph-mon[36452]: pgmap v5: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:22.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:22 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:22.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:22 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:22.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:22 smithi040 ceph-mon[29201]: pgmap v5: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:22.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:22 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:22.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:22 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:23 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:23 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:23 smithi163 ceph-mon[36452]: pgmap v6: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:23.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:23 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:23.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:23 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:23.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:23 smithi040 ceph-mon[29201]: pgmap v6: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:24.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:24 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:24.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:24 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:24.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:24 smithi163 ceph-mon[36452]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:26:24.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:24 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/3994236485' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:26:24.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:24 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/3994236485' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:26:24.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:24 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/3994236485' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:26:24.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:24 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/3994236485' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:26:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:24 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:24 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:24 smithi040 ceph-mon[29201]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:26:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:24 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/3994236485' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:26:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:24 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/3994236485' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:26:25.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:24 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/3994236485' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:26:25.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:24 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/3994236485' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:26:25.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:25 smithi040 ceph-mon[29201]: mgrmap e30: smithi040.jzygzo(active, since 7s), standbys: smithi163.wjfclz 2024-02-16T19:26:25.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:25 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi163.wjfclz", "id": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:26:25.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:25 smithi040 ceph-mon[29201]: pgmap v7: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-16T19:26:25.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:25 smithi163 ceph-mon[36452]: mgrmap e30: smithi040.jzygzo(active, since 7s), standbys: smithi163.wjfclz 2024-02-16T19:26:25.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:25 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi163.wjfclz", "id": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:26:25.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:25 smithi163 ceph-mon[36452]: pgmap v7: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-16T19:26:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:28 smithi040 ceph-mon[29201]: pgmap v8: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-16T19:26:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:28 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:28 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:28 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:28 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:28 smithi040 ceph-mon[29201]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:26:28.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:28 smithi040 ceph-mon[29201]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:26:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:28 smithi163 ceph-mon[36452]: pgmap v8: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-16T19:26:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:28 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:28 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:28 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:28.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:28 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:28.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:28 smithi163 ceph-mon[36452]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:26:28.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:28 smithi163 ceph-mon[36452]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:26:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:26:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:29 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:29 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:30 smithi040 ceph-mon[29201]: Reconfiguring prometheus.smithi040 (dependencies changed)... 2024-02-16T19:26:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:30 smithi040 ceph-mon[29201]: Reconfiguring daemon prometheus.smithi040 on smithi040 2024-02-16T19:26:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:30 smithi040 ceph-mon[29201]: pgmap v9: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-16T19:26:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:30 smithi163 ceph-mon[36452]: Reconfiguring prometheus.smithi040 (dependencies changed)... 2024-02-16T19:26:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:30 smithi163 ceph-mon[36452]: Reconfiguring daemon prometheus.smithi040 on smithi040 2024-02-16T19:26:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:30 smithi163 ceph-mon[36452]: pgmap v9: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-16T19:26:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:32 smithi040 ceph-mon[29201]: pgmap v10: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:26:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:32 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:32 smithi163 ceph-mon[36452]: pgmap v10: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:26:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:32 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:34 smithi163 ceph-mon[36452]: pgmap v11: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:26:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:34 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:34 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:34 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:26:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:34 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:34.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:34 smithi040 ceph-mon[29201]: pgmap v11: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:26:34.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:34 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:34.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:34 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:34.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:34 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:26:34.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:34 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:35 smithi163 ceph-mon[36452]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:26:35.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:35 smithi040 ceph-mon[29201]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:26:36.600 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:36 smithi163 ceph-mon[36452]: pgmap v12: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:26:36.600 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:36 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:26:36.600 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:36 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:26:36.600 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:36 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:36.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:36 smithi040 ceph-mon[29201]: pgmap v12: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:26:36.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:36 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:26:36.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:36 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:26:36.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:36 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:37.441 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:37 smithi163 ceph-mon[36452]: Upgrade: Updating mgr.smithi163.wjfclz 2024-02-16T19:26:37.441 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:37 smithi163 ceph-mon[36452]: Deploying daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:26:37.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:37 smithi040 ceph-mon[29201]: Upgrade: Updating mgr.smithi163.wjfclz 2024-02-16T19:26:37.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:37 smithi040 ceph-mon[29201]: Deploying daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:26:38.574 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:38 smithi163 ceph-mon[36452]: pgmap v13: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:38.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:38 smithi040 ceph-mon[29201]: pgmap v13: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:40 smithi163 ceph-mon[36452]: pgmap v14: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:40 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:40 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:40.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:40 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:40.741 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:40 smithi040 ceph-mon[29201]: pgmap v14: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:40.741 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:40 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:40.741 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:40 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:40.742 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:40 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:42.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:42 smithi163 ceph-mon[36452]: pgmap v15: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:42.760 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:42 smithi040 ceph-mon[29201]: pgmap v15: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:43 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:43 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:43 smithi163 ceph-mon[36452]: pgmap v16: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:44.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:43 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:44.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:43 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:44.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:43 smithi040 ceph-mon[29201]: pgmap v16: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.888+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3175977987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c01003a0 msgr2=0x7fa6c0102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.888+0000 7fa6c5b12700 1 --2- 172.21.15.40:0/3175977987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c01003a0 0x7fa6c0102780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4004940 tx=0x7fa6b4009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.888+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3175977987 shutdown_connections 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.888+0000 7fa6c5b12700 1 --2- 172.21.15.40:0/3175977987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c01003a0 0x7fa6c0102780 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.888+0000 7fa6c5b12700 1 --2- 172.21.15.40:0/3175977987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c00fda80 0x7fa6c00ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.888+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3175977987 >> 172.21.15.40:0/3175977987 conn(0x7fa6c00f7520 msgr2=0x7fa6c00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.888+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3175977987 shutdown_connections 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.888+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3175977987 wait complete. 2024-02-16T19:26:44.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6c5b12700 1 Processor -- start 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6c5b12700 1 -- start start 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6c5b12700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c00fda80 0x7fa6c00fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6c5b12700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01003a0 0x7fa6c00fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6c5b12700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00fbf90 con 0x7fa6c00fda80 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6c5b12700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00fc100 con 0x7fa6c01003a0 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6bf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c00fda80 0x7fa6c00fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6bf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c00fda80 0x7fa6c00fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36406/0 (socket says 172.21.15.40:36406) 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6bf7fe700 1 -- 172.21.15.40:0/3426127270 learned_addr learned my addr 172.21.15.40:0/3426127270 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.889+0000 7fa6beffd700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01003a0 0x7fa6c00fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6bf7fe700 1 -- 172.21.15.40:0/3426127270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01003a0 msgr2=0x7fa6c00fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6bf7fe700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01003a0 0x7fa6c00fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6bf7fe700 1 -- 172.21.15.40:0/3426127270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4005040 con 0x7fa6c00fda80 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6beffd700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01003a0 0x7fa6c00fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6bf7fe700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c00fda80 0x7fa6c00fd510 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000eb40 tx=0x7fa6b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6a7fff700 1 -- 172.21.15.40:0/3426127270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b0007cc0 con 0x7fa6c00fda80 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6a7fff700 1 -- 172.21.15.40:0/3426127270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6b000f040 con 0x7fa6c00fda80 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c00fc380 con 0x7fa6c00fda80 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6a7fff700 1 -- 172.21.15.40:0/3426127270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b000ca10 con 0x7fa6c00fda80 2024-02-16T19:26:44.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.890+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c0115900 con 0x7fa6c00fda80 2024-02-16T19:26:44.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.891+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c004bed0 con 0x7fa6c00fda80 2024-02-16T19:26:44.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.891+0000 7fa6a7fff700 1 -- 172.21.15.40:0/3426127270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 91309+0+0 (secure 0 0 0) 0x7fa6b000cb70 con 0x7fa6c00fda80 2024-02-16T19:26:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.892+0000 7fa6a7fff700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fa6a806d200 0x7fa6a806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.892+0000 7fa6a7fff700 1 -- 172.21.15.40:0/3426127270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa6b008c2b0 con 0x7fa6c00fda80 2024-02-16T19:26:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.892+0000 7fa6beffd700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fa6a806d200 0x7fa6a806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:44.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.892+0000 7fa6beffd700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fa6a806d200 0x7fa6a806f6b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4013740 tx=0x7fa6b4009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:44.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:44.897+0000 7fa6a7fff700 1 -- 172.21.15.40:0/3426127270 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa6b0059d90 con 0x7fa6c00fda80 2024-02-16T19:26:45.035 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:44 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:45.035 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:44 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:45.035 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:44 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:45.035 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:44 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:45.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.117+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 --> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6c0101000 con 0x7fa6a806d200 2024-02-16T19:26:45.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.118+0000 7fa6a7fff700 1 -- 172.21.15.40:0/3426127270 <== mgr.14764 v2:172.21.15.40:6800/1137681815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fa6c0101000 con 0x7fa6a806d200 2024-02-16T19:26:45.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fa6a806d200 msgr2=0x7fa6a806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fa6a806d200 0x7fa6a806f6b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4013740 tx=0x7fa6b4009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:45.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c00fda80 msgr2=0x7fa6c00fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c00fda80 0x7fa6c00fd510 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000eb40 tx=0x7fa6b000b040 comp rx=0 tx=0).stop 2024-02-16T19:26:45.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 shutdown_connections 2024-02-16T19:26:45.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6c00fda80 0x7fa6c00fd510 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fa6a806d200 0x7fa6a806f6b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 --2- 172.21.15.40:0/3426127270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01003a0 0x7fa6c00fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 >> 172.21.15.40:0/3426127270 conn(0x7fa6c00f7520 msgr2=0x7fa6c00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 shutdown_connections 2024-02-16T19:26:45.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.120+0000 7fa6c5b12700 1 -- 172.21.15.40:0/3426127270 wait complete. 2024-02-16T19:26:45.131 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:26:45.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:44 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:45.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:44 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:45.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:44 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:45.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:44 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.223+0000 7f975587b700 1 -- 172.21.15.40:0/4004379765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 msgr2=0x7f97500fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.223+0000 7f975587b700 1 --2- 172.21.15.40:0/4004379765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 0x7f97500fc250 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9740004940 tx=0x7f9740009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.223+0000 7f975587b700 1 -- 172.21.15.40:0/4004379765 shutdown_connections 2024-02-16T19:26:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.223+0000 7f975587b700 1 --2- 172.21.15.40:0/4004379765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97500fcfb0 0x7f97500fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.223+0000 7f975587b700 1 --2- 172.21.15.40:0/4004379765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 0x7f97500fc250 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.223+0000 7f975587b700 1 -- 172.21.15.40:0/4004379765 >> 172.21.15.40:0/4004379765 conn(0x7f97500f7520 msgr2=0x7f97500f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.223+0000 7f975587b700 1 -- 172.21.15.40:0/4004379765 shutdown_connections 2024-02-16T19:26:45.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.223+0000 7f975587b700 1 -- 172.21.15.40:0/4004379765 wait complete. 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f975587b700 1 Processor -- start 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f975587b700 1 -- start start 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f975587b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 0x7f975010fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f975587b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97500fcfb0 0x7f9750110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f975587b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9750110670 con 0x7f97500fbe40 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f975587b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9750110f10 con 0x7f97500fcfb0 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f974effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 0x7f975010fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f974effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 0x7f975010fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36428/0 (socket says 172.21.15.40:36428) 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f974effd700 1 -- 172.21.15.40:0/3128524136 learned_addr learned my addr 172.21.15.40:0/3128524136 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.224+0000 7f974e7fc700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97500fcfb0 0x7f9750110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f974effd700 1 -- 172.21.15.40:0/3128524136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97500fcfb0 msgr2=0x7f9750110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f974effd700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97500fcfb0 0x7f9750110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f974effd700 1 -- 172.21.15.40:0/3128524136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9740005040 con 0x7f97500fbe40 2024-02-16T19:26:45.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f974effd700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 0x7f975010fb30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9740009fd0 tx=0x7f9740009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f97377fe700 1 -- 172.21.15.40:0/3128524136 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974000e040 con 0x7f97500fbe40 2024-02-16T19:26:45.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9750111190 con 0x7f97500fbe40 2024-02-16T19:26:45.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f97377fe700 1 -- 172.21.15.40:0/3128524136 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f974000bcc0 con 0x7f97500fbe40 2024-02-16T19:26:45.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f97377fe700 1 -- 172.21.15.40:0/3128524136 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974001c630 con 0x7f97500fbe40 2024-02-16T19:26:45.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.225+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97501116d0 con 0x7f97500fbe40 2024-02-16T19:26:45.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.226+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975004bed0 con 0x7f97500fbe40 2024-02-16T19:26:45.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.226+0000 7f97377fe700 1 -- 172.21.15.40:0/3128524136 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f9740013420 con 0x7f97500fbe40 2024-02-16T19:26:45.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.227+0000 7f97377fe700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f973806d200 0x7f973806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.227+0000 7f97377fe700 1 -- 172.21.15.40:0/3128524136 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f974008dc40 con 0x7f97500fbe40 2024-02-16T19:26:45.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.227+0000 7f974e7fc700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f973806d200 0x7f973806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.227+0000 7f974e7fc700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f973806d200 0x7f973806f6b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9744004f40 tx=0x7f974400b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.231+0000 7f97377fe700 1 -- 172.21.15.40:0/3128524136 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f974005b670 con 0x7f97500fbe40 2024-02-16T19:26:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.455+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 --> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9750111d40 con 0x7f973806d200 2024-02-16T19:26:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.456+0000 7f97377fe700 1 -- 172.21.15.40:0/3128524136 <== mgr.14764 v2:172.21.15.40:6800/1137681815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9750111d40 con 0x7f973806d200 2024-02-16T19:26:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f973806d200 msgr2=0x7f973806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f973806d200 0x7f973806f6b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9744004f40 tx=0x7f974400b040 comp rx=0 tx=0).stop 2024-02-16T19:26:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 msgr2=0x7f975010fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 0x7f975010fb30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9740009fd0 tx=0x7f9740009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 shutdown_connections 2024-02-16T19:26:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97500fbe40 0x7f975010fb30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f973806d200 0x7f973806f6b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 --2- 172.21.15.40:0/3128524136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97500fcfb0 0x7f9750110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 >> 172.21.15.40:0/3128524136 conn(0x7f97500f7520 msgr2=0x7f9750065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 shutdown_connections 2024-02-16T19:26:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.458+0000 7f975587b700 1 -- 172.21.15.40:0/3128524136 wait complete. 2024-02-16T19:26:45.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.560+0000 7f2282842700 1 -- 172.21.15.40:0/413301649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 msgr2=0x7f227c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.560+0000 7f2282842700 1 --2- 172.21.15.40:0/413301649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 0x7f227c0fd320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f226c004940 tx=0x7f226c009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:45.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.560+0000 7f2282842700 1 -- 172.21.15.40:0/413301649 shutdown_connections 2024-02-16T19:26:45.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.560+0000 7f2282842700 1 --2- 172.21.15.40:0/413301649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227c0fe0e0 0x7f227c0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.560+0000 7f2282842700 1 --2- 172.21.15.40:0/413301649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 0x7f227c0fd320 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.560+0000 7f2282842700 1 -- 172.21.15.40:0/413301649 >> 172.21.15.40:0/413301649 conn(0x7f227c0f8620 msgr2=0x7f227c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.560+0000 7f2282842700 1 -- 172.21.15.40:0/413301649 shutdown_connections 2024-02-16T19:26:45.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.560+0000 7f2282842700 1 -- 172.21.15.40:0/413301649 wait complete. 2024-02-16T19:26:45.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f2282842700 1 Processor -- start 2024-02-16T19:26:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f2282842700 1 -- start start 2024-02-16T19:26:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f2282842700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 0x7f227c10d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f2282842700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227c0fe0e0 0x7f227c10de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f2282842700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c10e420 con 0x7f227c0fcef0 2024-02-16T19:26:45.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f2282842700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c10ecc0 con 0x7f227c0fe0e0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f227bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 0x7f227c10d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f227bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 0x7f227c10d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36442/0 (socket says 172.21.15.40:36442) 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f227bfff700 1 -- 172.21.15.40:0/311756413 learned_addr learned my addr 172.21.15.40:0/311756413 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f227b7fe700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227c0fe0e0 0x7f227c10de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f227bfff700 1 -- 172.21.15.40:0/311756413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227c0fe0e0 msgr2=0x7f227c10de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f227bfff700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227c0fe0e0 0x7f227c10de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f227bfff700 1 -- 172.21.15.40:0/311756413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c005040 con 0x7f227c0fcef0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.561+0000 7f227b7fe700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227c0fe0e0 0x7f227c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.562+0000 7f227bfff700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 0x7f227c10d8e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f226c000c00 tx=0x7f226c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.562+0000 7f2278ff9700 1 -- 172.21.15.40:0/311756413 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f226c00e070 con 0x7f227c0fcef0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.562+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c10ef40 con 0x7f227c0fcef0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.562+0000 7f2278ff9700 1 -- 172.21.15.40:0/311756413 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f226c01c400 con 0x7f227c0fcef0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.562+0000 7f2278ff9700 1 -- 172.21.15.40:0/311756413 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f226c00b670 con 0x7f227c0fcef0 2024-02-16T19:26:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.562+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c10f420 con 0x7f227c0fcef0 2024-02-16T19:26:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.563+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c04bed0 con 0x7f227c0fcef0 2024-02-16T19:26:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.563+0000 7f2278ff9700 1 -- 172.21.15.40:0/311756413 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f226c01c9e0 con 0x7f227c0fcef0 2024-02-16T19:26:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.564+0000 7f2278ff9700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f226406d200 0x7f226406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.564+0000 7f2278ff9700 1 -- 172.21.15.40:0/311756413 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f226c08de00 con 0x7f227c0fcef0 2024-02-16T19:26:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.564+0000 7f227b7fe700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f226406d200 0x7f226406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.564+0000 7f227b7fe700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f226406d200 0x7f226406f6b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2270009b80 tx=0x7f227000b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.570+0000 7f2278ff9700 1 -- 172.21.15.40:0/311756413 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f226c05b830 con 0x7f227c0fcef0 2024-02-16T19:26:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.782+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 --> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f227c10f840 con 0x7f226406d200 2024-02-16T19:26:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.792+0000 7f2278ff9700 1 -- 172.21.15.40:0/311756413 <== mgr.14764 v2:172.21.15.40:6800/1137681815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f227c10f840 con 0x7f226406d200 2024-02-16T19:26:45.793 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:26:45.793 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (88s) 0s ago 10m 16.2M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:26:45.793 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (10m) 0s ago 10m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:26:45.793 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (9m) 2s ago 9m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (53s) 0s ago 10m 80.0M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (4m) 0s ago 4m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (4m) 0s ago 4m 12.5M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (4m) 2s ago 4m 16.3M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (4m) 2s ago 4m 122M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (39s) 0s ago 11m 491M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (6s) 2s ago 9m 65.0M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (11m) 0s ago 11m 55.2M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 e62bfb93d328 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (9m) 2s ago 9m 41.4M 2048M 17.2.7-513-gb88fae66 870a0944ccb8 7679bf43e6a5 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (95s) 0s ago 10m 8363k - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (45s) 2s ago 9m 13.7M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (8m) 0s ago 8m 227M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (7m) 0s ago 7m 242M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (7m) 0s ago 7m 196M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:26:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (7m) 2s ago 7m 269M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:26:45.795 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (6m) 2s ago 6m 231M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:26:45.795 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (6m) 2s ago 6m 234M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:26:45.795 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (12s) 0s ago 9m 39.7M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:26:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f226406d200 msgr2=0x7f226406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f226406d200 0x7f226406f6b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2270009b80 tx=0x7f227000b040 comp rx=0 tx=0).stop 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 msgr2=0x7f227c10d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 0x7f227c10d8e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f226c000c00 tx=0x7f226c009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 shutdown_connections 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c0fcef0 0x7f227c10d8e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f226406d200 0x7f226406f6b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 --2- 172.21.15.40:0/311756413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227c0fe0e0 0x7f227c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 >> 172.21.15.40:0/311756413 conn(0x7f227c0f8620 msgr2=0x7f227c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 shutdown_connections 2024-02-16T19:26:45.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.794+0000 7f2282842700 1 -- 172.21.15.40:0/311756413 wait complete. 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.895+0000 7fe9c1a67700 1 -- 172.21.15.40:0/3487885433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fe2b0 msgr2=0x7fe9bc0fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.895+0000 7fe9c1a67700 1 --2- 172.21.15.40:0/3487885433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fe2b0 0x7fe9bc0fe700 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0004940 tx=0x7fe9b0009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.895+0000 7fe9c1a67700 1 -- 172.21.15.40:0/3487885433 shutdown_connections 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.895+0000 7fe9c1a67700 1 --2- 172.21.15.40:0/3487885433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fe2b0 0x7fe9bc0fe700 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.895+0000 7fe9c1a67700 1 --2- 172.21.15.40:0/3487885433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0fb220 0x7fe9bc0fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.895+0000 7fe9c1a67700 1 -- 172.21.15.40:0/3487885433 >> 172.21.15.40:0/3487885433 conn(0x7fe9bc0f6ca0 msgr2=0x7fe9bc0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.895+0000 7fe9c1a67700 1 -- 172.21.15.40:0/3487885433 shutdown_connections 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.895+0000 7fe9c1a67700 1 -- 172.21.15.40:0/3487885433 wait complete. 2024-02-16T19:26:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9c1a67700 1 Processor -- start 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9c1a67700 1 -- start start 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9c1a67700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fb220 0x7fe9bc0fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9c1a67700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0fe2b0 0x7fe9bc0fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9c1a67700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0fc7c0 con 0x7fe9bc0fb220 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9c1a67700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0fc930 con 0x7fe9bc0fe2b0 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9bb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fb220 0x7fe9bc0fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9bb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fb220 0x7fe9bc0fdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36464/0 (socket says 172.21.15.40:36464) 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9bb7fe700 1 -- 172.21.15.40:0/4159583789 learned_addr learned my addr 172.21.15.40:0/4159583789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.896+0000 7fe9baffd700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0fe2b0 0x7fe9bc0fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9bb7fe700 1 -- 172.21.15.40:0/4159583789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0fe2b0 msgr2=0x7fe9bc0fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9bb7fe700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0fe2b0 0x7fe9bc0fc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9bb7fe700 1 -- 172.21.15.40:0/4159583789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b0005040 con 0x7fe9bc0fb220 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9bb7fe700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fb220 0x7fe9bc0fdd40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00eb70 tx=0x7fe9ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9a3fff700 1 -- 172.21.15.40:0/4159583789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9ac007cc0 con 0x7fe9bc0fb220 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc0fcbb0 con 0x7fe9bc0fb220 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc115980 con 0x7fe9bc0fb220 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9a3fff700 1 -- 172.21.15.40:0/4159583789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9ac00f040 con 0x7fe9bc0fb220 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.897+0000 7fe9a3fff700 1 -- 172.21.15.40:0/4159583789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9ac00cb20 con 0x7fe9bc0fb220 2024-02-16T19:26:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.898+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a80052e0 con 0x7fe9bc0fb220 2024-02-16T19:26:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.899+0000 7fe9a3fff700 1 -- 172.21.15.40:0/4159583789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 91309+0+0 (secure 0 0 0) 0x7fe9ac00cc80 con 0x7fe9bc0fb220 2024-02-16T19:26:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.899+0000 7fe9a3fff700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fe9a406d200 0x7fe9a406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.899+0000 7fe9a3fff700 1 -- 172.21.15.40:0/4159583789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe9ac08c130 con 0x7fe9bc0fb220 2024-02-16T19:26:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.899+0000 7fe9baffd700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fe9a406d200 0x7fe9a406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.900+0000 7fe9baffd700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fe9a406d200 0x7fe9a406f6b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000b0d0 tx=0x7fe9b0009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:45.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:45.904+0000 7fe9a3fff700 1 -- 172.21.15.40:0/4159583789 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe9ac059c10 con 0x7fe9bc0fb220 2024-02-16T19:26:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.188+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe9a8005860 con 0x7fe9bc0fb220 2024-02-16T19:26:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.189+0000 7fe9a3fff700 1 -- 172.21.15.40:0/4159583789 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe9ac0597a0 con 0x7fe9bc0fb220 2024-02-16T19:26:46.190 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:26:46.191 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 12, 2024-02-16T19:26:46.192 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:26:46.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:26:46.192 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:26:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fe9a406d200 msgr2=0x7fe9a406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fe9a406d200 0x7fe9a406f6b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000b0d0 tx=0x7fe9b0009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fb220 msgr2=0x7fe9bc0fdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fb220 0x7fe9bc0fdd40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00eb70 tx=0x7fe9ac00b040 comp rx=0 tx=0).stop 2024-02-16T19:26:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 shutdown_connections 2024-02-16T19:26:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9bc0fb220 0x7fe9bc0fdd40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7fe9a406d200 0x7fe9a406f6b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 --2- 172.21.15.40:0/4159583789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0fe2b0 0x7fe9bc0fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 >> 172.21.15.40:0/4159583789 conn(0x7fe9bc0f6ca0 msgr2=0x7fe9bc1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 shutdown_connections 2024-02-16T19:26:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.191+0000 7fe9c1a67700 1 -- 172.21.15.40:0/4159583789 wait complete. 2024-02-16T19:26:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.293+0000 7f9cc59f3700 1 -- 172.21.15.40:0/1881496513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 msgr2=0x7f9cc0102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.293+0000 7f9cc59f3700 1 --2- 172.21.15.40:0/1881496513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 0x7f9cc0102520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0004940 tx=0x7f9cb0009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.293+0000 7f9cc59f3700 1 -- 172.21.15.40:0/1881496513 shutdown_connections 2024-02-16T19:26:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.293+0000 7f9cc59f3700 1 --2- 172.21.15.40:0/1881496513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 0x7f9cc0102520 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.293+0000 7f9cc59f3700 1 --2- 172.21.15.40:0/1881496513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc0066c50 0x7f9cc00ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.293+0000 7f9cc59f3700 1 -- 172.21.15.40:0/1881496513 >> 172.21.15.40:0/1881496513 conn(0x7f9cc00f7520 msgr2=0x7f9cc00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.293+0000 7f9cc59f3700 1 -- 172.21.15.40:0/1881496513 shutdown_connections 2024-02-16T19:26:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.293+0000 7f9cc59f3700 1 -- 172.21.15.40:0/1881496513 wait complete. 2024-02-16T19:26:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.294+0000 7f9cc59f3700 1 Processor -- start 2024-02-16T19:26:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.294+0000 7f9cc59f3700 1 -- start start 2024-02-16T19:26:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cc59f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc0066c50 0x7f9cc010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cc59f3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 0x7f9cc010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cc59f3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc010e4a0 con 0x7f9cc01000e0 2024-02-16T19:26:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cc59f3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc010ed40 con 0x7f9cc0066c50 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cbe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 0x7f9cc010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc0066c50 0x7f9cc010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cbe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 0x7f9cc010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36478/0 (socket says 172.21.15.40:36478) 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cbe7fc700 1 -- 172.21.15.40:0/3012889239 learned_addr learned my addr 172.21.15.40:0/3012889239 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cbe7fc700 1 -- 172.21.15.40:0/3012889239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc0066c50 msgr2=0x7f9cc010d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cbe7fc700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc0066c50 0x7f9cc010d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cbe7fc700 1 -- 172.21.15.40:0/3012889239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb0005040 con 0x7f9cc01000e0 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cbe7fc700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 0x7f9cc010dea0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0000c00 tx=0x7f9cb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cb77fe700 1 -- 172.21.15.40:0/3012889239 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cb000e040 con 0x7f9cc01000e0 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cb77fe700 1 -- 172.21.15.40:0/3012889239 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cb000bbf0 con 0x7f9cc01000e0 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc010efc0 con 0x7f9cc01000e0 2024-02-16T19:26:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.295+0000 7f9cb77fe700 1 -- 172.21.15.40:0/3012889239 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cb001c630 con 0x7f9cc01000e0 2024-02-16T19:26:46.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.296+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc010f500 con 0x7f9cc01000e0 2024-02-16T19:26:46.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.296+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc0107a50 con 0x7f9cc01000e0 2024-02-16T19:26:46.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.297+0000 7f9cb77fe700 1 -- 172.21.15.40:0/3012889239 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f9cb000bd60 con 0x7f9cc01000e0 2024-02-16T19:26:46.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.297+0000 7f9cb77fe700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f9cac06d200 0x7f9cac06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.298+0000 7f9cb77fe700 1 -- 172.21.15.40:0/3012889239 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9cb008da90 con 0x7f9cc01000e0 2024-02-16T19:26:46.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.298+0000 7f9cbeffd700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f9cac06d200 0x7f9cac06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.298+0000 7f9cbeffd700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f9cac06d200 0x7f9cac06f6b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8004f40 tx=0x7f9ca800b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.302+0000 7f9cb77fe700 1 -- 172.21.15.40:0/3012889239 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f9cb005b560 con 0x7f9cc01000e0 2024-02-16T19:26:46.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.550+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9cc004bed0 con 0x7f9cc01000e0 2024-02-16T19:26:46.552 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:46 smithi040 ceph-mon[29201]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:46.552 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:46 smithi040 ceph-mon[29201]: pgmap v17: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:46.552 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:46 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/4159583789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:26:46.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.551+0000 7f9cb77fe700 1 -- 172.21.15.40:0/3012889239 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1854 (secure 0 0 0) 0x7f9cb005b0f0 con 0x7f9cc01000e0 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:e19 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 19 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:24:09.512876+0000 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:26:46.553 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:26:46.554 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:46.555 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:46.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f9cac06d200 msgr2=0x7f9cac06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f9cac06d200 0x7f9cac06f6b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8004f40 tx=0x7f9ca800b040 comp rx=0 tx=0).stop 2024-02-16T19:26:46.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 msgr2=0x7f9cc010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 0x7f9cc010dea0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0000c00 tx=0x7f9cb0009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:46.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 shutdown_connections 2024-02-16T19:26:46.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f9cac06d200 0x7f9cac06f6b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc0066c50 0x7f9cc010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 --2- 172.21.15.40:0/3012889239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cc01000e0 0x7f9cc010dea0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 >> 172.21.15.40:0/3012889239 conn(0x7f9cc00f7520 msgr2=0x7f9cc00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:46.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 shutdown_connections 2024-02-16T19:26:46.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.553+0000 7f9cc59f3700 1 -- 172.21.15.40:0/3012889239 wait complete. 2024-02-16T19:26:46.557 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 19 2024-02-16T19:26:46.589 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:46 smithi163 ceph-mon[36452]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:46.589 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:46 smithi163 ceph-mon[36452]: pgmap v17: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:46.589 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:46 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/4159583789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:26:46.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.655+0000 7ff677999700 1 -- 172.21.15.40:0/3591692735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 msgr2=0x7ff6700fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.655+0000 7ff677999700 1 --2- 172.21.15.40:0/3591692735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 0x7ff6700fe550 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff660004940 tx=0x7ff660009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:46.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.655+0000 7ff677999700 1 -- 172.21.15.40:0/3591692735 shutdown_connections 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.655+0000 7ff677999700 1 --2- 172.21.15.40:0/3591692735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 0x7ff6700fe550 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.655+0000 7ff677999700 1 --2- 172.21.15.40:0/3591692735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6700fcef0 0x7ff6700fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.655+0000 7ff677999700 1 -- 172.21.15.40:0/3591692735 >> 172.21.15.40:0/3591692735 conn(0x7ff6700f8620 msgr2=0x7ff6700faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff677999700 1 -- 172.21.15.40:0/3591692735 shutdown_connections 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff677999700 1 -- 172.21.15.40:0/3591692735 wait complete. 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff677999700 1 Processor -- start 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff677999700 1 -- start start 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff677999700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6700fcef0 0x7ff67010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff677999700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 0x7ff67010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff677999700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67010e4a0 con 0x7ff6700fe0e0 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff677999700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67010ed40 con 0x7ff6700fcef0 2024-02-16T19:26:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff674f34700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 0x7ff67010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff674f34700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 0x7ff67010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36488/0 (socket says 172.21.15.40:36488) 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff674f34700 1 -- 172.21.15.40:0/2647447503 learned_addr learned my addr 172.21.15.40:0/2647447503 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.656+0000 7ff675735700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6700fcef0 0x7ff67010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff674f34700 1 -- 172.21.15.40:0/2647447503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6700fcef0 msgr2=0x7ff67010d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff674f34700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6700fcef0 0x7ff67010d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff674f34700 1 -- 172.21.15.40:0/2647447503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660005040 con 0x7ff6700fe0e0 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff674f34700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 0x7ff67010dea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff6600093b0 tx=0x7ff660009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff665ffb700 1 -- 172.21.15.40:0/2647447503 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff66000e040 con 0x7ff6700fe0e0 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff665ffb700 1 -- 172.21.15.40:0/2647447503 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff66000bcc0 con 0x7ff6700fe0e0 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67010efc0 con 0x7ff6700fe0e0 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff665ffb700 1 -- 172.21.15.40:0/2647447503 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff66001c630 con 0x7ff6700fe0e0 2024-02-16T19:26:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.657+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67010f500 con 0x7ff6700fe0e0 2024-02-16T19:26:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.658+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67004bed0 con 0x7ff6700fe0e0 2024-02-16T19:26:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.659+0000 7ff665ffb700 1 -- 172.21.15.40:0/2647447503 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 91309+0+0 (secure 0 0 0) 0x7ff660013980 con 0x7ff6700fe0e0 2024-02-16T19:26:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.659+0000 7ff665ffb700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7ff65c06d200 0x7ff65c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.659+0000 7ff665ffb700 1 -- 172.21.15.40:0/2647447503 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff66008dba0 con 0x7ff6700fe0e0 2024-02-16T19:26:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.659+0000 7ff675735700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7ff65c06d200 0x7ff65c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.660+0000 7ff675735700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7ff65c06d200 0x7ff65c06f6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff6700fcc30 tx=0x7ff66c00b040 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:46.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.663+0000 7ff665ffb700 1 -- 172.21.15.40:0/2647447503 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff66005b5d0 con 0x7ff6700fe0e0 2024-02-16T19:26:46.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.884+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 --> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff67004bed0 con 0x7ff65c06d200 2024-02-16T19:26:46.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.885+0000 7ff665ffb700 1 -- 172.21.15.40:0/2647447503 <== mgr.14764 v2:172.21.15.40:6800/1137681815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7ff67004bed0 con 0x7ff65c06d200 2024-02-16T19:26:46.886 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "2/21 daemons upgraded", 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading mgr daemons", 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:26:46.887 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7ff65c06d200 msgr2=0x7ff65c06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7ff65c06d200 0x7ff65c06f6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff6700fcc30 tx=0x7ff66c00b040 comp rx=0 tx=0).stop 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 msgr2=0x7ff67010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 0x7ff67010dea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff6600093b0 tx=0x7ff660009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 shutdown_connections 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7ff65c06d200 0x7ff65c06f6b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6700fcef0 0x7ff67010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 --2- 172.21.15.40:0/2647447503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6700fe0e0 0x7ff67010dea0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.887+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 >> 172.21.15.40:0/2647447503 conn(0x7ff6700f8620 msgr2=0x7ff6701013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.888+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 shutdown_connections 2024-02-16T19:26:46.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.888+0000 7ff677999700 1 -- 172.21.15.40:0/2647447503 wait complete. 2024-02-16T19:26:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.989+0000 7f8add1db700 1 -- 172.21.15.40:0/880134239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fe2b0 msgr2=0x7f8ad80fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.989+0000 7f8add1db700 1 --2- 172.21.15.40:0/880134239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fe2b0 0x7f8ad80fe700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0004940 tx=0x7f8ac0009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.989+0000 7f8add1db700 1 -- 172.21.15.40:0/880134239 shutdown_connections 2024-02-16T19:26:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 --2- 172.21.15.40:0/880134239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fe2b0 0x7f8ad80fe700 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 --2- 172.21.15.40:0/880134239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ad80fb220 0x7f8ad80fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 -- 172.21.15.40:0/880134239 >> 172.21.15.40:0/880134239 conn(0x7f8ad80f6ca0 msgr2=0x7f8ad80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:46.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 -- 172.21.15.40:0/880134239 shutdown_connections 2024-02-16T19:26:46.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 -- 172.21.15.40:0/880134239 wait complete. 2024-02-16T19:26:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 Processor -- start 2024-02-16T19:26:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 -- start start 2024-02-16T19:26:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fb220 0x7f8ad810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ad80fe2b0 0x7f8ad810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad810e550 con 0x7f8ad80fb220 2024-02-16T19:26:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8add1db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad810edf0 con 0x7f8ad80fe2b0 2024-02-16T19:26:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.990+0000 7f8ad6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fb220 0x7f8ad810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8ad6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fb220 0x7f8ad810da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36510/0 (socket says 172.21.15.40:36510) 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8ad6d9d700 1 -- 172.21.15.40:0/932066927 learned_addr learned my addr 172.21.15.40:0/932066927 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8ad659c700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ad80fe2b0 0x7f8ad810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8ad6d9d700 1 -- 172.21.15.40:0/932066927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ad80fe2b0 msgr2=0x7f8ad810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8ad6d9d700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ad80fe2b0 0x7f8ad810df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8ad6d9d700 1 -- 172.21.15.40:0/932066927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0005040 con 0x7f8ad80fb220 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8ad6d9d700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fb220 0x7f8ad810da10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800cb10 tx=0x7f8ac800ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8acf7fe700 1 -- 172.21.15.40:0/932066927 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac800a980 con 0x7f8ad80fb220 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8acf7fe700 1 -- 172.21.15.40:0/932066927 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ac8012420 con 0x7f8ad80fb220 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad810f0d0 con 0x7f8ad80fb220 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8acf7fe700 1 -- 172.21.15.40:0/932066927 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac80115c0 con 0x7f8ad80fb220 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.991+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad810f610 con 0x7f8ad80fb220 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.992+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad804bed0 con 0x7f8ad80fb220 2024-02-16T19:26:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.993+0000 7f8acf7fe700 1 -- 172.21.15.40:0/932066927 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 91309+0+0 (secure 0 0 0) 0x7f8ac8003020 con 0x7f8ad80fb220 2024-02-16T19:26:46.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.993+0000 7f8acf7fe700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f8ac406d200 0x7f8ac406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:26:46.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.993+0000 7f8acf7fe700 1 -- 172.21.15.40:0/932066927 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8ac808ca40 con 0x7f8ad80fb220 2024-02-16T19:26:46.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.993+0000 7f8ad659c700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f8ac406d200 0x7f8ac406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:26:46.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.994+0000 7f8ad659c700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f8ac406d200 0x7f8ac406f6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0004f90 tx=0x7f8ac0009e30 comp rx=0 tx=0).ready entity=mgr.14764 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:26:46.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:46.998+0000 7f8acf7fe700 1 -- 172.21.15.40:0/932066927 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f8ac805a500 con 0x7f8ad80fb220 2024-02-16T19:26:47.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.283+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8ad8105a40 con 0x7f8ad80fb220 2024-02-16T19:26:47.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.284+0000 7f8acf7fe700 1 -- 172.21.15.40:0/932066927 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f8ac8012590 con 0x7f8ad80fb220 2024-02-16T19:26:47.285 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f8ac406d200 msgr2=0x7f8ac406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f8ac406d200 0x7f8ac406f6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0004f90 tx=0x7f8ac0009e30 comp rx=0 tx=0).stop 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fb220 msgr2=0x7f8ad810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fb220 0x7f8ad810da10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800cb10 tx=0x7f8ac800ce20 comp rx=0 tx=0).stop 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 shutdown_connections 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ad80fb220 0x7f8ad810da10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.40:6800/1137681815,v1:172.21.15.40:6801/1137681815] conn(0x7f8ac406d200 0x7f8ac406f6b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 --2- 172.21.15.40:0/932066927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ad80fe2b0 0x7f8ad810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 >> 172.21.15.40:0/932066927 conn(0x7f8ad80f6ca0 msgr2=0x7f8ad81014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:26:47.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 shutdown_connections 2024-02-16T19:26:47.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:26:47.286+0000 7f8add1db700 1 -- 172.21.15.40:0/932066927 wait complete. 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/3012889239' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: Standby manager daemon smithi163.wjfclz restarted 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/4148783641' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/4148783641' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/4148783641' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='mgr.? 172.21.15.163:0/4148783641' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:47.422 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:47 smithi040 ceph-mon[29201]: from='client.? 172.21.15.40:0/932066927' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:26:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/3012889239' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:26:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: Standby manager daemon smithi163.wjfclz restarted 2024-02-16T19:26:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:26:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/4148783641' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:26:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/4148783641' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:26:47.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/4148783641' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:26:47.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/4148783641' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:26:47.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:26:47.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:47 smithi163 ceph-mon[36452]: from='client.? 172.21.15.40:0/932066927' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:26:48.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: pgmap v18: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: mgrmap e31: smithi040.jzygzo(active, since 30s), standbys: smithi163.wjfclz 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:48.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi040.jzygzo"}]: dispatch 2024-02-16T19:26:48.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi040.jzygzo"}]': finished 2024-02-16T19:26:48.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi163.wjfclz"}]: dispatch 2024-02-16T19:26:48.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi163.wjfclz"}]': finished 2024-02-16T19:26:48.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:48 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: pgmap v18: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: mgrmap e31: smithi040.jzygzo(active, since 30s), standbys: smithi163.wjfclz 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:26:48.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:26:48.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:48.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi040.jzygzo"}]: dispatch 2024-02-16T19:26:48.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi040.jzygzo"}]': finished 2024-02-16T19:26:48.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi163.wjfclz"}]: dispatch 2024-02-16T19:26:48.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi163.wjfclz"}]': finished 2024-02-16T19:26:48.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:48 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T19:26:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:49 smithi040 ceph-mon[29201]: Upgrade: Setting container_image for all mgr 2024-02-16T19:26:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:49 smithi163 ceph-mon[36452]: Upgrade: Setting container_image for all mgr 2024-02-16T19:26:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:50 smithi040 ceph-mon[29201]: pgmap v19: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:50 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:50 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:26:50.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:50 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:26:50.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:50 smithi040 ceph-mon[29201]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:50.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:50 smithi163 ceph-mon[36452]: pgmap v19: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:50.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:50 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' 2024-02-16T19:26:50.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:50 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:26:50.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:50 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:26:50.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:50 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:26:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:51 smithi040 ceph-mon[29201]: Upgrade: Updating mon.smithi040 2024-02-16T19:26:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:51 smithi040 ceph-mon[29201]: Deploying daemon mon.smithi040 on smithi040 2024-02-16T19:26:51.674 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:51 smithi040 systemd[1]: Stopping Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:26:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:51 smithi163 ceph-mon[36452]: Upgrade: Updating mon.smithi040 2024-02-16T19:26:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:51 smithi163 ceph-mon[36452]: Deploying daemon mon.smithi040 on smithi040 2024-02-16T19:26:51.949 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:51 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[29197]: 2024-02-16T19:26:51.672+0000 7f326c128700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi040 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:26:51.949 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:51 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[29197]: 2024-02-16T19:26:51.672+0000 7f326c128700 -1 mon.smithi040@0(leader) e2 *** Got Signal Terminated *** 2024-02-16T19:26:52.234 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:51 smithi040 podman[94034]: 2024-02-16 19:26:51.948831182 +0000 UTC m=+0.391182825 container died e62bfb93d3286cbab019c5836b3134790e2c8cf4477c07a7a15c60bccb0405c5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-b88fae6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:26:52.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:52 smithi040 podman[94034]: 2024-02-16 19:26:52.399119134 +0000 UTC m=+0.841470785 container cleanup e62bfb93d3286cbab019c5836b3134790e2c8cf4477c07a7a15c60bccb0405c5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=quincy-b88fae6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:26:52.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:52 smithi040 bash[94034]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040 2024-02-16T19:26:53.042 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:52 smithi040 podman[94059]: 2024-02-16 19:26:52.675601808 +0000 UTC m=+0.726765735 container remove e62bfb93d3286cbab019c5836b3134790e2c8cf4477c07a7a15c60bccb0405c5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=quincy-b88fae6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:26:53.042 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:52 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040.service: Deactivated successfully. 2024-02-16T19:26:53.042 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:52 smithi040 systemd[1]: Stopped Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:26:53.042 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:52 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040.service: Consumed 11.510s CPU time. 2024-02-16T19:26:53.323 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 systemd[1]: Starting Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:26:53.323 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 podman[94191]: 2024-02-16 19:26:53.160702256 +0000 UTC m=+0.018043331 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:26:53.323 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 podman[94191]: 2024-02-16 19:26:53.323241145 +0000 UTC m=+0.180582211 container create 1c1011f60db65020aba4245f7749afcb4a16a678cb3e2f4689b6113ca39fafa3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:26:53.575 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 podman[94191]: 2024-02-16 19:26:53.571554307 +0000 UTC m=+0.428895377 container init 1c1011f60db65020aba4245f7749afcb4a16a678cb3e2f4689b6113ca39fafa3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-16T19:26:53.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 podman[94191]: 2024-02-16 19:26:53.575405351 +0000 UTC m=+0.432746436 container start 1c1011f60db65020aba4245f7749afcb4a16a678cb3e2f4689b6113ca39fafa3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: pidfile_write: ignore empty --pid-file 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: load: jerasure load: lrc 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: RocksDB version: 7.9.2 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Git sha 0 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: DB SUMMARY 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: DB Session ID: 30IG3W1WRNZ47MQC26Y7 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: CURRENT file: CURRENT 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T19:26:53.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1170 Bytes 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi040/store.db dir, Total Num: 1, files: 000033.sst 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi040/store.db: 000031.log size: 1925120 ; 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.error_if_exists: 0 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.create_if_missing: 0 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.paranoid_checks: 1 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.env: 0x5586567f1ae0 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.info_log: 0x558657f73760 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.statistics: (nil) 2024-02-16T19:26:53.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.use_fsync: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_log_file_size: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.allow_fallocate: 1 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.use_direct_reads: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.db_log_dir: 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.wal_dir: 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T19:26:53.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.write_buffer_manager: 0x558657f47400 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T19:26:53.898 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.unordered_write: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.row_cache: None 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.wal_filter: None 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.two_write_queues: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.wal_compression: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.atomic_flush: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T19:26:53.899 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T19:26:53.901 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T19:26:53.901 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.log_readahead_size: 0 2024-02-16T19:26:53.901 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T19:26:53.901 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T19:26:53.901 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T19:26:53.901 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T19:26:53.901 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_background_jobs: 2 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_background_compactions: -1 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_subcompactions: 1 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_open_files: -1 2024-02-16T19:26:53.902 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_background_flushes: -1 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Compression algorithms supported: 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: kZSTD supported: 0 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: kXpressCompression supported: 0 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: kZlibCompression supported: 1 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: kSnappyCompression supported: 1 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: kLZ4Compression supported: 1 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: kBZip2Compression supported: 0 2024-02-16T19:26:53.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi040/store.db/MANIFEST-000015 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.merge_operator: 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_filter: None 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558657f73880) 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: cache_index_and_filter_blocks: 1 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T19:26:53.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: pin_top_level_index_and_filter: 1 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_type: 0 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_block_index_type: 0 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_shortening: 1 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: checksum: 4 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: no_block_cache: 0 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache: 0x558657f77090 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_name: BinnedLRUCache 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_options: 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: capacity : 536870912 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: num_shard_bits : 4 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: strict_capacity_limit : 0 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: high_pri_pool_ratio: 0.000 2024-02-16T19:26:53.905 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_compressed: (nil) 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: persistent_cache: (nil) 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_size: 4096 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_size_deviation: 10 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_restart_interval: 16 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_block_restart_interval: 1 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: metadata_block_size: 4096 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: partition_filters: 0 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: use_delta_encoding: 1 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: filter_policy: bloomfilter 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: whole_key_filtering: 1 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: verify_compression: 0 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: read_amp_bytes_per_bit: 0 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: format_version: 5 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: enable_index_compression: 1 2024-02-16T19:26:53.906 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_align: 0 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: max_auto_readahead_size: 262144 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: prepopulate_block_cache: 0 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: initial_auto_readahead_size: 8192 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression: NoCompression 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.num_levels: 7 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T19:26:53.907 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T19:26:53.908 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T19:26:53.909 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T19:26:53.910 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.table_properties_collectors: 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.inplace_update_support: 0 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.bloom_locality: 0 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.max_successive_merges: 0 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T19:26:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.ttl: 2592000 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.enable_blob_files: false 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.min_blob_size: 0 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T19:26:53.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi040/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12206, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fe8e5907-6f2f-4475-8992-38435b1b5eee 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111613604582, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111613612314, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1759598, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12207, "largest_seqno": 13018, "table_properties": {"data_size": 1755301, "index_size": 2323, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 965, "raw_key_size": 8633, "raw_average_key_size": 23, "raw_value_size": 1747427, "raw_average_value_size": 4774, "num_data_blocks": 106, "num_entries": 366, "num_filter_entries": 366, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708111613, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fe8e5907-6f2f-4475-8992-38435b1b5eee", "db_session_id": "30IG3W1WRNZ47MQC26Y7", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111613612401, "job": 1, "event": "recovery_finished"} 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T19:26:53.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi040/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558658066000 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: DB pointer 0x558658052000 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** DB Stats ** 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-02-16T19:26:53.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: L0 1/0 1.68 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 414.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: L6 1/0 5.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Sum 2/0 6.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 414.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 414.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 414.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:26:53.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative compaction: 0.00 GB write, 165.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval compaction: 0.00 GB write, 165.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Block cache BinnedLRUCache@0x558657f77090#2 capacity: 512.00 MB usage: 3.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.03 KB,0.000196695%) IndexBlock(1,2.58 KB,0.000491738%) Misc(1,0.00 KB,0%) 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: starting mon.smithi040 rank 0 at public addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] at bind addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi040 fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: mon.smithi040@-1(???) e2 preinit fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:26:53.916 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: mon.smithi040@-1(???).mds e19 new map 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: mon.smithi040@-1(???).mds e19 print_map 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: e19 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: legacy client fscid: 1 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Filesystem 'cephfs' (1) 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: fs_name cephfs 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: epoch 19 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: modified 2024-02-16T19:24:09.512876+0000 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: tableserver 0 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: root 0 2024-02-16T19:26:53.917 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: session_timeout 60 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: session_autoclose 300 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: max_file_size 1099511627776 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: max_xattr_size 65536 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: required_client_features {} 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: last_failure 0 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: last_failure_osd_epoch 46 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: max_mds 2 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: in 0,1 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: up {0=14486,1=24263} 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: failed 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: damaged 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: stopped 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_pools [3] 2024-02-16T19:26:53.918 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: metadata_pool 2 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: inline_data disabled 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: balancer 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: bal_rank_mask -1 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: standby_count_wanted 1 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: [mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: [mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Standby daemons: 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: [mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: [mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: mon.smithi040@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: mon.smithi040@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:26:53.919 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: mon.smithi040@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:26:53.920 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: mon.smithi040@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:26:53.920 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 ceph-mon[94210]: mon.smithi040@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-02-16T19:26:53.920 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 bash[94191]: 1c1011f60db65020aba4245f7749afcb4a16a678cb3e2f4689b6113ca39fafa3 2024-02-16T19:26:53.920 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:53 smithi040 systemd[1]: Started Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:26:55.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:26:55.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:26:55.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: pgmap v21: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:55.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: mon.smithi040 calling monitor election 2024-02-16T19:26:55.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: mon.smithi040 is new leader, mons smithi040,smithi163 in quorum (ranks 0,1) 2024-02-16T19:26:55.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: monmap e2: 2 mons at {smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],smithi163=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:26:55.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:26:55.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T19:26:55.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: mgrmap e31: smithi040.jzygzo(active, since 37s), standbys: smithi163.wjfclz 2024-02-16T19:26:55.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: overall HEALTH_OK 2024-02-16T19:26:55.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: from='mgr.14764 ' entity='' 2024-02-16T19:26:55.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:26:55 smithi040 ceph-mon[94210]: mgrmap e32: smithi040.jzygzo(active, since 37s), standbys: smithi163.wjfclz 2024-02-16T19:26:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:26:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: from='mgr.14764 172.21.15.40:0/3582922358' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:26:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: pgmap v21: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:26:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: mon.smithi040 calling monitor election 2024-02-16T19:26:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: mon.smithi040 is new leader, mons smithi040,smithi163 in quorum (ranks 0,1) 2024-02-16T19:26:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: monmap e2: 2 mons at {smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],smithi163=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:26:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:26:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: osdmap e48: 6 total, 6 up, 6 in 2024-02-16T19:26:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: mgrmap e31: smithi040.jzygzo(active, since 37s), standbys: smithi163.wjfclz 2024-02-16T19:26:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: overall HEALTH_OK 2024-02-16T19:26:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: from='mgr.14764 ' entity='' 2024-02-16T19:26:55.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:26:55 smithi163 ceph-mon[36452]: mgrmap e32: smithi040.jzygzo(active, since 37s), standbys: smithi163.wjfclz 2024-02-16T19:27:02.134 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:01 smithi040 ceph-mon[94210]: Standby manager daemon smithi163.wjfclz restarted 2024-02-16T19:27:02.135 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:01 smithi040 ceph-mon[94210]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:27:02.135 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:01 smithi040 ceph-mon[94210]: from='mgr.? 172.21.15.163:0/491008836' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:27:02.135 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:01 smithi040 ceph-mon[94210]: from='mgr.? 172.21.15.163:0/491008836' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:27:02.135 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:01 smithi040 ceph-mon[94210]: from='mgr.? 172.21.15.163:0/491008836' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:27:02.135 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:01 smithi040 ceph-mon[94210]: from='mgr.? 172.21.15.163:0/491008836' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:27:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:01 smithi163 ceph-mon[36452]: Standby manager daemon smithi163.wjfclz restarted 2024-02-16T19:27:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:01 smithi163 ceph-mon[36452]: Standby manager daemon smithi163.wjfclz started 2024-02-16T19:27:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:01 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/491008836' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/crt"}]: dispatch 2024-02-16T19:27:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:01 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/491008836' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-16T19:27:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:01 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/491008836' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.wjfclz/key"}]: dispatch 2024-02-16T19:27:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:01 smithi163 ceph-mon[36452]: from='mgr.? 172.21.15.163:0/491008836' entity='mgr.smithi163.wjfclz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-16T19:27:02.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163[36434]: 2024-02-16T19:27:02.148+0000 7f145fc56700 -1 mon.smithi163@1(peon).osd e49 update_from_paxos full map CRC mismatch, resetting to canonical 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: mgrmap e33: smithi040.jzygzo(active, since 44s), standbys: smithi163.wjfclz 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: Active manager daemon smithi040.jzygzo restarted 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: Activating manager daemon smithi040.jzygzo 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: mgrmap e34: smithi040.jzygzo(active, starting, since 0.00500411s), standbys: smithi163.wjfclz 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: failed to encode map e49 with expected crc 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:27:03.256 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi163.wjfclz", "id": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: failed to encode map e49 with expected crc 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: failed to encode map e49 with expected crc 2024-02-16T19:27:03.257 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: Manager daemon smithi040.jzygzo is now available 2024-02-16T19:27:03.258 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:03.258 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:03.258 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:03.258 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:27:03.258 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:02 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/trash_purge_schedule"}]: dispatch 2024-02-16T19:27:03.344 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: mgrmap e33: smithi040.jzygzo(active, since 44s), standbys: smithi163.wjfclz 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: Active manager daemon smithi040.jzygzo restarted 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: Activating manager daemon smithi040.jzygzo 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: mgrmap e34: smithi040.jzygzo(active, starting, since 0.00500411s), standbys: smithi163.wjfclz 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: failed to encode map e49 with expected crc 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi040.jzygzo", "id": "smithi040.jzygzo"}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr metadata", "who": "smithi163.wjfclz", "id": "smithi163.wjfclz"}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:27:03.345 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: failed to encode map e49 with expected crc 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: failed to encode map e49 with expected crc 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: Manager daemon smithi040.jzygzo is now available 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/mirror_snapshot_schedule"}]: dispatch 2024-02-16T19:27:03.346 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.jzygzo/trash_purge_schedule"}]: dispatch 2024-02-16T19:27:04.379 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:04 smithi040 ceph-mon[94210]: [16/Feb/2024:19:27:03] ENGINE Bus STARTING 2024-02-16T19:27:04.379 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:04 smithi040 ceph-mon[94210]: mgrmap e35: smithi040.jzygzo(active, since 1.00858s), standbys: smithi163.wjfclz 2024-02-16T19:27:04.379 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:04 smithi040 ceph-mon[94210]: pgmap v3: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:04.379 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:04 smithi040 ceph-mon[94210]: [16/Feb/2024:19:27:03] ENGINE Serving on https://172.21.15.40:7150 2024-02-16T19:27:04.379 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:04 smithi040 ceph-mon[94210]: [16/Feb/2024:19:27:03] ENGINE Serving on http://172.21.15.40:8765 2024-02-16T19:27:04.379 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:04 smithi040 ceph-mon[94210]: [16/Feb/2024:19:27:03] ENGINE Bus STARTED 2024-02-16T19:27:04.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:04 smithi163 ceph-mon[36452]: [16/Feb/2024:19:27:03] ENGINE Bus STARTING 2024-02-16T19:27:04.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:04 smithi163 ceph-mon[36452]: mgrmap e35: smithi040.jzygzo(active, since 1.00858s), standbys: smithi163.wjfclz 2024-02-16T19:27:04.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:04 smithi163 ceph-mon[36452]: pgmap v3: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:04.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:04 smithi163 ceph-mon[36452]: [16/Feb/2024:19:27:03] ENGINE Serving on https://172.21.15.40:7150 2024-02-16T19:27:04.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:04 smithi163 ceph-mon[36452]: [16/Feb/2024:19:27:03] ENGINE Serving on http://172.21.15.40:8765 2024-02-16T19:27:04.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:04 smithi163 ceph-mon[36452]: [16/Feb/2024:19:27:03] ENGINE Bus STARTED 2024-02-16T19:27:05.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:05 smithi163 ceph-mon[36452]: pgmap v4: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:05.570 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:05 smithi040 ceph-mon[94210]: pgmap v4: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:06.420 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:06 smithi040 ceph-mon[94210]: mgrmap e36: smithi040.jzygzo(active, since 3s), standbys: smithi163.wjfclz 2024-02-16T19:27:06.420 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:06 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:06.420 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:06 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:06.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:06 smithi163 ceph-mon[36452]: mgrmap e36: smithi040.jzygzo(active, since 3s), standbys: smithi163.wjfclz 2024-02-16T19:27:06.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:06 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:06.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:06 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:07.641 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:07 smithi163 ceph-mon[36452]: pgmap v5: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:07.641 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:07 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:07.641 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:07 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:07.642 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:07 smithi163 ceph-mon[36452]: mgrmap e37: smithi040.jzygzo(active, since 5s), standbys: smithi163.wjfclz 2024-02-16T19:27:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:07 smithi040 ceph-mon[94210]: pgmap v5: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:07 smithi040 ceph-mon[94210]: mgrmap e37: smithi040.jzygzo(active, since 5s), standbys: smithi163.wjfclz 2024-02-16T19:27:08.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:08 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:08.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:08 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:08 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:08 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:09 smithi163 ceph-mon[36452]: pgmap v6: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:09 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:09 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:09 smithi040 ceph-mon[94210]: pgmap v6: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:09 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:09 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:11.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:11.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: pgmap v7: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:27:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:11 smithi163 ceph-mon[36452]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:27:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:27:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: Updating smithi040:/etc/ceph/ceph.conf 2024-02-16T19:27:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: Updating smithi163:/etc/ceph/ceph.conf 2024-02-16T19:27:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: pgmap v7: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-16T19:27:11.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:27:11.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-16T19:27:11.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:27:11.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-02-16T19:27:11.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: Updating smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:27:11.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:11 smithi040 ceph-mon[94210]: Updating smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-16T19:27:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:12 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:12 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:12 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:12 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:12 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:12 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:12.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:12 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:12.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:12 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T19:27:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:12.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-16T19:27:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:13 smithi163 ceph-mon[36452]: pgmap v8: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-16T19:27:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:13 smithi163 ceph-mon[36452]: Upgrade: Updating mon.smithi163 2024-02-16T19:27:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:13 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:13 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:27:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:13 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:27:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:13 smithi163 ceph-mon[36452]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:13 smithi163 ceph-mon[36452]: Deploying daemon mon.smithi163 on smithi163 2024-02-16T19:27:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:13 smithi040 ceph-mon[94210]: pgmap v8: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-16T19:27:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:13 smithi040 ceph-mon[94210]: Upgrade: Updating mon.smithi163 2024-02-16T19:27:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:27:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:27:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:14.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:13 smithi040 ceph-mon[94210]: Deploying daemon mon.smithi163 on smithi163 2024-02-16T19:27:14.789 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:14 smithi163 systemd[1]: Stopping Ceph mon.smithi163 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:27:15.106 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:14 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163[36434]: 2024-02-16T19:27:14.787+0000 7f1465461700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi163 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:27:15.106 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:14 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163[36434]: 2024-02-16T19:27:14.787+0000 7f1465461700 -1 mon.smithi163@1(peon) e2 *** Got Signal Terminated *** 2024-02-16T19:27:15.106 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:14 smithi163 podman[81472]: 2024-02-16 19:27:14.808514819 +0000 UTC m=+0.141333699 container died 7679bf43e6a52cb951533cedfd5d7483d87dbee1cabd9b43ee7b28aca46cddf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163, io.buildah.version=1.33.5, ceph=True, RELEASE=quincy-b88fae6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-16T19:27:15.398 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:15 smithi163 podman[81472]: 2024-02-16 19:27:15.23020635 +0000 UTC m=+0.563025244 container cleanup 7679bf43e6a52cb951533cedfd5d7483d87dbee1cabd9b43ee7b28aca46cddf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=quincy-b88fae6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:27:15.399 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:15 smithi163 bash[81472]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163 2024-02-16T19:27:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:15 smithi163 podman[81486]: 2024-02-16 19:27:15.398465402 +0000 UTC m=+0.584503496 container remove 7679bf43e6a52cb951533cedfd5d7483d87dbee1cabd9b43ee7b28aca46cddf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=quincy-b88fae6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-02-16T19:27:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:15 smithi163 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi163.service: Deactivated successfully. 2024-02-16T19:27:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:15 smithi163 systemd[1]: Stopped Ceph mon.smithi163 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:27:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:15 smithi163 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi163.service: Consumed 5.931s CPU time. 2024-02-16T19:27:16.083 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:15 smithi163 systemd[1]: Starting Ceph mon.smithi163 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:27:16.083 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 podman[81619]: 2024-02-16 19:27:15.943900583 +0000 UTC m=+0.017876187 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:27:16.084 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 podman[81619]: 2024-02-16 19:27:16.083355966 +0000 UTC m=+0.157331565 container create 4840eea014c332aab5f5fcbb225fe81a40e6edd3c74d6cdd76fc63cc133a6cbf (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 podman[81619]: 2024-02-16 19:27:16.277702135 +0000 UTC m=+0.351677734 container init 4840eea014c332aab5f5fcbb225fe81a40e6edd3c74d6cdd76fc63cc133a6cbf (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 podman[81619]: 2024-02-16 19:27:16.280884077 +0000 UTC m=+0.354859687 container start 4840eea014c332aab5f5fcbb225fe81a40e6edd3c74d6cdd76fc63cc133a6cbf (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: set uid:gid to 167:167 (ceph:ceph) 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: pidfile_write: ignore empty --pid-file 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: load: jerasure load: lrc 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: RocksDB version: 7.9.2 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Git sha 0 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: DB SUMMARY 2024-02-16T19:27:16.335 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: DB Session ID: 6GYZQYFFQNUWHZNVVU6U 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: CURRENT file: CURRENT 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: IDENTITY file: IDENTITY 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1095 Bytes 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi163/store.db dir, Total Num: 1, files: 000027.sst 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi163/store.db: 000025.log size: 5692668 ; 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.error_if_exists: 0 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.create_if_missing: 0 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.paranoid_checks: 1 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.env: 0x55b5e04f6ae0 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.fs: PosixFileSystem 2024-02-16T19:27:16.336 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.info_log: 0x55b5e108d760 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_file_opening_threads: 16 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.statistics: (nil) 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.use_fsync: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_log_file_size: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.keep_log_file_num: 1000 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.recycle_log_file_num: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.allow_fallocate: 1 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.allow_mmap_reads: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.allow_mmap_writes: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.use_direct_reads: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.create_missing_column_families: 0 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.db_log_dir: 2024-02-16T19:27:16.337 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.wal_dir: 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.advise_random_on_open: 1 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.db_write_buffer_size: 0 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.write_buffer_manager: 0x55b5e1061400 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.rate_limiter: (nil) 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-16T19:27:16.338 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.wal_recovery_mode: 2 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.enable_thread_tracking: 0 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.enable_pipelined_write: 0 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.unordered_write: 0 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.row_cache: None 2024-02-16T19:27:16.339 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.wal_filter: None 2024-02-16T19:27:16.340 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-16T19:27:16.340 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.allow_ingest_behind: 0 2024-02-16T19:27:16.340 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.two_write_queues: 0 2024-02-16T19:27:16.340 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.manual_wal_flush: 0 2024-02-16T19:27:16.340 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.wal_compression: 0 2024-02-16T19:27:16.340 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.atomic_flush: 0 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.log_readahead_size: 0 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.best_efforts_recovery: 0 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.allow_data_in_errors: 0 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.db_host_id: __hostname__ 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_background_jobs: 2 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_background_compactions: -1 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_subcompactions: 1 2024-02-16T19:27:16.341 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_total_wal_size: 0 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_open_files: -1 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bytes_per_sync: 0 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_readahead_size: 0 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_background_flushes: -1 2024-02-16T19:27:16.342 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Compression algorithms supported: 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: kZSTD supported: 0 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: kXpressCompression supported: 0 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: kLZ4HCCompression supported: 1 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: kZlibCompression supported: 1 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: kSnappyCompression supported: 1 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: kLZ4Compression supported: 1 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: kBZip2Compression supported: 0 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi163/store.db/MANIFEST-000009 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-16T19:27:16.343 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.merge_operator: 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_filter: None 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_filter_factory: None 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.sst_partitioner_factory: None 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b5e108d880) 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: cache_index_and_filter_blocks: 1 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: pin_top_level_index_and_filter: 1 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: index_type: 0 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: data_block_index_type: 0 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: index_shortening: 1 2024-02-16T19:27:16.344 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: checksum: 4 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: no_block_cache: 0 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: block_cache: 0x55b5e1091090 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: block_cache_name: BinnedLRUCache 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: block_cache_options: 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: capacity : 536870912 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: num_shard_bits : 4 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: strict_capacity_limit : 0 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: high_pri_pool_ratio: 0.000 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: block_cache_compressed: (nil) 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: persistent_cache: (nil) 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: block_size: 4096 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: block_size_deviation: 10 2024-02-16T19:27:16.345 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: block_restart_interval: 16 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: index_block_restart_interval: 1 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: metadata_block_size: 4096 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: partition_filters: 0 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: use_delta_encoding: 1 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: filter_policy: bloomfilter 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: whole_key_filtering: 1 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: verify_compression: 0 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: read_amp_bytes_per_bit: 0 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: format_version: 5 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: enable_index_compression: 1 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: block_align: 0 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: max_auto_readahead_size: 262144 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: prepopulate_block_cache: 0 2024-02-16T19:27:16.346 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: initial_auto_readahead_size: 8192 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: num_file_reads_for_auto_readahead: 2 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.write_buffer_size: 33554432 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_write_buffer_number: 2 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression: NoCompression 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression: Disabled 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.prefix_extractor: nullptr 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.num_levels: 7 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-16T19:27:16.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.level: 32767 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.strategy: 0 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.enabled: false 2024-02-16T19:27:16.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.target_file_size_base: 67108864 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-16T19:27:16.349 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.arena_block_size: 1048576 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.disable_auto_compactions: 0 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-16T19:27:16.350 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.table_properties_collectors: 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.inplace_update_support: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.bloom_locality: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.max_successive_merges: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.paranoid_file_checks: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.force_consistency_checks: 1 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.report_bg_io_stats: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.ttl: 2592000 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-16T19:27:16.351 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.enable_blob_files: false 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.min_blob_size: 0 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.blob_file_size: 268435456 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.blob_file_starting_level: 0 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi163/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12319, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-16T19:27:16.352 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-02-16T19:27:16.353 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 88391043-b03d-4e83-8270-7d42fc6f4a4f 2024-02-16T19:27:16.353 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111636321042, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-02-16T19:27:16.353 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-02-16T19:27:16.353 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111636333387, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3435060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12324, "largest_seqno": 13686, "table_properties": {"data_size": 3428834, "index_size": 3735, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1477, "raw_key_size": 13235, "raw_average_key_size": 23, "raw_value_size": 3416700, "raw_average_value_size": 6057, "num_data_blocks": 170, "num_entries": 564, "num_filter_entries": 564, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708111636, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "88391043-b03d-4e83-8270-7d42fc6f4a4f", "db_session_id": "6GYZQYFFQNUWHZNVVU6U", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-02-16T19:27:16.353 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708111636333518, "job": 1, "event": "recovery_finished"} 2024-02-16T19:27:16.353 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-02-16T19:27:16.353 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-16T19:27:16.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:16 smithi040 ceph-mon[94210]: pgmap v9: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-16T19:27:16.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi163/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b5e1180000 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: DB pointer 0x55b5e116c000 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: ** DB Stats ** 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-16T19:27:16.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: ** Compaction Stats [default] ** 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: L0 1/0 3.28 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 681.1 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: L6 1/0 5.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Sum 2/0 8.50 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 681.1 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 681.1 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: ** Compaction Stats [default] ** 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 681.1 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Cumulative compaction: 0.00 GB write, 114.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Interval compaction: 0.00 GB write, 114.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Block cache BinnedLRUCache@0x55b5e1091090#2 capacity: 512.00 MB usage: 5.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.53 KB,0.000292063%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.740 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: starting mon.smithi163 rank 1 at public addrs [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] at bind addrs [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi163 fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: mon.smithi163@-1(???) e2 preinit fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: mon.smithi163@-1(???).mds e19 new map 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: mon.smithi163@-1(???).mds e19 print_map 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: e19 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: legacy client fscid: 1 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Filesystem 'cephfs' (1) 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: fs_name cephfs 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: epoch 19 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: modified 2024-02-16T19:24:09.512876+0000 2024-02-16T19:27:16.741 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: tableserver 0 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: root 0 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: session_timeout 60 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: session_autoclose 300 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: max_file_size 1099511627776 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: max_xattr_size 65536 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: required_client_features {} 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: last_failure 0 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: last_failure_osd_epoch 46 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: max_mds 2 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: in 0,1 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: up {0=14486,1=24263} 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: failed 2024-02-16T19:27:16.742 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: damaged 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: stopped 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: data_pools [3] 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: metadata_pool 2 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: inline_data disabled 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: balancer 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: bal_rank_mask -1 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: standby_count_wanted 1 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: [mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: [mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: Standby daemons: 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: [mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:16.743 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout: [mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:16.744 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: mon.smithi163@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-16T19:27:16.744 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: mon.smithi163@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:27:16.744 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: mon.smithi163@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:27:16.744 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: mon.smithi163@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-16T19:27:16.744 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: mon.smithi163@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-02-16T19:27:16.744 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 bash[81619]: 4840eea014c332aab5f5fcbb225fe81a40e6edd3c74d6cdd76fc63cc133a6cbf 2024-02-16T19:27:16.744 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 systemd[1]: Started Ceph mon.smithi163 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:27:16.744 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:16 smithi163 ceph-mon[81637]: pgmap v9: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-16T19:27:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.403+0000 7fe550777700 1 -- 172.21.15.40:0/2967261683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fbe40 msgr2=0x7fe5480fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.403+0000 7fe550777700 1 --2- 172.21.15.40:0/2967261683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fbe40 0x7fe5480fc250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe544004940 tx=0x7fe544009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 -- 172.21.15.40:0/2967261683 shutdown_connections 2024-02-16T19:27:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 --2- 172.21.15.40:0/2967261683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fcfb0 0x7fe5480fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 --2- 172.21.15.40:0/2967261683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fbe40 0x7fe5480fc250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 -- 172.21.15.40:0/2967261683 >> 172.21.15.40:0/2967261683 conn(0x7fe5480f74e0 msgr2=0x7fe5480f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 -- 172.21.15.40:0/2967261683 shutdown_connections 2024-02-16T19:27:17.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 -- 172.21.15.40:0/2967261683 wait complete. 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 Processor -- start 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 -- start start 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 0x7fe548112a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.404+0000 7fe550777700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fcfb0 0x7fe548114fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe550777700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe548066350 con 0x7fe5480fcfb0 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe550777700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe548066490 con 0x7fe5480fbe40 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54dd12700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fcfb0 0x7fe548114fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54dd12700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fcfb0 0x7fe548114fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44306/0 (socket says 172.21.15.40:44306) 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54dd12700 1 -- 172.21.15.40:0/2142695325 learned_addr learned my addr 172.21.15.40:0/2142695325 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54e513700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 0x7fe548112a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54e513700 1 -- 172.21.15.40:0/2142695325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 msgr2=0x7fe548112a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54e513700 1 -- 172.21.15.40:0/2142695325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 msgr2=0x7fe548112a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54e513700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 0x7fe548112a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-16T19:27:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54dd12700 1 -- 172.21.15.40:0/2142695325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 msgr2=0x7fe548112a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54e513700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 0x7fe548112a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54dd12700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 0x7fe548112a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54dd12700 1 -- 172.21.15.40:0/2142695325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544005040 con 0x7fe5480fcfb0 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe54dd12700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fcfb0 0x7fe548114fb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe538002a50 tx=0x7fe5380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe53effd700 1 -- 172.21.15.40:0/2142695325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe538007cc0 con 0x7fe5480fcfb0 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe53effd700 1 -- 172.21.15.40:0/2142695325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe53800f040 con 0x7fe5480fcfb0 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5481154f0 con 0x7fe5480fcfb0 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe53effd700 1 -- 172.21.15.40:0/2142695325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe538009640 con 0x7fe5480fcfb0 2024-02-16T19:27:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.405+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5481158f0 con 0x7fe5480fcfb0 2024-02-16T19:27:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.406+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54804bed0 con 0x7fe5480fcfb0 2024-02-16T19:27:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.407+0000 7fe53effd700 1 -- 172.21.15.40:0/2142695325 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fe5380097c0 con 0x7fe5480fcfb0 2024-02-16T19:27:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.407+0000 7fe53effd700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe53406d900 0x7fe53406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.407+0000 7fe53effd700 1 -- 172.21.15.40:0/2142695325 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe53808ced0 con 0x7fe5480fcfb0 2024-02-16T19:27:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.407+0000 7fe54e513700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe53406d900 0x7fe53406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.408+0000 7fe54e513700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe53406d900 0x7fe53406fdb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe54400b3b0 tx=0x7fe544009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:17.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.412+0000 7fe53effd700 1 -- 172.21.15.40:0/2142695325 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe538056500 con 0x7fe5480fcfb0 2024-02-16T19:27:17.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.636+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe548101990 con 0x7fe53406d900 2024-02-16T19:27:17.637 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: pgmap v10: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:27:17.637 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: mon.smithi040 calling monitor election 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: mon.smithi163 calling monitor election 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: mon.smithi040 is new leader, mons smithi040,smithi163 in quorum (ranks 0,1) 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: monmap e3: 2 mons at {smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],smithi163=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: mgrmap e37: smithi040.jzygzo(active, since 14s), standbys: smithi163.wjfclz 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: overall HEALTH_OK 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:17.638 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:17.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.637+0000 7fe53effd700 1 -- 172.21.15.40:0/2142695325 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe548101990 con 0x7fe53406d900 2024-02-16T19:27:17.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe53406d900 msgr2=0x7fe53406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:17.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe53406d900 0x7fe53406fdb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe54400b3b0 tx=0x7fe544009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:17.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fcfb0 msgr2=0x7fe548114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:17.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fcfb0 0x7fe548114fb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe538002a50 tx=0x7fe5380046b0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 shutdown_connections 2024-02-16T19:27:17.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5480fbe40 0x7fe548112a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe53406d900 0x7fe53406fdb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 --2- 172.21.15.40:0/2142695325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5480fcfb0 0x7fe548114fb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 >> 172.21.15.40:0/2142695325 conn(0x7fe5480f74e0 msgr2=0x7fe548100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:17.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 shutdown_connections 2024-02-16T19:27:17.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.639+0000 7fe550777700 1 -- 172.21.15.40:0/2142695325 wait complete. 2024-02-16T19:27:17.650 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:27:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: pgmap v10: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:27:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: mon.smithi040 calling monitor election 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: mon.smithi163 calling monitor election 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: mon.smithi040 is new leader, mons smithi040,smithi163 in quorum (ranks 0,1) 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: monmap e3: 2 mons at {smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],smithi163=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: osdmap e49: 6 total, 6 up, 6 in 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: mgrmap e37: smithi040.jzygzo(active, since 14s), standbys: smithi163.wjfclz 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: overall HEALTH_OK 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:17.739 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.742+0000 7f12c0909700 1 -- 172.21.15.40:0/1175450731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 msgr2=0x7f12bc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.742+0000 7f12c0909700 1 --2- 172.21.15.40:0/1175450731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 0x7f12bc0fd320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f12ac004940 tx=0x7f12ac009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.742+0000 7f12c0909700 1 -- 172.21.15.40:0/1175450731 shutdown_connections 2024-02-16T19:27:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.742+0000 7f12c0909700 1 --2- 172.21.15.40:0/1175450731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 0x7f12bc0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.742+0000 7f12c0909700 1 --2- 172.21.15.40:0/1175450731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 0x7f12bc0fd320 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.742+0000 7f12c0909700 1 -- 172.21.15.40:0/1175450731 >> 172.21.15.40:0/1175450731 conn(0x7f12bc0f8620 msgr2=0x7f12bc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.742+0000 7f12c0909700 1 -- 172.21.15.40:0/1175450731 shutdown_connections 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.742+0000 7f12c0909700 1 -- 172.21.15.40:0/1175450731 wait complete. 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12c0909700 1 Processor -- start 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12c0909700 1 -- start start 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12c0909700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 0x7f12bc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12c0909700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 0x7f12bc0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12c0909700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc0708e0 con 0x7f12bc0fcef0 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12c0909700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc070a50 con 0x7f12bc0fe0e0 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12ba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 0x7f12bc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12ba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 0x7f12bc071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34728/0 (socket says 172.21.15.40:34728) 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12ba59c700 1 -- 172.21.15.40:0/1595940744 learned_addr learned my addr 172.21.15.40:0/1595940744 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12b9d9b700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 0x7f12bc0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12b9d9b700 1 -- 172.21.15.40:0/1595940744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 msgr2=0x7f12bc0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12b9d9b700 1 -- 172.21.15.40:0/1595940744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 msgr2=0x7f12bc0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12b9d9b700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 0x7f12bc0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12ba59c700 1 -- 172.21.15.40:0/1595940744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 msgr2=0x7f12bc0702e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.743+0000 7f12b9d9b700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 0x7f12bc0702e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.744+0000 7f12ba59c700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 0x7f12bc0702e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.744+0000 7f12ba59c700 1 -- 172.21.15.40:0/1595940744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac005040 con 0x7f12bc0fcef0 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.744+0000 7f12ba59c700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 0x7f12bc071e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009360 tx=0x7f12ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.744+0000 7f12aaffd700 1 -- 172.21.15.40:0/1595940744 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ac00e040 con 0x7f12bc0fcef0 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.744+0000 7f12aaffd700 1 -- 172.21.15.40:0/1595940744 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12ac004290 con 0x7f12bc0fcef0 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.744+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc070cd0 con 0x7f12bc0fcef0 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.744+0000 7f12aaffd700 1 -- 172.21.15.40:0/1595940744 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ac01c630 con 0x7f12bc0fcef0 2024-02-16T19:27:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.744+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc1159a0 con 0x7f12bc0fcef0 2024-02-16T19:27:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.745+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12bc04bed0 con 0x7f12bc0fcef0 2024-02-16T19:27:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.745+0000 7f12aaffd700 1 -- 172.21.15.40:0/1595940744 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f12ac00bcc0 con 0x7f12bc0fcef0 2024-02-16T19:27:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.746+0000 7f12aaffd700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f12a406d900 0x7f12a406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.746+0000 7f12b9d9b700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f12a406d900 0x7f12a406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.746+0000 7f12aaffd700 1 -- 172.21.15.40:0/1595940744 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f12ac08e570 con 0x7f12bc0fcef0 2024-02-16T19:27:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.746+0000 7f12b9d9b700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f12a406d900 0x7f12a406fdb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009e50 tx=0x7f12b0009f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.750+0000 7f12aaffd700 1 -- 172.21.15.40:0/1595940744 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12ac05b750 con 0x7f12bc0fcef0 2024-02-16T19:27:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.972+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12bc102ac0 con 0x7f12a406d900 2024-02-16T19:27:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.973+0000 7f12aaffd700 1 -- 172.21.15.40:0/1595940744 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f12bc102ac0 con 0x7f12a406d900 2024-02-16T19:27:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f12a406d900 msgr2=0x7f12a406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f12a406d900 0x7f12a406fdb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009e50 tx=0x7f12b0009f10 comp rx=0 tx=0).stop 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 msgr2=0x7f12bc071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 0x7f12bc071e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009360 tx=0x7f12ac009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 shutdown_connections 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f12a406d900 0x7f12a406fdb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12bc0fcef0 0x7f12bc071e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 --2- 172.21.15.40:0/1595940744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12bc0fe0e0 0x7f12bc0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 >> 172.21.15.40:0/1595940744 conn(0x7f12bc0f8620 msgr2=0x7f12bc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 shutdown_connections 2024-02-16T19:27:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:17.975+0000 7f12c0909700 1 -- 172.21.15.40:0/1595940744 wait complete. 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.077+0000 7f9135be1700 1 -- 172.21.15.40:0/3414610216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 msgr2=0x7f91300fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.077+0000 7f9135be1700 1 --2- 172.21.15.40:0/3414610216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 0x7f91300fd840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9120004940 tx=0x7f9120009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.077+0000 7f9135be1700 1 -- 172.21.15.40:0/3414610216 shutdown_connections 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.077+0000 7f9135be1700 1 --2- 172.21.15.40:0/3414610216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91300fde10 0x7f91300faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.077+0000 7f9135be1700 1 --2- 172.21.15.40:0/3414610216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 0x7f91300fd840 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.077+0000 7f9135be1700 1 -- 172.21.15.40:0/3414610216 >> 172.21.15.40:0/3414610216 conn(0x7f91300f6ca0 msgr2=0x7f91300f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.077+0000 7f9135be1700 1 -- 172.21.15.40:0/3414610216 shutdown_connections 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.077+0000 7f9135be1700 1 -- 172.21.15.40:0/3414610216 wait complete. 2024-02-16T19:27:18.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f9135be1700 1 Processor -- start 2024-02-16T19:27:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f9135be1700 1 -- start start 2024-02-16T19:27:18.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f9135be1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 0x7f913010fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f9135be1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 0x7f9130110f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f9135be1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91301106d0 con 0x7f91300fd430 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f9135be1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130110810 con 0x7f9130110210 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f912f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 0x7f913010fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f912f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 0x7f913010fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34742/0 (socket says 172.21.15.40:34742) 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f912f7fe700 1 -- 172.21.15.40:0/4122999726 learned_addr learned my addr 172.21.15.40:0/4122999726 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.078+0000 7f912effd700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 0x7f9130110f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f912effd700 1 -- 172.21.15.40:0/4122999726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 msgr2=0x7f9130110f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f912effd700 1 -- 172.21.15.40:0/4122999726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 msgr2=0x7f9130110f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f912effd700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 0x7f9130110f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f912f7fe700 1 -- 172.21.15.40:0/4122999726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 msgr2=0x7f9130110f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f912effd700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 0x7f9130110f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f912f7fe700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 0x7f9130110f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f912f7fe700 1 -- 172.21.15.40:0/4122999726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9120005040 con 0x7f91300fd430 2024-02-16T19:27:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f912f7fe700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 0x7f913010fcd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f912000b3b0 tx=0x7f9120009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f9117fff700 1 -- 172.21.15.40:0/4122999726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912000e040 con 0x7f91300fd430 2024-02-16T19:27:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f9117fff700 1 -- 172.21.15.40:0/4122999726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f912000bbf0 con 0x7f91300fd430 2024-02-16T19:27:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91301115f0 con 0x7f91300fd430 2024-02-16T19:27:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f9117fff700 1 -- 172.21.15.40:0/4122999726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912001c410 con 0x7f91300fd430 2024-02-16T19:27:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.079+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9130111b30 con 0x7f91300fd430 2024-02-16T19:27:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.080+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9130109ca0 con 0x7f91300fd430 2024-02-16T19:27:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.080+0000 7f9117fff700 1 -- 172.21.15.40:0/4122999726 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f912000bd60 con 0x7f91300fd430 2024-02-16T19:27:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.081+0000 7f9117fff700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f911806d900 0x7f911806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.081+0000 7f9117fff700 1 -- 172.21.15.40:0/4122999726 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f912008e580 con 0x7f91300fd430 2024-02-16T19:27:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.081+0000 7f912effd700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f911806d900 0x7f911806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.081+0000 7f912effd700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f911806d900 0x7f911806fdb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9124007fd0 tx=0x7f9124009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.086+0000 7f9117fff700 1 -- 172.21.15.40:0/4122999726 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f912005b6e0 con 0x7f91300fd430 2024-02-16T19:27:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.302+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f91300fde10 con 0x7f911806d900 2024-02-16T19:27:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.313+0000 7f9117fff700 1 -- 172.21.15.40:0/4122999726 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f91300fde10 con 0x7f911806d900 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (2m) 10s ago 10m 16.6M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (11m) 10s ago 11m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (10m) 13s ago 9m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (85s) 10s ago 10m 80.1M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (5m) 10s ago 5m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (5m) 10s ago 5m 13.0M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (5m) 13s ago 5m 16.6M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (5m) 13s ago 5m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (72s) 10s ago 12m 471M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (39s) 13s ago 9m 446M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (25s) 10s ago 12m 28.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:27:18.315 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 starting - - - 2048M 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (2m) 10s ago 11m 8090k - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (77s) 13s ago 9m 15.9M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (9m) 10s ago 9m 227M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (8m) 10s ago 8m 242M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (8m) 10s ago 8m 197M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (7m) 13s ago 7m 270M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (7m) 13s ago 7m 232M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (6m) 13s ago 6m 235M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:27:18.316 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (44s) 10s ago 10m 46.0M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:27:18.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f911806d900 msgr2=0x7f911806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f911806d900 0x7f911806fdb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9124007fd0 tx=0x7f9124009040 comp rx=0 tx=0).stop 2024-02-16T19:27:18.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 msgr2=0x7f913010fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 0x7f913010fcd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f912000b3b0 tx=0x7f9120009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:18.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 shutdown_connections 2024-02-16T19:27:18.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f911806d900 0x7f911806fdb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91300fd430 0x7f913010fcd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 --2- 172.21.15.40:0/4122999726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9130110210 0x7f9130110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 >> 172.21.15.40:0/4122999726 conn(0x7f91300f6ca0 msgr2=0x7f91300f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:18.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 shutdown_connections 2024-02-16T19:27:18.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.316+0000 7f9135be1700 1 -- 172.21.15.40:0/4122999726 wait complete. 2024-02-16T19:27:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 -- 172.21.15.40:0/2011054514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fbe40 msgr2=0x7fe6640fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 --2- 172.21.15.40:0/2011054514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fbe40 0x7fe6640fc250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe660009ce0 tx=0x7fe660004d10 comp rx=0 tx=0).stop 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 -- 172.21.15.40:0/2011054514 shutdown_connections 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 --2- 172.21.15.40:0/2011054514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fcfb0 0x7fe6640fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 --2- 172.21.15.40:0/2011054514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fbe40 0x7fe6640fc250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 -- 172.21.15.40:0/2011054514 >> 172.21.15.40:0/2011054514 conn(0x7fe6640f7500 msgr2=0x7fe6640f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 -- 172.21.15.40:0/2011054514 shutdown_connections 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 -- 172.21.15.40:0/2011054514 wait complete. 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 Processor -- start 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.418+0000 7fe66b751700 1 -- start start 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe66b751700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 0x7fe66410daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe66b751700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fcfb0 0x7fe66410dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe66b751700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66410e5e0 con 0x7fe6640fcfb0 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe66b751700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66410ee80 con 0x7fe6640fbe40 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe668cec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fcfb0 0x7fe66410dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe668cec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fcfb0 0x7fe66410dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34768/0 (socket says 172.21.15.40:34768) 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe668cec700 1 -- 172.21.15.40:0/959265364 learned_addr learned my addr 172.21.15.40:0/959265364 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe6694ed700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 0x7fe66410daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe6694ed700 1 -- 172.21.15.40:0/959265364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 msgr2=0x7fe66410daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe6694ed700 1 -- 172.21.15.40:0/959265364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 msgr2=0x7fe66410daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe668cec700 1 -- 172.21.15.40:0/959265364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 msgr2=0x7fe66410daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe6694ed700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 0x7fe66410daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe6694ed700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 0x7fe66410daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe668cec700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 0x7fe66410daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe668cec700 1 -- 172.21.15.40:0/959265364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe660009940 con 0x7fe6640fcfb0 2024-02-16T19:27:18.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe668cec700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fcfb0 0x7fe66410dfe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe65400c830 tx=0x7fe654004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:18.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe659ffb700 1 -- 172.21.15.40:0/959265364 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65400a760 con 0x7fe6640fcfb0 2024-02-16T19:27:18.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe659ffb700 1 -- 172.21.15.40:0/959265364 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe65400ad90 con 0x7fe6640fcfb0 2024-02-16T19:27:18.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66410f160 con 0x7fe6640fcfb0 2024-02-16T19:27:18.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.419+0000 7fe659ffb700 1 -- 172.21.15.40:0/959265364 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6540113a0 con 0x7fe6640fcfb0 2024-02-16T19:27:18.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.420+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66410f6a0 con 0x7fe6640fcfb0 2024-02-16T19:27:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.421+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66404bed0 con 0x7fe6640fcfb0 2024-02-16T19:27:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.421+0000 7fe659ffb700 1 -- 172.21.15.40:0/959265364 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fe654012400 con 0x7fe6640fcfb0 2024-02-16T19:27:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.421+0000 7fe659ffb700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d900 0x7fe65006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.421+0000 7fe659ffb700 1 -- 172.21.15.40:0/959265364 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe65408d080 con 0x7fe6640fcfb0 2024-02-16T19:27:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.421+0000 7fe6694ed700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d900 0x7fe65006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.422+0000 7fe6694ed700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d900 0x7fe65006fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe66000b740 tx=0x7fe660004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:18.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.426+0000 7fe659ffb700 1 -- 172.21.15.40:0/959265364 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe65405a370 con 0x7fe6640fcfb0 2024-02-16T19:27:18.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:18 smithi040 ceph-mon[94210]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:18.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:18 smithi040 ceph-mon[94210]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:18.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.715+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe664104930 con 0x7fe6640fcfb0 2024-02-16T19:27:18.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.716+0000 7fe659ffb700 1 -- 172.21.15.40:0/959265364 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fe654059f00 con 0x7fe6640fcfb0 2024-02-16T19:27:18.717 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:27:18.717 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:27:18.717 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:27:18.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:27:18.717 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:27:18.717 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:27:18.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 10, 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:27:18.718 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:27:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d900 msgr2=0x7fe65006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d900 0x7fe65006fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe66000b740 tx=0x7fe660004d10 comp rx=0 tx=0).stop 2024-02-16T19:27:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fcfb0 msgr2=0x7fe66410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fcfb0 0x7fe66410dfe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe65400c830 tx=0x7fe654004470 comp rx=0 tx=0).stop 2024-02-16T19:27:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 shutdown_connections 2024-02-16T19:27:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6640fbe40 0x7fe66410daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d900 0x7fe65006fdb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 --2- 172.21.15.40:0/959265364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6640fcfb0 0x7fe66410dfe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 >> 172.21.15.40:0/959265364 conn(0x7fe6640f7500 msgr2=0x7fe664065f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:18.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 shutdown_connections 2024-02-16T19:27:18.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.718+0000 7fe66b751700 1 -- 172.21.15.40:0/959265364 wait complete. 2024-02-16T19:27:18.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:18 smithi163 ceph-mon[81637]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:18.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:18 smithi163 ceph-mon[81637]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 -- 172.21.15.40:0/4147801928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 msgr2=0x7fbb280fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 --2- 172.21.15.40:0/4147801928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 0x7fbb280fb9f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009ce0 tx=0x7fbb18004d10 comp rx=0 tx=0).stop 2024-02-16T19:27:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 -- 172.21.15.40:0/4147801928 shutdown_connections 2024-02-16T19:27:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 --2- 172.21.15.40:0/4147801928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb280fc7f0 0x7fbb280fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 --2- 172.21.15.40:0/4147801928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 0x7fbb280fb9f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 -- 172.21.15.40:0/4147801928 >> 172.21.15.40:0/4147801928 conn(0x7fbb280f6ca0 msgr2=0x7fbb280f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:18.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 -- 172.21.15.40:0/4147801928 shutdown_connections 2024-02-16T19:27:18.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 -- 172.21.15.40:0/4147801928 wait complete. 2024-02-16T19:27:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.820+0000 7fbb2d888700 1 Processor -- start 2024-02-16T19:27:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb2d888700 1 -- start start 2024-02-16T19:27:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb2d888700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 0x7fbb281007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb2d888700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb280fc7f0 0x7fbb280fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb2d888700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280ff2d0 con 0x7fbb280fb5e0 2024-02-16T19:27:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb2d888700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280ff440 con 0x7fbb280fc7f0 2024-02-16T19:27:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb26ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 0x7fbb281007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb26ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 0x7fbb281007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34776/0 (socket says 172.21.15.40:34776) 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb26ffd700 1 -- 172.21.15.40:0/253263183 learned_addr learned my addr 172.21.15.40:0/253263183 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb26ffd700 1 -- 172.21.15.40:0/253263183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb280fc7f0 msgr2=0x7fbb280fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb26ffd700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb280fc7f0 0x7fbb280fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb26ffd700 1 -- 172.21.15.40:0/253263183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18009940 con 0x7fbb280fb5e0 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb26ffd700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 0x7fbb281007c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800bf90 tx=0x7fbb18004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb1f7fe700 1 -- 172.21.15.40:0/253263183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb1800e040 con 0x7fbb280fb5e0 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb280ff6c0 con 0x7fbb280fb5e0 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb1f7fe700 1 -- 172.21.15.40:0/253263183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb180128e0 con 0x7fbb280fb5e0 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.821+0000 7fbb1f7fe700 1 -- 172.21.15.40:0/253263183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb1801c990 con 0x7fbb280fb5e0 2024-02-16T19:27:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.822+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb28115bc0 con 0x7fbb280fb5e0 2024-02-16T19:27:18.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.823+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb28108190 con 0x7fbb280fb5e0 2024-02-16T19:27:18.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.823+0000 7fbb1f7fe700 1 -- 172.21.15.40:0/253263183 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fbb18012a50 con 0x7fbb280fb5e0 2024-02-16T19:27:18.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.823+0000 7fbb1f7fe700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb1406d900 0x7fbb1406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:18.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.824+0000 7fbb1f7fe700 1 -- 172.21.15.40:0/253263183 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbb1808ef90 con 0x7fbb280fb5e0 2024-02-16T19:27:18.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.824+0000 7fbb267fc700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb1406d900 0x7fbb1406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:18.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.824+0000 7fbb267fc700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb1406d900 0x7fbb1406fdb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006fd0 tx=0x7fbb10006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:18.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:18.828+0000 7fbb1f7fe700 1 -- 172.21.15.40:0/253263183 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb1805c050 con 0x7fbb280fb5e0 2024-02-16T19:27:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.080+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbb28104140 con 0x7fbb280fb5e0 2024-02-16T19:27:19.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.080+0000 7fbb1f7fe700 1 -- 172.21.15.40:0/253263183 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1892 (secure 0 0 0) 0x7fbb18012d00 con 0x7fbb280fb5e0 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:e19 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 19 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:24:09.512876+0000 2024-02-16T19:27:19.082 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:27:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:27:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:27:19.085 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:27:19.085 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:19.085 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:19.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb1406d900 msgr2=0x7fbb1406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb1406d900 0x7fbb1406fdb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006fd0 tx=0x7fbb10006db0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 msgr2=0x7fbb281007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 0x7fbb281007c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800bf90 tx=0x7fbb18004d10 comp rx=0 tx=0).stop 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 shutdown_connections 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb1406d900 0x7fbb1406fdb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb280fb5e0 0x7fbb281007c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 --2- 172.21.15.40:0/253263183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb280fc7f0 0x7fbb280fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.082+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 >> 172.21.15.40:0/253263183 conn(0x7fbb280f6ca0 msgr2=0x7fbb280f8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.083+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 shutdown_connections 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.083+0000 7fbb2d888700 1 -- 172.21.15.40:0/253263183 wait complete. 2024-02-16T19:27:19.086 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 19 2024-02-16T19:27:19.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.186+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529114959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 msgr2=0x7f11640fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.186+0000 7f1169dd5700 1 --2- 172.21.15.40:0/1529114959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 0x7f11640fd320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1154004940 tx=0x7f1154009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:19.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.186+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529114959 shutdown_connections 2024-02-16T19:27:19.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.186+0000 7f1169dd5700 1 --2- 172.21.15.40:0/1529114959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11640fe0e0 0x7f11640fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.186+0000 7f1169dd5700 1 --2- 172.21.15.40:0/1529114959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 0x7f11640fd320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.186+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529114959 >> 172.21.15.40:0/1529114959 conn(0x7f11640f8620 msgr2=0x7f11640faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:19.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.186+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529114959 shutdown_connections 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.186+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529114959 wait complete. 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f1169dd5700 1 Processor -- start 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f1169dd5700 1 -- start start 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f1169dd5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 0x7f1164071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f1169dd5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11640fe0e0 0x7f11640702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f1169dd5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640708e0 con 0x7f11640fcef0 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f1169dd5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1164070a50 con 0x7f11640fe0e0 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f11637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 0x7f1164071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f11637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 0x7f1164071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34790/0 (socket says 172.21.15.40:34790) 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f11637fe700 1 -- 172.21.15.40:0/1529839016 learned_addr learned my addr 172.21.15.40:0/1529839016 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f1162ffd700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11640fe0e0 0x7f11640702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f11637fe700 1 -- 172.21.15.40:0/1529839016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11640fe0e0 msgr2=0x7f11640702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f11637fe700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11640fe0e0 0x7f11640702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f11637fe700 1 -- 172.21.15.40:0/1529839016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154005040 con 0x7f11640fcef0 2024-02-16T19:27:19.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.187+0000 7f1162ffd700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11640fe0e0 0x7f11640702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:27:19.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.188+0000 7f11637fe700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 0x7f1164071e60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f115400b3b0 tx=0x7f1154009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:19.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.188+0000 7f114bfff700 1 -- 172.21.15.40:0/1529839016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f115400e040 con 0x7f11640fcef0 2024-02-16T19:27:19.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.188+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1164070cd0 con 0x7f11640fcef0 2024-02-16T19:27:19.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.188+0000 7f114bfff700 1 -- 172.21.15.40:0/1529839016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f115400bcc0 con 0x7f11640fcef0 2024-02-16T19:27:19.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.188+0000 7f114bfff700 1 -- 172.21.15.40:0/1529839016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f115401c630 con 0x7f11640fcef0 2024-02-16T19:27:19.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.188+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11641159a0 con 0x7f11640fcef0 2024-02-16T19:27:19.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.189+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116404bed0 con 0x7f11640fcef0 2024-02-16T19:27:19.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.189+0000 7f114bfff700 1 -- 172.21.15.40:0/1529839016 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f11540139f0 con 0x7f11640fcef0 2024-02-16T19:27:19.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.190+0000 7f114bfff700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f114c06d900 0x7f114c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:19.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.190+0000 7f114bfff700 1 -- 172.21.15.40:0/1529839016 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f115408e5c0 con 0x7f11640fcef0 2024-02-16T19:27:19.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.190+0000 7f1162ffd700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f114c06d900 0x7f114c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:19.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.190+0000 7f1162ffd700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f114c06d900 0x7f114c06fdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1158007fd0 tx=0x7f1158009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:19.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.194+0000 7f114bfff700 1 -- 172.21.15.40:0/1529839016 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f115405b7a0 con 0x7f11640fcef0 2024-02-16T19:27:19.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.419+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1164102ac0 con 0x7f114c06d900 2024-02-16T19:27:19.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.420+0000 7f114bfff700 1 -- 172.21.15.40:0/1529839016 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f1164102ac0 con 0x7f114c06d900 2024-02-16T19:27:19.421 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "3/21 daemons upgraded", 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading mon daemons", 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:27:19.422 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f114c06d900 msgr2=0x7f114c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f114c06d900 0x7f114c06fdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1158007fd0 tx=0x7f1158009040 comp rx=0 tx=0).stop 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 msgr2=0x7f1164071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 0x7f1164071e60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f115400b3b0 tx=0x7f1154009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 shutdown_connections 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f114c06d900 0x7f114c06fdb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11640fcef0 0x7f1164071e60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 --2- 172.21.15.40:0/1529839016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11640fe0e0 0x7f11640702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.422+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 >> 172.21.15.40:0/1529839016 conn(0x7f11640f8620 msgr2=0x7f11641013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.423+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 shutdown_connections 2024-02-16T19:27:19.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.423+0000 7f1169dd5700 1 -- 172.21.15.40:0/1529839016 wait complete. 2024-02-16T19:27:19.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 -- 172.21.15.40:0/3112223392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 msgr2=0x7fab800fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 --2- 172.21.15.40:0/3112223392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 0x7fab800fe550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fab68004940 tx=0x7fab68009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 -- 172.21.15.40:0/3112223392 shutdown_connections 2024-02-16T19:27:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 --2- 172.21.15.40:0/3112223392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 0x7fab800fe550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 --2- 172.21.15.40:0/3112223392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab800fcef0 0x7fab800fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 -- 172.21.15.40:0/3112223392 >> 172.21.15.40:0/3112223392 conn(0x7fab800f8620 msgr2=0x7fab800faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 -- 172.21.15.40:0/3112223392 shutdown_connections 2024-02-16T19:27:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 -- 172.21.15.40:0/3112223392 wait complete. 2024-02-16T19:27:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.526+0000 7fab85ed8700 1 Processor -- start 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab85ed8700 1 -- start start 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab85ed8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab800fcef0 0x7fab8010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab85ed8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 0x7fab8010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab85ed8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8010e4d0 con 0x7fab800fe0e0 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab85ed8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8010ed70 con 0x7fab800fcef0 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 0x7fab8010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 0x7fab8010ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34808/0 (socket says 172.21.15.40:34808) 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab7effd700 1 -- 172.21.15.40:0/738452102 learned_addr learned my addr 172.21.15.40:0/738452102 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab7f7fe700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab800fcef0 0x7fab8010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab7effd700 1 -- 172.21.15.40:0/738452102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab800fcef0 msgr2=0x7fab8010d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab7effd700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab800fcef0 0x7fab8010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab7effd700 1 -- 172.21.15.40:0/738452102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab68005040 con 0x7fab800fe0e0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab7effd700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 0x7fab8010ded0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fab680093b0 tx=0x7fab68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab77fff700 1 -- 172.21.15.40:0/738452102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6800e040 con 0x7fab800fe0e0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.527+0000 7fab77fff700 1 -- 172.21.15.40:0/738452102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab6800bcc0 con 0x7fab800fe0e0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.528+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8010eff0 con 0x7fab800fe0e0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.528+0000 7fab77fff700 1 -- 172.21.15.40:0/738452102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6801c630 con 0x7fab800fe0e0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.528+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8010f530 con 0x7fab800fe0e0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.528+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8004bed0 con 0x7fab800fe0e0 2024-02-16T19:27:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.529+0000 7fab77fff700 1 -- 172.21.15.40:0/738452102 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fab680139f0 con 0x7fab800fe0e0 2024-02-16T19:27:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.529+0000 7fab77fff700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab6c06d900 0x7fab6c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.530+0000 7fab77fff700 1 -- 172.21.15.40:0/738452102 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fab6808e5b0 con 0x7fab800fe0e0 2024-02-16T19:27:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.530+0000 7fab7f7fe700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab6c06d900 0x7fab6c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.530+0000 7fab7f7fe700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab6c06d900 0x7fab6c06fdb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab800fcc30 tx=0x7fab7000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:19.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.534+0000 7fab77fff700 1 -- 172.21.15.40:0/738452102 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab6805b790 con 0x7fab800fe0e0 2024-02-16T19:27:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:19 smithi163 ceph-mon[81637]: pgmap v11: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:27:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:19 smithi163 ceph-mon[81637]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:19 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/959265364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:19 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/253263183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:27:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.816+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab80105a50 con 0x7fab800fe0e0 2024-02-16T19:27:19.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:19 smithi040 ceph-mon[94210]: pgmap v11: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:27:19.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:19 smithi040 ceph-mon[94210]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:19.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:19 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/959265364' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:19.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:19 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/253263183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:27:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.817+0000 7fab77fff700 1 -- 172.21.15.40:0/738452102 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fab6805b320 con 0x7fab800fe0e0 2024-02-16T19:27:19.819 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab6c06d900 msgr2=0x7fab6c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab6c06d900 0x7fab6c06fdb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab800fcc30 tx=0x7fab7000b040 comp rx=0 tx=0).stop 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 msgr2=0x7fab8010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 0x7fab8010ded0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fab680093b0 tx=0x7fab68009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 shutdown_connections 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab800fcef0 0x7fab8010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab6c06d900 0x7fab6c06fdb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 --2- 172.21.15.40:0/738452102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab800fe0e0 0x7fab8010ded0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 >> 172.21.15.40:0/738452102 conn(0x7fab800f8620 msgr2=0x7fab801013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 shutdown_connections 2024-02-16T19:27:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:19.820+0000 7fab85ed8700 1 -- 172.21.15.40:0/738452102 wait complete. 2024-02-16T19:27:20.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:20 smithi163 ceph-mon[81637]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:20.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:20 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/738452102' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:27:20.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:20 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:20.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:20 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:20.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:20 smithi040 ceph-mon[94210]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:20.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:20 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/738452102' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:27:20.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:20 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:20.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:20 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:22 smithi040 ceph-mon[94210]: pgmap v12: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:27:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:22 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:22 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:22 smithi163 ceph-mon[81637]: pgmap v12: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-16T19:27:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:22 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:22 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:24.364 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:24 smithi163 ceph-mon[81637]: pgmap v13: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:24.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:24 smithi040 ceph-mon[94210]: pgmap v13: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:25.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: pgmap v14: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:25.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:25.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:25.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:25.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:27:25.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:25.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: Reconfiguring mon.smithi040 (monmap changed)... 2024-02-16T19:27:25.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:27:25.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:27:25.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:25.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:25 smithi163 ceph-mon[81637]: Reconfiguring daemon mon.smithi040 on smithi040 2024-02-16T19:27:26.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: pgmap v14: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:26.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:26.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:26.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:26.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:27:26.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:26.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: Reconfiguring mon.smithi040 (monmap changed)... 2024-02-16T19:27:26.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:27:26.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:27:26.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:26.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:25 smithi040 ceph-mon[94210]: Reconfiguring daemon mon.smithi040 on smithi040 2024-02-16T19:27:27.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:27 smithi040 ceph-mon[94210]: pgmap v15: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:27 smithi040 ceph-mon[94210]: Reconfiguring mgr.smithi040.jzygzo (monmap changed)... 2024-02-16T19:27:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.jzygzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:27:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:27:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:27 smithi040 ceph-mon[94210]: Reconfiguring daemon mgr.smithi040.jzygzo on smithi040 2024-02-16T19:27:28.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:27 smithi163 ceph-mon[81637]: pgmap v15: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:28.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:28.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:28.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:27 smithi163 ceph-mon[81637]: Reconfiguring mgr.smithi040.jzygzo (monmap changed)... 2024-02-16T19:27:28.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.jzygzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:27:28.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:27:28.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:28.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:27 smithi163 ceph-mon[81637]: Reconfiguring daemon mgr.smithi040.jzygzo on smithi040 2024-02-16T19:27:29.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:29 smithi040 ceph-mon[94210]: pgmap v16: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:29.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:29.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:29.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:29 smithi040 ceph-mon[94210]: Reconfiguring crash.smithi040 (monmap changed)... 2024-02-16T19:27:29.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:27:29.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:29.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:29 smithi040 ceph-mon[94210]: Reconfiguring daemon crash.smithi040 on smithi040 2024-02-16T19:27:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:29 smithi163 ceph-mon[81637]: pgmap v16: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:29 smithi163 ceph-mon[81637]: Reconfiguring crash.smithi040 (monmap changed)... 2024-02-16T19:27:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:27:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:29 smithi163 ceph-mon[81637]: Reconfiguring daemon crash.smithi040 on smithi040 2024-02-16T19:27:31.347 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:31.347 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:31.347 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:31 smithi040 ceph-mon[94210]: Reconfiguring osd.0 (monmap changed)... 2024-02-16T19:27:31.347 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:27:31.347 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:31.347 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:31 smithi040 ceph-mon[94210]: Reconfiguring daemon osd.0 on smithi040 2024-02-16T19:27:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:31 smithi163 ceph-mon[81637]: Reconfiguring osd.0 (monmap changed)... 2024-02-16T19:27:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:27:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:31 smithi163 ceph-mon[81637]: Reconfiguring daemon osd.0 on smithi040 2024-02-16T19:27:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:32 smithi040 ceph-mon[94210]: pgmap v17: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:27:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:32.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:32 smithi163 ceph-mon[81637]: pgmap v17: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:32.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:32.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:32.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:27:32.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:33.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:33 smithi040 ceph-mon[94210]: Reconfiguring osd.1 (monmap changed)... 2024-02-16T19:27:33.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:33 smithi040 ceph-mon[94210]: Reconfiguring daemon osd.1 on smithi040 2024-02-16T19:27:33.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:33 smithi163 ceph-mon[81637]: Reconfiguring osd.1 (monmap changed)... 2024-02-16T19:27:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:33 smithi163 ceph-mon[81637]: Reconfiguring daemon osd.1 on smithi040 2024-02-16T19:27:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:34.388 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:34 smithi040 ceph-mon[94210]: pgmap v18: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:34.388 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:34.388 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:34.388 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:27:34.389 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:34 smithi163 ceph-mon[81637]: pgmap v18: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:27:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:35.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:35 smithi040 ceph-mon[94210]: Reconfiguring osd.2 (monmap changed)... 2024-02-16T19:27:35.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:35 smithi040 ceph-mon[94210]: Reconfiguring daemon osd.2 on smithi040 2024-02-16T19:27:35.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:35 smithi163 ceph-mon[81637]: Reconfiguring osd.2 (monmap changed)... 2024-02-16T19:27:35.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:35 smithi163 ceph-mon[81637]: Reconfiguring daemon osd.2 on smithi040 2024-02-16T19:27:36.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:36 smithi163 ceph-mon[81637]: pgmap v19: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:36.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:36 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:36.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:36 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:36.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:36 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.ezkppl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:27:36.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:36 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:36 smithi040 ceph-mon[94210]: pgmap v19: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:36 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:36 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:36 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.ezkppl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:27:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:36 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:37.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:37 smithi040 ceph-mon[94210]: Reconfiguring mds.cephfs.smithi040.ezkppl (monmap changed)... 2024-02-16T19:27:37.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:37 smithi040 ceph-mon[94210]: Reconfiguring daemon mds.cephfs.smithi040.ezkppl on smithi040 2024-02-16T19:27:37.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:37 smithi163 ceph-mon[81637]: Reconfiguring mds.cephfs.smithi040.ezkppl (monmap changed)... 2024-02-16T19:27:37.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:37 smithi163 ceph-mon[81637]: Reconfiguring daemon mds.cephfs.smithi040.ezkppl on smithi040 2024-02-16T19:27:38.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:38 smithi040 ceph-mon[94210]: pgmap v20: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:38.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:38.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:38.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.bmpjpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:27:38.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:38.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:38 smithi163 ceph-mon[81637]: pgmap v20: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:38.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:38.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.bmpjpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:27:38.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:39 smithi040 ceph-mon[94210]: Reconfiguring mds.cephfs.smithi040.bmpjpk (monmap changed)... 2024-02-16T19:27:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:39 smithi040 ceph-mon[94210]: Reconfiguring daemon mds.cephfs.smithi040.bmpjpk on smithi040 2024-02-16T19:27:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:39.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:39.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:27:39.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:39.430 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:39 smithi163 ceph-mon[81637]: Reconfiguring mds.cephfs.smithi040.bmpjpk (monmap changed)... 2024-02-16T19:27:39.430 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:39 smithi163 ceph-mon[81637]: Reconfiguring daemon mds.cephfs.smithi040.bmpjpk on smithi040 2024-02-16T19:27:39.430 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:39.430 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:39.430 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:27:39.430 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:40 smithi040 ceph-mon[94210]: pgmap v21: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:40 smithi040 ceph-mon[94210]: Reconfiguring crash.smithi163 (monmap changed)... 2024-02-16T19:27:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:40 smithi040 ceph-mon[94210]: Reconfiguring daemon crash.smithi163 on smithi163 2024-02-16T19:27:40.414 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:40 smithi163 ceph-mon[81637]: pgmap v21: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:40.414 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:40 smithi163 ceph-mon[81637]: Reconfiguring crash.smithi163 (monmap changed)... 2024-02-16T19:27:40.414 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:40 smithi163 ceph-mon[81637]: Reconfiguring daemon crash.smithi163 on smithi163 2024-02-16T19:27:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:41 smithi163 ceph-mon[81637]: pgmap v22: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:41 smithi163 ceph-mon[81637]: Reconfiguring mgr.smithi163.wjfclz (monmap changed)... 2024-02-16T19:27:41.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:27:41.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:27:41.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:41.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:41 smithi163 ceph-mon[81637]: Reconfiguring daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:27:41.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:41 smithi040 ceph-mon[94210]: pgmap v22: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:41.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:41.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:41.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:41 smithi040 ceph-mon[94210]: Reconfiguring mgr.smithi163.wjfclz (monmap changed)... 2024-02-16T19:27:41.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.wjfclz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-16T19:27:41.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-16T19:27:41.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:41.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:41 smithi040 ceph-mon[94210]: Reconfiguring daemon mgr.smithi163.wjfclz on smithi163 2024-02-16T19:27:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:43 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:43 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:43 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:27:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:43 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:27:43.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:43 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:43 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:43 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:43 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-16T19:27:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:43 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-16T19:27:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:43 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:44 smithi040 ceph-mon[94210]: Reconfiguring mon.smithi163 (monmap changed)... 2024-02-16T19:27:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:44 smithi040 ceph-mon[94210]: Reconfiguring daemon mon.smithi163 on smithi163 2024-02-16T19:27:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:44 smithi040 ceph-mon[94210]: pgmap v23: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:44 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:44 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:44 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:27:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:44 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:44 smithi163 ceph-mon[81637]: Reconfiguring mon.smithi163 (monmap changed)... 2024-02-16T19:27:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:44 smithi163 ceph-mon[81637]: Reconfiguring daemon mon.smithi163 on smithi163 2024-02-16T19:27:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:44 smithi163 ceph-mon[81637]: pgmap v23: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:44 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:44 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:44 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:27:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:44 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:45 smithi040 ceph-mon[94210]: Reconfiguring osd.3 (monmap changed)... 2024-02-16T19:27:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:45 smithi040 ceph-mon[94210]: Reconfiguring daemon osd.3 on smithi163 2024-02-16T19:27:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:45 smithi040 ceph-mon[94210]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:27:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:45 smithi040 ceph-mon[94210]: pgmap v24: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:45.454 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:45 smithi163 ceph-mon[81637]: Reconfiguring osd.3 (monmap changed)... 2024-02-16T19:27:45.454 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:45 smithi163 ceph-mon[81637]: Reconfiguring daemon osd.3 on smithi163 2024-02-16T19:27:45.454 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:45 smithi163 ceph-mon[81637]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:27:45.454 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:45 smithi163 ceph-mon[81637]: pgmap v24: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:46.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:46.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:46.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:46 smithi163 ceph-mon[81637]: Reconfiguring osd.4 (monmap changed)... 2024-02-16T19:27:46.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:27:46.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:46.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:46 smithi163 ceph-mon[81637]: Reconfiguring daemon osd.4 on smithi163 2024-02-16T19:27:46.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:46.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:46.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:46 smithi040 ceph-mon[94210]: Reconfiguring osd.4 (monmap changed)... 2024-02-16T19:27:46.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:27:46.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:46.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:46 smithi040 ceph-mon[94210]: Reconfiguring daemon osd.4 on smithi163 2024-02-16T19:27:47.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:47 smithi040 ceph-mon[94210]: pgmap v25: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:47.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:47.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:47.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:27:47.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:47.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:47.911 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:47 smithi163 ceph-mon[81637]: pgmap v25: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:47.911 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:47.911 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:47.911 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:27:47.911 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:47.911 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:27:48.851 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:48 smithi040 ceph-mon[94210]: Reconfiguring osd.5 (monmap changed)... 2024-02-16T19:27:48.851 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:48 smithi040 ceph-mon[94210]: Reconfiguring daemon osd.5 on smithi163 2024-02-16T19:27:48.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:48 smithi163 ceph-mon[81637]: Reconfiguring osd.5 (monmap changed)... 2024-02-16T19:27:48.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:48 smithi163 ceph-mon[81637]: Reconfiguring daemon osd.5 on smithi163 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.925+0000 7f025b2ef700 1 -- 172.21.15.40:0/2892082677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254066bb0 msgr2=0x7f02540ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.925+0000 7f025b2ef700 1 --2- 172.21.15.40:0/2892082677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254066bb0 0x7f02540ffbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0250004940 tx=0x7f0250009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.925+0000 7f025b2ef700 1 -- 172.21.15.40:0/2892082677 shutdown_connections 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.925+0000 7f025b2ef700 1 --2- 172.21.15.40:0/2892082677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0254100130 0x7f0254102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.925+0000 7f025b2ef700 1 --2- 172.21.15.40:0/2892082677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254066bb0 0x7f02540ffbf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.925+0000 7f025b2ef700 1 -- 172.21.15.40:0/2892082677 >> 172.21.15.40:0/2892082677 conn(0x7f02540f75c0 msgr2=0x7f02540f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025b2ef700 1 -- 172.21.15.40:0/2892082677 shutdown_connections 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025b2ef700 1 -- 172.21.15.40:0/2892082677 wait complete. 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025b2ef700 1 Processor -- start 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025b2ef700 1 -- start start 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025b2ef700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0254066bb0 0x7f025410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:49.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025b2ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254100130 0x7f025410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025b2ef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025410e460 con 0x7f0254100130 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025b2ef700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025410ed00 con 0x7f0254066bb0 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025888a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254100130 0x7f025410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025888a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254100130 0x7f025410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38624/0 (socket says 172.21.15.40:38624) 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025888a700 1 -- 172.21.15.40:0/1985169620 learned_addr learned my addr 172.21.15.40:0/1985169620 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.926+0000 7f025908b700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0254066bb0 0x7f025410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f025888a700 1 -- 172.21.15.40:0/1985169620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0254066bb0 msgr2=0x7f025410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f025888a700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0254066bb0 0x7f025410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:49.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f025888a700 1 -- 172.21.15.40:0/1985169620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250005040 con 0x7f0254100130 2024-02-16T19:27:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f025888a700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254100130 0x7f025410de60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0244002a50 tx=0x7f02440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f0249ffb700 1 -- 172.21.15.40:0/1985169620 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0244007cc0 con 0x7f0254100130 2024-02-16T19:27:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f0249ffb700 1 -- 172.21.15.40:0/1985169620 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f024400f040 con 0x7f0254100130 2024-02-16T19:27:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025410efe0 con 0x7f0254100130 2024-02-16T19:27:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f0249ffb700 1 -- 172.21.15.40:0/1985169620 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f024400db30 con 0x7f0254100130 2024-02-16T19:27:49.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.927+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025410f520 con 0x7f0254100130 2024-02-16T19:27:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.928+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025404bed0 con 0x7f0254100130 2024-02-16T19:27:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.929+0000 7f0249ffb700 1 -- 172.21.15.40:0/1985169620 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f024400dc90 con 0x7f0254100130 2024-02-16T19:27:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.929+0000 7f0249ffb700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f024006d900 0x7f024006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.929+0000 7f0249ffb700 1 -- 172.21.15.40:0/1985169620 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f024408cd80 con 0x7f0254100130 2024-02-16T19:27:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.929+0000 7f025908b700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f024006d900 0x7f024006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:49.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.929+0000 7f025908b700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f024006d900 0x7f024006fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0250004890 tx=0x7f0250009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:49.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:49.934+0000 7f0249ffb700 1 -- 172.21.15.40:0/1985169620 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0244059ee0 con 0x7f0254100130 2024-02-16T19:27:50.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.156+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f025410f970 con 0x7f024006d900 2024-02-16T19:27:50.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.157+0000 7f0249ffb700 1 -- 172.21.15.40:0/1985169620 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f025410f970 con 0x7f024006d900 2024-02-16T19:27:50.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f024006d900 msgr2=0x7f024006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f024006d900 0x7f024006fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0250004890 tx=0x7f0250009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:50.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254100130 msgr2=0x7f025410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254100130 0x7f025410de60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0244002a50 tx=0x7f02440046b0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 shutdown_connections 2024-02-16T19:27:50.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0254066bb0 0x7f025410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f024006d900 0x7f024006fdb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 --2- 172.21.15.40:0/1985169620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0254100130 0x7f025410de60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 >> 172.21.15.40:0/1985169620 conn(0x7f02540f75c0 msgr2=0x7f02540f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:50.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 shutdown_connections 2024-02-16T19:27:50.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.159+0000 7f025b2ef700 1 -- 172.21.15.40:0/1985169620 wait complete. 2024-02-16T19:27:50.170 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:27:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.261+0000 7fc1b3a54700 1 -- 172.21.15.40:0/311112283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 msgr2=0x7fc1ac070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.261+0000 7fc1b3a54700 1 --2- 172.21.15.40:0/311112283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 0x7fc1ac070cd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc19c004940 tx=0x7fc19c009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.261+0000 7fc1b3a54700 1 -- 172.21.15.40:0/311112283 shutdown_connections 2024-02-16T19:27:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.261+0000 7fc1b3a54700 1 --2- 172.21.15.40:0/311112283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 0x7fc1ac070cd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 --2- 172.21.15.40:0/311112283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1ac071eb0 0x7fc1ac070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 -- 172.21.15.40:0/311112283 >> 172.21.15.40:0/311112283 conn(0x7fc1ac0f8620 msgr2=0x7fc1ac0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 -- 172.21.15.40:0/311112283 shutdown_connections 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 -- 172.21.15.40:0/311112283 wait complete. 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 Processor -- start 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 -- start start 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 0x7fc1ac10fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1ac071eb0 0x7fc1ac110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac110680 con 0x7fc1ac070860 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b3a54700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac110f20 con 0x7fc1ac071eb0 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b17f0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 0x7fc1ac10fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b17f0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 0x7fc1ac10fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38644/0 (socket says 172.21.15.40:38644) 2024-02-16T19:27:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.262+0000 7fc1b17f0700 1 -- 172.21.15.40:0/3323004213 learned_addr learned my addr 172.21.15.40:0/3323004213 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1b0fef700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1ac071eb0 0x7fc1ac110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1b17f0700 1 -- 172.21.15.40:0/3323004213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1ac071eb0 msgr2=0x7fc1ac110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1b17f0700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1ac071eb0 0x7fc1ac110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1b17f0700 1 -- 172.21.15.40:0/3323004213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c005040 con 0x7fc1ac070860 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1b17f0700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 0x7fc1ac10fb40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8001d80 tx=0x7fc1a8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1a1ffb700 1 -- 172.21.15.40:0/3323004213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a8007d10 con 0x7fc1ac070860 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1a1ffb700 1 -- 172.21.15.40:0/3323004213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1a8001e80 con 0x7fc1ac070860 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac111200 con 0x7fc1ac070860 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1a1ffb700 1 -- 172.21.15.40:0/3323004213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a8014640 con 0x7fc1ac070860 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.263+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac111740 con 0x7fc1ac070860 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.264+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ac04bed0 con 0x7fc1ac070860 2024-02-16T19:27:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.265+0000 7fc1a1ffb700 1 -- 172.21.15.40:0/3323004213 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fc1a800e960 con 0x7fc1ac070860 2024-02-16T19:27:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.265+0000 7fc1a1ffb700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc19806d900 0x7fc19806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.265+0000 7fc1a1ffb700 1 -- 172.21.15.40:0/3323004213 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc1a808cd10 con 0x7fc1ac070860 2024-02-16T19:27:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.265+0000 7fc1b0fef700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc19806d900 0x7fc19806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.266+0000 7fc1b0fef700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc19806d900 0x7fc19806fdb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b3b0 tx=0x7fc19c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:50.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.270+0000 7fc1a1ffb700 1 -- 172.21.15.40:0/3323004213 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1a805a0b0 con 0x7fc1ac070860 2024-02-16T19:27:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:50 smithi040 ceph-mon[94210]: pgmap v26: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:50 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:50 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:50 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.rggdhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:27:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:50 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:50 smithi163 ceph-mon[81637]: pgmap v26: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:50 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:50 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:50 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.rggdhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:27:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:50 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:50.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.493+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1ac111c10 con 0x7fc19806d900 2024-02-16T19:27:50.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.494+0000 7fc1a1ffb700 1 -- 172.21.15.40:0/3323004213 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc1ac111c10 con 0x7fc19806d900 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc19806d900 msgr2=0x7fc19806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc19806d900 0x7fc19806fdb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b3b0 tx=0x7fc19c009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 msgr2=0x7fc1ac10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 0x7fc1ac10fb40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8001d80 tx=0x7fc1a8009ec0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 shutdown_connections 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc19806d900 0x7fc19806fdb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1ac070860 0x7fc1ac10fb40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 --2- 172.21.15.40:0/3323004213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1ac071eb0 0x7fc1ac110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 >> 172.21.15.40:0/3323004213 conn(0x7fc1ac0f8620 msgr2=0x7fc1ac102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 shutdown_connections 2024-02-16T19:27:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.496+0000 7fc1b3a54700 1 -- 172.21.15.40:0/3323004213 wait complete. 2024-02-16T19:27:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.597+0000 7f53ec3cc700 1 -- 172.21.15.40:0/288207694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 msgr2=0x7f53e40fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.597+0000 7f53ec3cc700 1 --2- 172.21.15.40:0/288207694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 0x7f53e40fd420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f53d4004940 tx=0x7f53d4009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.597+0000 7f53ec3cc700 1 -- 172.21.15.40:0/288207694 shutdown_connections 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.597+0000 7f53ec3cc700 1 --2- 172.21.15.40:0/288207694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 0x7f53e40fd420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.597+0000 7f53ec3cc700 1 --2- 172.21.15.40:0/288207694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53e40fbe40 0x7f53e40fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.597+0000 7f53ec3cc700 1 -- 172.21.15.40:0/288207694 >> 172.21.15.40:0/288207694 conn(0x7f53e40f7520 msgr2=0x7f53e40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.597+0000 7f53ec3cc700 1 -- 172.21.15.40:0/288207694 shutdown_connections 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.597+0000 7f53ec3cc700 1 -- 172.21.15.40:0/288207694 wait complete. 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53ec3cc700 1 Processor -- start 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53ec3cc700 1 -- start start 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53ec3cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53e40fbe40 0x7f53e410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53ec3cc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 0x7f53e410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53ec3cc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e410e4f0 con 0x7f53e40fcfb0 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53ec3cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e410ed30 con 0x7f53e40fbe40 2024-02-16T19:27:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53e9967700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 0x7f53e410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53e9967700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 0x7f53e410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38658/0 (socket says 172.21.15.40:38658) 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53e9967700 1 -- 172.21.15.40:0/1944510019 learned_addr learned my addr 172.21.15.40:0/1944510019 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53ea168700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53e40fbe40 0x7f53e410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53e9967700 1 -- 172.21.15.40:0/1944510019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53e40fbe40 msgr2=0x7f53e410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53e9967700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53e40fbe40 0x7f53e410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53e9967700 1 -- 172.21.15.40:0/1944510019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4005040 con 0x7f53e40fcfb0 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.598+0000 7f53ea168700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53e40fbe40 0x7f53e410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.599+0000 7f53e9967700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 0x7f53e410de60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f53d400b3b0 tx=0x7f53d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.599+0000 7f53daffd700 1 -- 172.21.15.40:0/1944510019 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d400e040 con 0x7f53e40fcfb0 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.599+0000 7f53daffd700 1 -- 172.21.15.40:0/1944510019 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53d400bd90 con 0x7f53e40fcfb0 2024-02-16T19:27:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.599+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e410efb0 con 0x7f53e40fcfb0 2024-02-16T19:27:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.599+0000 7f53daffd700 1 -- 172.21.15.40:0/1944510019 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d401c630 con 0x7f53e40fcfb0 2024-02-16T19:27:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.599+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e410f4f0 con 0x7f53e40fcfb0 2024-02-16T19:27:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.600+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e404bed0 con 0x7f53e40fcfb0 2024-02-16T19:27:50.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.600+0000 7f53daffd700 1 -- 172.21.15.40:0/1944510019 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f53d40139f0 con 0x7f53e40fcfb0 2024-02-16T19:27:50.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.601+0000 7f53daffd700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f53d006d900 0x7f53d006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.601+0000 7f53ea168700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f53d006d900 0x7f53d006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.601+0000 7f53daffd700 1 -- 172.21.15.40:0/1944510019 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f53d408e670 con 0x7f53e40fcfb0 2024-02-16T19:27:50.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.601+0000 7f53ea168700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f53d006d900 0x7f53d006fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.605+0000 7f53daffd700 1 -- 172.21.15.40:0/1944510019 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53d405b850 con 0x7f53e40fcfb0 2024-02-16T19:27:50.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.819+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f53e410f9d0 con 0x7f53d006d900 2024-02-16T19:27:50.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.829+0000 7f53daffd700 1 -- 172.21.15.40:0/1944510019 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f53e410f9d0 con 0x7f53d006d900 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (2m) 43s ago 11m 16.6M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (11m) 43s ago 11m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 e4595d67bcd6 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (10m) 30s ago 10m 6660k - 17.2.7-513-gb88fae66 870a0944ccb8 3204afd4e22e 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (118s) 43s ago 11m 80.1M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (5m) 43s ago 5m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (5m) 43s ago 5m 13.0M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (5m) 30s ago 5m 16.8M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (5m) 30s ago 5m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (104s) 43s ago 12m 471M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:27:50.831 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (71s) 30s ago 10m 447M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (57s) 43s ago 12m 28.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (34s) 30s ago 10m 23.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (2m) 43s ago 11m 8090k - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (110s) 30s ago 10m 15.6M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (9m) 43s ago 9m 227M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (9m) 43s ago 9m 242M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (8m) 43s ago 8m 197M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (8m) 30s ago 8m 270M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (7m) 30s ago 7m 232M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (7m) 30s ago 7m 235M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:27:50.832 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (77s) 43s ago 10m 46.0M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:27:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f53d006d900 msgr2=0x7f53d006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f53d006d900 0x7f53d006fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0008040 comp rx=0 tx=0).stop 2024-02-16T19:27:50.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 msgr2=0x7f53e410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 0x7f53e410de60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f53d400b3b0 tx=0x7f53d4009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 shutdown_connections 2024-02-16T19:27:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53e40fbe40 0x7f53e410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f53d006d900 0x7f53d006fdb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 --2- 172.21.15.40:0/1944510019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53e40fcfb0 0x7f53e410de60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.831+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 >> 172.21.15.40:0/1944510019 conn(0x7f53e40f7520 msgr2=0x7f53e4065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.832+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 shutdown_connections 2024-02-16T19:27:50.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.832+0000 7f53ec3cc700 1 -- 172.21.15.40:0/1944510019 wait complete. 2024-02-16T19:27:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.932+0000 7f8d1d922700 1 -- 172.21.15.40:0/3720731019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fe0e0 msgr2=0x7f8d180fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.932+0000 7f8d1d922700 1 --2- 172.21.15.40:0/3720731019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fe0e0 0x7f8d180fe550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c004940 tx=0x7f8d0c009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.932+0000 7f8d1d922700 1 -- 172.21.15.40:0/3720731019 shutdown_connections 2024-02-16T19:27:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.932+0000 7f8d1d922700 1 --2- 172.21.15.40:0/3720731019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fe0e0 0x7f8d180fe550 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.932+0000 7f8d1d922700 1 --2- 172.21.15.40:0/3720731019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d180fcef0 0x7f8d180fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.932+0000 7f8d1d922700 1 -- 172.21.15.40:0/3720731019 >> 172.21.15.40:0/3720731019 conn(0x7f8d180f8620 msgr2=0x7f8d180faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.932+0000 7f8d1d922700 1 -- 172.21.15.40:0/3720731019 shutdown_connections 2024-02-16T19:27:50.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.932+0000 7f8d1d922700 1 -- 172.21.15.40:0/3720731019 wait complete. 2024-02-16T19:27:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d1d922700 1 Processor -- start 2024-02-16T19:27:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d1d922700 1 -- start start 2024-02-16T19:27:50.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d1d922700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fcef0 0x7f8d1810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d1d922700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d180fe0e0 0x7f8d1810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d1d922700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1810e4a0 con 0x7f8d180fcef0 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d1d922700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1810ed40 con 0x7f8d180fe0e0 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d16ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fcef0 0x7f8d1810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d16ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fcef0 0x7f8d1810d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38682/0 (socket says 172.21.15.40:38682) 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d16ffd700 1 -- 172.21.15.40:0/312959948 learned_addr learned my addr 172.21.15.40:0/312959948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d167fc700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d180fe0e0 0x7f8d1810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d16ffd700 1 -- 172.21.15.40:0/312959948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d180fe0e0 msgr2=0x7f8d1810dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d16ffd700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d180fe0e0 0x7f8d1810dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d16ffd700 1 -- 172.21.15.40:0/312959948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c005040 con 0x7f8d180fcef0 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.933+0000 7f8d16ffd700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fcef0 0x7f8d1810d960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d08002a80 tx=0x7f8d08004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:50.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.934+0000 7f8cff7fe700 1 -- 172.21.15.40:0/312959948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d08007cc0 con 0x7f8d180fcef0 2024-02-16T19:27:50.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.934+0000 7f8cff7fe700 1 -- 172.21.15.40:0/312959948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d0800f040 con 0x7f8d180fcef0 2024-02-16T19:27:50.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.934+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1810f020 con 0x7f8d180fcef0 2024-02-16T19:27:50.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.934+0000 7f8cff7fe700 1 -- 172.21.15.40:0/312959948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d0800da30 con 0x7f8d180fcef0 2024-02-16T19:27:50.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.934+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1810f560 con 0x7f8d180fcef0 2024-02-16T19:27:50.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.935+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1804bed0 con 0x7f8d180fcef0 2024-02-16T19:27:50.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.935+0000 7f8cff7fe700 1 -- 172.21.15.40:0/312959948 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f8d0800db90 con 0x7f8d180fcef0 2024-02-16T19:27:50.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.935+0000 7f8cff7fe700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8d0006d900 0x7f8d0006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:50.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.936+0000 7f8cff7fe700 1 -- 172.21.15.40:0/312959948 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8d0808ce10 con 0x7f8d180fcef0 2024-02-16T19:27:50.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.936+0000 7f8d167fc700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8d0006d900 0x7f8d0006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:50.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.936+0000 7f8d167fc700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8d0006d900 0x7f8d0006fdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c004f90 tx=0x7f8d0c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:50.940+0000 7f8cff7fe700 1 -- 172.21.15.40:0/312959948 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d08059ff0 con 0x7f8d180fcef0 2024-02-16T19:27:51.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.231+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d1810f8f0 con 0x7f8d180fcef0 2024-02-16T19:27:51.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.231+0000 7f8cff7fe700 1 -- 172.21.15.40:0/312959948 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f8d08059b80 con 0x7f8d180fcef0 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:27:51.233 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:27:51.234 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 10, 2024-02-16T19:27:51.234 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:27:51.234 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:27:51.234 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:27:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8d0006d900 msgr2=0x7f8d0006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8d0006d900 0x7f8d0006fdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c004f90 tx=0x7f8d0c009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fcef0 msgr2=0x7f8d1810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fcef0 0x7f8d1810d960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d08002a80 tx=0x7f8d08004600 comp rx=0 tx=0).stop 2024-02-16T19:27:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 shutdown_connections 2024-02-16T19:27:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8d0006d900 0x7f8d0006fdb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d180fcef0 0x7f8d1810d960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 --2- 172.21.15.40:0/312959948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d180fe0e0 0x7f8d1810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 >> 172.21.15.40:0/312959948 conn(0x7f8d180f8620 msgr2=0x7f8d181013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 shutdown_connections 2024-02-16T19:27:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.234+0000 7f8d1d922700 1 -- 172.21.15.40:0/312959948 wait complete. 2024-02-16T19:27:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.338+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/910316303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 msgr2=0x7f8c28070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.338+0000 7f8c2cd1b700 1 --2- 172.21.15.40:0/910316303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 0x7f8c28070cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c004940 tx=0x7f8c1c009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.338+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/910316303 shutdown_connections 2024-02-16T19:27:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.338+0000 7f8c2cd1b700 1 --2- 172.21.15.40:0/910316303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 0x7f8c28070cd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.338+0000 7f8c2cd1b700 1 --2- 172.21.15.40:0/910316303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c28071eb0 0x7f8c28070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.338+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/910316303 >> 172.21.15.40:0/910316303 conn(0x7f8c280f8620 msgr2=0x7f8c280faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.338+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/910316303 shutdown_connections 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.338+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/910316303 wait complete. 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2cd1b700 1 Processor -- start 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2cd1b700 1 -- start start 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2cd1b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 0x7f8c2810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2cd1b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c28071eb0 0x7f8c28110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2cd1b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28110600 con 0x7f8c28070860 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2cd1b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28110ea0 con 0x7f8c28071eb0 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 0x7f8c2810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 0x7f8c2810fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38712/0 (socket says 172.21.15.40:38712) 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2659c700 1 -- 172.21.15.40:0/2178269199 learned_addr learned my addr 172.21.15.40:0/2178269199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c25d9b700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c28071eb0 0x7f8c28110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2659c700 1 -- 172.21.15.40:0/2178269199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c28071eb0 msgr2=0x7f8c28110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2659c700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c28071eb0 0x7f8c28110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2659c700 1 -- 172.21.15.40:0/2178269199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c005040 con 0x7f8c28070860 2024-02-16T19:27:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.339+0000 7f8c2659c700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 0x7f8c2810fac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800eb40 tx=0x7f8c1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.340+0000 7f8c16ffd700 1 -- 172.21.15.40:0/2178269199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c18007cc0 con 0x7f8c28070860 2024-02-16T19:27:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.340+0000 7f8c16ffd700 1 -- 172.21.15.40:0/2178269199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c1800f040 con 0x7f8c28070860 2024-02-16T19:27:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.340+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c28111120 con 0x7f8c28070860 2024-02-16T19:27:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.340+0000 7f8c16ffd700 1 -- 172.21.15.40:0/2178269199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1800ca10 con 0x7f8c28070860 2024-02-16T19:27:51.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.340+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c28111660 con 0x7f8c28070860 2024-02-16T19:27:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.341+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2804bed0 con 0x7f8c28070860 2024-02-16T19:27:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.341+0000 7f8c16ffd700 1 -- 172.21.15.40:0/2178269199 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f8c1800cc80 con 0x7f8c28070860 2024-02-16T19:27:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.342+0000 7f8c16ffd700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c1006d900 0x7f8c1006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.342+0000 7f8c16ffd700 1 -- 172.21.15.40:0/2178269199 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8c1808cd00 con 0x7f8c28070860 2024-02-16T19:27:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.342+0000 7f8c25d9b700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c1006d900 0x7f8c1006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.342+0000 7f8c25d9b700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c1006d900 0x7f8c1006fdb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009b30 tx=0x7f8c1c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.346+0000 7f8c16ffd700 1 -- 172.21.15.40:0/2178269199 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c18090030 con 0x7f8c28070860 2024-02-16T19:27:51.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:51 smithi163 ceph-mon[81637]: Reconfiguring mds.cephfs.smithi163.rggdhh (monmap changed)... 2024-02-16T19:27:51.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:51 smithi163 ceph-mon[81637]: Reconfiguring daemon mds.cephfs.smithi163.rggdhh on smithi163 2024-02-16T19:27:51.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:51.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:51.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.qovvde", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:27:51.388 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:51 smithi040 ceph-mon[94210]: Reconfiguring mds.cephfs.smithi163.rggdhh (monmap changed)... 2024-02-16T19:27:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:51 smithi040 ceph-mon[94210]: Reconfiguring daemon mds.cephfs.smithi163.rggdhh on smithi163 2024-02-16T19:27:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.qovvde", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:27:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:51.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.595+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8c28111a80 con 0x7f8c28070860 2024-02-16T19:27:51.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.595+0000 7f8c16ffd700 1 -- 172.21.15.40:0/2178269199 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f8c18059f10 con 0x7f8c28070860 2024-02-16T19:27:51.596 INFO:teuthology.orchestra.run.smithi040.stdout:e20 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 20 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:27:43.925056+0000 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:27:51.597 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:27:51.598 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:27:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c1006d900 msgr2=0x7f8c1006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c1006d900 0x7f8c1006fdb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009b30 tx=0x7f8c1c009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 msgr2=0x7f8c2810fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 0x7f8c2810fac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800eb40 tx=0x7f8c1800b040 comp rx=0 tx=0).stop 2024-02-16T19:27:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 shutdown_connections 2024-02-16T19:27:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c1006d900 0x7f8c1006fdb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c28070860 0x7f8c2810fac0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 --2- 172.21.15.40:0/2178269199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c28071eb0 0x7f8c28110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 >> 172.21.15.40:0/2178269199 conn(0x7f8c280f8620 msgr2=0x7f8c28102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:51.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 shutdown_connections 2024-02-16T19:27:51.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.597+0000 7f8c2cd1b700 1 -- 172.21.15.40:0/2178269199 wait complete. 2024-02-16T19:27:51.601 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-02-16T19:27:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.699+0000 7fa493c44700 1 -- 172.21.15.40:0/3068573401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fcfe0 msgr2=0x7fa48c0fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.699+0000 7fa493c44700 1 --2- 172.21.15.40:0/3068573401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fcfe0 0x7fa48c0fd450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa488004940 tx=0x7fa488009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.699+0000 7fa493c44700 1 -- 172.21.15.40:0/3068573401 shutdown_connections 2024-02-16T19:27:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.699+0000 7fa493c44700 1 --2- 172.21.15.40:0/3068573401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fcfe0 0x7fa48c0fd450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.699+0000 7fa493c44700 1 --2- 172.21.15.40:0/3068573401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c0fbe70 0x7fa48c0fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.699+0000 7fa493c44700 1 -- 172.21.15.40:0/3068573401 >> 172.21.15.40:0/3068573401 conn(0x7fa48c0f7530 msgr2=0x7fa48c0f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa493c44700 1 -- 172.21.15.40:0/3068573401 shutdown_connections 2024-02-16T19:27:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa493c44700 1 -- 172.21.15.40:0/3068573401 wait complete. 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa493c44700 1 Processor -- start 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa493c44700 1 -- start start 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa493c44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fbe70 0x7fa48c10d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa493c44700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c0fcfe0 0x7fa48c10ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa493c44700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c10e3d0 con 0x7fa48c0fbe70 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa493c44700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c10ec70 con 0x7fa48c0fcfe0 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa4919e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fbe70 0x7fa48c10d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa4919e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fbe70 0x7fa48c10d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38730/0 (socket says 172.21.15.40:38730) 2024-02-16T19:27:51.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.700+0000 7fa4919e0700 1 -- 172.21.15.40:0/2736860602 learned_addr learned my addr 172.21.15.40:0/2736860602 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa4911df700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c0fcfe0 0x7fa48c10ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa4919e0700 1 -- 172.21.15.40:0/2736860602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c0fcfe0 msgr2=0x7fa48c10ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa4919e0700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c0fcfe0 0x7fa48c10ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa4919e0700 1 -- 172.21.15.40:0/2736860602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa488005040 con 0x7fa48c0fbe70 2024-02-16T19:27:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa4919e0700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fbe70 0x7fa48c10d890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa480002a50 tx=0x7fa4800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa47e7fc700 1 -- 172.21.15.40:0/2736860602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa480007cc0 con 0x7fa48c0fbe70 2024-02-16T19:27:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa47e7fc700 1 -- 172.21.15.40:0/2736860602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa48000f040 con 0x7fa48c0fbe70 2024-02-16T19:27:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c10ef50 con 0x7fa48c0fbe70 2024-02-16T19:27:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa47e7fc700 1 -- 172.21.15.40:0/2736860602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa48000db70 con 0x7fa48c0fbe70 2024-02-16T19:27:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.701+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c10f490 con 0x7fa48c0fbe70 2024-02-16T19:27:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.702+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c04bed0 con 0x7fa48c0fbe70 2024-02-16T19:27:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.702+0000 7fa47e7fc700 1 -- 172.21.15.40:0/2736860602 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fa48000dd10 con 0x7fa48c0fbe70 2024-02-16T19:27:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.703+0000 7fa47e7fc700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa47806d900 0x7fa47806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.703+0000 7fa47e7fc700 1 -- 172.21.15.40:0/2736860602 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa48008c860 con 0x7fa48c0fbe70 2024-02-16T19:27:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.703+0000 7fa4911df700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa47806d900 0x7fa47806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.703+0000 7fa4911df700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa47806d900 0x7fa47806fdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa488004f90 tx=0x7fa488009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.708+0000 7fa47e7fc700 1 -- 172.21.15.40:0/2736860602 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa4800599c0 con 0x7fa48c0fbe70 2024-02-16T19:27:51.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.929+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa48c101930 con 0x7fa47806d900 2024-02-16T19:27:51.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.930+0000 7fa47e7fc700 1 -- 172.21.15.40:0/2736860602 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fa48c101930 con 0x7fa47806d900 2024-02-16T19:27:51.931 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:27:51.931 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:27:51.931 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:27:51.931 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:27:51.931 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:27:51.931 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:27:51.931 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:27:51.932 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:27:51.932 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "4/21 daemons upgraded", 2024-02-16T19:27:51.932 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading mon daemons", 2024-02-16T19:27:51.932 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:27:51.932 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa47806d900 msgr2=0x7fa47806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa47806d900 0x7fa47806fdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa488004f90 tx=0x7fa488009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fbe70 msgr2=0x7fa48c10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fbe70 0x7fa48c10d890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa480002a50 tx=0x7fa4800046b0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 shutdown_connections 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa47806d900 0x7fa47806fdb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0fbe70 0x7fa48c10d890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 --2- 172.21.15.40:0/2736860602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c0fcfe0 0x7fa48c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 >> 172.21.15.40:0/2736860602 conn(0x7fa48c0f7530 msgr2=0x7fa48c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 shutdown_connections 2024-02-16T19:27:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:51.932+0000 7fa493c44700 1 -- 172.21.15.40:0/2736860602 wait complete. 2024-02-16T19:27:52.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.034+0000 7fcddc2f9700 1 -- 172.21.15.40:0/92483525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 msgr2=0x7fcdd40fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:52.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.034+0000 7fcddc2f9700 1 --2- 172.21.15.40:0/92483525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 0x7fcdd40fd840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0004940 tx=0x7fcdd0009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 -- 172.21.15.40:0/92483525 shutdown_connections 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 --2- 172.21.15.40:0/92483525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdd40fde10 0x7fcdd40faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 --2- 172.21.15.40:0/92483525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 0x7fcdd40fd840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 -- 172.21.15.40:0/92483525 >> 172.21.15.40:0/92483525 conn(0x7fcdd40f6ca0 msgr2=0x7fcdd40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 -- 172.21.15.40:0/92483525 shutdown_connections 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 -- 172.21.15.40:0/92483525 wait complete. 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 Processor -- start 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 -- start start 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 0x7fcdd410fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdd4110210 0x7fcdd4110f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41106d0 con 0x7fcdd40fd430 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcddc2f9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd4110810 con 0x7fcdd4110210 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.035+0000 7fcdda095700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 0x7fcdd410fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:52.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdda095700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 0x7fcdd410fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38750/0 (socket says 172.21.15.40:38750) 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdda095700 1 -- 172.21.15.40:0/4138911389 learned_addr learned my addr 172.21.15.40:0/4138911389 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdd9894700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdd4110210 0x7fcdd4110f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdda095700 1 -- 172.21.15.40:0/4138911389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdd4110210 msgr2=0x7fcdd4110f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdda095700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdd4110210 0x7fcdd4110f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdda095700 1 -- 172.21.15.40:0/4138911389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd0005040 con 0x7fcdd40fd430 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdda095700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 0x7fcdd410fcd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000b3b0 tx=0x7fcdd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdcaffd700 1 -- 172.21.15.40:0/4138911389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd000e040 con 0x7fcdd40fd430 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdcaffd700 1 -- 172.21.15.40:0/4138911389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdd000bbf0 con 0x7fcdd40fd430 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd41115f0 con 0x7fcdd40fd430 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcdcaffd700 1 -- 172.21.15.40:0/4138911389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd001c630 con 0x7fcdd40fd430 2024-02-16T19:27:52.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.036+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd4111b30 con 0x7fcdd40fd430 2024-02-16T19:27:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.037+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd4109ca0 con 0x7fcdd40fd430 2024-02-16T19:27:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.038+0000 7fcdcaffd700 1 -- 172.21.15.40:0/4138911389 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fcdd000bd60 con 0x7fcdd40fd430 2024-02-16T19:27:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.038+0000 7fcdcaffd700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc006d900 0x7fcdc006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:27:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.038+0000 7fcdcaffd700 1 -- 172.21.15.40:0/4138911389 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcdd008e3f0 con 0x7fcdd40fd430 2024-02-16T19:27:52.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.038+0000 7fcdd9894700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc006d900 0x7fcdc006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:27:52.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.039+0000 7fcdd9894700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc006d900 0x7fcdc006fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40fe280 tx=0x7fcdc400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:27:52.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.043+0000 7fcdcaffd700 1 -- 172.21.15.40:0/4138911389 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdd005b720 con 0x7fcdd40fd430 2024-02-16T19:27:52.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:52 smithi163 ceph-mon[81637]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:52 smithi163 ceph-mon[81637]: pgmap v27: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:52 smithi163 ceph-mon[81637]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:52 smithi163 ceph-mon[81637]: Reconfiguring mds.cephfs.smithi163.qovvde (monmap changed)... 2024-02-16T19:27:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:52 smithi163 ceph-mon[81637]: Reconfiguring daemon mds.cephfs.smithi163.qovvde on smithi163 2024-02-16T19:27:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:52 smithi163 ceph-mon[81637]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:52 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/312959948' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:52 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2178269199' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:27:52.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.332+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcdd4107c60 con 0x7fcdd40fd430 2024-02-16T19:27:52.334 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:52 smithi040 ceph-mon[94210]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:52.334 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:52 smithi040 ceph-mon[94210]: pgmap v27: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:52.334 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:52 smithi040 ceph-mon[94210]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:52.334 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:52 smithi040 ceph-mon[94210]: Reconfiguring mds.cephfs.smithi163.qovvde (monmap changed)... 2024-02-16T19:27:52.334 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:52 smithi040 ceph-mon[94210]: Reconfiguring daemon mds.cephfs.smithi163.qovvde on smithi163 2024-02-16T19:27:52.334 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:52 smithi040 ceph-mon[94210]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:52.334 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:52 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/312959948' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:52.334 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:52 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2178269199' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:27:52.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.332+0000 7fcdcaffd700 1 -- 172.21.15.40:0/4138911389 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fcdd005b2b0 con 0x7fcdd40fd430 2024-02-16T19:27:52.335 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.334+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc006d900 msgr2=0x7fcdc006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.334+0000 7fcddc2f9700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc006d900 0x7fcdc006fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40fe280 tx=0x7fcdc400b040 comp rx=0 tx=0).stop 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.334+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 msgr2=0x7fcdd410fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.334+0000 7fcddc2f9700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 0x7fcdd410fcd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000b3b0 tx=0x7fcdd0009e30 comp rx=0 tx=0).stop 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.335+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 shutdown_connections 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.335+0000 7fcddc2f9700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc006d900 0x7fcdc006fdb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.335+0000 7fcddc2f9700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdd40fd430 0x7fcdd410fcd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.335+0000 7fcddc2f9700 1 --2- 172.21.15.40:0/4138911389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdd4110210 0x7fcdd4110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:27:52.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.335+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 >> 172.21.15.40:0/4138911389 conn(0x7fcdd40f6ca0 msgr2=0x7fcdd40f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:27:52.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.335+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 shutdown_connections 2024-02-16T19:27:52.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:27:52.335+0000 7fcddc2f9700 1 -- 172.21.15.40:0/4138911389 wait complete. 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/4138911389' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi040"}]: dispatch 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi040"}]': finished 2024-02-16T19:27:53.109 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi163"}]: dispatch 2024-02-16T19:27:53.110 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi163"}]': finished 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/4138911389' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi040"}]: dispatch 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi040"}]': finished 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi163"}]: dispatch 2024-02-16T19:27:53.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi163"}]': finished 2024-02-16T19:27:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:54 smithi040 ceph-mon[94210]: pgmap v28: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:54 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all mon 2024-02-16T19:27:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:54 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:54 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:27:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:54 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:54 smithi163 ceph-mon[81637]: pgmap v28: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:54 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all mon 2024-02-16T19:27:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:54 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:54 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:27:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:54 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:27:55.286 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:55 smithi040 ceph-mon[94210]: Upgrade: Updating crash.smithi040 (1/2) 2024-02-16T19:27:55.286 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:55 smithi040 ceph-mon[94210]: Deploying daemon crash.smithi040 on smithi040 2024-02-16T19:27:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:55 smithi163 ceph-mon[81637]: Upgrade: Updating crash.smithi040 (1/2) 2024-02-16T19:27:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:55 smithi163 ceph-mon[81637]: Deploying daemon crash.smithi040 on smithi040 2024-02-16T19:27:56.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:56 smithi040 ceph-mon[94210]: pgmap v29: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:56.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:56 smithi163 ceph-mon[81637]: pgmap v29: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:58 smithi040 ceph-mon[94210]: pgmap v30: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:58 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:27:58 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:58.427 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:58 smithi163 ceph-mon[81637]: pgmap v30: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:27:58.427 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:58 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:27:58.427 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:27:58 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:00.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:00 smithi040 ceph-mon[94210]: pgmap v31: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:00.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:00 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:00.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:00 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:28:00.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:00 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:00.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:00 smithi163 ceph-mon[81637]: pgmap v31: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:00.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:00 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:00.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:00 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-16T19:28:00.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:00 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:01.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:01 smithi040 ceph-mon[94210]: Upgrade: Updating crash.smithi163 (2/2) 2024-02-16T19:28:01.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:01 smithi040 ceph-mon[94210]: Deploying daemon crash.smithi163 on smithi163 2024-02-16T19:28:01.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:01 smithi163 ceph-mon[81637]: Upgrade: Updating crash.smithi163 (2/2) 2024-02-16T19:28:01.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:01 smithi163 ceph-mon[81637]: Deploying daemon crash.smithi163 on smithi163 2024-02-16T19:28:02.314 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:02 smithi163 ceph-mon[81637]: pgmap v32: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:02.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:02 smithi040 ceph-mon[94210]: pgmap v32: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:03.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:03.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:03.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:03.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:03.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:03.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:03.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:03.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:04.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:04 smithi040 ceph-mon[94210]: pgmap v33: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:04.399 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:04 smithi163 ceph-mon[81637]: pgmap v33: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:05.433 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:05 smithi040 ceph-mon[94210]: pgmap v34: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:05.462 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:05 smithi163 ceph-mon[81637]: pgmap v34: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:07.297 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:06 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:07.297 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:06 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:07.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:06 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:07.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:06 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:08 smithi163 ceph-mon[81637]: pgmap v35: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:08 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:08 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:08 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:08 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:08 smithi040 ceph-mon[94210]: pgmap v35: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:08 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:08 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:08 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:08 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:10 smithi040 ceph-mon[94210]: pgmap v36: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:10 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:10 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:10.347 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:10 smithi163 ceph-mon[81637]: pgmap v36: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:10.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:10 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:10.348 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:10 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:11 smithi040 ceph-mon[94210]: pgmap v37: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:12.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:11 smithi163 ceph-mon[81637]: pgmap v37: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:12.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:12.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.537 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: pgmap v38: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:13.537 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all crash 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi040"}]: dispatch 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi040"}]': finished 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi163"}]: dispatch 2024-02-16T19:28:13.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi163"}]': finished 2024-02-16T19:28:13.539 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T19:28:13.539 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T19:28:13.539 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:13 smithi040 ceph-mon[94210]: Upgrade: osd.0 is safe to restart 2024-02-16T19:28:13.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: pgmap v38: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all crash 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi040"}]: dispatch 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi040"}]': finished 2024-02-16T19:28:13.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi163"}]: dispatch 2024-02-16T19:28:13.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi163"}]': finished 2024-02-16T19:28:13.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T19:28:13.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-16T19:28:13.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:13 smithi163 ceph-mon[81637]: Upgrade: osd.0 is safe to restart 2024-02-16T19:28:15.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:15 smithi040 ceph-mon[94210]: pgmap v39: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:15.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:15 smithi040 ceph-mon[94210]: Upgrade: Updating osd.0 2024-02-16T19:28:15.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:15 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:15.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:15 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:28:15.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:15 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:15.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:15 smithi040 ceph-mon[94210]: Deploying daemon osd.0 on smithi040 2024-02-16T19:28:15.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:15 smithi163 ceph-mon[81637]: pgmap v39: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:15.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:15 smithi163 ceph-mon[81637]: Upgrade: Updating osd.0 2024-02-16T19:28:15.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:15 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:15.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:15 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-16T19:28:15.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:15 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:15.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:15 smithi163 ceph-mon[81637]: Deploying daemon osd.0 on smithi040 2024-02-16T19:28:17.297 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:17 smithi040 ceph-mon[94210]: pgmap v40: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:17.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:17 smithi163 ceph-mon[81637]: pgmap v40: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:17.644 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:17 smithi040 systemd[1]: Stopping Ceph osd.0 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:28:18.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:17 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[48162]: 2024-02-16T19:28:17.684+0000 7fe9dce15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:28:18.145 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:17 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[48162]: 2024-02-16T19:28:17.684+0000 7fe9dce15700 -1 osd.0 49 *** Got signal Terminated *** 2024-02-16T19:28:18.145 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:17 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[48162]: 2024-02-16T19:28:17.684+0000 7fe9dce15700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:28:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:18 smithi163 ceph-mon[81637]: osd.0 marked itself down and dead 2024-02-16T19:28:18.613 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:18.613 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:18 smithi040 ceph-mon[94210]: osd.0 marked itself down and dead 2024-02-16T19:28:18.867 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:18 smithi040 podman[102454]: 2024-02-16 19:28:18.613354211 +0000 UTC m=+1.039270002 container died b9845390ca29694692c58dcc1e7fc45bba35ab7ffaf53cc0bc1297580ccf4318 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=quincy-b88fae6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:28:19.123 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:19 smithi040 podman[102454]: 2024-02-16 19:28:19.110222445 +0000 UTC m=+1.536138236 container cleanup b9845390ca29694692c58dcc1e7fc45bba35ab7ffaf53cc0bc1297580ccf4318 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=quincy-b88fae6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-16T19:28:19.123 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:19 smithi040 bash[102454]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0 2024-02-16T19:28:19.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:19 smithi163 ceph-mon[81637]: pgmap v41: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:19.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:19 smithi163 ceph-mon[81637]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:28:19.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:19 smithi163 ceph-mon[81637]: failed to encode map e50 with expected crc 2024-02-16T19:28:19.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:19 smithi163 ceph-mon[81637]: osdmap e50: 6 total, 5 up, 6 in 2024-02-16T19:28:19.644 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:19 smithi040 podman[102472]: 2024-02-16 19:28:19.361935387 +0000 UTC m=+0.753118225 container remove b9845390ca29694692c58dcc1e7fc45bba35ab7ffaf53cc0bc1297580ccf4318 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=quincy-b88fae6) 2024-02-16T19:28:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:19 smithi040 ceph-mon[94210]: pgmap v41: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:19 smithi040 ceph-mon[94210]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:28:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:19 smithi040 ceph-mon[94210]: failed to encode map e50 with expected crc 2024-02-16T19:28:19.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:19 smithi040 ceph-mon[94210]: osdmap e50: 6 total, 5 up, 6 in 2024-02-16T19:28:19.974 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:19 smithi040 podman[102576]: 2024-02-16 19:28:19.614576403 +0000 UTC m=+0.021925065 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:28:19.974 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:19 smithi040 podman[102576]: 2024-02-16 19:28:19.763170574 +0000 UTC m=+0.170519235 container create 62ef2fcef285faf60cf526b6f9f369cacbb2baadfcaccc7f204a4ce21701507b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-16T19:28:19.974 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:19 smithi040 podman[102576]: 2024-02-16 19:28:19.973914206 +0000 UTC m=+0.381262870 container init 62ef2fcef285faf60cf526b6f9f369cacbb2baadfcaccc7f204a4ce21701507b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:28:20.236 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:19 smithi040 podman[102576]: 2024-02-16 19:28:19.977078583 +0000 UTC m=+0.384427246 container start 62ef2fcef285faf60cf526b6f9f369cacbb2baadfcaccc7f204a4ce21701507b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-16T19:28:20.236 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:20 smithi040 podman[102576]: 2024-02-16 19:28:20.065687281 +0000 UTC m=+0.473035948 container attach 62ef2fcef285faf60cf526b6f9f369cacbb2baadfcaccc7f204a4ce21701507b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-16T19:28:20.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:20 smithi163 ceph-mon[81637]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:20 smithi163 ceph-mon[81637]: osdmap e51: 6 total, 5 up, 6 in 2024-02-16T19:28:20.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:20 smithi163 ceph-mon[81637]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:20 smithi163 ceph-mon[81637]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:20 smithi163 ceph-mon[81637]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:20 smithi163 ceph-mon[81637]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:20 smithi163 ceph-mon[81637]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:20 smithi040 ceph-mon[94210]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:20 smithi040 ceph-mon[94210]: osdmap e51: 6 total, 5 up, 6 in 2024-02-16T19:28:20.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:20 smithi040 ceph-mon[94210]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:20 smithi040 ceph-mon[94210]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:20 smithi040 ceph-mon[94210]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:20 smithi040 ceph-mon[94210]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:20 smithi040 ceph-mon[94210]: failed to encode map e51 with expected crc 2024-02-16T19:28:20.645 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:20 smithi040 podman[102576]: 2024-02-16 19:28:20.281764593 +0000 UTC m=+0.689113257 container died 62ef2fcef285faf60cf526b6f9f369cacbb2baadfcaccc7f204a4ce21701507b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, ceph=True) 2024-02-16T19:28:21.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:20 smithi040 podman[102603]: 2024-02-16 19:28:20.919940765 +0000 UTC m=+0.633270270 container remove 62ef2fcef285faf60cf526b6f9f369cacbb2baadfcaccc7f204a4ce21701507b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-16T19:28:21.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:20 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.0.service: Deactivated successfully. 2024-02-16T19:28:21.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:20 smithi040 systemd[1]: Stopped Ceph osd.0 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:28:21.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:20 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.0.service: Consumed 56.436s CPU time. 2024-02-16T19:28:21.404 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:21 smithi040 ceph-mon[94210]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:21.404 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:21 smithi040 systemd[1]: Starting Ceph osd.0 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:28:21.404 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:21 smithi040 podman[102706]: 2024-02-16 19:28:21.264858085 +0000 UTC m=+0.018313877 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:28:21.405 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:21 smithi040 podman[102706]: 2024-02-16 19:28:21.404380542 +0000 UTC m=+0.157836325 container create c05e1b7b835844d089ad698bed7bc4fc7fd9ac9b8f228dd2a83c7cd985445117 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-16T19:28:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:21 smithi163 ceph-mon[81637]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:22.143 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:21 smithi040 podman[102706]: 2024-02-16 19:28:21.691038294 +0000 UTC m=+0.444494089 container init c05e1b7b835844d089ad698bed7bc4fc7fd9ac9b8f228dd2a83c7cd985445117 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-16T19:28:22.143 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:21 smithi040 podman[102706]: 2024-02-16 19:28:21.694375475 +0000 UTC m=+0.447831268 container start c05e1b7b835844d089ad698bed7bc4fc7fd9ac9b8f228dd2a83c7cd985445117 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-16T19:28:22.143 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:21 smithi040 podman[102706]: 2024-02-16 19:28:21.748898315 +0000 UTC m=+0.502354102 container attach c05e1b7b835844d089ad698bed7bc4fc7fd9ac9b8f228dd2a83c7cd985445117 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0) 2024-02-16T19:28:22.394 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate[102721]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T19:28:22.394 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 bash[102706]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T19:28:22.394 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 bash[102706]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T19:28:22.394 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 bash[102706]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 bash[102706]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 bash[102706]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 bash[102706]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 bash[102706]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate[102721]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate[102721]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate[102721]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate[102721]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate[102721]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate[102721]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-16T19:28:22.395 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 podman[102706]: 2024-02-16 19:28:22.166891909 +0000 UTC m=+0.920347696 container died c05e1b7b835844d089ad698bed7bc4fc7fd9ac9b8f228dd2a83c7cd985445117 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:28:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.448+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/3435429796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fcef0 msgr2=0x7fbb980fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.448+0000 7fbb9fa3c700 1 --2- 172.21.15.40:0/3435429796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fcef0 0x7fbb980fd320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009ce0 tx=0x7fbb88004d10 comp rx=0 tx=0).stop 2024-02-16T19:28:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.448+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/3435429796 shutdown_connections 2024-02-16T19:28:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.448+0000 7fbb9fa3c700 1 --2- 172.21.15.40:0/3435429796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb980fe0e0 0x7fbb980fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.448+0000 7fbb9fa3c700 1 --2- 172.21.15.40:0/3435429796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fcef0 0x7fbb980fd320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.448+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/3435429796 >> 172.21.15.40:0/3435429796 conn(0x7fbb980f8620 msgr2=0x7fbb980faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.448+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/3435429796 shutdown_connections 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.448+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/3435429796 wait complete. 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9fa3c700 1 Processor -- start 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9fa3c700 1 -- start start 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9fa3c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb980fcef0 0x7fbb9810fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9fa3c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fe0e0 0x7fbb981102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9fa3c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98111080 con 0x7fbb980fe0e0 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9fa3c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb981111f0 con 0x7fbb980fcef0 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9cfd7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fe0e0 0x7fbb981102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9cfd7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fe0e0 0x7fbb981102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43726/0 (socket says 172.21.15.40:43726) 2024-02-16T19:28:22.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9cfd7700 1 -- 172.21.15.40:0/1183086201 learned_addr learned my addr 172.21.15.40:0/1183086201 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9d7d8700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb980fcef0 0x7fbb9810fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9cfd7700 1 -- 172.21.15.40:0/1183086201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb980fcef0 msgr2=0x7fbb9810fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9cfd7700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb980fcef0 0x7fbb9810fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9cfd7700 1 -- 172.21.15.40:0/1183086201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88009940 con 0x7fbb980fe0e0 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.449+0000 7fbb9cfd7700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fe0e0 0x7fbb981102c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00b790 tx=0x7fbb8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.450+0000 7fbb95ffb700 1 -- 172.21.15.40:0/1183086201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8c00f770 con 0x7fbb980fe0e0 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.450+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb98111470 con 0x7fbb980fe0e0 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.450+0000 7fbb95ffb700 1 -- 172.21.15.40:0/1183086201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb8c00fda0 con 0x7fbb980fe0e0 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.450+0000 7fbb95ffb700 1 -- 172.21.15.40:0/1183086201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8c010420 con 0x7fbb980fe0e0 2024-02-16T19:28:22.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.450+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb981119b0 con 0x7fbb980fe0e0 2024-02-16T19:28:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.451+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb98102210 con 0x7fbb980fe0e0 2024-02-16T19:28:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.451+0000 7fbb95ffb700 1 -- 172.21.15.40:0/1183086201 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fbb8c01a7f0 con 0x7fbb980fe0e0 2024-02-16T19:28:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.451+0000 7fbb95ffb700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb8006d900 0x7fbb8006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.452+0000 7fbb95ffb700 1 -- 172.21.15.40:0/1183086201 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbb8c08cff0 con 0x7fbb980fe0e0 2024-02-16T19:28:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.452+0000 7fbb9d7d8700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb8006d900 0x7fbb8006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.452+0000 7fbb9d7d8700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb8006d900 0x7fbb8006fdb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009cb0 tx=0x7fbb88004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:22.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.456+0000 7fbb95ffb700 1 -- 172.21.15.40:0/1183086201 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb8c0567b0 con 0x7fbb980fe0e0 2024-02-16T19:28:22.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.680+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb98111cd0 con 0x7fbb8006d900 2024-02-16T19:28:22.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.681+0000 7fbb95ffb700 1 -- 172.21.15.40:0/1183086201 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbb98111cd0 con 0x7fbb8006d900 2024-02-16T19:28:22.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb8006d900 msgr2=0x7fbb8006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb8006d900 0x7fbb8006fdb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009cb0 tx=0x7fbb88004d10 comp rx=0 tx=0).stop 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fe0e0 msgr2=0x7fbb981102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fe0e0 0x7fbb981102c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00b790 tx=0x7fbb8c009250 comp rx=0 tx=0).stop 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 shutdown_connections 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb980fcef0 0x7fbb9810fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbb8006d900 0x7fbb8006fdb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 --2- 172.21.15.40:0/1183086201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb980fe0e0 0x7fbb981102c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 >> 172.21.15.40:0/1183086201 conn(0x7fbb980f8620 msgr2=0x7fbb980ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.683+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 shutdown_connections 2024-02-16T19:28:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.684+0000 7fbb9fa3c700 1 -- 172.21.15.40:0/1183086201 wait complete. 2024-02-16T19:28:22.695 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:28:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.787+0000 7f98cff50700 1 -- 172.21.15.40:0/801071029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 msgr2=0x7f98c80fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.787+0000 7f98cff50700 1 --2- 172.21.15.40:0/801071029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 0x7f98c80fd430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98bc004940 tx=0x7f98bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.787+0000 7f98cff50700 1 -- 172.21.15.40:0/801071029 shutdown_connections 2024-02-16T19:28:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.787+0000 7f98cff50700 1 --2- 172.21.15.40:0/801071029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 0x7f98c80fd430 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.787+0000 7f98cff50700 1 --2- 172.21.15.40:0/801071029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c80fbe50 0x7f98c80fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.787+0000 7f98cff50700 1 -- 172.21.15.40:0/801071029 >> 172.21.15.40:0/801071029 conn(0x7f98c80f7510 msgr2=0x7f98c80f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.787+0000 7f98cff50700 1 -- 172.21.15.40:0/801071029 shutdown_connections 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.787+0000 7f98cff50700 1 -- 172.21.15.40:0/801071029 wait complete. 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.788+0000 7f98cff50700 1 Processor -- start 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.788+0000 7f98cff50700 1 -- start start 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.788+0000 7f98cff50700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c80fbe50 0x7f98c810c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.788+0000 7f98cff50700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 0x7f98c810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.788+0000 7f98cff50700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c810d430 con 0x7f98c80fcfc0 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.788+0000 7f98cff50700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c810dc70 con 0x7f98c80fbe50 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cd4eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 0x7f98c810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cd4eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 0x7f98c810cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43738/0 (socket says 172.21.15.40:43738) 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cd4eb700 1 -- 172.21.15.40:0/934198076 learned_addr learned my addr 172.21.15.40:0/934198076 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cdcec700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c80fbe50 0x7f98c810c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cd4eb700 1 -- 172.21.15.40:0/934198076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c80fbe50 msgr2=0x7f98c810c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cd4eb700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c80fbe50 0x7f98c810c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cd4eb700 1 -- 172.21.15.40:0/934198076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc005040 con 0x7f98c80fcfc0 2024-02-16T19:28:22.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cd4eb700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 0x7f98c810cda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0093b0 tx=0x7f98bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:22.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98c67fc700 1 -- 172.21.15.40:0/934198076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98bc00e040 con 0x7f98c80fcfc0 2024-02-16T19:28:22.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98c67fc700 1 -- 172.21.15.40:0/934198076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98bc00bcc0 con 0x7f98c80fcfc0 2024-02-16T19:28:22.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c810def0 con 0x7f98c80fcfc0 2024-02-16T19:28:22.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98c67fc700 1 -- 172.21.15.40:0/934198076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98bc01c630 con 0x7f98c80fcfc0 2024-02-16T19:28:22.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.789+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c810e430 con 0x7f98c80fcfc0 2024-02-16T19:28:22.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.790+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c804bed0 con 0x7f98c80fcfc0 2024-02-16T19:28:22.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.791+0000 7f98c67fc700 1 -- 172.21.15.40:0/934198076 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f98bc013980 con 0x7f98c80fcfc0 2024-02-16T19:28:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.791+0000 7f98c67fc700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98b006d900 0x7f98b006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.791+0000 7f98c67fc700 1 -- 172.21.15.40:0/934198076 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f98bc08e4d0 con 0x7f98c80fcfc0 2024-02-16T19:28:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.791+0000 7f98cdcec700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98b006d900 0x7f98b006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.792+0000 7f98cdcec700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98b006d900 0x7f98b006fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f98c0007580 tx=0x7f98c0009780 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:22.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:22.796+0000 7f98c67fc700 1 -- 172.21.15.40:0/934198076 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f98bc05b6b0 con 0x7f98c80fcfc0 2024-02-16T19:28:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.032+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98c810e930 con 0x7f98b006d900 2024-02-16T19:28:23.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.033+0000 7f98c67fc700 1 -- 172.21.15.40:0/934198076 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f98c810e930 con 0x7f98b006d900 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.035+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98b006d900 msgr2=0x7f98b006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.035+0000 7f98cff50700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98b006d900 0x7f98b006fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f98c0007580 tx=0x7f98c0009780 comp rx=0 tx=0).stop 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.035+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 msgr2=0x7f98c810cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.035+0000 7f98cff50700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 0x7f98c810cda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0093b0 tx=0x7f98bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.036+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 shutdown_connections 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.036+0000 7f98cff50700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c80fbe50 0x7f98c810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.036+0000 7f98cff50700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98b006d900 0x7f98b006fdb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.036+0000 7f98cff50700 1 --2- 172.21.15.40:0/934198076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c80fcfc0 0x7f98c810cda0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.036+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 >> 172.21.15.40:0/934198076 conn(0x7f98c80f7510 msgr2=0x7f98c8100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.036+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 shutdown_connections 2024-02-16T19:28:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.036+0000 7f98cff50700 1 -- 172.21.15.40:0/934198076 wait complete. 2024-02-16T19:28:23.046 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:22 smithi040 podman[102855]: 2024-02-16 19:28:22.829097794 +0000 UTC m=+0.657419767 container remove c05e1b7b835844d089ad698bed7bc4fc7fd9ac9b8f228dd2a83c7cd985445117 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD) 2024-02-16T19:28:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 -- 172.21.15.40:0/2514206617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fcef0 msgr2=0x7fd2ec0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 --2- 172.21.15.40:0/2514206617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fcef0 0x7fd2ec0fd320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc004940 tx=0x7fd2dc009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 -- 172.21.15.40:0/2514206617 shutdown_connections 2024-02-16T19:28:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 --2- 172.21.15.40:0/2514206617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2ec0fe0e0 0x7fd2ec0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 --2- 172.21.15.40:0/2514206617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fcef0 0x7fd2ec0fd320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 -- 172.21.15.40:0/2514206617 >> 172.21.15.40:0/2514206617 conn(0x7fd2ec0f8620 msgr2=0x7fd2ec0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 -- 172.21.15.40:0/2514206617 shutdown_connections 2024-02-16T19:28:23.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 -- 172.21.15.40:0/2514206617 wait complete. 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 Processor -- start 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.140+0000 7fd2f28c1700 1 -- start start 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2f28c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2ec0fcef0 0x7fd2ec10fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2f28c1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fe0e0 0x7fd2ec1100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2f28c1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec110740 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2f28c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec110f80 con 0x7fd2ec0fcef0 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fe0e0 0x7fd2ec1100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fe0e0 0x7fd2ec1100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43742/0 (socket says 172.21.15.40:43742) 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2eb7fe700 1 -- 172.21.15.40:0/3465893123 learned_addr learned my addr 172.21.15.40:0/3465893123 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2ebfff700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2ec0fcef0 0x7fd2ec10fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2eb7fe700 1 -- 172.21.15.40:0/3465893123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2ec0fcef0 msgr2=0x7fd2ec10fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2eb7fe700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2ec0fcef0 0x7fd2ec10fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2eb7fe700 1 -- 172.21.15.40:0/3465893123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc005040 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2ebfff700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2ec0fcef0 0x7fd2ec10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:28:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2eb7fe700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fe0e0 0x7fd2ec1100b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000b790 tx=0x7fd2e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2e8ff9700 1 -- 172.21.15.40:0/3465893123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2e000f770 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec111260 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2e8ff9700 1 -- 172.21.15.40:0/3465893123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2e000fda0 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2e8ff9700 1 -- 172.21.15.40:0/3465893123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2e000d310 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.141+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1117a0 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.142+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec102210 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.143+0000 7fd2e8ff9700 1 -- 172.21.15.40:0/3465893123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fd2e0010420 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.143+0000 7fd2e8ff9700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd2d406d900 0x7fd2d406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.143+0000 7fd2e8ff9700 1 -- 172.21.15.40:0/3465893123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd2e008d090 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.143+0000 7fd2ebfff700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd2d406d900 0x7fd2d406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.144+0000 7fd2ebfff700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd2d406d900 0x7fd2d406fdb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00b3b0 tx=0x7fd2dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:23.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.148+0000 7fd2e8ff9700 1 -- 172.21.15.40:0/3465893123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd2e005a380 con 0x7fd2ec0fe0e0 2024-02-16T19:28:23.315 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:23 smithi040 ceph-mon[94210]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:23.315 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:23 smithi040 ceph-mon[94210]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:23.315 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:23 smithi040 ceph-mon[94210]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:23.316 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:23 smithi040 podman[102968]: 2024-02-16 19:28:22.948904772 +0000 UTC m=+0.017860073 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:28:23.316 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:23 smithi040 podman[102968]: 2024-02-16 19:28:23.054357635 +0000 UTC m=+0.123312925 container create 607e375ba25bb58ab7af0699f0e46b134ccdddbd0289f24d2f39c1b8fa542a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-16T19:28:23.316 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:23 smithi040 podman[102968]: 2024-02-16 19:28:23.315337281 +0000 UTC m=+0.384292574 container init 607e375ba25bb58ab7af0699f0e46b134ccdddbd0289f24d2f39c1b8fa542a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-16T19:28:23.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.367+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2ec111a90 con 0x7fd2d406d900 2024-02-16T19:28:23.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.377+0000 7fd2e8ff9700 1 -- 172.21.15.40:0/3465893123 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd2ec111a90 con 0x7fd2d406d900 2024-02-16T19:28:23.378 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:28:23.378 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (3m) 15s ago 11m 16.7M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:28:23.378 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (26s) 15s ago 12m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:28:23.378 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (20s) 17s ago 11m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (2m) 15s ago 11m 81.5M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (6m) 15s ago 6m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (6m) 15s ago 6m 13.4M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (6m) 17s ago 6m 17.0M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (6m) 17s ago 6m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (2m) 15s ago 13m 494M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (104s) 17s ago 10m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (90s) 15s ago 13m 41.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (67s) 17s ago 10m 32.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (3m) 15s ago 12m 15.6M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (2m) 17s ago 10m 16.0M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (10m) 15s ago 10m 228M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 b9845390ca29 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (9m) 15s ago 9m 242M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (9m) 15s ago 9m 197M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:28:23.379 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (8m) 17s ago 8m 270M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:28:23.380 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (8m) 17s ago 8m 233M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:28:23.380 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (7m) 17s ago 7m 235M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:28:23.380 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (109s) 15s ago 11m 50.7M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:28:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd2d406d900 msgr2=0x7fd2d406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd2d406d900 0x7fd2d406fdb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00b3b0 tx=0x7fd2dc009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fe0e0 msgr2=0x7fd2ec1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fe0e0 0x7fd2ec1100b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000b790 tx=0x7fd2e0009250 comp rx=0 tx=0).stop 2024-02-16T19:28:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 shutdown_connections 2024-02-16T19:28:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2ec0fcef0 0x7fd2ec10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd2d406d900 0x7fd2d406fdb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 --2- 172.21.15.40:0/3465893123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2ec0fe0e0 0x7fd2ec1100b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 >> 172.21.15.40:0/3465893123 conn(0x7fd2ec0f8620 msgr2=0x7fd2ec0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 shutdown_connections 2024-02-16T19:28:23.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.379+0000 7fd2f28c1700 1 -- 172.21.15.40:0/3465893123 wait complete. 2024-02-16T19:28:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 -- 172.21.15.40:0/2021849623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 msgr2=0x7f34980fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 --2- 172.21.15.40:0/2021849623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 0x7f34980fd320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3488009ce0 tx=0x7f3488004d10 comp rx=0 tx=0).stop 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 -- 172.21.15.40:0/2021849623 shutdown_connections 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 --2- 172.21.15.40:0/2021849623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34980fe0e0 0x7f34980fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 --2- 172.21.15.40:0/2021849623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 0x7f34980fd320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 -- 172.21.15.40:0/2021849623 >> 172.21.15.40:0/2021849623 conn(0x7f34980f8620 msgr2=0x7f34980faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 -- 172.21.15.40:0/2021849623 shutdown_connections 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 -- 172.21.15.40:0/2021849623 wait complete. 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 Processor -- start 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.486+0000 7f349ff9b700 1 -- start start 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349ff9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 0x7f3498071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349ff9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34980fe0e0 0x7f34980702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349ff9b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980708e0 con 0x7f34980fcef0 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349ff9b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498070a50 con 0x7f34980fe0e0 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349dd37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 0x7f3498071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349dd37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 0x7f3498071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43766/0 (socket says 172.21.15.40:43766) 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349dd37700 1 -- 172.21.15.40:0/608144891 learned_addr learned my addr 172.21.15.40:0/608144891 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:23.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349d536700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34980fe0e0 0x7f34980702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349dd37700 1 -- 172.21.15.40:0/608144891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34980fe0e0 msgr2=0x7f34980702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349dd37700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34980fe0e0 0x7f34980702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349dd37700 1 -- 172.21.15.40:0/608144891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488009940 con 0x7f34980fcef0 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349d536700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34980fe0e0 0x7f34980702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349dd37700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 0x7f3498071e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3488000f80 tx=0x7f3488004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f34967fc700 1 -- 172.21.15.40:0/608144891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348800e040 con 0x7f34980fcef0 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3498070cd0 con 0x7f34980fcef0 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f34967fc700 1 -- 172.21.15.40:0/608144891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3488008ea0 con 0x7f34980fcef0 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f34967fc700 1 -- 172.21.15.40:0/608144891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348801ca50 con 0x7f34980fcef0 2024-02-16T19:28:23.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.487+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3498115b10 con 0x7f34980fcef0 2024-02-16T19:28:23.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.488+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349804bed0 con 0x7f34980fcef0 2024-02-16T19:28:23.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.489+0000 7f34967fc700 1 -- 172.21.15.40:0/608144891 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f34880125f0 con 0x7f34980fcef0 2024-02-16T19:28:23.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.489+0000 7f34967fc700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f348006d900 0x7f348006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.489+0000 7f34967fc700 1 -- 172.21.15.40:0/608144891 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f348808eee0 con 0x7f34980fcef0 2024-02-16T19:28:23.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.489+0000 7f349d536700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f348006d900 0x7f348006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.490+0000 7f349d536700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f348006d900 0x7f348006fdb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f348c007fd0 tx=0x7f348c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.494+0000 7f34967fc700 1 -- 172.21.15.40:0/608144891 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f348805c040 con 0x7f34980fcef0 2024-02-16T19:28:23.644 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:23 smithi040 podman[102968]: 2024-02-16 19:28:23.319240484 +0000 UTC m=+0.388195784 container start 607e375ba25bb58ab7af0699f0e46b134ccdddbd0289f24d2f39c1b8fa542a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:28:23.644 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:23 smithi040 ceph-osd[103010]: -- 172.21.15.40:0/2052024558 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f0edc0b1e0 con 0x55f0edbdd000 2024-02-16T19:28:23.644 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:23 smithi040 bash[102968]: 607e375ba25bb58ab7af0699f0e46b134ccdddbd0289f24d2f39c1b8fa542a9b 2024-02-16T19:28:23.645 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:23 smithi040 systemd[1]: Started Ceph osd.0 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:28:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:23 smithi163 ceph-mon[81637]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:23 smithi163 ceph-mon[81637]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:23 smithi163 ceph-mon[81637]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.784+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3498115ca0 con 0x7f34980fcef0 2024-02-16T19:28:23.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.785+0000 7f34967fc700 1 -- 172.21.15.40:0/608144891 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f3498115ca0 con 0x7f34980fcef0 2024-02-16T19:28:23.786 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:28:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:28:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:28:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:28:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 5 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 9, 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:28:23.787 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:28:23.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f348006d900 msgr2=0x7f348006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f348006d900 0x7f348006fdb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f348c007fd0 tx=0x7f348c009040 comp rx=0 tx=0).stop 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 msgr2=0x7f3498071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 0x7f3498071e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3488000f80 tx=0x7f3488004d10 comp rx=0 tx=0).stop 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 shutdown_connections 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f348006d900 0x7f348006fdb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34980fcef0 0x7f3498071e60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 --2- 172.21.15.40:0/608144891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34980fe0e0 0x7f34980702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 >> 172.21.15.40:0/608144891 conn(0x7f34980f8620 msgr2=0x7f34981013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 shutdown_connections 2024-02-16T19:28:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.787+0000 7f349ff9b700 1 -- 172.21.15.40:0/608144891 wait complete. 2024-02-16T19:28:23.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 -- 172.21.15.40:0/319629570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fcfb0 msgr2=0x7fae5c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 --2- 172.21.15.40:0/319629570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fcfb0 0x7fae5c0fd420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae4c004940 tx=0x7fae4c009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 -- 172.21.15.40:0/319629570 shutdown_connections 2024-02-16T19:28:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 --2- 172.21.15.40:0/319629570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fcfb0 0x7fae5c0fd420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 --2- 172.21.15.40:0/319629570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae5c0fbe40 0x7fae5c0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 -- 172.21.15.40:0/319629570 >> 172.21.15.40:0/319629570 conn(0x7fae5c0f74e0 msgr2=0x7fae5c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 -- 172.21.15.40:0/319629570 shutdown_connections 2024-02-16T19:28:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 -- 172.21.15.40:0/319629570 wait complete. 2024-02-16T19:28:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 Processor -- start 2024-02-16T19:28:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.893+0000 7fae62acf700 1 -- start start 2024-02-16T19:28:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae62acf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fbe40 0x7fae5c10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae62acf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae5c0fcfb0 0x7fae5c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae62acf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c10e490 con 0x7fae5c0fbe40 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae62acf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c10ed30 con 0x7fae5c0fcfb0 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae6086b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fbe40 0x7fae5c10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae6086b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fbe40 0x7fae5c10d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43784/0 (socket says 172.21.15.40:43784) 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae6086b700 1 -- 172.21.15.40:0/3675846225 learned_addr learned my addr 172.21.15.40:0/3675846225 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae5bfff700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae5c0fcfb0 0x7fae5c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae6086b700 1 -- 172.21.15.40:0/3675846225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae5c0fcfb0 msgr2=0x7fae5c10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae6086b700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae5c0fcfb0 0x7fae5c10de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae6086b700 1 -- 172.21.15.40:0/3675846225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c005040 con 0x7fae5c0fbe40 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae5bfff700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae5c0fcfb0 0x7fae5c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.894+0000 7fae6086b700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fbe40 0x7fae5c10d950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fae44002a50 tx=0x7fae440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.895+0000 7fae597fa700 1 -- 172.21.15.40:0/3675846225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae44007cc0 con 0x7fae5c0fbe40 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.895+0000 7fae597fa700 1 -- 172.21.15.40:0/3675846225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae4400f040 con 0x7fae5c0fbe40 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.895+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c10f010 con 0x7fae5c0fbe40 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.895+0000 7fae597fa700 1 -- 172.21.15.40:0/3675846225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae44009640 con 0x7fae5c0fbe40 2024-02-16T19:28:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.895+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c10f550 con 0x7fae5c0fbe40 2024-02-16T19:28:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.896+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c107980 con 0x7fae5c0fbe40 2024-02-16T19:28:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.896+0000 7fae597fa700 1 -- 172.21.15.40:0/3675846225 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fae440097a0 con 0x7fae5c0fbe40 2024-02-16T19:28:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.897+0000 7fae597fa700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fae4806d900 0x7fae4806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.897+0000 7fae597fa700 1 -- 172.21.15.40:0/3675846225 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fae4408cf70 con 0x7fae5c0fbe40 2024-02-16T19:28:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.897+0000 7fae5bfff700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fae4806d900 0x7fae4806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.897+0000 7fae5bfff700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fae4806d900 0x7fae4806fdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae4c004560 tx=0x7fae4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:23.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:23.902+0000 7fae597fa700 1 -- 172.21.15.40:0/3675846225 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae440c0d20 con 0x7fae5c0fbe40 2024-02-16T19:28:24.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:23 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[103005]: 2024-02-16T19:28:23.858+0000 7f8ec72317c0 -1 Falling back to public interface 2024-02-16T19:28:24.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.176+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fae5c10f7f0 con 0x7fae5c0fbe40 2024-02-16T19:28:24.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.176+0000 7fae597fa700 1 -- 172.21.15.40:0/3675846225 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fae4405a070 con 0x7fae5c0fbe40 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:e20 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 20 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:27:43.925056+0000 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:28:24.178 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:28:24.179 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:28:24.180 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fae4806d900 msgr2=0x7fae4806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fae4806d900 0x7fae4806fdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae4c004560 tx=0x7fae4c009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fbe40 msgr2=0x7fae5c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fbe40 0x7fae5c10d950 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fae44002a50 tx=0x7fae440046b0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 shutdown_connections 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fae4806d900 0x7fae4806fdb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae5c0fbe40 0x7fae5c10d950 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 --2- 172.21.15.40:0/3675846225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae5c0fcfb0 0x7fae5c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 >> 172.21.15.40:0/3675846225 conn(0x7fae5c0f74e0 msgr2=0x7fae5c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 shutdown_connections 2024-02-16T19:28:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.178+0000 7fae62acf700 1 -- 172.21.15.40:0/3675846225 wait complete. 2024-02-16T19:28:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-02-16T19:28:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 -- 172.21.15.40:0/2431645632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40100060 msgr2=0x7fdf40102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 --2- 172.21.15.40:0/2431645632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40100060 0x7fdf40102480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf28004940 tx=0x7fdf28009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 -- 172.21.15.40:0/2431645632 shutdown_connections 2024-02-16T19:28:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 --2- 172.21.15.40:0/2431645632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40100060 0x7fdf40102480 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 --2- 172.21.15.40:0/2431645632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf40066c50 0x7fdf400ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 -- 172.21.15.40:0/2431645632 >> 172.21.15.40:0/2431645632 conn(0x7fdf400f7590 msgr2=0x7fdf400f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 -- 172.21.15.40:0/2431645632 shutdown_connections 2024-02-16T19:28:24.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 -- 172.21.15.40:0/2431645632 wait complete. 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.283+0000 7fdf454c1700 1 Processor -- start 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf454c1700 1 -- start start 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf454c1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40066c50 0x7fdf400fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf454c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf40100060 0x7fdf400fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf454c1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400fc020 con 0x7fdf40066c50 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf454c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400fc190 con 0x7fdf40100060 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40066c50 0x7fdf400fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40066c50 0x7fdf400fd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43804/0 (socket says 172.21.15.40:43804) 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf3effd700 1 -- 172.21.15.40:0/4219138002 learned_addr learned my addr 172.21.15.40:0/4219138002 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf3effd700 1 -- 172.21.15.40:0/4219138002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf40100060 msgr2=0x7fdf400fba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf3e7fc700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf40100060 0x7fdf400fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf3effd700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf40100060 0x7fdf400fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf3effd700 1 -- 172.21.15.40:0/4219138002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28005040 con 0x7fdf40066c50 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf3effd700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40066c50 0x7fdf400fd5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf38001d00 tx=0x7fdf38009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf377fe700 1 -- 172.21.15.40:0/4219138002 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf38007d10 con 0x7fdf40066c50 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.284+0000 7fdf377fe700 1 -- 172.21.15.40:0/4219138002 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf38007e70 con 0x7fdf40066c50 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.285+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf400fc470 con 0x7fdf40066c50 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.285+0000 7fdf377fe700 1 -- 172.21.15.40:0/4219138002 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf38014640 con 0x7fdf40066c50 2024-02-16T19:28:24.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.285+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf401159c0 con 0x7fdf40066c50 2024-02-16T19:28:24.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.285+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4004bed0 con 0x7fdf40066c50 2024-02-16T19:28:24.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.286+0000 7fdf377fe700 1 -- 172.21.15.40:0/4219138002 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fdf3800e900 con 0x7fdf40066c50 2024-02-16T19:28:24.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.286+0000 7fdf377fe700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdf2c06d900 0x7fdf2c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:24.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.286+0000 7fdf377fe700 1 -- 172.21.15.40:0/4219138002 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdf3808ce90 con 0x7fdf40066c50 2024-02-16T19:28:24.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.286+0000 7fdf3e7fc700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdf2c06d900 0x7fdf2c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:24.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.287+0000 7fdf3e7fc700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdf2c06d900 0x7fdf2c06fdb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf28004f90 tx=0x7fdf28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.291+0000 7fdf377fe700 1 -- 172.21.15.40:0/4219138002 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf3805a1b0 con 0x7fdf40066c50 2024-02-16T19:28:24.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.523+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf40100cc0 con 0x7fdf2c06d900 2024-02-16T19:28:24.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.525+0000 7fdf377fe700 1 -- 172.21.15.40:0/4219138002 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdf40100cc0 con 0x7fdf2c06d900 2024-02-16T19:28:24.526 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:28:24.526 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:28:24.526 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:28:24.526 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:28:24.526 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:28:24.526 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:28:24.527 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:28:24.527 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:28:24.527 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:28:24.527 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "6/21 daemons upgraded", 2024-02-16T19:28:24.527 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:28:24.527 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:28:24.527 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:28:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdf2c06d900 msgr2=0x7fdf2c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdf2c06d900 0x7fdf2c06fdb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf28004f90 tx=0x7fdf28009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40066c50 msgr2=0x7fdf400fd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40066c50 0x7fdf400fd5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf38001d00 tx=0x7fdf38009e40 comp rx=0 tx=0).stop 2024-02-16T19:28:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 shutdown_connections 2024-02-16T19:28:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdf2c06d900 0x7fdf2c06fdb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf40066c50 0x7fdf400fd5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 --2- 172.21.15.40:0/4219138002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf40100060 0x7fdf400fba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 >> 172.21.15.40:0/4219138002 conn(0x7fdf400f7590 msgr2=0x7fdf400f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 shutdown_connections 2024-02-16T19:28:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.528+0000 7fdf454c1700 1 -- 172.21.15.40:0/4219138002 wait complete. 2024-02-16T19:28:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.635+0000 7f3628b9d700 1 -- 172.21.15.40:0/2471931216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 msgr2=0x7f36240fe3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.635+0000 7f3628b9d700 1 --2- 172.21.15.40:0/2471931216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 0x7f36240fe3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3614004940 tx=0x7f3614009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.635+0000 7f3628b9d700 1 -- 172.21.15.40:0/2471931216 shutdown_connections 2024-02-16T19:28:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.635+0000 7f3628b9d700 1 --2- 172.21.15.40:0/2471931216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 0x7f36240fe3f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.635+0000 7f3628b9d700 1 --2- 172.21.15.40:0/2471931216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36240fcd90 0x7f36240fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.635+0000 7f3628b9d700 1 -- 172.21.15.40:0/2471931216 >> 172.21.15.40:0/2471931216 conn(0x7f36240f8450 msgr2=0x7f36240fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.635+0000 7f3628b9d700 1 -- 172.21.15.40:0/2471931216 shutdown_connections 2024-02-16T19:28:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.635+0000 7f3628b9d700 1 -- 172.21.15.40:0/2471931216 wait complete. 2024-02-16T19:28:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3628b9d700 1 Processor -- start 2024-02-16T19:28:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3628b9d700 1 -- start start 2024-02-16T19:28:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3628b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36240fcd90 0x7f362410d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3628b9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 0x7f362410dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3628b9d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362410e260 con 0x7f36240fdfa0 2024-02-16T19:28:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3628b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362410eb00 con 0x7f36240fcd90 2024-02-16T19:28:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3622ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 0x7f362410dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3622ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 0x7f362410dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43822/0 (socket says 172.21.15.40:43822) 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f3622ffd700 1 -- 172.21.15.40:0/3918343196 learned_addr learned my addr 172.21.15.40:0/3918343196 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.636+0000 7f36237fe700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36240fcd90 0x7f362410d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f3622ffd700 1 -- 172.21.15.40:0/3918343196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36240fcd90 msgr2=0x7f362410d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f3622ffd700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36240fcd90 0x7f362410d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f3622ffd700 1 -- 172.21.15.40:0/3918343196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614005040 con 0x7f36240fdfa0 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f3622ffd700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 0x7f362410dc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3614009fd0 tx=0x7f3614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f360bfff700 1 -- 172.21.15.40:0/3918343196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361400e040 con 0x7f36240fdfa0 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f360bfff700 1 -- 172.21.15.40:0/3918343196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f361400bcc0 con 0x7f36240fdfa0 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362410ed80 con 0x7f36240fdfa0 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f360bfff700 1 -- 172.21.15.40:0/3918343196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361401c630 con 0x7f36240fdfa0 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.637+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362410f2c0 con 0x7f36240fdfa0 2024-02-16T19:28:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.638+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362404bed0 con 0x7f36240fdfa0 2024-02-16T19:28:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.638+0000 7f360bfff700 1 -- 172.21.15.40:0/3918343196 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 37) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f3614013420 con 0x7f36240fdfa0 2024-02-16T19:28:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.639+0000 7f360bfff700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f360c06d900 0x7f360c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.639+0000 7f360bfff700 1 -- 172.21.15.40:0/3918343196 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f361408e580 con 0x7f36240fdfa0 2024-02-16T19:28:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.639+0000 7f36237fe700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f360c06d900 0x7f360c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.639+0000 7f36237fe700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f360c06d900 0x7f360c06fdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f36240fde00 tx=0x7f361800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:24.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.644+0000 7f360bfff700 1 -- 172.21.15.40:0/3918343196 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f361405b760 con 0x7f36240fdfa0 2024-02-16T19:28:24.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:24 smithi163 ceph-mon[81637]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:24.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:24 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:24.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:24 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:24.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:24 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:24.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:24 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/608144891' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:24.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:24 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3675846225' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:28:24.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:24 smithi040 ceph-mon[94210]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:24.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:24 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:24.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:24 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:24.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:24 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:24.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:24 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/608144891' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:24.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:24 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3675846225' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:28:24.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.938+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3624105870 con 0x7f36240fdfa0 2024-02-16T19:28:24.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.939+0000 7f360bfff700 1 -- 172.21.15.40:0/3918343196 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1328 (secure 0 0 0) 0x7f361405b2f0 con 0x7f36240fdfa0 2024-02-16T19:28:24.940 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 69/360 objects degraded (19.167%), 19 pgs degraded 2024-02-16T19:28:24.940 INFO:teuthology.orchestra.run.smithi040.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-16T19:28:24.940 INFO:teuthology.orchestra.run.smithi040.stdout: osd.0 (root=default,host=smithi040) is down 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 69/360 objects degraded (19.167%), 19 pgs degraded 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-02-16T19:28:24.941 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f360c06d900 msgr2=0x7f360c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f360c06d900 0x7f360c06fdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f36240fde00 tx=0x7f361800b040 comp rx=0 tx=0).stop 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 msgr2=0x7f362410dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 0x7f362410dc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3614009fd0 tx=0x7f3614009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 shutdown_connections 2024-02-16T19:28:24.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36240fcd90 0x7f362410d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f360c06d900 0x7f360c06fdb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 --2- 172.21.15.40:0/3918343196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36240fdfa0 0x7f362410dc60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:24.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 >> 172.21.15.40:0/3918343196 conn(0x7f36240f8450 msgr2=0x7f36241011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:24.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 shutdown_connections 2024-02-16T19:28:24.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:24.941+0000 7f3628b9d700 1 -- 172.21.15.40:0/3918343196 wait complete. 2024-02-16T19:28:25.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:25 smithi040 ceph-mon[94210]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:25.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:25 smithi040 ceph-mon[94210]: Health check failed: Degraded data redundancy: 69/360 objects degraded (19.167%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T19:28:25.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:25 smithi040 ceph-mon[94210]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:25.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:25 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3918343196' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:28:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:25 smithi163 ceph-mon[81637]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:25 smithi163 ceph-mon[81637]: Health check failed: Degraded data redundancy: 69/360 objects degraded (19.167%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T19:28:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:25 smithi163 ceph-mon[81637]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:25 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3918343196' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:28:27.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:27 smithi040 ceph-mon[94210]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:27 smithi163 ceph-mon[81637]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:28.344 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:28 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[103005]: 2024-02-16T19:28:28.083+0000 7f8ec72317c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-02-16T19:28:28.644 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:28 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[103005]: 2024-02-16T19:28:28.528+0000 7f8ec72317c0 -1 osd.0 49 log_to_monitors true 2024-02-16T19:28:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:29 smithi163 ceph-mon[81637]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:29 smithi163 ceph-mon[81637]: from='osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T19:28:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:29.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:29 smithi040 ceph-mon[94210]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:29.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:29 smithi040 ceph-mon[94210]: from='osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-16T19:28:29.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:29.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:30.635 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:28:30 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[103005]: 2024-02-16T19:28:30.335+0000 7f8eb4681700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:28:30.635 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:30 smithi040 ceph-mon[94210]: from='osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T19:28:30.635 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:30 smithi040 ceph-mon[94210]: failed to encode map e52 with expected crc 2024-02-16T19:28:30.635 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:30 smithi040 ceph-mon[94210]: osdmap e52: 6 total, 5 up, 6 in 2024-02-16T19:28:30.635 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:30 smithi040 ceph-mon[94210]: from='osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:28:30.636 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:30 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:30.636 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:30 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:30.636 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:30 smithi040 ceph-mon[94210]: from='osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162]' entity='osd.0' 2024-02-16T19:28:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:30 smithi163 ceph-mon[81637]: from='osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-16T19:28:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:30 smithi163 ceph-mon[81637]: failed to encode map e52 with expected crc 2024-02-16T19:28:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:30 smithi163 ceph-mon[81637]: osdmap e52: 6 total, 5 up, 6 in 2024-02-16T19:28:30.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:30 smithi163 ceph-mon[81637]: from='osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:28:30.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:30 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:30.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:30 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:30.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:30 smithi163 ceph-mon[81637]: from='osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162]' entity='osd.0' 2024-02-16T19:28:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:31 smithi163 ceph-mon[81637]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:31 smithi163 ceph-mon[81637]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:28:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:31 smithi163 ceph-mon[81637]: osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162] boot 2024-02-16T19:28:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:31 smithi163 ceph-mon[81637]: osdmap e53: 6 total, 6 up, 6 in 2024-02-16T19:28:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:28:32.269 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:31 smithi040 ceph-mon[94210]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:32.269 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:31 smithi040 ceph-mon[94210]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:28:32.269 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:31 smithi040 ceph-mon[94210]: osd.0 [v2:172.21.15.40:6802/3792625162,v1:172.21.15.40:6803/3792625162] boot 2024-02-16T19:28:32.269 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:31 smithi040 ceph-mon[94210]: osdmap e53: 6 total, 6 up, 6 in 2024-02-16T19:28:32.269 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-16T19:28:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: failed to encode map e53 with expected crc 2024-02-16T19:28:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: failed to encode map e53 with expected crc 2024-02-16T19:28:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: failed to encode map e54 with expected crc 2024-02-16T19:28:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: osdmap e54: 6 total, 6 up, 6 in 2024-02-16T19:28:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: failed to encode map e54 with expected crc 2024-02-16T19:28:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: failed to encode map e54 with expected crc 2024-02-16T19:28:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:33 smithi163 ceph-mon[81637]: failed to encode map e54 with expected crc 2024-02-16T19:28:33.566 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: failed to encode map e53 with expected crc 2024-02-16T19:28:33.566 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: failed to encode map e53 with expected crc 2024-02-16T19:28:33.566 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 69/360 objects degraded (19.167%) 2024-02-16T19:28:33.566 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:33.566 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:33.566 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: failed to encode map e54 with expected crc 2024-02-16T19:28:33.567 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: osdmap e54: 6 total, 6 up, 6 in 2024-02-16T19:28:33.567 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: failed to encode map e54 with expected crc 2024-02-16T19:28:33.567 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: failed to encode map e54 with expected crc 2024-02-16T19:28:33.567 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:33 smithi040 ceph-mon[94210]: failed to encode map e54 with expected crc 2024-02-16T19:28:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: failed to encode map e54 with expected crc 2024-02-16T19:28:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: failed to encode map e54 with expected crc 2024-02-16T19:28:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: failed to encode map e54 with expected crc 2024-02-16T19:28:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: mgrmap e38: smithi040.jzygzo(active, since 91s), standbys: smithi163.wjfclz 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:28:34.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:34 smithi163 ceph-mon[81637]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-16T19:28:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: failed to encode map e54 with expected crc 2024-02-16T19:28:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: failed to encode map e54 with expected crc 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: failed to encode map e54 with expected crc 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: mgrmap e38: smithi040.jzygzo(active, since 91s), standbys: smithi163.wjfclz 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:28:34.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:34 smithi040 ceph-mon[94210]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-16T19:28:35.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:35 smithi163 ceph-mon[81637]: pgmap v54: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s; 12/360 objects degraded (3.333%) 2024-02-16T19:28:35.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:35 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 12/360 objects degraded (3.333%), 3 pgs degraded (PG_DEGRADED) 2024-02-16T19:28:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:35 smithi040 ceph-mon[94210]: pgmap v54: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s; 12/360 objects degraded (3.333%) 2024-02-16T19:28:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:35 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 12/360 objects degraded (3.333%), 3 pgs degraded (PG_DEGRADED) 2024-02-16T19:28:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:36 smithi040 ceph-mon[94210]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/360 objects degraded (3.333%), 3 pgs degraded) 2024-02-16T19:28:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:36 smithi040 ceph-mon[94210]: Cluster is now healthy 2024-02-16T19:28:36.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:36 smithi163 ceph-mon[81637]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/360 objects degraded (3.333%), 3 pgs degraded) 2024-02-16T19:28:36.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:36 smithi163 ceph-mon[81637]: Cluster is now healthy 2024-02-16T19:28:37.639 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:37 smithi163 ceph-mon[81637]: pgmap v55: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 917 B/s wr, 0 op/s 2024-02-16T19:28:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:37 smithi040 ceph-mon[94210]: pgmap v55: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 917 B/s wr, 0 op/s 2024-02-16T19:28:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:39 smithi040 ceph-mon[94210]: pgmap v56: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T19:28:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:39 smithi163 ceph-mon[81637]: pgmap v56: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-16T19:28:41.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:41 smithi040 ceph-mon[94210]: pgmap v57: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 694 B/s wr, 0 op/s 2024-02-16T19:28:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:41 smithi163 ceph-mon[81637]: pgmap v57: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 694 B/s wr, 0 op/s 2024-02-16T19:28:43.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:43 smithi040 ceph-mon[94210]: pgmap v58: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 614 B/s wr, 0 op/s 2024-02-16T19:28:43.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:43 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:43.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:43 smithi163 ceph-mon[81637]: pgmap v58: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 614 B/s wr, 0 op/s 2024-02-16T19:28:43.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:43 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:45.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:45 smithi040 ceph-mon[94210]: pgmap v59: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 518 B/s wr, 0 op/s 2024-02-16T19:28:45.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:45 smithi163 ceph-mon[81637]: pgmap v59: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 518 B/s wr, 0 op/s 2024-02-16T19:28:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:47 smithi040 ceph-mon[94210]: pgmap v60: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:47 smithi163 ceph-mon[81637]: pgmap v60: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:28:49.592 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:49 smithi040 ceph-mon[94210]: pgmap v61: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:49.592 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:28:49.593 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:49 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:28:49.593 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:49 smithi040 ceph-mon[94210]: Upgrade: osd.1 is safe to restart 2024-02-16T19:28:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:49 smithi163 ceph-mon[81637]: pgmap v61: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:28:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:49 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-16T19:28:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:49 smithi163 ceph-mon[81637]: Upgrade: osd.1 is safe to restart 2024-02-16T19:28:51.771 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:51 smithi040 ceph-mon[94210]: pgmap v62: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:51.771 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:51 smithi040 ceph-mon[94210]: Upgrade: Updating osd.1 2024-02-16T19:28:51.771 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:51.771 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:28:51.771 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:51.771 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:51 smithi040 ceph-mon[94210]: Deploying daemon osd.1 on smithi040 2024-02-16T19:28:51.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:51 smithi163 ceph-mon[81637]: pgmap v62: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:51.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:51 smithi163 ceph-mon[81637]: Upgrade: Updating osd.1 2024-02-16T19:28:51.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:28:51.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-16T19:28:51.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:28:51.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:51 smithi163 ceph-mon[81637]: Deploying daemon osd.1 on smithi040 2024-02-16T19:28:53.861 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:53 smithi040 ceph-mon[94210]: pgmap v63: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:53 smithi163 ceph-mon[81637]: pgmap v63: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:54.118 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:53 smithi040 systemd[1]: Stopping Ceph osd.1 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:28:54.394 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:54 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[55736]: 2024-02-16T19:28:54.116+0000 7f3831663700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:28:54.394 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:54 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[55736]: 2024-02-16T19:28:54.116+0000 7f3831663700 -1 osd.1 54 *** Got signal Terminated *** 2024-02-16T19:28:54.394 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:54 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[55736]: 2024-02-16T19:28:54.116+0000 7f3831663700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:28:54.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:54 smithi040 ceph-mon[94210]: osd.1 marked itself down and dead 2024-02-16T19:28:54.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:54 smithi163 ceph-mon[81637]: osd.1 marked itself down and dead 2024-02-16T19:28:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.061+0000 7ff0256a1700 1 -- 172.21.15.40:0/357869857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0201000e0 msgr2=0x7ff020102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.061+0000 7ff0256a1700 1 --2- 172.21.15.40:0/357869857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0201000e0 0x7ff020102520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff008004940 tx=0x7ff008009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.061+0000 7ff0256a1700 1 -- 172.21.15.40:0/357869857 shutdown_connections 2024-02-16T19:28:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.061+0000 7ff0256a1700 1 --2- 172.21.15.40:0/357869857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0201000e0 0x7ff020102520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.061+0000 7ff0256a1700 1 --2- 172.21.15.40:0/357869857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff020066c50 0x7ff0200ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.061+0000 7ff0256a1700 1 -- 172.21.15.40:0/357869857 >> 172.21.15.40:0/357869857 conn(0x7ff0200f7520 msgr2=0x7ff0200f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.061+0000 7ff0256a1700 1 -- 172.21.15.40:0/357869857 shutdown_connections 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.061+0000 7ff0256a1700 1 -- 172.21.15.40:0/357869857 wait complete. 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff0256a1700 1 Processor -- start 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff0256a1700 1 -- start start 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff0256a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff020066c50 0x7ff02010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff0256a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0201000e0 0x7ff02010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff0256a1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02010e490 con 0x7ff020066c50 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff0256a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02010ed30 con 0x7ff0201000e0 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff01effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff020066c50 0x7ff02010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff01effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff020066c50 0x7ff02010d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47966/0 (socket says 172.21.15.40:47966) 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff01effd700 1 -- 172.21.15.40:0/1587646862 learned_addr learned my addr 172.21.15.40:0/1587646862 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff01e7fc700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0201000e0 0x7ff02010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff01effd700 1 -- 172.21.15.40:0/1587646862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0201000e0 msgr2=0x7ff02010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff01effd700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0201000e0 0x7ff02010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.062+0000 7ff01effd700 1 -- 172.21.15.40:0/1587646862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008005040 con 0x7ff020066c50 2024-02-16T19:28:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.063+0000 7ff01effd700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff020066c50 0x7ff02010d950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff018002a50 tx=0x7ff0180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.063+0000 7ff0177fe700 1 -- 172.21.15.40:0/1587646862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff018007cc0 con 0x7ff020066c50 2024-02-16T19:28:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.063+0000 7ff0177fe700 1 -- 172.21.15.40:0/1587646862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff01800f040 con 0x7ff020066c50 2024-02-16T19:28:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.063+0000 7ff0177fe700 1 -- 172.21.15.40:0/1587646862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff018009640 con 0x7ff020066c50 2024-02-16T19:28:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.063+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02010f010 con 0x7ff020066c50 2024-02-16T19:28:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.063+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02010f550 con 0x7ff020066c50 2024-02-16T19:28:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.064+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02004bed0 con 0x7ff020066c50 2024-02-16T19:28:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.065+0000 7ff0177fe700 1 -- 172.21.15.40:0/1587646862 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff0180097a0 con 0x7ff020066c50 2024-02-16T19:28:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.065+0000 7ff0177fe700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff00c06d7e0 0x7ff00c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.066+0000 7ff0177fe700 1 -- 172.21.15.40:0/1587646862 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff01808cbd0 con 0x7ff020066c50 2024-02-16T19:28:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.066+0000 7ff01e7fc700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff00c06d7e0 0x7ff00c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.066+0000 7ff01e7fc700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff00c06d7e0 0x7ff00c06fc90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff00800b3b0 tx=0x7ff008009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.071+0000 7ff0177fe700 1 -- 172.21.15.40:0/1587646862 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff018059fd0 con 0x7ff020066c50 2024-02-16T19:28:55.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.299+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff020100d60 con 0x7ff00c06d7e0 2024-02-16T19:28:55.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.300+0000 7ff0177fe700 1 -- 172.21.15.40:0/1587646862 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff020100d60 con 0x7ff00c06d7e0 2024-02-16T19:28:55.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff00c06d7e0 msgr2=0x7ff00c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff00c06d7e0 0x7ff00c06fc90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff00800b3b0 tx=0x7ff008009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff020066c50 msgr2=0x7ff02010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff020066c50 0x7ff02010d950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff018002a50 tx=0x7ff0180046b0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 shutdown_connections 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff00c06d7e0 0x7ff00c06fc90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff020066c50 0x7ff02010d950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 --2- 172.21.15.40:0/1587646862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0201000e0 0x7ff02010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 >> 172.21.15.40:0/1587646862 conn(0x7ff0200f7520 msgr2=0x7ff0200f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 shutdown_connections 2024-02-16T19:28:55.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.302+0000 7ff0256a1700 1 -- 172.21.15.40:0/1587646862 wait complete. 2024-02-16T19:28:55.314 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:28:55.314 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:55 smithi040 podman[108022]: 2024-02-16 19:28:55.096239424 +0000 UTC m=+1.092377683 container died bcce2fd16e5e2705b27fca6e40966b462f52725df4a7702c1a00ff08ba5a7ace (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=quincy-b88fae6) 2024-02-16T19:28:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.407+0000 7f4dd1a61700 1 -- 172.21.15.40:0/3938546378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fcef0 msgr2=0x7f4dcc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.407+0000 7f4dd1a61700 1 --2- 172.21.15.40:0/3938546378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fcef0 0x7f4dcc0fd320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc004940 tx=0x7f4dbc009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.407+0000 7f4dd1a61700 1 -- 172.21.15.40:0/3938546378 shutdown_connections 2024-02-16T19:28:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.407+0000 7f4dd1a61700 1 --2- 172.21.15.40:0/3938546378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dcc0fe0e0 0x7f4dcc0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.407+0000 7f4dd1a61700 1 --2- 172.21.15.40:0/3938546378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fcef0 0x7f4dcc0fd320 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.407+0000 7f4dd1a61700 1 -- 172.21.15.40:0/3938546378 >> 172.21.15.40:0/3938546378 conn(0x7f4dcc0f8620 msgr2=0x7f4dcc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.407+0000 7f4dd1a61700 1 -- 172.21.15.40:0/3938546378 shutdown_connections 2024-02-16T19:28:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.407+0000 7f4dd1a61700 1 -- 172.21.15.40:0/3938546378 wait complete. 2024-02-16T19:28:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dd1a61700 1 Processor -- start 2024-02-16T19:28:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dd1a61700 1 -- start start 2024-02-16T19:28:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dd1a61700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dcc0fcef0 0x7f4dcc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dd1a61700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fe0e0 0x7f4dcc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dd1a61700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc10e4b0 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dd1a61700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc10ed50 con 0x7f4dcc0fcef0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dca7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fe0e0 0x7f4dcc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dca7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fe0e0 0x7f4dcc10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47974/0 (socket says 172.21.15.40:47974) 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.408+0000 7f4dca7fc700 1 -- 172.21.15.40:0/1850072756 learned_addr learned my addr 172.21.15.40:0/1850072756 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4dcaffd700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dcc0fcef0 0x7f4dcc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4dca7fc700 1 -- 172.21.15.40:0/1850072756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dcc0fcef0 msgr2=0x7f4dcc10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4dca7fc700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dcc0fcef0 0x7f4dcc10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4dca7fc700 1 -- 172.21.15.40:0/1850072756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc005040 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4dca7fc700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fe0e0 0x7f4dcc10deb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0002a50 tx=0x7f4dc00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4db37fe700 1 -- 172.21.15.40:0/1850072756 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dc0007cc0 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dcc10f030 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4db37fe700 1 -- 172.21.15.40:0/1850072756 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dc000f040 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4db37fe700 1 -- 172.21.15.40:0/1850072756 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dc000da30 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.409+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dcc10f570 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.410+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc04bed0 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.411+0000 7f4db37fe700 1 -- 172.21.15.40:0/1850072756 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4dc000db90 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.411+0000 7f4db37fe700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4db406d7e0 0x7f4db406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.411+0000 7f4db37fe700 1 -- 172.21.15.40:0/1850072756 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4dc008cc70 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.411+0000 7f4dcaffd700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4db406d7e0 0x7f4db406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.411+0000 7f4dcaffd700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4db406d7e0 0x7f4db406fc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc000c00 tx=0x7f4dbc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.416+0000 7f4db37fe700 1 -- 172.21.15.40:0/1850072756 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4dc0059ff0 con 0x7f4dcc0fe0e0 2024-02-16T19:28:55.644 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:55 smithi040 podman[108022]: 2024-02-16 19:28:55.469580439 +0000 UTC m=+1.465718703 container cleanup bcce2fd16e5e2705b27fca6e40966b462f52725df4a7702c1a00ff08ba5a7ace (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=quincy-b88fae6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:28:55.644 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:55 smithi040 bash[108022]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1 2024-02-16T19:28:55.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:55 smithi040 ceph-mon[94210]: pgmap v64: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:55.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:55 smithi040 ceph-mon[94210]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:28:55.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:55 smithi040 ceph-mon[94210]: failed to encode map e55 with expected crc 2024-02-16T19:28:55.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:55 smithi040 ceph-mon[94210]: failed to encode map e55 with expected crc 2024-02-16T19:28:55.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:55 smithi040 ceph-mon[94210]: failed to encode map e55 with expected crc 2024-02-16T19:28:55.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:55 smithi040 ceph-mon[94210]: osdmap e55: 6 total, 5 up, 6 in 2024-02-16T19:28:55.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:55 smithi040 ceph-mon[94210]: failed to encode map e55 with expected crc 2024-02-16T19:28:55.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.684+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dcc10f9c0 con 0x7f4db406d7e0 2024-02-16T19:28:55.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.685+0000 7f4db37fe700 1 -- 172.21.15.40:0/1850072756 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4dcc10f9c0 con 0x7f4db406d7e0 2024-02-16T19:28:55.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4db406d7e0 msgr2=0x7f4db406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4db406d7e0 0x7f4db406fc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc000c00 tx=0x7f4dbc009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:55.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fe0e0 msgr2=0x7f4dcc10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fe0e0 0x7f4dcc10deb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0002a50 tx=0x7f4dc00046b0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 shutdown_connections 2024-02-16T19:28:55.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dcc0fcef0 0x7f4dcc10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4db406d7e0 0x7f4db406fc90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 --2- 172.21.15.40:0/1850072756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc0fe0e0 0x7f4dcc10deb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 >> 172.21.15.40:0/1850072756 conn(0x7f4dcc0f8620 msgr2=0x7f4dcc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:55.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 shutdown_connections 2024-02-16T19:28:55.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.687+0000 7f4dd1a61700 1 -- 172.21.15.40:0/1850072756 wait complete. 2024-02-16T19:28:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.790+0000 7fef172cf700 1 -- 172.21.15.40:0/803907219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fe0e0 msgr2=0x7fef100fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.790+0000 7fef172cf700 1 --2- 172.21.15.40:0/803907219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fe0e0 0x7fef100fe550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef04004940 tx=0x7fef04009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.790+0000 7fef172cf700 1 -- 172.21.15.40:0/803907219 shutdown_connections 2024-02-16T19:28:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.790+0000 7fef172cf700 1 --2- 172.21.15.40:0/803907219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fe0e0 0x7fef100fe550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.790+0000 7fef172cf700 1 --2- 172.21.15.40:0/803907219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef100fcef0 0x7fef100fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.790+0000 7fef172cf700 1 -- 172.21.15.40:0/803907219 >> 172.21.15.40:0/803907219 conn(0x7fef100f8620 msgr2=0x7fef100faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.790+0000 7fef172cf700 1 -- 172.21.15.40:0/803907219 shutdown_connections 2024-02-16T19:28:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef172cf700 1 -- 172.21.15.40:0/803907219 wait complete. 2024-02-16T19:28:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef172cf700 1 Processor -- start 2024-02-16T19:28:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef172cf700 1 -- start start 2024-02-16T19:28:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef172cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fcef0 0x7fef1010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef172cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef100fe0e0 0x7fef1010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef172cf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1010e560 con 0x7fef100fcef0 2024-02-16T19:28:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef172cf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1010eda0 con 0x7fef100fe0e0 2024-02-16T19:28:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef1506b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fcef0 0x7fef1010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef1506b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fcef0 0x7fef1010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47996/0 (socket says 172.21.15.40:47996) 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.791+0000 7fef1506b700 1 -- 172.21.15.40:0/916890130 learned_addr learned my addr 172.21.15.40:0/916890130 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef1486a700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef100fe0e0 0x7fef1010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef1506b700 1 -- 172.21.15.40:0/916890130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef100fe0e0 msgr2=0x7fef1010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef1506b700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef100fe0e0 0x7fef1010ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef1506b700 1 -- 172.21.15.40:0/916890130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04005040 con 0x7fef100fcef0 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef1506b700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fcef0 0x7fef1010d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00b790 tx=0x7fef0c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef01ffb700 1 -- 172.21.15.40:0/916890130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef0c00f6f0 con 0x7fef100fcef0 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef01ffb700 1 -- 172.21.15.40:0/916890130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef0c00fd20 con 0x7fef100fcef0 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1010f080 con 0x7fef100fcef0 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef01ffb700 1 -- 172.21.15.40:0/916890130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef0c0103a0 con 0x7fef100fcef0 2024-02-16T19:28:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.792+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1010f5c0 con 0x7fef100fcef0 2024-02-16T19:28:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.793+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1004bed0 con 0x7fef100fcef0 2024-02-16T19:28:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.793+0000 7fef01ffb700 1 -- 172.21.15.40:0/916890130 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fef0c011400 con 0x7fef100fcef0 2024-02-16T19:28:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.794+0000 7fef01ffb700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7feefc06d7e0 0x7feefc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.794+0000 7fef01ffb700 1 -- 172.21.15.40:0/916890130 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fef0c08cd30 con 0x7fef100fcef0 2024-02-16T19:28:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.794+0000 7fef1486a700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7feefc06d7e0 0x7feefc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.794+0000 7fef1486a700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7feefc06d7e0 0x7feefc06fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef04004f90 tx=0x7fef04009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:55.799+0000 7fef01ffb700 1 -- 172.21.15.40:0/916890130 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef0c05a0b0 con 0x7fef100fcef0 2024-02-16T19:28:55.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:55 smithi163 ceph-mon[81637]: pgmap v64: 65 pgs: 65 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:55.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:55 smithi163 ceph-mon[81637]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:28:55.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:55 smithi163 ceph-mon[81637]: failed to encode map e55 with expected crc 2024-02-16T19:28:55.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:55 smithi163 ceph-mon[81637]: failed to encode map e55 with expected crc 2024-02-16T19:28:55.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:55 smithi163 ceph-mon[81637]: failed to encode map e55 with expected crc 2024-02-16T19:28:55.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:55 smithi163 ceph-mon[81637]: osdmap e55: 6 total, 5 up, 6 in 2024-02-16T19:28:55.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:55 smithi163 ceph-mon[81637]: failed to encode map e55 with expected crc 2024-02-16T19:28:56.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.062+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef10102ac0 con 0x7feefc06d7e0 2024-02-16T19:28:56.064 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:55 smithi040 podman[108065]: 2024-02-16 19:28:55.763170579 +0000 UTC m=+0.662735020 container remove bcce2fd16e5e2705b27fca6e40966b462f52725df4a7702c1a00ff08ba5a7ace (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=quincy-b88fae6, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-16T19:28:56.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.073+0000 7fef01ffb700 1 -- 172.21.15.40:0/916890130 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fef10102ac0 con 0x7feefc06d7e0 2024-02-16T19:28:56.074 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:28:56.074 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (3m) 27s ago 12m 17.9M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:28:56.074 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (58s) 27s ago 12m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:28:56.074 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (53s) 50s ago 11m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:28:56.074 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (3m) 27s ago 12m 81.5M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (6m) 27s ago 6m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (6m) 27s ago 6m 13.5M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (6m) 50s ago 6m 17.0M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (6m) 50s ago 6m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (2m) 27s ago 13m 497M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (2m) 50s ago 11m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (2m) 27s ago 14m 42.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (100s) 50s ago 11m 32.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (3m) 27s ago 12m 15.6M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (2m) 50s ago 11m 16.0M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (33s) 27s ago 10m 15.0M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (10m) 27s ago 10m 243M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 bcce2fd16e5e 2024-02-16T19:28:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (9m) 27s ago 9m 197M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:28:56.076 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (9m) 50s ago 9m 270M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:28:56.076 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (8m) 50s ago 8m 233M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:28:56.076 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (8m) 50s ago 8m 235M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:28:56.076 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (2m) 27s ago 11m 50.8M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:28:56.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7feefc06d7e0 msgr2=0x7feefc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7feefc06d7e0 0x7feefc06fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef04004f90 tx=0x7fef04009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fcef0 msgr2=0x7fef1010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fcef0 0x7fef1010d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00b790 tx=0x7fef0c009220 comp rx=0 tx=0).stop 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 shutdown_connections 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7feefc06d7e0 0x7feefc06fc90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef100fcef0 0x7fef1010d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 --2- 172.21.15.40:0/916890130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef100fe0e0 0x7fef1010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 >> 172.21.15.40:0/916890130 conn(0x7fef100f8620 msgr2=0x7fef101013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 shutdown_connections 2024-02-16T19:28:56.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.075+0000 7fef172cf700 1 -- 172.21.15.40:0/916890130 wait complete. 2024-02-16T19:28:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.177+0000 7f45bd086700 1 -- 172.21.15.40:0/3603851716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 msgr2=0x7f45b80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.177+0000 7f45bd086700 1 --2- 172.21.15.40:0/3603851716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 0x7f45b80ffe60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f45a8004940 tx=0x7f45a8009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.177+0000 7f45bd086700 1 -- 172.21.15.40:0/3603851716 shutdown_connections 2024-02-16T19:28:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.177+0000 7f45bd086700 1 --2- 172.21.15.40:0/3603851716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b81003a0 0x7f45b8102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.177+0000 7f45bd086700 1 --2- 172.21.15.40:0/3603851716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 0x7f45b80ffe60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.177+0000 7f45bd086700 1 -- 172.21.15.40:0/3603851716 >> 172.21.15.40:0/3603851716 conn(0x7f45b80f7520 msgr2=0x7f45b80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.177+0000 7f45bd086700 1 -- 172.21.15.40:0/3603851716 shutdown_connections 2024-02-16T19:28:56.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.177+0000 7f45bd086700 1 -- 172.21.15.40:0/3603851716 wait complete. 2024-02-16T19:28:56.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.178+0000 7f45bd086700 1 Processor -- start 2024-02-16T19:28:56.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.178+0000 7f45bd086700 1 -- start start 2024-02-16T19:28:56.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.178+0000 7f45bd086700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 0x7f45b810d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.178+0000 7f45bd086700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b81003a0 0x7f45b810de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.178+0000 7f45bd086700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b810e430 con 0x7f45b80fda80 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.178+0000 7f45bd086700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b810ecd0 con 0x7f45b81003a0 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.178+0000 7f45b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 0x7f45b810d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.179+0000 7f45b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 0x7f45b810d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48020/0 (socket says 172.21.15.40:48020) 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.179+0000 7f45b6d9d700 1 -- 172.21.15.40:0/1979160082 learned_addr learned my addr 172.21.15.40:0/1979160082 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.179+0000 7f45b659c700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b81003a0 0x7f45b810de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.179+0000 7f45b6d9d700 1 -- 172.21.15.40:0/1979160082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b81003a0 msgr2=0x7f45b810de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.179+0000 7f45b6d9d700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b81003a0 0x7f45b810de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.179+0000 7f45b6d9d700 1 -- 172.21.15.40:0/1979160082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a8005040 con 0x7f45b80fda80 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.179+0000 7f45b6d9d700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 0x7f45b810d8f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f45a80093b0 tx=0x7f45a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.179+0000 7f459f7fe700 1 -- 172.21.15.40:0/1979160082 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45a800e040 con 0x7f45b80fda80 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.180+0000 7f459f7fe700 1 -- 172.21.15.40:0/1979160082 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45a800bcc0 con 0x7f45b80fda80 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.180+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b810ef50 con 0x7f45b80fda80 2024-02-16T19:28:56.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.180+0000 7f459f7fe700 1 -- 172.21.15.40:0/1979160082 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45a801c630 con 0x7f45b80fda80 2024-02-16T19:28:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.180+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b810f490 con 0x7f45b80fda80 2024-02-16T19:28:56.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.181+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b804bed0 con 0x7f45b80fda80 2024-02-16T19:28:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.181+0000 7f459f7fe700 1 -- 172.21.15.40:0/1979160082 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f45a8013980 con 0x7f45b80fda80 2024-02-16T19:28:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.182+0000 7f459f7fe700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f45a006d7e0 0x7f45a006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.182+0000 7f459f7fe700 1 -- 172.21.15.40:0/1979160082 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f45a808e250 con 0x7f45b80fda80 2024-02-16T19:28:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.182+0000 7f45b659c700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f45a006d7e0 0x7f45a006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.182+0000 7f45b659c700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f45a006d7e0 0x7f45a006fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45ac004f60 tx=0x7f45ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:56.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.186+0000 7f459f7fe700 1 -- 172.21.15.40:0/1979160082 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f45a805b650 con 0x7f45b80fda80 2024-02-16T19:28:56.327 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:56 smithi040 podman[108203]: 2024-02-16 19:28:56.057671349 +0000 UTC m=+0.021899664 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:28:56.327 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:56 smithi040 podman[108203]: 2024-02-16 19:28:56.189373159 +0000 UTC m=+0.153601474 container create 0433972f4f8331e3883057aa2bce86cd04cfe501b12a730666f674527a166cf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS) 2024-02-16T19:28:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.485+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f45b810f840 con 0x7f45b80fda80 2024-02-16T19:28:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.486+0000 7f459f7fe700 1 -- 172.21.15.40:0/1979160082 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f45a805b1e0 con 0x7f45b80fda80 2024-02-16T19:28:56.487 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4, 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:28:56.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:28:56.489 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:28:56.489 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 8, 2024-02-16T19:28:56.489 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-16T19:28:56.489 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:28:56.489 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f45a006d7e0 msgr2=0x7f45a006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f45a006d7e0 0x7f45a006fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45ac004f60 tx=0x7f45ac00b040 comp rx=0 tx=0).stop 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 msgr2=0x7f45b810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 0x7f45b810d8f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f45a80093b0 tx=0x7f45a8009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 shutdown_connections 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f45a006d7e0 0x7f45a006fc90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45b80fda80 0x7f45b810d8f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 --2- 172.21.15.40:0/1979160082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b81003a0 0x7f45b810de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 >> 172.21.15.40:0/1979160082 conn(0x7f45b80f7520 msgr2=0x7f45b80f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 shutdown_connections 2024-02-16T19:28:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.488+0000 7f45bd086700 1 -- 172.21.15.40:0/1979160082 wait complete. 2024-02-16T19:28:56.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.593+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/2113545543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fbe40 msgr2=0x7fbc580fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.593+0000 7fbc5e6e0700 1 --2- 172.21.15.40:0/2113545543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fbe40 0x7fbc580fc250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbc48004940 tx=0x7fbc48009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:56.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.593+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/2113545543 shutdown_connections 2024-02-16T19:28:56.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.593+0000 7fbc5e6e0700 1 --2- 172.21.15.40:0/2113545543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc580fcfb0 0x7fbc580fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.593+0000 7fbc5e6e0700 1 --2- 172.21.15.40:0/2113545543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fbe40 0x7fbc580fc250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.593+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/2113545543 >> 172.21.15.40:0/2113545543 conn(0x7fbc580f74e0 msgr2=0x7fbc580f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:56.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.593+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/2113545543 shutdown_connections 2024-02-16T19:28:56.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.593+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/2113545543 wait complete. 2024-02-16T19:28:56.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.594+0000 7fbc5e6e0700 1 Processor -- start 2024-02-16T19:28:56.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.594+0000 7fbc5e6e0700 1 -- start start 2024-02-16T19:28:56.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc5e6e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc580fbe40 0x7fbc5810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc5e6e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fcfb0 0x7fbc5810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc5e6e0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5810e480 con 0x7fbc580fcfb0 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc5e6e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5810ed20 con 0x7fbc580fbe40 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fcfb0 0x7fbc5810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fcfb0 0x7fbc5810de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48042/0 (socket says 172.21.15.40:48042) 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc577fe700 1 -- 172.21.15.40:0/1510012154 learned_addr learned my addr 172.21.15.40:0/1510012154 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc57fff700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc580fbe40 0x7fbc5810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc577fe700 1 -- 172.21.15.40:0/1510012154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc580fbe40 msgr2=0x7fbc5810d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc577fe700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc580fbe40 0x7fbc5810d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc577fe700 1 -- 172.21.15.40:0/1510012154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48005040 con 0x7fbc580fcfb0 2024-02-16T19:28:56.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc57fff700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc580fbe40 0x7fbc5810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:28:56.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc577fe700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fcfb0 0x7fbc5810de80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c8c0 tx=0x7fbc4c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:56.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc54ff9700 1 -- 172.21.15.40:0/1510012154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc4c00a980 con 0x7fbc580fcfb0 2024-02-16T19:28:56.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc54ff9700 1 -- 172.21.15.40:0/1510012154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc4c00ce60 con 0x7fbc580fcfb0 2024-02-16T19:28:56.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5810f000 con 0x7fbc580fcfb0 2024-02-16T19:28:56.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc54ff9700 1 -- 172.21.15.40:0/1510012154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc4c0115e0 con 0x7fbc580fcfb0 2024-02-16T19:28:56.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.595+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5810f540 con 0x7fbc580fcfb0 2024-02-16T19:28:56.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.597+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5804bed0 con 0x7fbc580fcfb0 2024-02-16T19:28:56.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.597+0000 7fbc54ff9700 1 -- 172.21.15.40:0/1510012154 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbc4c012400 con 0x7fbc580fcfb0 2024-02-16T19:28:56.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.598+0000 7fbc54ff9700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbc4006d7e0 0x7fbc4006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.598+0000 7fbc57fff700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbc4006d7e0 0x7fbc4006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.598+0000 7fbc54ff9700 1 -- 172.21.15.40:0/1510012154 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbc4c08ceb0 con 0x7fbc580fcfb0 2024-02-16T19:28:56.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.598+0000 7fbc57fff700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbc4006d7e0 0x7fbc4006fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc48000c00 tx=0x7fbc48009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:56.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.602+0000 7fbc54ff9700 1 -- 172.21.15.40:0/1510012154 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc4c05a350 con 0x7fbc580fcfb0 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: osdmap e56: 6 total, 5 up, 6 in 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:56.775 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:56 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1979160082' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:56.776 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:56 smithi040 podman[108203]: 2024-02-16 19:28:56.460380157 +0000 UTC m=+0.424608476 container init 0433972f4f8331e3883057aa2bce86cd04cfe501b12a730666f674527a166cf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-16T19:28:56.776 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:56 smithi040 podman[108203]: 2024-02-16 19:28:56.463672266 +0000 UTC m=+0.427900582 container start 0433972f4f8331e3883057aa2bce86cd04cfe501b12a730666f674527a166cf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-16T19:28:56.776 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:56 smithi040 podman[108203]: 2024-02-16 19:28:56.559719763 +0000 UTC m=+0.523948093 container attach 0433972f4f8331e3883057aa2bce86cd04cfe501b12a730666f674527a166cf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-16T19:28:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.849+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbc58104930 con 0x7fbc580fcfb0 2024-02-16T19:28:56.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.850+0000 7fbc54ff9700 1 -- 172.21.15.40:0/1510012154 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fbc4c0126b0 con 0x7fbc580fcfb0 2024-02-16T19:28:56.851 INFO:teuthology.orchestra.run.smithi040.stdout:e20 2024-02-16T19:28:56.851 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:28:56.851 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:28:56.851 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 20 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:27:43.925056+0000 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:28:56.852 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:28:56.853 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:28:56.854 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:28:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbc4006d7e0 msgr2=0x7fbc4006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbc4006d7e0 0x7fbc4006fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc48000c00 tx=0x7fbc48009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fcfb0 msgr2=0x7fbc5810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fcfb0 0x7fbc5810de80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c8c0 tx=0x7fbc4c0044c0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 shutdown_connections 2024-02-16T19:28:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc580fbe40 0x7fbc5810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbc4006d7e0 0x7fbc4006fc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 --2- 172.21.15.40:0/1510012154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc580fcfb0 0x7fbc5810de80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 >> 172.21.15.40:0/1510012154 conn(0x7fbc580f74e0 msgr2=0x7fbc580656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:56.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 shutdown_connections 2024-02-16T19:28:56.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.852+0000 7fbc5e6e0700 1 -- 172.21.15.40:0/1510012154 wait complete. 2024-02-16T19:28:56.856 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.956+0000 7fe63c093700 1 -- 172.21.15.40:0/2299497205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fb220 msgr2=0x7fe6340fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.956+0000 7fe63c093700 1 --2- 172.21.15.40:0/2299497205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fb220 0x7fe6340fb630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe630004940 tx=0x7fe630009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.956+0000 7fe63c093700 1 -- 172.21.15.40:0/2299497205 shutdown_connections 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.956+0000 7fe63c093700 1 --2- 172.21.15.40:0/2299497205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6340fe2b0 0x7fe6340fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.956+0000 7fe63c093700 1 --2- 172.21.15.40:0/2299497205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fb220 0x7fe6340fb630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.956+0000 7fe63c093700 1 -- 172.21.15.40:0/2299497205 >> 172.21.15.40:0/2299497205 conn(0x7fe6340f6ca0 msgr2=0x7fe6340f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63c093700 1 -- 172.21.15.40:0/2299497205 shutdown_connections 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63c093700 1 -- 172.21.15.40:0/2299497205 wait complete. 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63c093700 1 Processor -- start 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63c093700 1 -- start start 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63c093700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6340fb220 0x7fe6340fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63c093700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fe2b0 0x7fe6340fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63c093700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340fc7c0 con 0x7fe6340fe2b0 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63c093700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340fc930 con 0x7fe6340fb220 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63962e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fe2b0 0x7fe6340fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63962e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fe2b0 0x7fe6340fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48066/0 (socket says 172.21.15.40:48066) 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.957+0000 7fe63962e700 1 -- 172.21.15.40:0/2465850533 learned_addr learned my addr 172.21.15.40:0/2465850533 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe639e2f700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6340fb220 0x7fe6340fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe63962e700 1 -- 172.21.15.40:0/2465850533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6340fb220 msgr2=0x7fe6340fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe63962e700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6340fb220 0x7fe6340fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe63962e700 1 -- 172.21.15.40:0/2465850533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe630005040 con 0x7fe6340fe2b0 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe639e2f700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6340fb220 0x7fe6340fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe63962e700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fe2b0 0x7fe6340fc1c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe624002a50 tx=0x7fe6240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe62a7fc700 1 -- 172.21.15.40:0/2465850533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe624007cc0 con 0x7fe6340fe2b0 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe62a7fc700 1 -- 172.21.15.40:0/2465850533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe62400f040 con 0x7fe6340fe2b0 2024-02-16T19:28:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6340fcc10 con 0x7fe6340fe2b0 2024-02-16T19:28:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe62a7fc700 1 -- 172.21.15.40:0/2465850533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe624009640 con 0x7fe6340fe2b0 2024-02-16T19:28:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.958+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe634115a60 con 0x7fe6340fe2b0 2024-02-16T19:28:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.959+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63404bed0 con 0x7fe6340fe2b0 2024-02-16T19:28:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.960+0000 7fe62a7fc700 1 -- 172.21.15.40:0/2465850533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe6240097a0 con 0x7fe6340fe2b0 2024-02-16T19:28:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.960+0000 7fe62a7fc700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe62006d8c0 0x7fe62006fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.960+0000 7fe62a7fc700 1 -- 172.21.15.40:0/2465850533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe62408cc70 con 0x7fe6340fe2b0 2024-02-16T19:28:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.960+0000 7fe639e2f700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe62006d8c0 0x7fe62006fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.961+0000 7fe639e2f700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe62006d8c0 0x7fe62006fd70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe63000b3b0 tx=0x7fe630009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:56.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:56.965+0000 7fe62a7fc700 1 -- 172.21.15.40:0/2465850533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe624059fd0 con 0x7fe6340fe2b0 2024-02-16T19:28:56.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:56.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: osdmap e56: 6 total, 5 up, 6 in 2024-02-16T19:28:56.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: failed to encode map e56 with expected crc 2024-02-16T19:28:56.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:56.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:56.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:56 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1979160082' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:28:57.130 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:56 smithi040 podman[108203]: 2024-02-16 19:28:56.774454668 +0000 UTC m=+0.738682983 container died 0433972f4f8331e3883057aa2bce86cd04cfe501b12a730666f674527a166cf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-16T19:28:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.186+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe634102c20 con 0x7fe62006d8c0 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.188+0000 7fe62a7fc700 1 -- 172.21.15.40:0/2465850533 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe634102c20 con 0x7fe62006d8c0 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:28:57.189 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:28:57.190 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:28:57.190 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "7/21 daemons upgraded", 2024-02-16T19:28:57.190 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:28:57.190 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:28:57.190 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:28:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe62006d8c0 msgr2=0x7fe62006fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe62006d8c0 0x7fe62006fd70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe63000b3b0 tx=0x7fe630009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fe2b0 msgr2=0x7fe6340fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fe2b0 0x7fe6340fc1c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe624002a50 tx=0x7fe6240046b0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 shutdown_connections 2024-02-16T19:28:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6340fb220 0x7fe6340fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe62006d8c0 0x7fe62006fd70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 --2- 172.21.15.40:0/2465850533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6340fe2b0 0x7fe6340fc1c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 >> 172.21.15.40:0/2465850533 conn(0x7fe6340f6ca0 msgr2=0x7fe6341014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 shutdown_connections 2024-02-16T19:28:57.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.190+0000 7fe63c093700 1 -- 172.21.15.40:0/2465850533 wait complete. 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.293+0000 7fc275e5e700 1 -- 172.21.15.40:0/2859131380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fe0e0 msgr2=0x7fc2700fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.293+0000 7fc275e5e700 1 --2- 172.21.15.40:0/2859131380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fe0e0 0x7fc2700fe550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc264004940 tx=0x7fc264009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.293+0000 7fc275e5e700 1 -- 172.21.15.40:0/2859131380 shutdown_connections 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.293+0000 7fc275e5e700 1 --2- 172.21.15.40:0/2859131380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fe0e0 0x7fc2700fe550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.293+0000 7fc275e5e700 1 --2- 172.21.15.40:0/2859131380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2700fcef0 0x7fc2700fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.293+0000 7fc275e5e700 1 -- 172.21.15.40:0/2859131380 >> 172.21.15.40:0/2859131380 conn(0x7fc2700f8620 msgr2=0x7fc2700faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.293+0000 7fc275e5e700 1 -- 172.21.15.40:0/2859131380 shutdown_connections 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.293+0000 7fc275e5e700 1 -- 172.21.15.40:0/2859131380 wait complete. 2024-02-16T19:28:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.294+0000 7fc275e5e700 1 Processor -- start 2024-02-16T19:28:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.294+0000 7fc275e5e700 1 -- start start 2024-02-16T19:28:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.294+0000 7fc275e5e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fcef0 0x7fc270071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.294+0000 7fc275e5e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2700fe0e0 0x7fc2700702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.294+0000 7fc275e5e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700708e0 con 0x7fc2700fcef0 2024-02-16T19:28:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.294+0000 7fc275e5e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc270070a50 con 0x7fc2700fe0e0 2024-02-16T19:28:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.294+0000 7fc26f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fcef0 0x7fc270071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc26f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fcef0 0x7fc270071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48080/0 (socket says 172.21.15.40:48080) 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc26f7fe700 1 -- 172.21.15.40:0/2156836948 learned_addr learned my addr 172.21.15.40:0/2156836948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc26effd700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2700fe0e0 0x7fc2700702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc26f7fe700 1 -- 172.21.15.40:0/2156836948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2700fe0e0 msgr2=0x7fc2700702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc26f7fe700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2700fe0e0 0x7fc2700702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc26f7fe700 1 -- 172.21.15.40:0/2156836948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264005040 con 0x7fc2700fcef0 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc26f7fe700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fcef0 0x7fc270071e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc260002a80 tx=0x7fc260004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc257fff700 1 -- 172.21.15.40:0/2156836948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc260007cc0 con 0x7fc2700fcef0 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc257fff700 1 -- 172.21.15.40:0/2156836948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc26000f040 con 0x7fc2700fcef0 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc270070d30 con 0x7fc2700fcef0 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc257fff700 1 -- 172.21.15.40:0/2156836948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc26000da30 con 0x7fc2700fcef0 2024-02-16T19:28:57.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.295+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2701159a0 con 0x7fc2700fcef0 2024-02-16T19:28:57.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.296+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27004bed0 con 0x7fc2700fcef0 2024-02-16T19:28:57.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.297+0000 7fc257fff700 1 -- 172.21.15.40:0/2156836948 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc26000db90 con 0x7fc2700fcef0 2024-02-16T19:28:57.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.297+0000 7fc257fff700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc25806d7f0 0x7fc25806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:28:57.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.297+0000 7fc257fff700 1 -- 172.21.15.40:0/2156836948 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc26008cc20 con 0x7fc2700fcef0 2024-02-16T19:28:57.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.297+0000 7fc26effd700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc25806d7f0 0x7fc25806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:28:57.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.298+0000 7fc26effd700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc25806d7f0 0x7fc25806fca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b7a0 tx=0x7fc264009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:28:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.302+0000 7fc257fff700 1 -- 172.21.15.40:0/2156836948 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc260059fa0 con 0x7fc2700fcef0 2024-02-16T19:28:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.603+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc270115d80 con 0x7fc2700fcef0 2024-02-16T19:28:57.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.604+0000 7fc257fff700 1 -- 172.21.15.40:0/2156836948 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7fc260059b30 con 0x7fc2700fcef0 2024-02-16T19:28:57.606 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_WARN 1 osds down 2024-02-16T19:28:57.606 INFO:teuthology.orchestra.run.smithi040.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-16T19:28:57.606 INFO:teuthology.orchestra.run.smithi040.stdout: osd.1 (root=default,host=smithi040) is down 2024-02-16T19:28:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc25806d7f0 msgr2=0x7fc25806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc25806d7f0 0x7fc25806fca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b7a0 tx=0x7fc264009e30 comp rx=0 tx=0).stop 2024-02-16T19:28:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fcef0 msgr2=0x7fc270071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:28:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fcef0 0x7fc270071e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc260002a80 tx=0x7fc260004600 comp rx=0 tx=0).stop 2024-02-16T19:28:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 shutdown_connections 2024-02-16T19:28:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc25806d7f0 0x7fc25806fca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2700fcef0 0x7fc270071e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 --2- 172.21.15.40:0/2156836948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2700fe0e0 0x7fc2700702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:28:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 >> 172.21.15.40:0/2156836948 conn(0x7fc2700f8620 msgr2=0x7fc2701013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:28:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 shutdown_connections 2024-02-16T19:28:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:28:57.606+0000 7fc275e5e700 1 -- 172.21.15.40:0/2156836948 wait complete. 2024-02-16T19:28:57.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:57 smithi040 ceph-mon[94210]: pgmap v67: 65 pgs: 12 stale+active+clean, 53 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:57.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:57 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1510012154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:28:57.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:57 smithi040 podman[108275]: 2024-02-16 19:28:57.473257423 +0000 UTC m=+0.693414842 container remove 0433972f4f8331e3883057aa2bce86cd04cfe501b12a730666f674527a166cf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0) 2024-02-16T19:28:57.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:57 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-16T19:28:57.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:57 smithi040 systemd[1]: Stopped Ceph osd.1 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:28:57.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:57 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.1.service: Consumed 1min 2.760s CPU time. 2024-02-16T19:28:57.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:57 smithi040 systemd[1]: Starting Ceph osd.1 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:28:57.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:57 smithi163 ceph-mon[81637]: pgmap v67: 65 pgs: 12 stale+active+clean, 53 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:28:57.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:57 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1510012154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:28:58.306 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:57 smithi040 podman[108428]: 2024-02-16 19:28:57.816098329 +0000 UTC m=+0.018132096 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:28:58.306 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:57 smithi040 podman[108428]: 2024-02-16 19:28:57.926216969 +0000 UTC m=+0.128250765 container create dfc21207db4fa07ea0ba0c06af6d38ac87e349b95984233ada4e9f5b7d179c65 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:28:58.306 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 podman[108428]: 2024-02-16 19:28:58.135689523 +0000 UTC m=+0.337723281 container init dfc21207db4fa07ea0ba0c06af6d38ac87e349b95984233ada4e9f5b7d179c65 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-16T19:28:58.306 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 podman[108428]: 2024-02-16 19:28:58.138768046 +0000 UTC m=+0.340801814 container start dfc21207db4fa07ea0ba0c06af6d38ac87e349b95984233ada4e9f5b7d179c65 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-16T19:28:58.306 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 podman[108428]: 2024-02-16 19:28:58.305284511 +0000 UTC m=+0.507318288 container attach dfc21207db4fa07ea0ba0c06af6d38ac87e349b95984233ada4e9f5b7d179c65 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-16T19:28:58.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-mon[94210]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:58.586 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2156836948' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:28:58.894 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate[108444]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T19:28:58.894 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate[108444]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 bash[108428]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 bash[108428]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 bash[108428]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 bash[108428]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 bash[108428]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 bash[108428]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 bash[108428]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate[108444]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate[108444]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate[108444]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate[108444]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate[108444]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-16T19:28:58.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:58 smithi040 podman[108428]: 2024-02-16 19:28:58.610151281 +0000 UTC m=+0.812185050 container died dfc21207db4fa07ea0ba0c06af6d38ac87e349b95984233ada4e9f5b7d179c65 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-16T19:28:58.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:58 smithi163 ceph-mon[81637]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:28:58.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:58 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2156836948' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:28:59.543 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:59 smithi040 podman[108567]: 2024-02-16 19:28:59.18965289 +0000 UTC m=+0.574291774 container remove dfc21207db4fa07ea0ba0c06af6d38ac87e349b95984233ada4e9f5b7d179c65 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:28:59.543 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:59 smithi040 podman[108632]: 2024-02-16 19:28:59.307280706 +0000 UTC m=+0.018770068 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:28:59.543 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:59 smithi040 podman[108632]: 2024-02-16 19:28:59.432367073 +0000 UTC m=+0.143856486 container create 2ba47d6ce1c92a64fc3936cde1520f654caddb48ca0c4fb810aae783925cc55e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-16T19:28:59.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:59 smithi040 ceph-mon[94210]: pgmap v68: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 49/360 objects degraded (13.611%) 2024-02-16T19:28:59.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:28:59 smithi040 ceph-mon[94210]: Health check failed: Degraded data redundancy: 49/360 objects degraded (13.611%), 14 pgs degraded (PG_DEGRADED) 2024-02-16T19:28:59.894 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:59 smithi040 podman[108632]: 2024-02-16 19:28:59.635383512 +0000 UTC m=+0.346872867 container init 2ba47d6ce1c92a64fc3936cde1520f654caddb48ca0c4fb810aae783925cc55e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux ) 2024-02-16T19:28:59.894 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:59 smithi040 podman[108632]: 2024-02-16 19:28:59.638313708 +0000 UTC m=+0.349803064 container start 2ba47d6ce1c92a64fc3936cde1520f654caddb48ca0c4fb810aae783925cc55e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-16T19:28:59.894 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:59 smithi040 ceph-osd[108651]: -- 172.21.15.40:0/322037162 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555cfc24b1e0 con 0x555cfc21d000 2024-02-16T19:28:59.894 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:59 smithi040 bash[108632]: 2ba47d6ce1c92a64fc3936cde1520f654caddb48ca0c4fb810aae783925cc55e 2024-02-16T19:28:59.895 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:28:59 smithi040 systemd[1]: Started Ceph osd.1 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:28:59.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:59 smithi163 ceph-mon[81637]: pgmap v68: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 49/360 objects degraded (13.611%) 2024-02-16T19:28:59.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:28:59 smithi163 ceph-mon[81637]: Health check failed: Degraded data redundancy: 49/360 objects degraded (13.611%), 14 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:00.616 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:29:00 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[108647]: 2024-02-16T19:29:00.181+0000 7f88feebe7c0 -1 Falling back to public interface 2024-02-16T19:29:00.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:00 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:00.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:00 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:00.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:00 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:01.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:00 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:01.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:00 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:01.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:00 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:02.095 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:01 smithi040 ceph-mon[94210]: pgmap v69: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:29:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:01 smithi163 ceph-mon[81637]: pgmap v69: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:29:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:03 smithi163 ceph-mon[81637]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:29:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:03.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:03 smithi040 ceph-mon[94210]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:29:03.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:03.569 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:04.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:04 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 56/360 objects degraded (15.556%), 16 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:04.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:04 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 56/360 objects degraded (15.556%), 16 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:04.894 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:29:04 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[108647]: 2024-02-16T19:29:04.708+0000 7f88feebe7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-02-16T19:29:05.644 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:29:05 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[108647]: 2024-02-16T19:29:05.342+0000 7f88feebe7c0 -1 osd.1 54 log_to_monitors true 2024-02-16T19:29:06.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:06 smithi040 ceph-mon[94210]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:29:06.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:06 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:06.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:06 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:06.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:06 smithi040 ceph-mon[94210]: from='osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T19:29:06.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:06 smithi163 ceph-mon[81637]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:29:06.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:06 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:06.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:06 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:06.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:06 smithi163 ceph-mon[81637]: from='osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-16T19:29:07.171 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:29:06 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[108647]: 2024-02-16T19:29:06.904+0000 7f88ec30e700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:29:07.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:07 smithi040 ceph-mon[94210]: from='osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T19:29:07.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:07 smithi040 ceph-mon[94210]: osdmap e57: 6 total, 5 up, 6 in 2024-02-16T19:29:07.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:07 smithi040 ceph-mon[94210]: from='osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:29:07.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:07.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:07.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:07 smithi040 ceph-mon[94210]: failed to encode map e57 with expected crc 2024-02-16T19:29:07.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:07 smithi040 ceph-mon[94210]: from='osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765]' entity='osd.1' 2024-02-16T19:29:07.389 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:07 smithi163 ceph-mon[81637]: from='osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-16T19:29:07.389 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:07 smithi163 ceph-mon[81637]: osdmap e57: 6 total, 5 up, 6 in 2024-02-16T19:29:07.389 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:07 smithi163 ceph-mon[81637]: from='osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:29:07.389 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:07.389 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:07.389 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:07 smithi163 ceph-mon[81637]: failed to encode map e57 with expected crc 2024-02-16T19:29:07.389 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:07 smithi163 ceph-mon[81637]: from='osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765]' entity='osd.1' 2024-02-16T19:29:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:08 smithi040 ceph-mon[94210]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:29:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:08 smithi040 ceph-mon[94210]: failed to encode map e57 with expected crc 2024-02-16T19:29:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:08 smithi040 ceph-mon[94210]: failed to encode map e57 with expected crc 2024-02-16T19:29:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:08 smithi040 ceph-mon[94210]: failed to encode map e57 with expected crc 2024-02-16T19:29:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:08 smithi040 ceph-mon[94210]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:29:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:08 smithi040 ceph-mon[94210]: osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765] boot 2024-02-16T19:29:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:08 smithi040 ceph-mon[94210]: osdmap e58: 6 total, 6 up, 6 in 2024-02-16T19:29:08.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:08 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:29:08.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:08 smithi163 ceph-mon[81637]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:29:08.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:08 smithi163 ceph-mon[81637]: failed to encode map e57 with expected crc 2024-02-16T19:29:08.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:08 smithi163 ceph-mon[81637]: failed to encode map e57 with expected crc 2024-02-16T19:29:08.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:08 smithi163 ceph-mon[81637]: failed to encode map e57 with expected crc 2024-02-16T19:29:08.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:08 smithi163 ceph-mon[81637]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:29:08.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:08 smithi163 ceph-mon[81637]: osd.1 [v2:172.21.15.40:6810/3029722765,v1:172.21.15.40:6811/3029722765] boot 2024-02-16T19:29:08.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:08 smithi163 ceph-mon[81637]: osdmap e58: 6 total, 6 up, 6 in 2024-02-16T19:29:08.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:08 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-16T19:29:09.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:09 smithi040 ceph-mon[94210]: failed to encode map e58 with expected crc 2024-02-16T19:29:09.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:09 smithi040 ceph-mon[94210]: osdmap e59: 6 total, 6 up, 6 in 2024-02-16T19:29:09.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:09 smithi163 ceph-mon[81637]: failed to encode map e58 with expected crc 2024-02-16T19:29:09.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:09 smithi163 ceph-mon[81637]: osdmap e59: 6 total, 6 up, 6 in 2024-02-16T19:29:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: pgmap v75: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/360 objects degraded (6.389%) 2024-02-16T19:29:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:10 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 23/360 objects degraded (6.389%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:10.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: pgmap v75: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/360 objects degraded (6.389%) 2024-02-16T19:29:10.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: failed to encode map e59 with expected crc 2024-02-16T19:29:10.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:10 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 23/360 objects degraded (6.389%), 7 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:11.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:11.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:11.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:11.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:11.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:11.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:11.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:29:11.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:11.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:29:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:12 smithi040 ceph-mon[94210]: pgmap v77: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/360 objects degraded (6.389%) 2024-02-16T19:29:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:12 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:29:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:12 smithi040 ceph-mon[94210]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-16T19:29:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:12 smithi163 ceph-mon[81637]: pgmap v77: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/360 objects degraded (6.389%) 2024-02-16T19:29:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:12 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:29:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:12 smithi163 ceph-mon[81637]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-16T19:29:13.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:13 smithi163 ceph-mon[81637]: pgmap v78: 65 pgs: 20 peering, 45 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:13.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:13 smithi163 ceph-mon[81637]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/360 objects degraded (6.389%), 7 pgs degraded) 2024-02-16T19:29:13.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:13 smithi163 ceph-mon[81637]: Cluster is now healthy 2024-02-16T19:29:13.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:13 smithi040 ceph-mon[94210]: pgmap v78: 65 pgs: 20 peering, 45 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:13.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:13 smithi040 ceph-mon[94210]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/360 objects degraded (6.389%), 7 pgs degraded) 2024-02-16T19:29:13.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:13 smithi040 ceph-mon[94210]: Cluster is now healthy 2024-02-16T19:29:15.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:15 smithi040 ceph-mon[94210]: pgmap v79: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:15 smithi163 ceph-mon[81637]: pgmap v79: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:17 smithi040 ceph-mon[94210]: pgmap v80: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:17 smithi163 ceph-mon[81637]: pgmap v80: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:19 smithi040 ceph-mon[94210]: pgmap v81: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:19 smithi163 ceph-mon[81637]: pgmap v81: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:21.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:21 smithi040 ceph-mon[94210]: pgmap v82: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:21.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:21 smithi163 ceph-mon[81637]: pgmap v82: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:23 smithi040 ceph-mon[94210]: pgmap v83: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:23 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:23 smithi163 ceph-mon[81637]: pgmap v83: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:23 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:25 smithi040 ceph-mon[94210]: pgmap v84: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:25 smithi163 ceph-mon[81637]: pgmap v84: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:26 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:29:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:26 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:29:26.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:26 smithi040 ceph-mon[94210]: Upgrade: osd.2 is safe to restart 2024-02-16T19:29:26.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:26 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:29:26.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:26 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-16T19:29:26.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:26 smithi163 ceph-mon[81637]: Upgrade: osd.2 is safe to restart 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.713+0000 7f410f946700 1 -- 172.21.15.40:0/210869101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 msgr2=0x7f4108070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.713+0000 7f410f946700 1 --2- 172.21.15.40:0/210869101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 0x7f4108070380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4100009ce0 tx=0x7f4100004d10 comp rx=0 tx=0).stop 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.713+0000 7f410f946700 1 -- 172.21.15.40:0/210869101 shutdown_connections 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.713+0000 7f410f946700 1 --2- 172.21.15.40:0/210869101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41080708c0 0x7f4108070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.713+0000 7f410f946700 1 --2- 172.21.15.40:0/210869101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 0x7f4108070380 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.713+0000 7f410f946700 1 -- 172.21.15.40:0/210869101 >> 172.21.15.40:0/210869101 conn(0x7f41080f8620 msgr2=0x7f41080faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.713+0000 7f410f946700 1 -- 172.21.15.40:0/210869101 shutdown_connections 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.713+0000 7f410f946700 1 -- 172.21.15.40:0/210869101 wait complete. 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410f946700 1 Processor -- start 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410f946700 1 -- start start 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410f946700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41080708c0 0x7f410810fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:27.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410f946700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 0x7f41081100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410f946700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081106b0 con 0x7f4108071f10 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410f946700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4108110f50 con 0x7f41080708c0 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410cee1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 0x7f41081100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410cee1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 0x7f41081100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36466/0 (socket says 172.21.15.40:36466) 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410cee1700 1 -- 172.21.15.40:0/3718414119 learned_addr learned my addr 172.21.15.40:0/3718414119 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.714+0000 7f410d6e2700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41080708c0 0x7f410810fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f410cee1700 1 -- 172.21.15.40:0/3718414119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41080708c0 msgr2=0x7f410810fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f410cee1700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41080708c0 0x7f410810fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f410cee1700 1 -- 172.21.15.40:0/3718414119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009940 con 0x7f4108071f10 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f410cee1700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 0x7f41081100b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00eb70 tx=0x7f40fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f4105ffb700 1 -- 172.21.15.40:0/3718414119 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40fc007cc0 con 0x7f4108071f10 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41081111d0 con 0x7f4108071f10 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f4105ffb700 1 -- 172.21.15.40:0/3718414119 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40fc00f040 con 0x7f4108071f10 2024-02-16T19:29:27.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f4105ffb700 1 -- 172.21.15.40:0/3718414119 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40fc00ca90 con 0x7f4108071f10 2024-02-16T19:29:27.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.715+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4108111710 con 0x7f4108071f10 2024-02-16T19:29:27.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.716+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410804bed0 con 0x7f4108071f10 2024-02-16T19:29:27.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.717+0000 7f4105ffb700 1 -- 172.21.15.40:0/3718414119 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f40fc00cbf0 con 0x7f4108071f10 2024-02-16T19:29:27.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.717+0000 7f4105ffb700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f40f006d7e0 0x7f40f006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:27.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.717+0000 7f410d6e2700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f40f006d7e0 0x7f40f006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:27.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.717+0000 7f4105ffb700 1 -- 172.21.15.40:0/3718414119 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f40fc08ca10 con 0x7f4108071f10 2024-02-16T19:29:27.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.718+0000 7f410d6e2700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f40f006d7e0 0x7f40f006fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4100015040 tx=0x7f4100004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:27.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.723+0000 7f4105ffb700 1 -- 172.21.15.40:0/3718414119 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40fc059d90 con 0x7f4108071f10 2024-02-16T19:29:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.951+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4108111b20 con 0x7f40f006d7e0 2024-02-16T19:29:27.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.952+0000 7f4105ffb700 1 -- 172.21.15.40:0/3718414119 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4108111b20 con 0x7f40f006d7e0 2024-02-16T19:29:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f40f006d7e0 msgr2=0x7f40f006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f40f006d7e0 0x7f40f006fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4100015040 tx=0x7f4100004d10 comp rx=0 tx=0).stop 2024-02-16T19:29:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 msgr2=0x7f41081100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 0x7f41081100b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00eb70 tx=0x7f40fc00b040 comp rx=0 tx=0).stop 2024-02-16T19:29:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 shutdown_connections 2024-02-16T19:29:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41080708c0 0x7f410810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f40f006d7e0 0x7f40f006fc90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 --2- 172.21.15.40:0/3718414119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4108071f10 0x7f41081100b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 >> 172.21.15.40:0/3718414119 conn(0x7f41080f8620 msgr2=0x7f4108102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 shutdown_connections 2024-02-16T19:29:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:27.954+0000 7f410f946700 1 -- 172.21.15.40:0/3718414119 wait complete. 2024-02-16T19:29:27.965 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 -- 172.21.15.40:0/1704154077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 msgr2=0x7ff288102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 --2- 172.21.15.40:0/1704154077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 0x7ff288102520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff27c004940 tx=0x7ff27c009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 -- 172.21.15.40:0/1704154077 shutdown_connections 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 --2- 172.21.15.40:0/1704154077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 0x7ff288102520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 --2- 172.21.15.40:0/1704154077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff288066c70 0x7ff2880ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 -- 172.21.15.40:0/1704154077 >> 172.21.15.40:0/1704154077 conn(0x7ff2880f7520 msgr2=0x7ff2880f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 -- 172.21.15.40:0/1704154077 shutdown_connections 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 -- 172.21.15.40:0/1704154077 wait complete. 2024-02-16T19:29:28.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 Processor -- start 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.058+0000 7ff28ed81700 1 -- start start 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff28ed81700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff288066c70 0x7ff28810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff28ed81700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 0x7ff28810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff28ed81700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28810e540 con 0x7ff2881000c0 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff28ed81700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28810ed80 con 0x7ff288066c70 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff287fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 0x7ff28810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff287fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 0x7ff28810deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36480/0 (socket says 172.21.15.40:36480) 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff287fff700 1 -- 172.21.15.40:0/3577620014 learned_addr learned my addr 172.21.15.40:0/3577620014 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff28cb1d700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff288066c70 0x7ff28810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff287fff700 1 -- 172.21.15.40:0/3577620014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff288066c70 msgr2=0x7ff28810d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff287fff700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff288066c70 0x7ff28810d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff287fff700 1 -- 172.21.15.40:0/3577620014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c005040 con 0x7ff2881000c0 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff287fff700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 0x7ff28810deb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff27c000c00 tx=0x7ff27c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:28.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff2857fa700 1 -- 172.21.15.40:0/3577620014 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff27c00e040 con 0x7ff2881000c0 2024-02-16T19:29:28.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28810f000 con 0x7ff2881000c0 2024-02-16T19:29:28.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff2857fa700 1 -- 172.21.15.40:0/3577620014 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff27c00bbf0 con 0x7ff2881000c0 2024-02-16T19:29:28.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff2857fa700 1 -- 172.21.15.40:0/3577620014 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff27c01c630 con 0x7ff2881000c0 2024-02-16T19:29:28.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.059+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28810f540 con 0x7ff2881000c0 2024-02-16T19:29:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.060+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28804bed0 con 0x7ff2881000c0 2024-02-16T19:29:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.061+0000 7ff2857fa700 1 -- 172.21.15.40:0/3577620014 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff27c00bd60 con 0x7ff2881000c0 2024-02-16T19:29:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.061+0000 7ff2857fa700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff27006d7f0 0x7ff27006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.061+0000 7ff2857fa700 1 -- 172.21.15.40:0/3577620014 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff27c08e190 con 0x7ff2881000c0 2024-02-16T19:29:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.061+0000 7ff28cb1d700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff27006d7f0 0x7ff27006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.062+0000 7ff28cb1d700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff27006d7f0 0x7ff27006fca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff278009bf0 tx=0x7ff2780091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:28.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.066+0000 7ff2857fa700 1 -- 172.21.15.40:0/3577620014 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff27c05b590 con 0x7ff2881000c0 2024-02-16T19:29:28.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:27 smithi040 ceph-mon[94210]: pgmap v85: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:28.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:27 smithi040 ceph-mon[94210]: Upgrade: Updating osd.2 2024-02-16T19:29:28.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:28.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:29:28.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:28.224 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:27 smithi040 ceph-mon[94210]: Deploying daemon osd.2 on smithi040 2024-02-16T19:29:28.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:27 smithi163 ceph-mon[81637]: pgmap v85: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:28.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:27 smithi163 ceph-mon[81637]: Upgrade: Updating osd.2 2024-02-16T19:29:28.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:28.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-16T19:29:28.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:28.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:27 smithi163 ceph-mon[81637]: Deploying daemon osd.2 on smithi040 2024-02-16T19:29:28.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.289+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff28810f9d0 con 0x7ff27006d7f0 2024-02-16T19:29:28.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.290+0000 7ff2857fa700 1 -- 172.21.15.40:0/3577620014 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff28810f9d0 con 0x7ff27006d7f0 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff27006d7f0 msgr2=0x7ff27006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff27006d7f0 0x7ff27006fca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff278009bf0 tx=0x7ff2780091a0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 msgr2=0x7ff28810deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 0x7ff28810deb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff27c000c00 tx=0x7ff27c009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 shutdown_connections 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff288066c70 0x7ff28810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff27006d7f0 0x7ff27006fca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 --2- 172.21.15.40:0/3577620014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2881000c0 0x7ff28810deb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 >> 172.21.15.40:0/3577620014 conn(0x7ff2880f7520 msgr2=0x7ff2880f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 shutdown_connections 2024-02-16T19:29:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.292+0000 7ff28ed81700 1 -- 172.21.15.40:0/3577620014 wait complete. 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.396+0000 7f90d2b8b700 1 -- 172.21.15.40:0/532443018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 msgr2=0x7f90cc0fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.396+0000 7f90d2b8b700 1 --2- 172.21.15.40:0/532443018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 0x7f90cc0fcce0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f90b8004940 tx=0x7f90b8009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.396+0000 7f90d2b8b700 1 -- 172.21.15.40:0/532443018 shutdown_connections 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.396+0000 7f90d2b8b700 1 --2- 172.21.15.40:0/532443018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 0x7f90cc0fcce0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.396+0000 7f90d2b8b700 1 --2- 172.21.15.40:0/532443018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0fb5c0 0x7f90cc0fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.396+0000 7f90d2b8b700 1 -- 172.21.15.40:0/532443018 >> 172.21.15.40:0/532443018 conn(0x7f90cc0f6d20 msgr2=0x7f90cc0f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.396+0000 7f90d2b8b700 1 -- 172.21.15.40:0/532443018 shutdown_connections 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.396+0000 7f90d2b8b700 1 -- 172.21.15.40:0/532443018 wait complete. 2024-02-16T19:29:28.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90d2b8b700 1 Processor -- start 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90d2b8b700 1 -- start start 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90d2b8b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0fb5c0 0x7f90cc1109b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90d2b8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 0x7f90cc112f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90d2b8b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0fe830 con 0x7f90cc0fc870 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90d2b8b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0fe970 con 0x7f90cc0fb5c0 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 0x7f90cc112f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 0x7f90cc112f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36500/0 (socket says 172.21.15.40:36500) 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90cbfff700 1 -- 172.21.15.40:0/3410013697 learned_addr learned my addr 172.21.15.40:0/3410013697 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90d0927700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0fb5c0 0x7f90cc1109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90cbfff700 1 -- 172.21.15.40:0/3410013697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0fb5c0 msgr2=0x7f90cc1109b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90cbfff700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0fb5c0 0x7f90cc1109b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.397+0000 7f90cbfff700 1 -- 172.21.15.40:0/3410013697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b8005040 con 0x7f90cc0fc870 2024-02-16T19:29:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.398+0000 7f90cbfff700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 0x7f90cc112f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f90b80093b0 tx=0x7f90b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.398+0000 7f90c97fa700 1 -- 172.21.15.40:0/3410013697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90b800e040 con 0x7f90cc0fc870 2024-02-16T19:29:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.398+0000 7f90c97fa700 1 -- 172.21.15.40:0/3410013697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90b800bcc0 con 0x7f90cc0fc870 2024-02-16T19:29:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.398+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc113500 con 0x7f90cc0fc870 2024-02-16T19:29:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.398+0000 7f90c97fa700 1 -- 172.21.15.40:0/3410013697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90b801c630 con 0x7f90cc0fc870 2024-02-16T19:29:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.398+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc113a40 con 0x7f90cc0fc870 2024-02-16T19:29:28.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.399+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc04bed0 con 0x7f90cc0fc870 2024-02-16T19:29:28.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.399+0000 7f90c97fa700 1 -- 172.21.15.40:0/3410013697 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f90b8013980 con 0x7f90cc0fc870 2024-02-16T19:29:28.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.400+0000 7f90c97fa700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f90bc06d7e0 0x7f90bc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.400+0000 7f90c97fa700 1 -- 172.21.15.40:0/3410013697 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f90b808e270 con 0x7f90cc0fc870 2024-02-16T19:29:28.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.400+0000 7f90d0927700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f90bc06d7e0 0x7f90bc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.400+0000 7f90d0927700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f90bc06d7e0 0x7f90bc06fc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f90c00041d0 tx=0x7f90c00043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.404+0000 7f90c97fa700 1 -- 172.21.15.40:0/3410013697 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f90b8057b40 con 0x7f90cc0fc870 2024-02-16T19:29:28.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.618+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f90cc1012c0 con 0x7f90bc06d7e0 2024-02-16T19:29:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.628+0000 7f90c97fa700 1 -- 172.21.15.40:0/3410013697 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f90cc1012c0 con 0x7f90bc06d7e0 2024-02-16T19:29:28.628 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:29:28.628 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (4m) 23s ago 13m 26.8M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:29:28.628 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (91s) 23s ago 13m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (86s) 82s ago 12m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (3m) 23s ago 12m 81.1M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (7m) 23s ago 7m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (7m) 23s ago 7m 13.7M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (7m) 82s ago 7m 17.0M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (7m) 82s ago 7m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (3m) 23s ago 14m 498M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (2m) 82s ago 12m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (2m) 23s ago 14m 45.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (2m) 82s ago 11m 32.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (4m) 23s ago 13m 15.6M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (3m) 82s ago 12m 16.0M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (65s) 23s ago 11m 96.2M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:29:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (29s) 23s ago 10m 12.5M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:29:28.630 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (10m) 23s ago 10m 198M 1559M 17.2.7-513-gb88fae66 870a0944ccb8 4e55adf1505a 2024-02-16T19:29:28.630 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (9m) 82s ago 9m 270M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:29:28.630 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (9m) 82s ago 9m 233M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:29:28.630 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (8m) 82s ago 8m 235M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:29:28.630 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (2m) 23s ago 12m 53.7M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:29:28.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f90bc06d7e0 msgr2=0x7f90bc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f90bc06d7e0 0x7f90bc06fc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f90c00041d0 tx=0x7f90c00043c0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 msgr2=0x7f90cc112f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 0x7f90cc112f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f90b80093b0 tx=0x7f90b8009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 shutdown_connections 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0fb5c0 0x7f90cc1109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f90bc06d7e0 0x7f90bc06fc90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 --2- 172.21.15.40:0/3410013697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90cc0fc870 0x7f90cc112f00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 >> 172.21.15.40:0/3410013697 conn(0x7f90cc0f6d20 msgr2=0x7f90cc100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 shutdown_connections 2024-02-16T19:29:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.630+0000 7f90d2b8b700 1 -- 172.21.15.40:0/3410013697 wait complete. 2024-02-16T19:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.740+0000 7f9489d61700 1 -- 172.21.15.40:0/1205794506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fe1e0 msgr2=0x7f94840fe650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.740+0000 7f9489d61700 1 --2- 172.21.15.40:0/1205794506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fe1e0 0x7f94840fe650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9478004940 tx=0x7f9478009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.740+0000 7f9489d61700 1 -- 172.21.15.40:0/1205794506 shutdown_connections 2024-02-16T19:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.740+0000 7f9489d61700 1 --2- 172.21.15.40:0/1205794506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fe1e0 0x7f94840fe650 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.740+0000 7f9489d61700 1 --2- 172.21.15.40:0/1205794506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94840fcf30 0x7f94840fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.740+0000 7f9489d61700 1 -- 172.21.15.40:0/1205794506 >> 172.21.15.40:0/1205794506 conn(0x7f94840f86b0 msgr2=0x7f94840faac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.740+0000 7f9489d61700 1 -- 172.21.15.40:0/1205794506 shutdown_connections 2024-02-16T19:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.740+0000 7f9489d61700 1 -- 172.21.15.40:0/1205794506 wait complete. 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f9489d61700 1 Processor -- start 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f9489d61700 1 -- start start 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f9489d61700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fcf30 0x7f948410d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f9489d61700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94840fe1e0 0x7f948410ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f9489d61700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948410e3b0 con 0x7f94840fcf30 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f9489d61700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948410ec50 con 0x7f94840fe1e0 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f94837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fcf30 0x7f948410d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f94837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fcf30 0x7f948410d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36518/0 (socket says 172.21.15.40:36518) 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f94837fe700 1 -- 172.21.15.40:0/3547187470 learned_addr learned my addr 172.21.15.40:0/3547187470 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f9482ffd700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94840fe1e0 0x7f948410ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f94837fe700 1 -- 172.21.15.40:0/3547187470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94840fe1e0 msgr2=0x7f948410ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.741+0000 7f94837fe700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94840fe1e0 0x7f948410ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.742+0000 7f94837fe700 1 -- 172.21.15.40:0/3547187470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9478005040 con 0x7f94840fcf30 2024-02-16T19:29:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.742+0000 7f94837fe700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fcf30 0x7f948410d870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f947400eb40 tx=0x7f947400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.742+0000 7f946bfff700 1 -- 172.21.15.40:0/3547187470 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9474007cc0 con 0x7f94840fcf30 2024-02-16T19:29:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.742+0000 7f946bfff700 1 -- 172.21.15.40:0/3547187470 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f947400f040 con 0x7f94840fcf30 2024-02-16T19:29:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.742+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948410eed0 con 0x7f94840fcf30 2024-02-16T19:29:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.742+0000 7f946bfff700 1 -- 172.21.15.40:0/3547187470 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9474009620 con 0x7f94840fcf30 2024-02-16T19:29:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.742+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948410f410 con 0x7f94840fcf30 2024-02-16T19:29:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.743+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948404bed0 con 0x7f94840fcf30 2024-02-16T19:29:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.743+0000 7f946bfff700 1 -- 172.21.15.40:0/3547187470 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9474009780 con 0x7f94840fcf30 2024-02-16T19:29:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.744+0000 7f946bfff700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f946c06d7e0 0x7f946c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.744+0000 7f946bfff700 1 -- 172.21.15.40:0/3547187470 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f947408cb70 con 0x7f94840fcf30 2024-02-16T19:29:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.744+0000 7f9482ffd700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f946c06d7e0 0x7f946c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.744+0000 7f9482ffd700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f946c06d7e0 0x7f946c06fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9478000c00 tx=0x7f9478009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:28.749+0000 7f946bfff700 1 -- 172.21.15.40:0/3547187470 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9474059fa0 con 0x7f94840fcf30 2024-02-16T19:29:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.061+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f948410f800 con 0x7f94840fcf30 2024-02-16T19:29:29.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.061+0000 7f946bfff700 1 -- 172.21.15.40:0/3547187470 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f9474059b30 con 0x7f94840fcf30 2024-02-16T19:29:29.062 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:29:29.062 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:29:29.062 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:29:29.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4, 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 8, 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:29:29.063 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:29:29.064 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:29:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f946c06d7e0 msgr2=0x7f946c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f946c06d7e0 0x7f946c06fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9478000c00 tx=0x7f9478009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fcf30 msgr2=0x7f948410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fcf30 0x7f948410d870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f947400eb40 tx=0x7f947400b040 comp rx=0 tx=0).stop 2024-02-16T19:29:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 shutdown_connections 2024-02-16T19:29:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f946c06d7e0 0x7f946c06fc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94840fcf30 0x7f948410d870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 --2- 172.21.15.40:0/3547187470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94840fe1e0 0x7f948410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 >> 172.21.15.40:0/3547187470 conn(0x7f94840f86b0 msgr2=0x7f9484101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 shutdown_connections 2024-02-16T19:29:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.064+0000 7f9489d61700 1 -- 172.21.15.40:0/3547187470 wait complete. 2024-02-16T19:29:29.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:28 smithi040 ceph-mon[94210]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:29.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.169+0000 7f3df51cd700 1 -- 172.21.15.40:0/4082291741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 msgr2=0x7f3df00fe9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.169+0000 7f3df51cd700 1 --2- 172.21.15.40:0/4082291741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 0x7f3df00fe9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3de4004940 tx=0x7f3de4009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:29.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.169+0000 7f3df51cd700 1 -- 172.21.15.40:0/4082291741 shutdown_connections 2024-02-16T19:29:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.169+0000 7f3df51cd700 1 --2- 172.21.15.40:0/4082291741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df00fefa0 0x7f3df00fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.169+0000 7f3df51cd700 1 --2- 172.21.15.40:0/4082291741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 0x7f3df00fe9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.169+0000 7f3df51cd700 1 -- 172.21.15.40:0/4082291741 >> 172.21.15.40:0/4082291741 conn(0x7f3df00f6ca0 msgr2=0x7f3df00f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.169+0000 7f3df51cd700 1 -- 172.21.15.40:0/4082291741 shutdown_connections 2024-02-16T19:29:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.169+0000 7f3df51cd700 1 -- 172.21.15.40:0/4082291741 wait complete. 2024-02-16T19:29:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3df51cd700 1 Processor -- start 2024-02-16T19:29:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3df51cd700 1 -- start start 2024-02-16T19:29:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3df51cd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 0x7f3df010da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3df51cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df010df90 0x7f3df010ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3df51cd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df010e4e0 con 0x7f3df00fe5c0 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3df51cd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df010e620 con 0x7f3df010df90 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3deed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 0x7f3df010da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3deed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 0x7f3df010da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36536/0 (socket says 172.21.15.40:36536) 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3deed9d700 1 -- 172.21.15.40:0/397637251 learned_addr learned my addr 172.21.15.40:0/397637251 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3deed9d700 1 -- 172.21.15.40:0/397637251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df010df90 msgr2=0x7f3df010ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3deed9d700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df010df90 0x7f3df010ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.170+0000 7f3deed9d700 1 -- 172.21.15.40:0/397637251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de4005040 con 0x7f3df00fe5c0 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.171+0000 7f3deed9d700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 0x7f3df010da50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009150 tx=0x7f3de4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.171+0000 7f3dd77fe700 1 -- 172.21.15.40:0/397637251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3de400e040 con 0x7f3df00fe5c0 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.171+0000 7f3dd77fe700 1 -- 172.21.15.40:0/397637251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3de400bbf0 con 0x7f3df00fe5c0 2024-02-16T19:29:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.171+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df010f4b0 con 0x7f3df00fe5c0 2024-02-16T19:29:29.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.172+0000 7f3dd77fe700 1 -- 172.21.15.40:0/397637251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3de401c630 con 0x7f3df00fe5c0 2024-02-16T19:29:29.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.172+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df010f9f0 con 0x7f3df00fe5c0 2024-02-16T19:29:29.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.172+0000 7f3dd77fe700 1 -- 172.21.15.40:0/397637251 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3de401c8a0 con 0x7f3df00fe5c0 2024-02-16T19:29:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.172+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df00fa940 con 0x7f3df00fe5c0 2024-02-16T19:29:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.173+0000 7f3dd77fe700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3dd806d7e0 0x7f3dd806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.173+0000 7f3dd77fe700 1 -- 172.21.15.40:0/397637251 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3de408d9b0 con 0x7f3df00fe5c0 2024-02-16T19:29:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.173+0000 7f3dee59c700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3dd806d7e0 0x7f3dd806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:29.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.173+0000 7f3dee59c700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3dd806d7e0 0x7f3dd806fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3de0006fd0 tx=0x7f3de0006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.178+0000 7f3dd77fe700 1 -- 172.21.15.40:0/397637251 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3de405ae80 con 0x7f3df00fe5c0 2024-02-16T19:29:29.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:28 smithi163 ceph-mon[81637]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:29.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.428+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3df0105af0 con 0x7f3df00fe5c0 2024-02-16T19:29:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.429+0000 7f3dd77fe700 1 -- 172.21.15.40:0/397637251 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f3de405aa10 con 0x7f3df00fe5c0 2024-02-16T19:29:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:e20 2024-02-16T19:29:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:29:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:29:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:29:29.429 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:29:29.429 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 20 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:27:43.925056+0000 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:29:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:29:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:29:29.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:29:29.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:29:29.432 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:29:29.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:29:29.432 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:29:29.432 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.431+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3dd806d7e0 msgr2=0x7f3dd806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3dd806d7e0 0x7f3dd806fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3de0006fd0 tx=0x7f3de0006db0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 msgr2=0x7f3df010da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 0x7f3df010da50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009150 tx=0x7f3de4009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 shutdown_connections 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3dd806d7e0 0x7f3dd806fc90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00fe5c0 0x7f3df010da50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 --2- 172.21.15.40:0/397637251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df010df90 0x7f3df010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 >> 172.21.15.40:0/397637251 conn(0x7f3df00f6ca0 msgr2=0x7f3df01015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 shutdown_connections 2024-02-16T19:29:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.432+0000 7f3df51cd700 1 -- 172.21.15.40:0/397637251 wait complete. 2024-02-16T19:29:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-02-16T19:29:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 -- 172.21.15.40:0/1400694592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 msgr2=0x7f1f400fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 --2- 172.21.15.40:0/1400694592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 0x7f1f400fd350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1f34004940 tx=0x7f1f34009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 -- 172.21.15.40:0/1400694592 shutdown_connections 2024-02-16T19:29:29.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 --2- 172.21.15.40:0/1400694592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f400fe1f0 0x7f1f400fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 --2- 172.21.15.40:0/1400694592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 0x7f1f400fd350 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 -- 172.21.15.40:0/1400694592 >> 172.21.15.40:0/1400694592 conn(0x7f1f400f8680 msgr2=0x7f1f400faad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:29.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 -- 172.21.15.40:0/1400694592 shutdown_connections 2024-02-16T19:29:29.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 -- 172.21.15.40:0/1400694592 wait complete. 2024-02-16T19:29:29.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 Processor -- start 2024-02-16T19:29:29.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.537+0000 7f1f46d54700 1 -- start start 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f46d54700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 0x7f1f4010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f46d54700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f400fe1f0 0x7f1f4010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f46d54700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4010e4d0 con 0x7f1f400fcf40 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f46d54700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4010ed70 con 0x7f1f400fe1f0 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f44af0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 0x7f1f4010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f44af0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 0x7f1f4010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36548/0 (socket says 172.21.15.40:36548) 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f44af0700 1 -- 172.21.15.40:0/370378713 learned_addr learned my addr 172.21.15.40:0/370378713 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f3ffff700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f400fe1f0 0x7f1f4010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f44af0700 1 -- 172.21.15.40:0/370378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f400fe1f0 msgr2=0x7f1f4010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f44af0700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f400fe1f0 0x7f1f4010ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f44af0700 1 -- 172.21.15.40:0/370378713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34005040 con 0x7f1f400fcf40 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f44af0700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 0x7f1f4010d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1f340093b0 tx=0x7f1f34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f3d7fa700 1 -- 172.21.15.40:0/370378713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f3400e040 con 0x7f1f400fcf40 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f3d7fa700 1 -- 172.21.15.40:0/370378713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f3400bcc0 con 0x7f1f400fcf40 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4010eff0 con 0x7f1f400fcf40 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f3d7fa700 1 -- 172.21.15.40:0/370378713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f3401c630 con 0x7f1f400fcf40 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.538+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4010f530 con 0x7f1f400fcf40 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.539+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4004bed0 con 0x7f1f400fcf40 2024-02-16T19:29:29.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.540+0000 7f1f3d7fa700 1 -- 172.21.15.40:0/370378713 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1f340139f0 con 0x7f1f400fcf40 2024-02-16T19:29:29.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.540+0000 7f1f3d7fa700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1f3006d7e0 0x7f1f3006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.540+0000 7f1f3d7fa700 1 -- 172.21.15.40:0/370378713 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1f3408e210 con 0x7f1f400fcf40 2024-02-16T19:29:29.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.540+0000 7f1f3ffff700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1f3006d7e0 0x7f1f3006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:29.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.541+0000 7f1f3ffff700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1f3006d7e0 0x7f1f3006fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c007fd0 tx=0x7f1f2c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:29.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.545+0000 7f1f3d7fa700 1 -- 172.21.15.40:0/370378713 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f3405b6b0 con 0x7f1f400fcf40 2024-02-16T19:29:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.767+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f40102b40 con 0x7f1f3006d7e0 2024-02-16T19:29:29.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.768+0000 7f1f3d7fa700 1 -- 172.21.15.40:0/370378713 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1f40102b40 con 0x7f1f3006d7e0 2024-02-16T19:29:29.768 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:29:29.768 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:29:29.768 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:29:29.768 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "8/21 daemons upgraded", 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:29:29.769 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:29:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1f3006d7e0 msgr2=0x7f1f3006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1f3006d7e0 0x7f1f3006fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c007fd0 tx=0x7f1f2c009040 comp rx=0 tx=0).stop 2024-02-16T19:29:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 msgr2=0x7f1f4010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 0x7f1f4010d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1f340093b0 tx=0x7f1f34009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 shutdown_connections 2024-02-16T19:29:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1f3006d7e0 0x7f1f3006fc90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f400fcf40 0x7f1f4010d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 --2- 172.21.15.40:0/370378713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f400fe1f0 0x7f1f4010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 >> 172.21.15.40:0/370378713 conn(0x7f1f400f8680 msgr2=0x7f1f40101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 shutdown_connections 2024-02-16T19:29:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.770+0000 7f1f46d54700 1 -- 172.21.15.40:0/370378713 wait complete. 2024-02-16T19:29:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.873+0000 7fdacfd4f700 1 -- 172.21.15.40:0/3170469798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 msgr2=0x7fdac80fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.873+0000 7fdacfd4f700 1 --2- 172.21.15.40:0/3170469798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 0x7fdac80fd420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdab8004940 tx=0x7fdab8009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 -- 172.21.15.40:0/3170469798 shutdown_connections 2024-02-16T19:29:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 --2- 172.21.15.40:0/3170469798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 0x7fdac80fd420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 --2- 172.21.15.40:0/3170469798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac80fbe40 0x7fdac80fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 -- 172.21.15.40:0/3170469798 >> 172.21.15.40:0/3170469798 conn(0x7fdac80f7520 msgr2=0x7fdac80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 -- 172.21.15.40:0/3170469798 shutdown_connections 2024-02-16T19:29:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 -- 172.21.15.40:0/3170469798 wait complete. 2024-02-16T19:29:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 Processor -- start 2024-02-16T19:29:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 -- start start 2024-02-16T19:29:29.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac80fbe40 0x7fdac810c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 0x7fdac810cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac810d380 con 0x7fdac80fcfb0 2024-02-16T19:29:29.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacfd4f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac810dc20 con 0x7fdac80fbe40 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.874+0000 7fdacd2ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 0x7fdac810cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacd2ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 0x7fdac810cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36578/0 (socket says 172.21.15.40:36578) 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacd2ea700 1 -- 172.21.15.40:0/4065899691 learned_addr learned my addr 172.21.15.40:0/4065899691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacdaeb700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac80fbe40 0x7fdac810c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacd2ea700 1 -- 172.21.15.40:0/4065899691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac80fbe40 msgr2=0x7fdac810c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacd2ea700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac80fbe40 0x7fdac810c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacd2ea700 1 -- 172.21.15.40:0/4065899691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8005040 con 0x7fdac80fcfb0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacd2ea700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 0x7fdac810cd80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdab80093b0 tx=0x7fdab8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdabe7fc700 1 -- 172.21.15.40:0/4065899691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdab800e040 con 0x7fdac80fcfb0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdabe7fc700 1 -- 172.21.15.40:0/4065899691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdab800bcc0 con 0x7fdac80fcfb0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac810dea0 con 0x7fdac80fcfb0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdabe7fc700 1 -- 172.21.15.40:0/4065899691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdab801c630 con 0x7fdac80fcfb0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.875+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac810e3e0 con 0x7fdac80fcfb0 2024-02-16T19:29:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.876+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac804bed0 con 0x7fdac80fcfb0 2024-02-16T19:29:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.877+0000 7fdabe7fc700 1 -- 172.21.15.40:0/4065899691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdab8013980 con 0x7fdac80fcfb0 2024-02-16T19:29:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.877+0000 7fdabe7fc700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdab406d7e0 0x7fdab406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:29:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.877+0000 7fdabe7fc700 1 -- 172.21.15.40:0/4065899691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdab808e170 con 0x7fdac80fcfb0 2024-02-16T19:29:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.877+0000 7fdacdaeb700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdab406d7e0 0x7fdab406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:29:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.878+0000 7fdacdaeb700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdab406d7e0 0x7fdab406fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdac4004f40 tx=0x7fdac400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:29:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:29.882+0000 7fdabe7fc700 1 -- 172.21.15.40:0/4065899691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdab805b610 con 0x7fdac80fcfb0 2024-02-16T19:29:30.090 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:29 smithi040 ceph-mon[94210]: pgmap v86: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:30.090 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:29 smithi040 ceph-mon[94210]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:30.091 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:29 smithi040 ceph-mon[94210]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:30.091 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:29 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3547187470' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:30.091 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:29 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/397637251' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:29:30.180 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:29:30.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.178+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdac8104930 con 0x7fdac80fcfb0 2024-02-16T19:29:30.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.178+0000 7fdabe7fc700 1 -- 172.21.15.40:0/4065899691 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fdab805b1a0 con 0x7fdac80fcfb0 2024-02-16T19:29:30.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdab406d7e0 msgr2=0x7fdab406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:30.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdab406d7e0 0x7fdab406fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdac4004f40 tx=0x7fdac400b040 comp rx=0 tx=0).stop 2024-02-16T19:29:30.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 msgr2=0x7fdac810cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:29:30.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 0x7fdac810cd80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdab80093b0 tx=0x7fdab8009e30 comp rx=0 tx=0).stop 2024-02-16T19:29:30.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 shutdown_connections 2024-02-16T19:29:30.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac80fbe40 0x7fdac810c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:30.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fdab406d7e0 0x7fdab406fc90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:30.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 --2- 172.21.15.40:0/4065899691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdac80fcfb0 0x7fdac810cd80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:29:30.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 >> 172.21.15.40:0/4065899691 conn(0x7fdac80f7520 msgr2=0x7fdac8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:29:30.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 shutdown_connections 2024-02-16T19:29:30.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:29:30.181+0000 7fdacfd4f700 1 -- 172.21.15.40:0/4065899691 wait complete. 2024-02-16T19:29:30.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:29 smithi163 ceph-mon[81637]: pgmap v86: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:30.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:29 smithi163 ceph-mon[81637]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:30.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:29 smithi163 ceph-mon[81637]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:30.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:29 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3547187470' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:30.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:29 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/397637251' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:29:30.394 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:30 smithi040 systemd[1]: Stopping Ceph osd.2 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:29:30.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:30 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[63510]: 2024-02-16T19:29:30.537+0000 7fb3cd4c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:29:30.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:30 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[63510]: 2024-02-16T19:29:30.537+0000 7fb3cd4c8700 -1 osd.2 59 *** Got signal Terminated *** 2024-02-16T19:29:30.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:30 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[63510]: 2024-02-16T19:29:30.537+0000 7fb3cd4c8700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:29:31.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:30 smithi163 ceph-mon[81637]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:31.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:30 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/4065899691' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:29:31.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:30 smithi163 ceph-mon[81637]: osd.2 marked itself down and dead 2024-02-16T19:29:31.306 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:30 smithi040 ceph-mon[94210]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:29:31.307 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:30 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/4065899691' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:29:31.307 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:30 smithi040 ceph-mon[94210]: osd.2 marked itself down and dead 2024-02-16T19:29:31.581 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:31 smithi040 podman[114013]: 2024-02-16 19:29:31.306402357 +0000 UTC m=+0.883351764 container died 4e55adf1505a2671101b676a281d6e79e51f4be9ce69a1e22d82ca3fb5721e50 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-b88fae6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-16T19:29:31.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:31 smithi040 podman[114013]: 2024-02-16 19:29:31.679490269 +0000 UTC m=+1.256439662 container cleanup 4e55adf1505a2671101b676a281d6e79e51f4be9ce69a1e22d82ca3fb5721e50 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=quincy-b88fae6, GIT_BRANCH=HEAD) 2024-02-16T19:29:31.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:31 smithi040 bash[114013]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2 2024-02-16T19:29:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:31 smithi163 ceph-mon[81637]: pgmap v87: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:32.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:31 smithi163 ceph-mon[81637]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:29:32.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:31 smithi163 ceph-mon[81637]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:31 smithi163 ceph-mon[81637]: osdmap e60: 6 total, 5 up, 6 in 2024-02-16T19:29:32.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:31 smithi163 ceph-mon[81637]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:31 smithi163 ceph-mon[81637]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:31 smithi163 ceph-mon[81637]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:31 smithi163 ceph-mon[81637]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.259 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:31 smithi040 podman[114032]: 2024-02-16 19:29:31.964432687 +0000 UTC m=+0.654889255 container remove 4e55adf1505a2671101b676a281d6e79e51f4be9ce69a1e22d82ca3fb5721e50 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=quincy-b88fae6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, GIT_CLEAN=True) 2024-02-16T19:29:32.259 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:31 smithi040 ceph-mon[94210]: pgmap v87: 65 pgs: 65 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:32.259 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:31 smithi040 ceph-mon[94210]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:29:32.259 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:31 smithi040 ceph-mon[94210]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.260 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:31 smithi040 ceph-mon[94210]: osdmap e60: 6 total, 5 up, 6 in 2024-02-16T19:29:32.260 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:31 smithi040 ceph-mon[94210]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.260 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:31 smithi040 ceph-mon[94210]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.260 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:31 smithi040 ceph-mon[94210]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.260 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:31 smithi040 ceph-mon[94210]: failed to encode map e60 with expected crc 2024-02-16T19:29:32.544 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:32 smithi040 podman[114121]: 2024-02-16 19:29:32.162009062 +0000 UTC m=+0.019077605 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:29:32.544 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:32 smithi040 podman[114121]: 2024-02-16 19:29:32.290888407 +0000 UTC m=+0.147956976 container create 700b9c532aecb7b3a37a53b901dd64df311a81326eccbb2433b3958a89fa4b94 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-16T19:29:32.545 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:32 smithi040 podman[114121]: 2024-02-16 19:29:32.543946807 +0000 UTC m=+0.401015343 container init 700b9c532aecb7b3a37a53b901dd64df311a81326eccbb2433b3958a89fa4b94 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True) 2024-02-16T19:29:32.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:32 smithi040 podman[114121]: 2024-02-16 19:29:32.547320948 +0000 UTC m=+0.404389485 container start 700b9c532aecb7b3a37a53b901dd64df311a81326eccbb2433b3958a89fa4b94 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:29:32.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:32 smithi040 podman[114121]: 2024-02-16 19:29:32.618508939 +0000 UTC m=+0.475577480 container attach 700b9c532aecb7b3a37a53b901dd64df311a81326eccbb2433b3958a89fa4b94 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-02-16T19:29:33.144 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:32 smithi040 podman[114121]: 2024-02-16 19:29:32.839140463 +0000 UTC m=+0.696208999 container died 700b9c532aecb7b3a37a53b901dd64df311a81326eccbb2433b3958a89fa4b94 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:29:33.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: osdmap e61: 6 total, 5 up, 6 in 2024-02-16T19:29:33.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:33.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:33.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: osdmap e61: 6 total, 5 up, 6 in 2024-02-16T19:29:33.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: failed to encode map e61 with expected crc 2024-02-16T19:29:33.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:33.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:33.883 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:33 smithi040 podman[114147]: 2024-02-16 19:29:33.430732652 +0000 UTC m=+0.586398362 container remove 700b9c532aecb7b3a37a53b901dd64df311a81326eccbb2433b3958a89fa4b94 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:29:33.883 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:33 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-16T19:29:33.883 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:33 smithi040 systemd[1]: Stopped Ceph osd.2 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:29:33.883 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:33 smithi040 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.2.service: Consumed 50.123s CPU time. 2024-02-16T19:29:33.883 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:33 smithi040 systemd[1]: Starting Ceph osd.2 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:29:34.135 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:33 smithi040 ceph-mon[94210]: pgmap v90: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:34.135 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:33 smithi040 ceph-mon[94210]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-02-16T19:29:34.136 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:33 smithi040 podman[114248]: 2024-02-16 19:29:33.882157747 +0000 UTC m=+0.114877294 container create a984b38bc82750ee234d0103eda3a3f5a139d0951ff8ced653e9bd25dc8f3341 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-16T19:29:34.136 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:33 smithi040 podman[114248]: 2024-02-16 19:29:33.785243974 +0000 UTC m=+0.017963480 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:29:34.136 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 podman[114248]: 2024-02-16 19:29:34.076279573 +0000 UTC m=+0.308999079 container init a984b38bc82750ee234d0103eda3a3f5a139d0951ff8ced653e9bd25dc8f3341 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:29:34.136 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 podman[114248]: 2024-02-16 19:29:34.07932663 +0000 UTC m=+0.312046137 container start a984b38bc82750ee234d0103eda3a3f5a139d0951ff8ced653e9bd25dc8f3341 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-16T19:29:34.136 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 podman[114248]: 2024-02-16 19:29:34.134563012 +0000 UTC m=+0.367282520 container attach a984b38bc82750ee234d0103eda3a3f5a139d0951ff8ced653e9bd25dc8f3341 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-16T19:29:34.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:33 smithi163 ceph-mon[81637]: pgmap v90: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-02-16T19:29:34.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:33 smithi163 ceph-mon[81637]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate[114263]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate[114263]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 bash[114248]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 bash[114248]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 bash[114248]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 bash[114248]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 bash[114248]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 bash[114248]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 bash[114248]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate[114263]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T19:29:34.839 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate[114263]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T19:29:34.840 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate[114263]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-16T19:29:34.840 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate[114263]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-16T19:29:34.840 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate[114263]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-16T19:29:34.840 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:34 smithi040 podman[114248]: 2024-02-16 19:29:34.55758926 +0000 UTC m=+0.790308783 container died a984b38bc82750ee234d0103eda3a3f5a139d0951ff8ced653e9bd25dc8f3341 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-16T19:29:35.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:34 smithi040 ceph-mon[94210]: Health check failed: Degraded data redundancy: 17/360 objects degraded (4.722%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:35.463 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:35 smithi040 podman[114402]: 2024-02-16 19:29:35.15541659 +0000 UTC m=+0.592425400 container remove a984b38bc82750ee234d0103eda3a3f5a139d0951ff8ced653e9bd25dc8f3341 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-16T19:29:35.463 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:35 smithi040 podman[114466]: 2024-02-16 19:29:35.364682178 +0000 UTC m=+0.106514968 container create 36685277f4cb24a0668214441427eef646f96680723cdf9085591d7e0ce5cf3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-16T19:29:35.463 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:35 smithi040 podman[114466]: 2024-02-16 19:29:35.276763213 +0000 UTC m=+0.018595968 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:29:35.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:34 smithi163 ceph-mon[81637]: Health check failed: Degraded data redundancy: 17/360 objects degraded (4.722%), 5 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:35.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:35 smithi040 podman[114466]: 2024-02-16 19:29:35.617955853 +0000 UTC m=+0.359788601 container init 36685277f4cb24a0668214441427eef646f96680723cdf9085591d7e0ce5cf3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-02-16T19:29:35.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:35 smithi040 podman[114466]: 2024-02-16 19:29:35.621153663 +0000 UTC m=+0.362986416 container start 36685277f4cb24a0668214441427eef646f96680723cdf9085591d7e0ce5cf3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-16T19:29:35.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:35 smithi040 ceph-osd[114485]: -- 172.21.15.40:0/1770680465 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5622a12531e0 con 0x5622a1225000 2024-02-16T19:29:35.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:35 smithi040 bash[114466]: 36685277f4cb24a0668214441427eef646f96680723cdf9085591d7e0ce5cf3b 2024-02-16T19:29:35.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:35 smithi040 systemd[1]: Started Ceph osd.2 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:29:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:35 smithi040 ceph-mon[94210]: pgmap v91: 65 pgs: 5 active+undersized+degraded, 7 peering, 3 stale+active+clean, 7 active+undersized, 43 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17/360 objects degraded (4.722%) 2024-02-16T19:29:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:35 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:35 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:35 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:36.395 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:36 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[114481]: 2024-02-16T19:29:36.159+0000 7f86e11887c0 -1 Falling back to public interface 2024-02-16T19:29:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:35 smithi163 ceph-mon[81637]: pgmap v91: 65 pgs: 5 active+undersized+degraded, 7 peering, 3 stale+active+clean, 7 active+undersized, 43 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17/360 objects degraded (4.722%) 2024-02-16T19:29:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:35 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:35 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:35 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:37.349 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:37 smithi040 ceph-mon[94210]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-02-16T19:29:37.387 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:37 smithi163 ceph-mon[81637]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-02-16T19:29:38.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:38 smithi040 ceph-mon[94210]: pgmap v92: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:29:38.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:38 smithi163 ceph-mon[81637]: pgmap v92: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:29:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:40 smithi040 ceph-mon[94210]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:29:40.395 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:40 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[114481]: 2024-02-16T19:29:40.110+0000 7f86e11887c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-02-16T19:29:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:40 smithi163 ceph-mon[81637]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:29:40.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:40 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[114481]: 2024-02-16T19:29:40.517+0000 7f86e11887c0 -1 osd.2 59 log_to_monitors true 2024-02-16T19:29:41.394 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:29:41 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[114481]: 2024-02-16T19:29:41.078+0000 7f86d25e0700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:29:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:41 smithi040 ceph-mon[94210]: from='osd.2 [v2:172.21.15.40:6818/1990065416,v1:172.21.15.40:6819/1990065416]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T19:29:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:41 smithi163 ceph-mon[81637]: from='osd.2 [v2:172.21.15.40:6818/1990065416,v1:172.21.15.40:6819/1990065416]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-16T19:29:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:42 smithi040 ceph-mon[94210]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:29:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:42 smithi040 ceph-mon[94210]: from='osd.2 [v2:172.21.15.40:6818/1990065416,v1:172.21.15.40:6819/1990065416]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T19:29:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:42 smithi040 ceph-mon[94210]: osdmap e62: 6 total, 5 up, 6 in 2024-02-16T19:29:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:42 smithi040 ceph-mon[94210]: from='osd.2 [v2:172.21.15.40:6818/1990065416,v1:172.21.15.40:6819/1990065416]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:29:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:42 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:42 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:42 smithi163 ceph-mon[81637]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 307 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:29:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:42 smithi163 ceph-mon[81637]: from='osd.2 [v2:172.21.15.40:6818/1990065416,v1:172.21.15.40:6819/1990065416]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-16T19:29:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:42 smithi163 ceph-mon[81637]: osdmap e62: 6 total, 5 up, 6 in 2024-02-16T19:29:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:42 smithi163 ceph-mon[81637]: from='osd.2 [v2:172.21.15.40:6818/1990065416,v1:172.21.15.40:6819/1990065416]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-02-16T19:29:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:42 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:42.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:42 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: failed to encode map e62 with expected crc 2024-02-16T19:29:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:29:43.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: osd.2 [v2:172.21.15.40:6818/1990065416,v1:172.21.15.40:6819/1990065416] boot 2024-02-16T19:29:43.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: osdmap e63: 6 total, 6 up, 6 in 2024-02-16T19:29:43.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:29:43.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: failed to encode map e62 with expected crc 2024-02-16T19:29:43.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: failed to encode map e62 with expected crc 2024-02-16T19:29:43.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: failed to encode map e63 with expected crc 2024-02-16T19:29:43.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:43 smithi040 ceph-mon[94210]: osdmap e64: 6 total, 6 up, 6 in 2024-02-16T19:29:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: failed to encode map e62 with expected crc 2024-02-16T19:29:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:29:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: osd.2 [v2:172.21.15.40:6818/1990065416,v1:172.21.15.40:6819/1990065416] boot 2024-02-16T19:29:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: osdmap e63: 6 total, 6 up, 6 in 2024-02-16T19:29:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-16T19:29:43.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: failed to encode map e62 with expected crc 2024-02-16T19:29:43.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: failed to encode map e62 with expected crc 2024-02-16T19:29:43.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: failed to encode map e63 with expected crc 2024-02-16T19:29:43.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:43 smithi163 ceph-mon[81637]: osdmap e64: 6 total, 6 up, 6 in 2024-02-16T19:29:44.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:44 smithi040 ceph-mon[94210]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:29:44.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:44 smithi040 ceph-mon[94210]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:44 smithi040 ceph-mon[94210]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:44 smithi040 ceph-mon[94210]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:44 smithi040 ceph-mon[94210]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:44 smithi040 ceph-mon[94210]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:44 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 43/360 objects degraded (11.944%), 12 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:44 smithi163 ceph-mon[81637]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:29:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:44 smithi163 ceph-mon[81637]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:44 smithi163 ceph-mon[81637]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:44 smithi163 ceph-mon[81637]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:44 smithi163 ceph-mon[81637]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:44 smithi163 ceph-mon[81637]: failed to encode map e64 with expected crc 2024-02-16T19:29:44.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:44 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 43/360 objects degraded (11.944%), 12 pgs degraded (PG_DEGRADED) 2024-02-16T19:29:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: pgmap v99: 65 pgs: 7 active+undersized+degraded, 10 active+undersized, 48 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 26/360 objects degraded (7.222%) 2024-02-16T19:29:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:46.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:46.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:46.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:46.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:46.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:46.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:46.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:46.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:29:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: pgmap v99: 65 pgs: 7 active+undersized+degraded, 10 active+undersized, 48 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 26/360 objects degraded (7.222%) 2024-02-16T19:29:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:29:46.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:29:47.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:47 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:29:47.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:47 smithi040 ceph-mon[94210]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-02-16T19:29:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:47 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:29:47.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:47 smithi163 ceph-mon[81637]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-02-16T19:29:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:48 smithi040 ceph-mon[94210]: pgmap v100: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:48 smithi040 ceph-mon[94210]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 26/360 objects degraded (7.222%), 7 pgs degraded) 2024-02-16T19:29:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:48 smithi040 ceph-mon[94210]: Cluster is now healthy 2024-02-16T19:29:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:48 smithi163 ceph-mon[81637]: pgmap v100: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:48 smithi163 ceph-mon[81637]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 26/360 objects degraded (7.222%), 7 pgs degraded) 2024-02-16T19:29:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:48 smithi163 ceph-mon[81637]: Cluster is now healthy 2024-02-16T19:29:48.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:48.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:29:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:50 smithi040 ceph-mon[94210]: pgmap v101: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:50 smithi163 ceph-mon[81637]: pgmap v101: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:52.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:52 smithi040 ceph-mon[94210]: pgmap v102: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:52.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:52 smithi163 ceph-mon[81637]: pgmap v102: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:53.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:53 smithi040 ceph-mon[94210]: pgmap v103: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:53.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:53 smithi163 ceph-mon[81637]: pgmap v103: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:29:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:55 smithi040 ceph-mon[94210]: pgmap v104: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:55 smithi163 ceph-mon[81637]: pgmap v104: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:57 smithi040 ceph-mon[94210]: pgmap v105: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:57 smithi163 ceph-mon[81637]: pgmap v105: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:29:59 smithi040 ceph-mon[94210]: pgmap v106: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:29:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:29:59 smithi163 ceph-mon[81637]: pgmap v106: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.299+0000 7f47e2d8e700 1 -- 172.21.15.40:0/2366433167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fe0e0 msgr2=0x7f47dc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.299+0000 7f47e2d8e700 1 --2- 172.21.15.40:0/2366433167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fe0e0 0x7f47dc0fe550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f47c8004940 tx=0x7f47c8009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.299+0000 7f47e2d8e700 1 -- 172.21.15.40:0/2366433167 shutdown_connections 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.299+0000 7f47e2d8e700 1 --2- 172.21.15.40:0/2366433167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fe0e0 0x7f47dc0fe550 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.299+0000 7f47e2d8e700 1 --2- 172.21.15.40:0/2366433167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47dc0fcef0 0x7f47dc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.299+0000 7f47e2d8e700 1 -- 172.21.15.40:0/2366433167 >> 172.21.15.40:0/2366433167 conn(0x7f47dc0f8620 msgr2=0x7f47dc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.299+0000 7f47e2d8e700 1 -- 172.21.15.40:0/2366433167 shutdown_connections 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.299+0000 7f47e2d8e700 1 -- 172.21.15.40:0/2366433167 wait complete. 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.300+0000 7f47e2d8e700 1 Processor -- start 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.300+0000 7f47e2d8e700 1 -- start start 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.300+0000 7f47e2d8e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fcef0 0x7f47dc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.300+0000 7f47e2d8e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47dc0fe0e0 0x7f47dc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.300+0000 7f47e2d8e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc10e540 con 0x7f47dc0fcef0 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.300+0000 7f47e2d8e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc10ed80 con 0x7f47dc0fe0e0 2024-02-16T19:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.300+0000 7f47e0b2a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fcef0 0x7f47dc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47e0b2a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fcef0 0x7f47dc10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50214/0 (socket says 172.21.15.40:50214) 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47e0b2a700 1 -- 172.21.15.40:0/54918384 learned_addr learned my addr 172.21.15.40:0/54918384 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47dbfff700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47dc0fe0e0 0x7f47dc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47e0b2a700 1 -- 172.21.15.40:0/54918384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47dc0fe0e0 msgr2=0x7f47dc10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47e0b2a700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47dc0fe0e0 0x7f47dc10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47e0b2a700 1 -- 172.21.15.40:0/54918384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8005040 con 0x7f47dc0fcef0 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47e0b2a700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fcef0 0x7f47dc10d970 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47d000b790 tx=0x7f47d0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47d97fa700 1 -- 172.21.15.40:0/54918384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d000f6f0 con 0x7f47dc0fcef0 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47d97fa700 1 -- 172.21.15.40:0/54918384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47d000fd20 con 0x7f47dc0fcef0 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc10f060 con 0x7f47dc0fcef0 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47d97fa700 1 -- 172.21.15.40:0/54918384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d00103a0 con 0x7f47dc0fcef0 2024-02-16T19:30:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.301+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc10f5a0 con 0x7f47dc0fcef0 2024-02-16T19:30:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.302+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc04bed0 con 0x7f47dc0fcef0 2024-02-16T19:30:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.303+0000 7f47d97fa700 1 -- 172.21.15.40:0/54918384 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f47d0011400 con 0x7f47dc0fcef0 2024-02-16T19:30:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.303+0000 7f47d97fa700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f47cc06d7f0 0x7f47cc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.304+0000 7f47dbfff700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f47cc06d7f0 0x7f47cc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.304+0000 7f47d97fa700 1 -- 172.21.15.40:0/54918384 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f47d008ccb0 con 0x7f47dc0fcef0 2024-02-16T19:30:00.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.304+0000 7f47dbfff700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f47cc06d7f0 0x7f47cc06fca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47c80043d0 tx=0x7f47c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.308+0000 7f47d97fa700 1 -- 172.21.15.40:0/54918384 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47d005a0b0 con 0x7f47dc0fcef0 2024-02-16T19:30:00.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.537+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47dc102ac0 con 0x7f47cc06d7f0 2024-02-16T19:30:00.538 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:00 smithi040 ceph-mon[94210]: overall HEALTH_OK 2024-02-16T19:30:00.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.538+0000 7f47d97fa700 1 -- 172.21.15.40:0/54918384 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f47dc102ac0 con 0x7f47cc06d7f0 2024-02-16T19:30:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f47cc06d7f0 msgr2=0x7f47cc06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f47cc06d7f0 0x7f47cc06fca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47c80043d0 tx=0x7f47c8009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fcef0 msgr2=0x7f47dc10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fcef0 0x7f47dc10d970 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47d000b790 tx=0x7f47d0009220 comp rx=0 tx=0).stop 2024-02-16T19:30:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 shutdown_connections 2024-02-16T19:30:00.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f47cc06d7f0 0x7f47cc06fca0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc0fcef0 0x7f47dc10d970 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 --2- 172.21.15.40:0/54918384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47dc0fe0e0 0x7f47dc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 >> 172.21.15.40:0/54918384 conn(0x7f47dc0f8620 msgr2=0x7f47dc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:00.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 shutdown_connections 2024-02-16T19:30:00.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.540+0000 7f47e2d8e700 1 -- 172.21.15.40:0/54918384 wait complete. 2024-02-16T19:30:00.552 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:30:00.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.644+0000 7f5a18315700 1 -- 172.21.15.40:0/3273724530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 msgr2=0x7f5a100fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.644+0000 7f5a18315700 1 --2- 172.21.15.40:0/3273724530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 0x7f5a100fe550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5a00004940 tx=0x7f5a00009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:00.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 -- 172.21.15.40:0/3273724530 shutdown_connections 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 --2- 172.21.15.40:0/3273724530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 0x7f5a100fe550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 --2- 172.21.15.40:0/3273724530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a100fcef0 0x7f5a100fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 -- 172.21.15.40:0/3273724530 >> 172.21.15.40:0/3273724530 conn(0x7f5a100f8620 msgr2=0x7f5a100faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 -- 172.21.15.40:0/3273724530 shutdown_connections 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 -- 172.21.15.40:0/3273724530 wait complete. 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 Processor -- start 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 -- start start 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a100fcef0 0x7f5a1010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 0x7f5a1010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1010e430 con 0x7f5a100fe0e0 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a18315700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1010ecd0 con 0x7f5a100fcef0 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a158b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 0x7f5a1010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.645+0000 7f5a158b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 0x7f5a1010de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50230/0 (socket says 172.21.15.40:50230) 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a158b0700 1 -- 172.21.15.40:0/3773607825 learned_addr learned my addr 172.21.15.40:0/3773607825 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a160b1700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a100fcef0 0x7f5a1010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a158b0700 1 -- 172.21.15.40:0/3773607825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a100fcef0 msgr2=0x7f5a1010d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a158b0700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a100fcef0 0x7f5a1010d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a158b0700 1 -- 172.21.15.40:0/3773607825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a00005040 con 0x7f5a100fe0e0 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a158b0700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 0x7f5a1010de30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000c00 tx=0x7f5a00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a06ffd700 1 -- 172.21.15.40:0/3773607825 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a0000e070 con 0x7f5a100fe0e0 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a06ffd700 1 -- 172.21.15.40:0/3773607825 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a0001c400 con 0x7f5a100fe0e0 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a06ffd700 1 -- 172.21.15.40:0/3773607825 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a0000b650 con 0x7f5a100fe0e0 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1010ef50 con 0x7f5a100fe0e0 2024-02-16T19:30:00.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.646+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1010f430 con 0x7f5a100fe0e0 2024-02-16T19:30:00.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.647+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1004bed0 con 0x7f5a100fe0e0 2024-02-16T19:30:00.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.648+0000 7f5a06ffd700 1 -- 172.21.15.40:0/3773607825 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5a0001ca50 con 0x7f5a100fe0e0 2024-02-16T19:30:00.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.648+0000 7f5a06ffd700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f59fc06d7e0 0x7f59fc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.648+0000 7f5a06ffd700 1 -- 172.21.15.40:0/3773607825 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5a0008e450 con 0x7f5a100fe0e0 2024-02-16T19:30:00.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.648+0000 7f5a160b1700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f59fc06d7e0 0x7f59fc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.649+0000 7f5a160b1700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f59fc06d7e0 0x7f59fc06fc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009b80 tx=0x7f5a0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:00.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.653+0000 7f5a06ffd700 1 -- 172.21.15.40:0/3773607825 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a0005b8f0 con 0x7f5a100fe0e0 2024-02-16T19:30:00.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:00 smithi163 ceph-mon[81637]: overall HEALTH_OK 2024-02-16T19:30:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.880+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a1010f8d0 con 0x7f59fc06d7e0 2024-02-16T19:30:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.882+0000 7f5a06ffd700 1 -- 172.21.15.40:0/3773607825 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5a1010f8d0 con 0x7f59fc06d7e0 2024-02-16T19:30:00.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f59fc06d7e0 msgr2=0x7f59fc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f59fc06d7e0 0x7f59fc06fc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009b80 tx=0x7f5a0c00b040 comp rx=0 tx=0).stop 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 msgr2=0x7f5a1010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 0x7f5a1010de30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000c00 tx=0x7f5a00009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 shutdown_connections 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a100fcef0 0x7f5a1010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f59fc06d7e0 0x7f59fc06fc90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 --2- 172.21.15.40:0/3773607825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a100fe0e0 0x7f5a1010de30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 >> 172.21.15.40:0/3773607825 conn(0x7f5a100f8620 msgr2=0x7f5a101013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 shutdown_connections 2024-02-16T19:30:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.884+0000 7f5a18315700 1 -- 172.21.15.40:0/3773607825 wait complete. 2024-02-16T19:30:00.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/1928663961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 msgr2=0x7fe3dc0fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 --2- 172.21.15.40:0/1928663961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 0x7fe3dc0fd450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4004940 tx=0x7fe3c4009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:00.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/1928663961 shutdown_connections 2024-02-16T19:30:00.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 --2- 172.21.15.40:0/1928663961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 0x7fe3dc0fd450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 --2- 172.21.15.40:0/1928663961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3dc0fbe70 0x7fe3dc0fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/1928663961 >> 172.21.15.40:0/1928663961 conn(0x7fe3dc0f7530 msgr2=0x7fe3dc0f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/1928663961 shutdown_connections 2024-02-16T19:30:00.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/1928663961 wait complete. 2024-02-16T19:30:00.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.988+0000 7fe3e0c4d700 1 Processor -- start 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3e0c4d700 1 -- start start 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3e0c4d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3dc0fbe70 0x7fe3dc10d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3e0c4d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 0x7fe3dc10dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3e0c4d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc10e3e0 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3e0c4d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc10ec80 con 0x7fe3dc0fbe70 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3d9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 0x7fe3dc10dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3d9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 0x7fe3dc10dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50262/0 (socket says 172.21.15.40:50262) 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3da59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3dc0fbe70 0x7fe3dc10d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3d9d9b700 1 -- 172.21.15.40:0/790524691 learned_addr learned my addr 172.21.15.40:0/790524691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3d9d9b700 1 -- 172.21.15.40:0/790524691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3dc0fbe70 msgr2=0x7fe3dc10d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3d9d9b700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3dc0fbe70 0x7fe3dc10d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:00.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3d9d9b700 1 -- 172.21.15.40:0/790524691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c4005040 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3d9d9b700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 0x7fe3dc10dde0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40093b0 tx=0x7fe3c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:00.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.989+0000 7fe3d2ffd700 1 -- 172.21.15.40:0/790524691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3c400e040 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.990+0000 7fe3d2ffd700 1 -- 172.21.15.40:0/790524691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3c400bcc0 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.990+0000 7fe3d2ffd700 1 -- 172.21.15.40:0/790524691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3c401c630 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.990+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc10ef00 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.990+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc10f3f0 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.991+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc04bed0 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.991+0000 7fe3d2ffd700 1 -- 172.21.15.40:0/790524691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe3c4013980 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.991+0000 7fe3d2ffd700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe3c806d7e0 0x7fe3c806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:00.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.992+0000 7fe3d2ffd700 1 -- 172.21.15.40:0/790524691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe3c408e250 con 0x7fe3dc0fcfe0 2024-02-16T19:30:00.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.992+0000 7fe3da59c700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe3c806d7e0 0x7fe3c806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:00.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.992+0000 7fe3da59c700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe3c806d7e0 0x7fe3c806fc90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4007a40 tx=0x7fe3d4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:00.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:00.996+0000 7fe3d2ffd700 1 -- 172.21.15.40:0/790524691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3c405b650 con 0x7fe3dc0fcfe0 2024-02-16T19:30:01.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.209+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3dc10fb20 con 0x7fe3c806d7e0 2024-02-16T19:30:01.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.219+0000 7fe3d2ffd700 1 -- 172.21.15.40:0/790524691 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fe3dc10fb20 con 0x7fe3c806d7e0 2024-02-16T19:30:01.220 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:30:01.220 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (4m) 20s ago 13m 25.3M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:30:01.220 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (2m) 20s ago 13m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:30:01.220 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (118s) 115s ago 12m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:30:01.220 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (4m) 20s ago 13m 81.7M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (7m) 20s ago 7m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (8m) 20s ago 7m 13.8M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (7m) 115s ago 7m 17.0M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (7m) 115s ago 7m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (3m) 20s ago 14m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (3m) 115s ago 12m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (3m) 20s ago 15m 47.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (2m) 115s ago 12m 32.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (4m) 20s ago 13m 15.7M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (4m) 115s ago 12m 16.0M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (98s) 20s ago 11m 96.1M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (61s) 20s ago 11m 99.2M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (25s) 20s ago 10m 12.4M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:30:01.221 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (10m) 115s ago 10m 270M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 02cfcc7865e3 2024-02-16T19:30:01.222 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (9m) 115s ago 9m 233M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:30:01.222 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (9m) 115s ago 9m 235M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:30:01.222 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (3m) 20s ago 12m 53.8M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:30:01.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe3c806d7e0 msgr2=0x7fe3c806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe3c806d7e0 0x7fe3c806fc90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4007a40 tx=0x7fe3d4009da0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 msgr2=0x7fe3dc10dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 0x7fe3dc10dde0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40093b0 tx=0x7fe3c4009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:01.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 shutdown_connections 2024-02-16T19:30:01.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3dc0fbe70 0x7fe3dc10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe3c806d7e0 0x7fe3c806fc90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 --2- 172.21.15.40:0/790524691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3dc0fcfe0 0x7fe3dc10dde0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 >> 172.21.15.40:0/790524691 conn(0x7fe3dc0f7530 msgr2=0x7fe3dc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:01.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 shutdown_connections 2024-02-16T19:30:01.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.221+0000 7fe3e0c4d700 1 -- 172.21.15.40:0/790524691 wait complete. 2024-02-16T19:30:01.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.322+0000 7ff4008aa700 1 -- 172.21.15.40:0/4131366848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 msgr2=0x7ff3fc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.322+0000 7ff4008aa700 1 --2- 172.21.15.40:0/4131366848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 0x7ff3fc0fe550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4004940 tx=0x7ff3e4009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:01.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.322+0000 7ff4008aa700 1 -- 172.21.15.40:0/4131366848 shutdown_connections 2024-02-16T19:30:01.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.322+0000 7ff4008aa700 1 --2- 172.21.15.40:0/4131366848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 0x7ff3fc0fe550 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.322+0000 7ff4008aa700 1 --2- 172.21.15.40:0/4131366848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3fc0fcef0 0x7ff3fc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.322+0000 7ff4008aa700 1 -- 172.21.15.40:0/4131366848 >> 172.21.15.40:0/4131366848 conn(0x7ff3fc0f8620 msgr2=0x7ff3fc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:01.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.322+0000 7ff4008aa700 1 -- 172.21.15.40:0/4131366848 shutdown_connections 2024-02-16T19:30:01.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.322+0000 7ff4008aa700 1 -- 172.21.15.40:0/4131366848 wait complete. 2024-02-16T19:30:01.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff4008aa700 1 Processor -- start 2024-02-16T19:30:01.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff4008aa700 1 -- start start 2024-02-16T19:30:01.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff4008aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3fc0fcef0 0x7ff3fc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:01.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff4008aa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 0x7ff3fc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:01.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff4008aa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc10e4b0 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff4008aa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc10ed50 con 0x7ff3fc0fcef0 2024-02-16T19:30:01.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff3f9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 0x7ff3fc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:01.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff3f9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 0x7ff3fc10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50276/0 (socket says 172.21.15.40:50276) 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff3f9d9b700 1 -- 172.21.15.40:0/4191833070 learned_addr learned my addr 172.21.15.40:0/4191833070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff3fa59c700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3fc0fcef0 0x7ff3fc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff3f9d9b700 1 -- 172.21.15.40:0/4191833070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3fc0fcef0 msgr2=0x7ff3fc10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff3f9d9b700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3fc0fcef0 0x7ff3fc10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.323+0000 7ff3f9d9b700 1 -- 172.21.15.40:0/4191833070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e4005040 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.324+0000 7ff3f9d9b700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 0x7ff3fc10deb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009fd0 tx=0x7ff3e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.324+0000 7ff3f2ffd700 1 -- 172.21.15.40:0/4191833070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3e400e040 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.324+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc10efd0 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.324+0000 7ff3f2ffd700 1 -- 172.21.15.40:0/4191833070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3e400bcc0 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.324+0000 7ff3f2ffd700 1 -- 172.21.15.40:0/4191833070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3e401c630 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.324+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc10f510 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.325+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc04bed0 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.325+0000 7ff3f2ffd700 1 -- 172.21.15.40:0/4191833070 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff3e4013420 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.326+0000 7ff3f2ffd700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3e806d840 0x7ff3e806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.326+0000 7ff3f2ffd700 1 -- 172.21.15.40:0/4191833070 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff3e408e3a0 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.326+0000 7ff3fa59c700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3e806d840 0x7ff3e806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.326+0000 7ff3fa59c700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3e806d840 0x7ff3e806fcf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0fcc30 tx=0x7ff3ec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:01.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.330+0000 7ff3f2ffd700 1 -- 172.21.15.40:0/4191833070 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff3e405b700 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:01 smithi163 ceph-mon[81637]: pgmap v107: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:01.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:01 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:30:01.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:01 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:30:01.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:01 smithi163 ceph-mon[81637]: Upgrade: osd.3 is safe to restart 2024-02-16T19:30:01.404 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:01 smithi163 ceph-mon[81637]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:01.404 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:01 smithi163 ceph-mon[81637]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:01.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.616+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff3fc105a30 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.617 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:01 smithi040 ceph-mon[94210]: pgmap v107: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:01.617 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:01 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:30:01.617 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:01 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-16T19:30:01.618 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:01 smithi040 ceph-mon[94210]: Upgrade: osd.3 is safe to restart 2024-02-16T19:30:01.618 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:01 smithi040 ceph-mon[94210]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:01.618 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:01 smithi040 ceph-mon[94210]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.616+0000 7ff3f2ffd700 1 -- 172.21.15.40:0/4191833070 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7ff3e405b290 con 0x7ff3fc0fe0e0 2024-02-16T19:30:01.618 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:30:01.618 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:30:01.618 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:30:01.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 3, 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 7, 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 7 2024-02-16T19:30:01.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:30:01.620 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:30:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3e806d840 msgr2=0x7ff3e806fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3e806d840 0x7ff3e806fcf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0fcc30 tx=0x7ff3ec00b040 comp rx=0 tx=0).stop 2024-02-16T19:30:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 msgr2=0x7ff3fc10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 0x7ff3fc10deb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009fd0 tx=0x7ff3e4009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 shutdown_connections 2024-02-16T19:30:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3fc0fcef0 0x7ff3fc10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3e806d840 0x7ff3e806fcf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 --2- 172.21.15.40:0/4191833070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc0fe0e0 0x7ff3fc10deb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 >> 172.21.15.40:0/4191833070 conn(0x7ff3fc0f8620 msgr2=0x7ff3fc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 shutdown_connections 2024-02-16T19:30:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.618+0000 7ff4008aa700 1 -- 172.21.15.40:0/4191833070 wait complete. 2024-02-16T19:30:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.720+0000 7fc4338f8700 1 -- 172.21.15.40:0/1660187719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c0fda80 msgr2=0x7fc42c0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.720+0000 7fc4338f8700 1 --2- 172.21.15.40:0/1660187719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c0fda80 0x7fc42c0ffe60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc428004940 tx=0x7fc428009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.720+0000 7fc4338f8700 1 -- 172.21.15.40:0/1660187719 shutdown_connections 2024-02-16T19:30:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 --2- 172.21.15.40:0/1660187719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c1003a0 0x7fc42c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 --2- 172.21.15.40:0/1660187719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c0fda80 0x7fc42c0ffe60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 -- 172.21.15.40:0/1660187719 >> 172.21.15.40:0/1660187719 conn(0x7fc42c0f7520 msgr2=0x7fc42c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 -- 172.21.15.40:0/1660187719 shutdown_connections 2024-02-16T19:30:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 -- 172.21.15.40:0/1660187719 wait complete. 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 Processor -- start 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 -- start start 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c0fda80 0x7fc42c0fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c1003a0 0x7fc42c0fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0fbf90 con 0x7fc42c1003a0 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc4338f8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0fc100 con 0x7fc42c0fda80 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.721+0000 7fc430e93700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c1003a0 0x7fc42c0fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc431694700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c0fda80 0x7fc42c0fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc430e93700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c1003a0 0x7fc42c0fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50300/0 (socket says 172.21.15.40:50300) 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc430e93700 1 -- 172.21.15.40:0/946261313 learned_addr learned my addr 172.21.15.40:0/946261313 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc430e93700 1 -- 172.21.15.40:0/946261313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c0fda80 msgr2=0x7fc42c0fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc430e93700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c0fda80 0x7fc42c0fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc430e93700 1 -- 172.21.15.40:0/946261313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428005040 con 0x7fc42c1003a0 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc430e93700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c1003a0 0x7fc42c0fb990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc41c002a50 tx=0x7fc41c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc421ffb700 1 -- 172.21.15.40:0/946261313 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc41c007cc0 con 0x7fc42c1003a0 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc421ffb700 1 -- 172.21.15.40:0/946261313 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc41c00f040 con 0x7fc42c1003a0 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c0fc3e0 con 0x7fc42c1003a0 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc421ffb700 1 -- 172.21.15.40:0/946261313 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc41c00da30 con 0x7fc42c1003a0 2024-02-16T19:30:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.722+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c115900 con 0x7fc42c1003a0 2024-02-16T19:30:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.723+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42c04bed0 con 0x7fc42c1003a0 2024-02-16T19:30:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.724+0000 7fc421ffb700 1 -- 172.21.15.40:0/946261313 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc41c00db90 con 0x7fc42c1003a0 2024-02-16T19:30:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.724+0000 7fc421ffb700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc41806d7e0 0x7fc41806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.724+0000 7fc421ffb700 1 -- 172.21.15.40:0/946261313 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc41c08cb60 con 0x7fc42c1003a0 2024-02-16T19:30:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.724+0000 7fc431694700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc41806d7e0 0x7fc41806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.725+0000 7fc431694700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc41806d7e0 0x7fc41806fc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc428004890 tx=0x7fc428009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:01.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.729+0000 7fc421ffb700 1 -- 172.21.15.40:0/946261313 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc41c059f60 con 0x7fc42c1003a0 2024-02-16T19:30:01.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.974+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc42c105a10 con 0x7fc42c1003a0 2024-02-16T19:30:01.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.975+0000 7fc421ffb700 1 -- 172.21.15.40:0/946261313 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fc41c059af0 con 0x7fc42c1003a0 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:e20 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 20 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:30:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:27:43.925056+0000 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:30:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:30:01.979 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:30:01.980 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:30:01.980 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:30:01.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc41806d7e0 msgr2=0x7fc41806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc41806d7e0 0x7fc41806fc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc428004890 tx=0x7fc428009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c1003a0 msgr2=0x7fc42c0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c1003a0 0x7fc42c0fb990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc41c002a50 tx=0x7fc41c0046b0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 shutdown_connections 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c0fda80 0x7fc42c0fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc41806d7e0 0x7fc41806fc90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 --2- 172.21.15.40:0/946261313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc42c1003a0 0x7fc42c0fb990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 >> 172.21.15.40:0/946261313 conn(0x7fc42c0f7520 msgr2=0x7fc42c0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 shutdown_connections 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:01.977+0000 7fc4338f8700 1 -- 172.21.15.40:0/946261313 wait complete. 2024-02-16T19:30:01.981 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-02-16T19:30:02.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.080+0000 7fd02aae3700 1 -- 172.21.15.40:0/1765021645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 msgr2=0x7fd0240fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.080+0000 7fd02aae3700 1 --2- 172.21.15.40:0/1765021645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 0x7fd0240fd320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd018004940 tx=0x7fd018009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.080+0000 7fd02aae3700 1 -- 172.21.15.40:0/1765021645 shutdown_connections 2024-02-16T19:30:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.080+0000 7fd02aae3700 1 --2- 172.21.15.40:0/1765021645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0240fe0e0 0x7fd0240fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.080+0000 7fd02aae3700 1 --2- 172.21.15.40:0/1765021645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 0x7fd0240fd320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.080+0000 7fd02aae3700 1 -- 172.21.15.40:0/1765021645 >> 172.21.15.40:0/1765021645 conn(0x7fd0240f8620 msgr2=0x7fd0240faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.080+0000 7fd02aae3700 1 -- 172.21.15.40:0/1765021645 shutdown_connections 2024-02-16T19:30:02.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.080+0000 7fd02aae3700 1 -- 172.21.15.40:0/1765021645 wait complete. 2024-02-16T19:30:02.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02aae3700 1 Processor -- start 2024-02-16T19:30:02.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02aae3700 1 -- start start 2024-02-16T19:30:02.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02aae3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 0x7fd02410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:02.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02aae3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0240fe0e0 0x7fd02410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02aae3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02410e420 con 0x7fd0240fcef0 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02aae3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02410ecc0 con 0x7fd0240fe0e0 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02887f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 0x7fd02410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02887f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 0x7fd02410d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50320/0 (socket says 172.21.15.40:50320) 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.081+0000 7fd02887f700 1 -- 172.21.15.40:0/2835947873 learned_addr learned my addr 172.21.15.40:0/2835947873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd023fff700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0240fe0e0 0x7fd02410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd02887f700 1 -- 172.21.15.40:0/2835947873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0240fe0e0 msgr2=0x7fd02410de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd02887f700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0240fe0e0 0x7fd02410de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd02887f700 1 -- 172.21.15.40:0/2835947873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018005040 con 0x7fd0240fcef0 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd023fff700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0240fe0e0 0x7fd02410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd02887f700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 0x7fd02410d8e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd018009040 tx=0x7fd018009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd0217fa700 1 -- 172.21.15.40:0/2835947873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd01800e070 con 0x7fd0240fcef0 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd0217fa700 1 -- 172.21.15.40:0/2835947873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd01801c400 con 0x7fd0240fcef0 2024-02-16T19:30:02.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02410ef40 con 0x7fd0240fcef0 2024-02-16T19:30:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd0217fa700 1 -- 172.21.15.40:0/2835947873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd01800b650 con 0x7fd0240fcef0 2024-02-16T19:30:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.082+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02410f420 con 0x7fd0240fcef0 2024-02-16T19:30:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.083+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd02404bed0 con 0x7fd0240fcef0 2024-02-16T19:30:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.084+0000 7fd0217fa700 1 -- 172.21.15.40:0/2835947873 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd018013af0 con 0x7fd0240fcef0 2024-02-16T19:30:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.084+0000 7fd0217fa700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd01406d7f0 0x7fd01406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.084+0000 7fd0217fa700 1 -- 172.21.15.40:0/2835947873 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd01808e650 con 0x7fd0240fcef0 2024-02-16T19:30:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.084+0000 7fd023fff700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd01406d7f0 0x7fd01406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.085+0000 7fd023fff700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd01406d7f0 0x7fd01406fca0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd010004600 tx=0x7fd01000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:02.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.089+0000 7fd0217fa700 1 -- 172.21.15.40:0/2835947873 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd01805ba50 con 0x7fd0240fcef0 2024-02-16T19:30:02.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.330+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd02410f840 con 0x7fd01406d7f0 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.331+0000 7fd0217fa700 1 -- 172.21.15.40:0/2835947873 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd02410f840 con 0x7fd01406d7f0 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:30:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:30:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:30:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "9/21 daemons upgraded", 2024-02-16T19:30:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:30:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:30:02.333 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:30:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd01406d7f0 msgr2=0x7fd01406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd01406d7f0 0x7fd01406fca0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd010004600 tx=0x7fd01000b040 comp rx=0 tx=0).stop 2024-02-16T19:30:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 msgr2=0x7fd02410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 0x7fd02410d8e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd018009040 tx=0x7fd018009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 shutdown_connections 2024-02-16T19:30:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd01406d7f0 0x7fd01406fca0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0240fcef0 0x7fd02410d8e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 --2- 172.21.15.40:0/2835947873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0240fe0e0 0x7fd02410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 >> 172.21.15.40:0/2835947873 conn(0x7fd0240f8620 msgr2=0x7fd0241013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 shutdown_connections 2024-02-16T19:30:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.333+0000 7fd02aae3700 1 -- 172.21.15.40:0/2835947873 wait complete. 2024-02-16T19:30:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.435+0000 7fcde6382700 1 -- 172.21.15.40:0/4217121004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fe0e0 msgr2=0x7fcde00fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.435+0000 7fcde6382700 1 --2- 172.21.15.40:0/4217121004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fe0e0 0x7fcde00fe550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4004940 tx=0x7fcdd4009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.435+0000 7fcde6382700 1 -- 172.21.15.40:0/4217121004 shutdown_connections 2024-02-16T19:30:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.435+0000 7fcde6382700 1 --2- 172.21.15.40:0/4217121004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fe0e0 0x7fcde00fe550 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.435+0000 7fcde6382700 1 --2- 172.21.15.40:0/4217121004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde00fcef0 0x7fcde00fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.435+0000 7fcde6382700 1 -- 172.21.15.40:0/4217121004 >> 172.21.15.40:0/4217121004 conn(0x7fcde00f8620 msgr2=0x7fcde00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.436+0000 7fcde6382700 1 -- 172.21.15.40:0/4217121004 shutdown_connections 2024-02-16T19:30:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.436+0000 7fcde6382700 1 -- 172.21.15.40:0/4217121004 wait complete. 2024-02-16T19:30:02.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.436+0000 7fcde6382700 1 Processor -- start 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.436+0000 7fcde6382700 1 -- start start 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcde6382700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fcef0 0x7fcde010d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcde6382700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde00fe0e0 0x7fcde010de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcde6382700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde010e410 con 0x7fcde00fcef0 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcde6382700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde010ecb0 con 0x7fcde00fe0e0 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fcef0 0x7fcde010d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fcef0 0x7fcde010d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50340/0 (socket says 172.21.15.40:50340) 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddffff700 1 -- 172.21.15.40:0/2972949525 learned_addr learned my addr 172.21.15.40:0/2972949525 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddf7fe700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde00fe0e0 0x7fcde010de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddffff700 1 -- 172.21.15.40:0/2972949525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde00fe0e0 msgr2=0x7fcde010de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddffff700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde00fe0e0 0x7fcde010de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddffff700 1 -- 172.21.15.40:0/2972949525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd4005040 con 0x7fcde00fcef0 2024-02-16T19:30:02.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddffff700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fcef0 0x7fcde010d8d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000eb70 tx=0x7fcdd000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:02.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddcff9700 1 -- 172.21.15.40:0/2972949525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd0007cc0 con 0x7fcde00fcef0 2024-02-16T19:30:02.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddcff9700 1 -- 172.21.15.40:0/2972949525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdd000f040 con 0x7fcde00fcef0 2024-02-16T19:30:02.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcddcff9700 1 -- 172.21.15.40:0/2972949525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd000cb20 con 0x7fcde00fcef0 2024-02-16T19:30:02.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde010ef30 con 0x7fcde00fcef0 2024-02-16T19:30:02.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.437+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde010f470 con 0x7fcde00fcef0 2024-02-16T19:30:02.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.438+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde004bed0 con 0x7fcde00fcef0 2024-02-16T19:30:02.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.439+0000 7fcddcff9700 1 -- 172.21.15.40:0/2972949525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcdd000cc80 con 0x7fcde00fcef0 2024-02-16T19:30:02.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.439+0000 7fcddcff9700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc806d7e0 0x7fcdc806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:02.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.439+0000 7fcddcff9700 1 -- 172.21.15.40:0/2972949525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcdd008c8e0 con 0x7fcde00fcef0 2024-02-16T19:30:02.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.439+0000 7fcddf7fe700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc806d7e0 0x7fcdc806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:02.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.440+0000 7fcddf7fe700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc806d7e0 0x7fcdc806fc90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009b30 tx=0x7fcdd4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:02.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.444+0000 7fcddcff9700 1 -- 172.21.15.40:0/2972949525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdd0059d10 con 0x7fcde00fcef0 2024-02-16T19:30:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:02 smithi163 ceph-mon[81637]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:02 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/4191833070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:02 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/946261313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:30:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:02 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:02 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:02 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:30:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:02 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:02 smithi040 ceph-mon[94210]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:02 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/4191833070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:02 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/946261313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:30:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-16T19:30:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:02.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.732+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcde0105a30 con 0x7fcde00fcef0 2024-02-16T19:30:02.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.732+0000 7fcddcff9700 1 -- 172.21.15.40:0/2972949525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fcdd00598a0 con 0x7fcde00fcef0 2024-02-16T19:30:02.733 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc806d7e0 msgr2=0x7fcdc806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc806d7e0 0x7fcdc806fc90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009b30 tx=0x7fcdd4009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fcef0 msgr2=0x7fcde010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fcef0 0x7fcde010d8d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000eb70 tx=0x7fcdd000b040 comp rx=0 tx=0).stop 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 shutdown_connections 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcdc806d7e0 0x7fcdc806fc90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcde00fcef0 0x7fcde010d8d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 --2- 172.21.15.40:0/2972949525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde00fe0e0 0x7fcde010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 >> 172.21.15.40:0/2972949525 conn(0x7fcde00f8620 msgr2=0x7fcde01013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 shutdown_connections 2024-02-16T19:30:02.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:02.734+0000 7fcde6382700 1 -- 172.21.15.40:0/2972949525 wait complete. 2024-02-16T19:30:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:03 smithi163 ceph-mon[81637]: pgmap v108: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:03.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:03 smithi163 ceph-mon[81637]: Upgrade: Updating osd.3 2024-02-16T19:30:03.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:03 smithi163 ceph-mon[81637]: Deploying daemon osd.3 on smithi163 2024-02-16T19:30:03.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:03 smithi163 ceph-mon[81637]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:03.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:03 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2972949525' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:30:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:03 smithi040 ceph-mon[94210]: pgmap v108: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:03 smithi040 ceph-mon[94210]: Upgrade: Updating osd.3 2024-02-16T19:30:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:03 smithi040 ceph-mon[94210]: Deploying daemon osd.3 on smithi163 2024-02-16T19:30:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:03 smithi040 ceph-mon[94210]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:03 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2972949525' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:30:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:05 smithi163 ceph-mon[81637]: pgmap v109: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:05 smithi040 ceph-mon[94210]: pgmap v109: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:05.737 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:05 smithi163 systemd[1]: Stopping Ceph osd.3 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:30:06.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:05 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[42055]: 2024-02-16T19:30:05.757+0000 7fbd708ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:30:06.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:05 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[42055]: 2024-02-16T19:30:05.757+0000 7fbd708ff700 -1 osd.3 64 *** Got signal Terminated *** 2024-02-16T19:30:06.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:05 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[42055]: 2024-02-16T19:30:05.757+0000 7fbd708ff700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:30:06.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:06 smithi040 ceph-mon[94210]: osd.3 marked itself down and dead 2024-02-16T19:30:06.682 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:06 smithi163 ceph-mon[81637]: osd.3 marked itself down and dead 2024-02-16T19:30:06.987 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:06 smithi163 podman[90611]: 2024-02-16 19:30:06.755393054 +0000 UTC m=+1.108748590 container died 02cfcc7865e3903af3890253adab0a85bdb62dc543c155c89884a53443cbdcf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-b88fae6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-16T19:30:07.397 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:07 smithi163 ceph-mon[81637]: pgmap v110: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:07.397 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:07 smithi163 ceph-mon[81637]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:30:07.397 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:07 smithi163 ceph-mon[81637]: failed to encode map e65 with expected crc 2024-02-16T19:30:07.397 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:07 smithi163 ceph-mon[81637]: osdmap e65: 6 total, 5 up, 6 in 2024-02-16T19:30:07.397 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:07 smithi163 podman[90611]: 2024-02-16 19:30:07.153385939 +0000 UTC m=+1.506741482 container cleanup 02cfcc7865e3903af3890253adab0a85bdb62dc543c155c89884a53443cbdcf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=quincy-b88fae6, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-02-16T19:30:07.398 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:07 smithi163 bash[90611]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3 2024-02-16T19:30:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:07 smithi040 ceph-mon[94210]: pgmap v110: 65 pgs: 65 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:07 smithi040 ceph-mon[94210]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:30:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:07 smithi040 ceph-mon[94210]: failed to encode map e65 with expected crc 2024-02-16T19:30:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:07 smithi040 ceph-mon[94210]: osdmap e65: 6 total, 5 up, 6 in 2024-02-16T19:30:07.737 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:07 smithi163 podman[90641]: 2024-02-16 19:30:07.396804373 +0000 UTC m=+0.637996299 container remove 02cfcc7865e3903af3890253adab0a85bdb62dc543c155c89884a53443cbdcf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-b88fae6, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-16T19:30:08.042 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:07 smithi163 podman[90735]: 2024-02-16 19:30:07.6601535 +0000 UTC m=+0.018849231 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:30:08.042 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:07 smithi163 podman[90735]: 2024-02-16 19:30:07.798238968 +0000 UTC m=+0.156934733 container create dbdd2b693b03cf571dccc4c4452c6ea38896770618810a7048a897f8e089832e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-16T19:30:08.042 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:07 smithi163 podman[90735]: 2024-02-16 19:30:07.967389201 +0000 UTC m=+0.326084924 container init dbdd2b693b03cf571dccc4c4452c6ea38896770618810a7048a897f8e089832e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-02-16T19:30:08.043 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:07 smithi163 podman[90735]: 2024-02-16 19:30:07.970700178 +0000 UTC m=+0.329395903 container start dbdd2b693b03cf571dccc4c4452c6ea38896770618810a7048a897f8e089832e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD) 2024-02-16T19:30:08.043 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:08 smithi163 podman[90735]: 2024-02-16 19:30:08.04229685 +0000 UTC m=+0.400992581 container attach dbdd2b693b03cf571dccc4c4452c6ea38896770618810a7048a897f8e089832e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-16T19:30:08.308 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:08 smithi163 podman[90735]: 2024-02-16 19:30:08.264750794 +0000 UTC m=+0.623446518 container died dbdd2b693b03cf571dccc4c4452c6ea38896770618810a7048a897f8e089832e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS) 2024-02-16T19:30:08.613 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:08 smithi163 ceph-mon[81637]: failed to encode map e66 with expected crc 2024-02-16T19:30:08.613 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:08 smithi163 ceph-mon[81637]: failed to encode map e66 with expected crc 2024-02-16T19:30:08.613 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:08 smithi163 ceph-mon[81637]: osdmap e66: 6 total, 5 up, 6 in 2024-02-16T19:30:08.613 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:08 smithi163 ceph-mon[81637]: failed to encode map e66 with expected crc 2024-02-16T19:30:08.613 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:08 smithi163 ceph-mon[81637]: failed to encode map e66 with expected crc 2024-02-16T19:30:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:08 smithi040 ceph-mon[94210]: failed to encode map e66 with expected crc 2024-02-16T19:30:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:08 smithi040 ceph-mon[94210]: failed to encode map e66 with expected crc 2024-02-16T19:30:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:08 smithi040 ceph-mon[94210]: osdmap e66: 6 total, 5 up, 6 in 2024-02-16T19:30:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:08 smithi040 ceph-mon[94210]: failed to encode map e66 with expected crc 2024-02-16T19:30:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:08 smithi040 ceph-mon[94210]: failed to encode map e66 with expected crc 2024-02-16T19:30:09.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:08 smithi163 podman[90761]: 2024-02-16 19:30:08.896579589 +0000 UTC m=+0.627070308 container remove dbdd2b693b03cf571dccc4c4452c6ea38896770618810a7048a897f8e089832e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:30:09.238 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:08 smithi163 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-16T19:30:09.238 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:08 smithi163 systemd[1]: Stopped Ceph osd.3 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:30:09.238 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:08 smithi163 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.3.service: Consumed 1min 9.077s CPU time. 2024-02-16T19:30:09.238 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:09 smithi163 systemd[1]: Starting Ceph osd.3 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:30:09.570 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:09 smithi163 ceph-mon[81637]: pgmap v113: 65 pgs: 9 active+undersized+degraded, 6 stale+active+clean, 9 active+undersized, 41 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:30:09.571 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:09 smithi163 ceph-mon[81637]: Health check failed: Degraded data redundancy: 43/360 objects degraded (11.944%), 9 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:09.571 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:09 smithi163 podman[90863]: 2024-02-16 19:30:09.225736099 +0000 UTC m=+0.017733169 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:30:09.571 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:09 smithi163 podman[90863]: 2024-02-16 19:30:09.348586533 +0000 UTC m=+0.140583646 container create 240aa2bee3acddcded3ed4d3a729dcc9413af69bcab86efa7c2128a90ef8e9a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, ceph=True) 2024-02-16T19:30:09.571 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:09 smithi163 podman[90863]: 2024-02-16 19:30:09.570219392 +0000 UTC m=+0.362216472 container init 240aa2bee3acddcded3ed4d3a729dcc9413af69bcab86efa7c2128a90ef8e9a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-02-16T19:30:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:09 smithi040 ceph-mon[94210]: pgmap v113: 65 pgs: 9 active+undersized+degraded, 6 stale+active+clean, 9 active+undersized, 41 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 43/360 objects degraded (11.944%) 2024-02-16T19:30:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:09 smithi040 ceph-mon[94210]: Health check failed: Degraded data redundancy: 43/360 objects degraded (11.944%), 9 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:09.987 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:09 smithi163 podman[90863]: 2024-02-16 19:30:09.573796393 +0000 UTC m=+0.365793470 container start 240aa2bee3acddcded3ed4d3a729dcc9413af69bcab86efa7c2128a90ef8e9a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-16T19:30:09.988 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:09 smithi163 podman[90863]: 2024-02-16 19:30:09.667965173 +0000 UTC m=+0.459962254 container attach 240aa2bee3acddcded3ed4d3a729dcc9413af69bcab86efa7c2128a90ef8e9a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-16T19:30:10.338 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate[90877]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T19:30:10.338 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate[90877]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T19:30:10.338 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 bash[90863]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T19:30:10.338 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 bash[90863]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-16T19:30:10.338 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 bash[90863]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T19:30:10.338 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 bash[90863]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T19:30:10.338 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 bash[90863]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-16T19:30:10.338 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 bash[90863]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T19:30:10.339 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 bash[90863]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-16T19:30:10.339 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate[90877]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-16T19:30:10.339 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate[90877]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-16T19:30:10.339 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate[90877]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-16T19:30:10.339 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate[90877]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-16T19:30:10.339 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate[90877]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-16T19:30:10.339 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 podman[90863]: 2024-02-16 19:30:10.037261965 +0000 UTC m=+0.829259029 container died 240aa2bee3acddcded3ed4d3a729dcc9413af69bcab86efa7c2128a90ef8e9a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux ) 2024-02-16T19:30:10.973 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 podman[91007]: 2024-02-16 19:30:10.638889486 +0000 UTC m=+0.595937801 container remove 240aa2bee3acddcded3ed4d3a729dcc9413af69bcab86efa7c2128a90ef8e9a2 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-16T19:30:10.973 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 podman[91073]: 2024-02-16 19:30:10.753111353 +0000 UTC m=+0.017815574 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:30:10.973 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:10 smithi163 podman[91073]: 2024-02-16 19:30:10.873363772 +0000 UTC m=+0.138067984 container create d4b9a7b78af11c512f5d3f8dc6df4bc2cc236ec1653c94dc872dfb9aaad57cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-16T19:30:11.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:11 smithi163 podman[91073]: 2024-02-16 19:30:11.059229355 +0000 UTC m=+0.323933569 container init d4b9a7b78af11c512f5d3f8dc6df4bc2cc236ec1653c94dc872dfb9aaad57cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:30:11.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:11 smithi163 podman[91073]: 2024-02-16 19:30:11.062213607 +0000 UTC m=+0.326917823 container start d4b9a7b78af11c512f5d3f8dc6df4bc2cc236ec1653c94dc872dfb9aaad57cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:30:11.238 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:11 smithi163 ceph-osd[91092]: -- 172.21.15.163:0/2917686916 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a56566b1e0 con 0x55a56563cc00 2024-02-16T19:30:11.238 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:11 smithi163 bash[91073]: d4b9a7b78af11c512f5d3f8dc6df4bc2cc236ec1653c94dc872dfb9aaad57cb3 2024-02-16T19:30:11.238 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:11 smithi163 systemd[1]: Started Ceph osd.3 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:30:11.615 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:11 smithi163 ceph-mon[81637]: pgmap v114: 65 pgs: 12 active+undersized+degraded, 3 stale+active+clean, 13 active+undersized, 37 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 52/360 objects degraded (14.444%) 2024-02-16T19:30:11.615 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:11.615 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:11.615 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:11 smithi040 ceph-mon[94210]: pgmap v114: 65 pgs: 12 active+undersized+degraded, 3 stale+active+clean, 13 active+undersized, 37 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 52/360 objects degraded (14.444%) 2024-02-16T19:30:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:11.987 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:11 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:30:11.614+0000 7fec14d117c0 -1 Falling back to public interface 2024-02-16T19:30:13.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:13 smithi040 ceph-mon[94210]: pgmap v115: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 75/360 objects degraded (20.833%) 2024-02-16T19:30:13.671 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:13 smithi163 ceph-mon[81637]: pgmap v115: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 75/360 objects degraded (20.833%) 2024-02-16T19:30:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:14 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 75/360 objects degraded (20.833%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:14 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:14.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:14 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 75/360 objects degraded (20.833%), 19 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:14.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:14 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:15.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:15 smithi040 ceph-mon[94210]: pgmap v116: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 75/360 objects degraded (20.833%) 2024-02-16T19:30:15.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:15 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:15 smithi163 ceph-mon[81637]: pgmap v116: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 75/360 objects degraded (20.833%) 2024-02-16T19:30:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:15 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:16.358 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:16 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:30:16.176+0000 7fec14d117c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-02-16T19:30:16.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:16 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:16.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:16 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:16.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:16 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:16.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:16 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:17.078 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:16 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:30:16.737+0000 7fec14d117c0 -1 osd.3 64 log_to_monitors true 2024-02-16T19:30:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:17 smithi040 ceph-mon[94210]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 75/360 objects degraded (20.833%) 2024-02-16T19:30:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:17 smithi040 ceph-mon[94210]: from='osd.3 [v2:172.21.15.163:6800/346064807,v1:172.21.15.163:6801/346064807]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:30:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:17 smithi040 ceph-mon[94210]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:30:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:17.737 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:30:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:30:17.490+0000 7fec06169700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:30:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:17 smithi163 ceph-mon[81637]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 307 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 75/360 objects degraded (20.833%) 2024-02-16T19:30:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:17 smithi163 ceph-mon[81637]: from='osd.3 [v2:172.21.15.163:6800/346064807,v1:172.21.15.163:6801/346064807]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:30:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:17 smithi163 ceph-mon[81637]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-16T19:30:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:18.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:18 smithi163 ceph-mon[81637]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T19:30:18.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:18 smithi163 ceph-mon[81637]: osdmap e67: 6 total, 5 up, 6 in 2024-02-16T19:30:18.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:18 smithi163 ceph-mon[81637]: from='osd.3 [v2:172.21.15.163:6800/346064807,v1:172.21.15.163:6801/346064807]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:30:18.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:18 smithi163 ceph-mon[81637]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:30:18.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:18 smithi163 ceph-mon[81637]: failed to encode map e67 with expected crc 2024-02-16T19:30:18.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:18 smithi163 ceph-mon[81637]: failed to encode map e67 with expected crc 2024-02-16T19:30:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:18 smithi040 ceph-mon[94210]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-16T19:30:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:18 smithi040 ceph-mon[94210]: osdmap e67: 6 total, 5 up, 6 in 2024-02-16T19:30:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:18 smithi040 ceph-mon[94210]: from='osd.3 [v2:172.21.15.163:6800/346064807,v1:172.21.15.163:6801/346064807]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:30:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:18 smithi040 ceph-mon[94210]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:30:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:18 smithi040 ceph-mon[94210]: failed to encode map e67 with expected crc 2024-02-16T19:30:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:18 smithi040 ceph-mon[94210]: failed to encode map e67 with expected crc 2024-02-16T19:30:19.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 75/360 objects degraded (20.833%) 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: failed to encode map e68 with expected crc 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: osd.3 [v2:172.21.15.163:6800/346064807,v1:172.21.15.163:6801/346064807] boot 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: osdmap e68: 6 total, 6 up, 6 in 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:30:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:30:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:19 smithi040 ceph-mon[94210]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-16T19:30:19.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 75/360 objects degraded (20.833%) 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: failed to encode map e68 with expected crc 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: osd.3 [v2:172.21.15.163:6800/346064807,v1:172.21.15.163:6801/346064807] boot 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: osdmap e68: 6 total, 6 up, 6 in 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:30:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:30:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:19 smithi163 ceph-mon[81637]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-16T19:30:20.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:20 smithi040 ceph-mon[94210]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:20 smithi040 ceph-mon[94210]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:20 smithi040 ceph-mon[94210]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:20 smithi040 ceph-mon[94210]: osdmap e69: 6 total, 6 up, 6 in 2024-02-16T19:30:20.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:20 smithi040 ceph-mon[94210]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:20 smithi040 ceph-mon[94210]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:20 smithi040 ceph-mon[94210]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:20 smithi040 ceph-mon[94210]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:20 smithi163 ceph-mon[81637]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:20 smithi163 ceph-mon[81637]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:20 smithi163 ceph-mon[81637]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:20 smithi163 ceph-mon[81637]: osdmap e69: 6 total, 6 up, 6 in 2024-02-16T19:30:20.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:20 smithi163 ceph-mon[81637]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:20 smithi163 ceph-mon[81637]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:20 smithi163 ceph-mon[81637]: failed to encode map e69 with expected crc 2024-02-16T19:30:20.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:20 smithi163 ceph-mon[81637]: failed to encode map e69 with expected crc 2024-02-16T19:30:21.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:21 smithi040 ceph-mon[94210]: pgmap v122: 65 pgs: 1 activating, 18 active+undersized+degraded, 16 active+undersized, 30 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/360 objects degraded (19.722%) 2024-02-16T19:30:21.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:21 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 71/360 objects degraded (19.722%), 18 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:21.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:21 smithi163 ceph-mon[81637]: pgmap v122: 65 pgs: 1 activating, 18 active+undersized+degraded, 16 active+undersized, 30 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/360 objects degraded (19.722%) 2024-02-16T19:30:21.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:21 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 71/360 objects degraded (19.722%), 18 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:23.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:23 smithi040 ceph-mon[94210]: pgmap v123: 65 pgs: 1 activating, 13 active+undersized+degraded, 11 active+undersized, 40 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:30:23.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:23 smithi163 ceph-mon[81637]: pgmap v123: 65 pgs: 1 activating, 13 active+undersized+degraded, 11 active+undersized, 40 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:30:24.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:24 smithi040 ceph-mon[94210]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 56/360 objects degraded (15.556%), 13 pgs degraded) 2024-02-16T19:30:24.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:24 smithi040 ceph-mon[94210]: Cluster is now healthy 2024-02-16T19:30:24.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:24 smithi163 ceph-mon[81637]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 56/360 objects degraded (15.556%), 13 pgs degraded) 2024-02-16T19:30:24.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:24 smithi163 ceph-mon[81637]: Cluster is now healthy 2024-02-16T19:30:25.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:25 smithi040 ceph-mon[94210]: pgmap v124: 65 pgs: 1 activating, 64 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:25.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:25 smithi163 ceph-mon[81637]: pgmap v124: 65 pgs: 1 activating, 64 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:27.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:27 smithi040 ceph-mon[94210]: pgmap v125: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:27.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:27 smithi163 ceph-mon[81637]: pgmap v125: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:29.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:29 smithi040 ceph-mon[94210]: pgmap v126: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:29.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:29 smithi163 ceph-mon[81637]: pgmap v126: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:31.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:31 smithi040 ceph-mon[94210]: pgmap v127: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:31.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:31 smithi163 ceph-mon[81637]: pgmap v127: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.851+0000 7fc2c16a2700 1 -- 172.21.15.40:0/3210458973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fe1f0 msgr2=0x7fc2bc0fe660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:32.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.851+0000 7fc2c16a2700 1 --2- 172.21.15.40:0/3210458973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fe1f0 0x7fc2bc0fe660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4004940 tx=0x7fc2a4009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.851+0000 7fc2c16a2700 1 -- 172.21.15.40:0/3210458973 shutdown_connections 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.851+0000 7fc2c16a2700 1 --2- 172.21.15.40:0/3210458973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fe1f0 0x7fc2bc0fe660 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.851+0000 7fc2c16a2700 1 --2- 172.21.15.40:0/3210458973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0fcf40 0x7fc2bc0fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.851+0000 7fc2c16a2700 1 -- 172.21.15.40:0/3210458973 >> 172.21.15.40:0/3210458973 conn(0x7fc2bc0f8680 msgr2=0x7fc2bc0faad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.851+0000 7fc2c16a2700 1 -- 172.21.15.40:0/3210458973 shutdown_connections 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.851+0000 7fc2c16a2700 1 -- 172.21.15.40:0/3210458973 wait complete. 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.852+0000 7fc2c16a2700 1 Processor -- start 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.852+0000 7fc2c16a2700 1 -- start start 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.852+0000 7fc2c16a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fcf40 0x7fc2bc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.852+0000 7fc2c16a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0fe1f0 0x7fc2bc0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:32.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.852+0000 7fc2c16a2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc070970 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.852+0000 7fc2c16a2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc070ae0 con 0x7fc2bc0fe1f0 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.852+0000 7fc2baffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fcf40 0x7fc2bc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2baffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fcf40 0x7fc2bc071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52630/0 (socket says 172.21.15.40:52630) 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2baffd700 1 -- 172.21.15.40:0/1332162740 learned_addr learned my addr 172.21.15.40:0/1332162740 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2ba7fc700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0fe1f0 0x7fc2bc0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2baffd700 1 -- 172.21.15.40:0/1332162740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0fe1f0 msgr2=0x7fc2bc0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2baffd700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0fe1f0 0x7fc2bc0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2baffd700 1 -- 172.21.15.40:0/1332162740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a4005040 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2baffd700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fcf40 0x7fc2bc071e60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00bbb0 tx=0x7fc2ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2b37fe700 1 -- 172.21.15.40:0/1332162740 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2ac0092e0 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2b37fe700 1 -- 172.21.15.40:0/1332162740 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2ac00cb20 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2bc070dc0 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2b37fe700 1 -- 172.21.15.40:0/1332162740 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2ac015d70 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.853+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2bc071140 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.855+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2bc04bed0 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.855+0000 7fc2b37fe700 1 -- 172.21.15.40:0/1332162740 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc2ac012070 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.855+0000 7fc2b37fe700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc2a806d7e0 0x7fc2a806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:32.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.856+0000 7fc2b37fe700 1 -- 172.21.15.40:0/1332162740 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc2ac08cbe0 con 0x7fc2bc0fcf40 2024-02-16T19:30:32.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.856+0000 7fc2ba7fc700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc2a806d7e0 0x7fc2a806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:32.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.856+0000 7fc2ba7fc700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc2a806d7e0 0x7fc2a806fc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b3b0 tx=0x7fc2a4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:32.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:32.860+0000 7fc2b37fe700 1 -- 172.21.15.40:0/1332162740 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2ac059f60 con 0x7fc2bc0fcf40 2024-02-16T19:30:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.083+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2bc102b40 con 0x7fc2a806d7e0 2024-02-16T19:30:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.085+0000 7fc2b37fe700 1 -- 172.21.15.40:0/1332162740 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc2bc102b40 con 0x7fc2a806d7e0 2024-02-16T19:30:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc2a806d7e0 msgr2=0x7fc2a806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc2a806d7e0 0x7fc2a806fc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b3b0 tx=0x7fc2a4009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fcf40 msgr2=0x7fc2bc071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fcf40 0x7fc2bc071e60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00bbb0 tx=0x7fc2ac009840 comp rx=0 tx=0).stop 2024-02-16T19:30:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 shutdown_connections 2024-02-16T19:30:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc2a806d7e0 0x7fc2a806fc90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2bc0fcf40 0x7fc2bc071e60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 --2- 172.21.15.40:0/1332162740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0fe1f0 0x7fc2bc0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 >> 172.21.15.40:0/1332162740 conn(0x7fc2bc0f8680 msgr2=0x7fc2bc101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 shutdown_connections 2024-02-16T19:30:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.087+0000 7fc2c16a2700 1 -- 172.21.15.40:0/1332162740 wait complete. 2024-02-16T19:30:33.098 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:30:33.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 -- 172.21.15.40:0/3291946281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b80663c0 msgr2=0x7f28b80ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 --2- 172.21.15.40:0/3291946281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b80663c0 0x7f28b80ffb10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28a8004940 tx=0x7f28a8009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 -- 172.21.15.40:0/3291946281 shutdown_connections 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 --2- 172.21.15.40:0/3291946281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b81000e0 0x7f28b8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 --2- 172.21.15.40:0/3291946281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b80663c0 0x7f28b80ffb10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 -- 172.21.15.40:0/3291946281 >> 172.21.15.40:0/3291946281 conn(0x7f28b80f74e0 msgr2=0x7f28b80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 -- 172.21.15.40:0/3291946281 shutdown_connections 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 -- 172.21.15.40:0/3291946281 wait complete. 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 Processor -- start 2024-02-16T19:30:33.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.191+0000 7f28bc971700 1 -- start start 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28bc971700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b80663c0 0x7f28b80fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28bc971700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b81000e0 0x7f28b80fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28bc971700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80fc010 con 0x7f28b81000e0 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28bc971700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80fc180 con 0x7f28b80663c0 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28b5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b81000e0 0x7f28b80fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28b5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b81000e0 0x7f28b80fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52642/0 (socket says 172.21.15.40:52642) 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28b5d9b700 1 -- 172.21.15.40:0/4229826723 learned_addr learned my addr 172.21.15.40:0/4229826723 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28b659c700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b80663c0 0x7f28b80fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28b5d9b700 1 -- 172.21.15.40:0/4229826723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b80663c0 msgr2=0x7f28b80fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28b5d9b700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b80663c0 0x7f28b80fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28b5d9b700 1 -- 172.21.15.40:0/4229826723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a8005040 con 0x7f28b81000e0 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28b5d9b700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b81000e0 0x7f28b80fba10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28ac002a50 tx=0x7f28ac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:33.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28a6ffd700 1 -- 172.21.15.40:0/4229826723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ac007cc0 con 0x7f28b81000e0 2024-02-16T19:30:33.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b80fc460 con 0x7f28b81000e0 2024-02-16T19:30:33.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28a6ffd700 1 -- 172.21.15.40:0/4229826723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28ac00f040 con 0x7f28b81000e0 2024-02-16T19:30:33.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.192+0000 7f28a6ffd700 1 -- 172.21.15.40:0/4229826723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ac00d9c0 con 0x7f28b81000e0 2024-02-16T19:30:33.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.193+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8115980 con 0x7f28b81000e0 2024-02-16T19:30:33.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.193+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b8107a70 con 0x7f28b81000e0 2024-02-16T19:30:33.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.194+0000 7f28a6ffd700 1 -- 172.21.15.40:0/4229826723 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f28ac009730 con 0x7f28b81000e0 2024-02-16T19:30:33.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.194+0000 7f28a6ffd700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f28a006d7f0 0x7f28a006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.194+0000 7f28a6ffd700 1 -- 172.21.15.40:0/4229826723 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f28ac08cc40 con 0x7f28b81000e0 2024-02-16T19:30:33.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.195+0000 7f28b659c700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f28a006d7f0 0x7f28a006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.195+0000 7f28b659c700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f28a006d7f0 0x7f28a006fca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28a800b3b0 tx=0x7f28a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:33.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.199+0000 7f28a6ffd700 1 -- 172.21.15.40:0/4229826723 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28ac059fc0 con 0x7f28b81000e0 2024-02-16T19:30:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.422+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28b802b410 con 0x7f28a006d7f0 2024-02-16T19:30:33.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.423+0000 7f28a6ffd700 1 -- 172.21.15.40:0/4229826723 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f28b802b410 con 0x7f28a006d7f0 2024-02-16T19:30:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f28a006d7f0 msgr2=0x7f28a006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f28a006d7f0 0x7f28a006fca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28a800b3b0 tx=0x7f28a8009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b81000e0 msgr2=0x7f28b80fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b81000e0 0x7f28b80fba10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28ac002a50 tx=0x7f28ac004600 comp rx=0 tx=0).stop 2024-02-16T19:30:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 shutdown_connections 2024-02-16T19:30:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b80663c0 0x7f28b80fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f28a006d7f0 0x7f28a006fca0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 --2- 172.21.15.40:0/4229826723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b81000e0 0x7f28b80fba10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 >> 172.21.15.40:0/4229826723 conn(0x7f28b80f74e0 msgr2=0x7f28b80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 shutdown_connections 2024-02-16T19:30:33.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.425+0000 7f28bc971700 1 -- 172.21.15.40:0/4229826723 wait complete. 2024-02-16T19:30:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:33 smithi163 ceph-mon[81637]: pgmap v128: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:33 smithi163 ceph-mon[81637]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:33.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.528+0000 7f5f44d9d700 1 -- 172.21.15.40:0/1390567262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f400fd8d0 msgr2=0x7f5f400ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.528+0000 7f5f44d9d700 1 --2- 172.21.15.40:0/1390567262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f400fd8d0 0x7f5f400ffcb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f30004940 tx=0x7f5f30009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.528+0000 7f5f44d9d700 1 -- 172.21.15.40:0/1390567262 shutdown_connections 2024-02-16T19:30:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.528+0000 7f5f44d9d700 1 --2- 172.21.15.40:0/1390567262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f401001f0 0x7f5f401025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.528+0000 7f5f44d9d700 1 --2- 172.21.15.40:0/1390567262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f400fd8d0 0x7f5f400ffcb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.528+0000 7f5f44d9d700 1 -- 172.21.15.40:0/1390567262 >> 172.21.15.40:0/1390567262 conn(0x7f5f400f7370 msgr2=0x7f5f400f9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.528+0000 7f5f44d9d700 1 -- 172.21.15.40:0/1390567262 shutdown_connections 2024-02-16T19:30:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.528+0000 7f5f44d9d700 1 -- 172.21.15.40:0/1390567262 wait complete. 2024-02-16T19:30:33.530 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:33 smithi040 ceph-mon[94210]: pgmap v128: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:33.530 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:33.531 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:33.531 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:33.531 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:33 smithi040 ceph-mon[94210]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f44d9d700 1 Processor -- start 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f44d9d700 1 -- start start 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f44d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f400fd8d0 0x7f5f400fd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f44d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f401001f0 0x7f5f400fb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f44d9d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400fbe70 con 0x7f5f401001f0 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f44d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400fbfe0 con 0x7f5f400fd8d0 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f3effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f401001f0 0x7f5f400fb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f3effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f401001f0 0x7f5f400fb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52666/0 (socket says 172.21.15.40:52666) 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f3effd700 1 -- 172.21.15.40:0/3229264052 learned_addr learned my addr 172.21.15.40:0/3229264052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.529+0000 7f5f3f7fe700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f400fd8d0 0x7f5f400fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f3effd700 1 -- 172.21.15.40:0/3229264052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f400fd8d0 msgr2=0x7f5f400fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f3effd700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f400fd8d0 0x7f5f400fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f3effd700 1 -- 172.21.15.40:0/3229264052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30005040 con 0x7f5f401001f0 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f3effd700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f401001f0 0x7f5f400fb7e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400b790 tx=0x7f5f34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f27fff700 1 -- 172.21.15.40:0/3229264052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3400f770 con 0x7f5f401001f0 2024-02-16T19:30:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f27fff700 1 -- 172.21.15.40:0/3229264052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f3400fda0 con 0x7f5f401001f0 2024-02-16T19:30:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f400fc2c0 con 0x7f5f401001f0 2024-02-16T19:30:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f27fff700 1 -- 172.21.15.40:0/3229264052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3400d310 con 0x7f5f401001f0 2024-02-16T19:30:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.530+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f40115940 con 0x7f5f401001f0 2024-02-16T19:30:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.531+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f40107920 con 0x7f5f401001f0 2024-02-16T19:30:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.532+0000 7f5f27fff700 1 -- 172.21.15.40:0/3229264052 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f34010420 con 0x7f5f401001f0 2024-02-16T19:30:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.532+0000 7f5f27fff700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f2806d7e0 0x7f5f2806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.532+0000 7f5f27fff700 1 -- 172.21.15.40:0/3229264052 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5f3408cf20 con 0x7f5f401001f0 2024-02-16T19:30:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.532+0000 7f5f3f7fe700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f2806d7e0 0x7f5f2806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.533+0000 7f5f3f7fe700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f2806d7e0 0x7f5f2806fc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b3b0 tx=0x7f5f30009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.537+0000 7f5f27fff700 1 -- 172.21.15.40:0/3229264052 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f340c0d20 con 0x7f5f401001f0 2024-02-16T19:30:33.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.748+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f4002b310 con 0x7f5f2806d7e0 2024-02-16T19:30:33.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.759+0000 7f5f27fff700 1 -- 172.21.15.40:0/3229264052 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5f4002b310 con 0x7f5f2806d7e0 2024-02-16T19:30:33.760 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:30:33.760 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (5m) 52s ago 14m 25.3M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:30:33.760 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (2m) 52s ago 14m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:30:33.760 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (2m) 19s ago 13m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:30:33.760 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (4m) 52s ago 13m 81.7M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:30:33.760 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (8m) 52s ago 8m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (8m) 52s ago 8m 13.8M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (8m) 19s ago 8m 17.5M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (8m) 19s ago 8m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (4m) 52s ago 15m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (3m) 19s ago 13m 450M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (3m) 52s ago 15m 47.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (3m) 19s ago 13m 34.8M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (5m) 52s ago 14m 15.7M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (4m) 19s ago 13m 15.3M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (2m) 52s ago 12m 96.1M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:30:33.761 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (94s) 52s ago 11m 99.2M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:30:33.762 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (58s) 52s ago 11m 12.4M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:30:33.762 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (22s) 19s ago 10m 12.4M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:30:33.762 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (10m) 19s ago 10m 236M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 7e2bfb0602a6 2024-02-16T19:30:33.762 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (10m) 19s ago 10m 237M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:30:33.762 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (4m) 52s ago 13m 53.8M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f2806d7e0 msgr2=0x7f5f2806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f2806d7e0 0x7f5f2806fc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b3b0 tx=0x7f5f30009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f401001f0 msgr2=0x7f5f400fb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f401001f0 0x7f5f400fb7e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400b790 tx=0x7f5f34009250 comp rx=0 tx=0).stop 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 shutdown_connections 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f400fd8d0 0x7f5f400fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f2806d7e0 0x7f5f2806fc90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 --2- 172.21.15.40:0/3229264052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f401001f0 0x7f5f400fb7e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 >> 172.21.15.40:0/3229264052 conn(0x7f5f400f7370 msgr2=0x7f5f400f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 shutdown_connections 2024-02-16T19:30:33.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.761+0000 7f5f44d9d700 1 -- 172.21.15.40:0/3229264052 wait complete. 2024-02-16T19:30:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.862+0000 7fe66a62f700 1 -- 172.21.15.40:0/1002991014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6641000c0 msgr2=0x7fe6641024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.862+0000 7fe66a62f700 1 --2- 172.21.15.40:0/1002991014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6641000c0 0x7fe6641024a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe64c004940 tx=0x7fe64c009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.862+0000 7fe66a62f700 1 -- 172.21.15.40:0/1002991014 shutdown_connections 2024-02-16T19:30:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.862+0000 7fe66a62f700 1 --2- 172.21.15.40:0/1002991014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6641000c0 0x7fe6641024a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.862+0000 7fe66a62f700 1 --2- 172.21.15.40:0/1002991014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe664066c30 0x7fe6640ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.862+0000 7fe66a62f700 1 -- 172.21.15.40:0/1002991014 >> 172.21.15.40:0/1002991014 conn(0x7fe6640f7590 msgr2=0x7fe6640f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.862+0000 7fe66a62f700 1 -- 172.21.15.40:0/1002991014 shutdown_connections 2024-02-16T19:30:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.862+0000 7fe66a62f700 1 -- 172.21.15.40:0/1002991014 wait complete. 2024-02-16T19:30:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.863+0000 7fe66a62f700 1 Processor -- start 2024-02-16T19:30:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.863+0000 7fe66a62f700 1 -- start start 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.863+0000 7fe66a62f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe664066c30 0x7fe6640fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.863+0000 7fe66a62f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6641000c0 0x7fe6640fba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.863+0000 7fe66a62f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640fc080 con 0x7fe664066c30 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.863+0000 7fe66a62f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640fc1f0 con 0x7fe6641000c0 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe663fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe664066c30 0x7fe6640fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe663fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe664066c30 0x7fe6640fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52680/0 (socket says 172.21.15.40:52680) 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe663fff700 1 -- 172.21.15.40:0/1104164099 learned_addr learned my addr 172.21.15.40:0/1104164099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe6637fe700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6641000c0 0x7fe6640fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe663fff700 1 -- 172.21.15.40:0/1104164099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6641000c0 msgr2=0x7fe6640fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe663fff700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6641000c0 0x7fe6640fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe663fff700 1 -- 172.21.15.40:0/1104164099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c005040 con 0x7fe664066c30 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe6637fe700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6641000c0 0x7fe6640fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe663fff700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe664066c30 0x7fe6640fd600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe65400c8b0 tx=0x7fe654004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe660ff9700 1 -- 172.21.15.40:0/1104164099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65400a760 con 0x7fe664066c30 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6640fc4d0 con 0x7fe664066c30 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe660ff9700 1 -- 172.21.15.40:0/1104164099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe65400ad90 con 0x7fe664066c30 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe660ff9700 1 -- 172.21.15.40:0/1104164099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6540113a0 con 0x7fe664066c30 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.864+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe664115930 con 0x7fe664066c30 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.866+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66404bed0 con 0x7fe664066c30 2024-02-16T19:30:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.866+0000 7fe660ff9700 1 -- 172.21.15.40:0/1104164099 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe65400a8c0 con 0x7fe664066c30 2024-02-16T19:30:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.866+0000 7fe660ff9700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d7e0 0x7fe65006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.866+0000 7fe660ff9700 1 -- 172.21.15.40:0/1104164099 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe65408cee0 con 0x7fe664066c30 2024-02-16T19:30:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.866+0000 7fe6637fe700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d7e0 0x7fe65006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.867+0000 7fe6637fe700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d7e0 0x7fe65006fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe64c000c00 tx=0x7fe64c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:33.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:33.871+0000 7fe660ff9700 1 -- 172.21.15.40:0/1104164099 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe65405a2e0 con 0x7fe664066c30 2024-02-16T19:30:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.158+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe664115da0 con 0x7fe664066c30 2024-02-16T19:30:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.159+0000 7fe660ff9700 1 -- 172.21.15.40:0/1104164099 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fe654059e70 con 0x7fe664066c30 2024-02-16T19:30:34.160 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:30:34.160 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:30:34.160 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 2, 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:30:34.161 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:30:34.162 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 6, 2024-02-16T19:30:34.162 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 8 2024-02-16T19:30:34.162 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:30:34.162 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d7e0 msgr2=0x7fe65006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d7e0 0x7fe65006fc90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe64c000c00 tx=0x7fe64c009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe664066c30 msgr2=0x7fe6640fd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe664066c30 0x7fe6640fd600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe65400c8b0 tx=0x7fe654004470 comp rx=0 tx=0).stop 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 shutdown_connections 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe65006d7e0 0x7fe65006fc90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe664066c30 0x7fe6640fd600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 --2- 172.21.15.40:0/1104164099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6641000c0 0x7fe6640fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 >> 172.21.15.40:0/1104164099 conn(0x7fe6640f7590 msgr2=0x7fe6640f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 shutdown_connections 2024-02-16T19:30:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.161+0000 7fe66a62f700 1 -- 172.21.15.40:0/1104164099 wait complete. 2024-02-16T19:30:34.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 -- 172.21.15.40:0/1529154947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fc870 msgr2=0x7fe9340fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 --2- 172.21.15.40:0/1529154947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fc870 0x7fe9340fcce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe928004940 tx=0x7fe928009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 -- 172.21.15.40:0/1529154947 shutdown_connections 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 --2- 172.21.15.40:0/1529154947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fc870 0x7fe9340fcce0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 --2- 172.21.15.40:0/1529154947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340fb5c0 0x7fe9340fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 -- 172.21.15.40:0/1529154947 >> 172.21.15.40:0/1529154947 conn(0x7fe9340f6d20 msgr2=0x7fe9340f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 -- 172.21.15.40:0/1529154947 shutdown_connections 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 -- 172.21.15.40:0/1529154947 wait complete. 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 Processor -- start 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 -- start start 2024-02-16T19:30:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fb5c0 0x7fe934100840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340fc870 0x7fe9340fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340ff2c0 con 0x7fe9340fb5c0 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.265+0000 7fe93b336700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340ff430 con 0x7fe9340fc870 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe9390d2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fb5c0 0x7fe934100840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe9390d2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fb5c0 0x7fe934100840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52688/0 (socket says 172.21.15.40:52688) 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe9390d2700 1 -- 172.21.15.40:0/2354945816 learned_addr learned my addr 172.21.15.40:0/2354945816 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe9388d1700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340fc870 0x7fe9340fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe9390d2700 1 -- 172.21.15.40:0/2354945816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340fc870 msgr2=0x7fe9340fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe9390d2700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340fc870 0x7fe9340fecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe9390d2700 1 -- 172.21.15.40:0/2354945816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe928005040 con 0x7fe9340fb5c0 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe9390d2700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fb5c0 0x7fe934100840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe92c002a50 tx=0x7fe92c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe931ffb700 1 -- 172.21.15.40:0/2354945816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe92c007cc0 con 0x7fe9340fb5c0 2024-02-16T19:30:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9340ff710 con 0x7fe9340fb5c0 2024-02-16T19:30:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe931ffb700 1 -- 172.21.15.40:0/2354945816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe92c00f040 con 0x7fe9340fb5c0 2024-02-16T19:30:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.266+0000 7fe931ffb700 1 -- 172.21.15.40:0/2354945816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe92c00da70 con 0x7fe9340fb5c0 2024-02-16T19:30:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.267+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe934115ce0 con 0x7fe9340fb5c0 2024-02-16T19:30:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.268+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93404bed0 con 0x7fe9340fb5c0 2024-02-16T19:30:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.268+0000 7fe931ffb700 1 -- 172.21.15.40:0/2354945816 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe92c00dc10 con 0x7fe9340fb5c0 2024-02-16T19:30:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.268+0000 7fe931ffb700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe91c06d7e0 0x7fe91c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.268+0000 7fe931ffb700 1 -- 172.21.15.40:0/2354945816 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe92c08c540 con 0x7fe9340fb5c0 2024-02-16T19:30:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.269+0000 7fe9388d1700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe91c06d7e0 0x7fe91c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.269+0000 7fe9388d1700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe91c06d7e0 0x7fe91c06fc90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b3b0 tx=0x7fe928009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:34.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.273+0000 7fe931ffb700 1 -- 172.21.15.40:0/2354945816 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe92c0598c0 con 0x7fe9340fb5c0 2024-02-16T19:30:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:34 smithi163 ceph-mon[81637]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:34 smithi163 ceph-mon[81637]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:30:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:34 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:30:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:34 smithi163 ceph-mon[81637]: Upgrade: osd.4 is safe to restart 2024-02-16T19:30:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:34 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1104164099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.523+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe934115ed0 con 0x7fe9340fb5c0 2024-02-16T19:30:34.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:34 smithi040 ceph-mon[94210]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:34.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:34 smithi040 ceph-mon[94210]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:34.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:30:34.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:34 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-16T19:30:34.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:34 smithi040 ceph-mon[94210]: Upgrade: osd.4 is safe to restart 2024-02-16T19:30:34.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:34 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1104164099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.523+0000 7fe931ffb700 1 -- 172.21.15.40:0/2354945816 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fe92c059450 con 0x7fe9340fb5c0 2024-02-16T19:30:34.525 INFO:teuthology.orchestra.run.smithi040.stdout:e20 2024-02-16T19:30:34.525 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:30:34.525 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 20 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:27:43.925056+0000 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:30:34.526 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:30:34.527 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:30:34.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:30:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe91c06d7e0 msgr2=0x7fe91c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe91c06d7e0 0x7fe91c06fc90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b3b0 tx=0x7fe928009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fb5c0 msgr2=0x7fe934100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fb5c0 0x7fe934100840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe92c002a50 tx=0x7fe92c0046b0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 shutdown_connections 2024-02-16T19:30:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe91c06d7e0 0x7fe91c06fc90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340fb5c0 0x7fe934100840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 --2- 172.21.15.40:0/2354945816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340fc870 0x7fe9340fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 >> 172.21.15.40:0/2354945816 conn(0x7fe9340f6d20 msgr2=0x7fe9340f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 shutdown_connections 2024-02-16T19:30:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.526+0000 7fe93b336700 1 -- 172.21.15.40:0/2354945816 wait complete. 2024-02-16T19:30:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 -- 172.21.15.40:0/1461452528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 msgr2=0x7fbfec0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 --2- 172.21.15.40:0/1461452528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 0x7fbfec0fe550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc004940 tx=0x7fbfdc009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 -- 172.21.15.40:0/1461452528 shutdown_connections 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 --2- 172.21.15.40:0/1461452528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 0x7fbfec0fe550 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 --2- 172.21.15.40:0/1461452528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfec0fcef0 0x7fbfec0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 -- 172.21.15.40:0/1461452528 >> 172.21.15.40:0/1461452528 conn(0x7fbfec0f8620 msgr2=0x7fbfec0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 -- 172.21.15.40:0/1461452528 shutdown_connections 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 -- 172.21.15.40:0/1461452528 wait complete. 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 Processor -- start 2024-02-16T19:30:34.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 -- start start 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfec0fcef0 0x7fbfec10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 0x7fbfec10dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec10e4a0 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.631+0000 7fbff4575700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec10ed40 con 0x7fbfec0fcef0 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff1b10700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 0x7fbfec10dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff1b10700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 0x7fbfec10dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52706/0 (socket says 172.21.15.40:52706) 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff1b10700 1 -- 172.21.15.40:0/4205000592 learned_addr learned my addr 172.21.15.40:0/4205000592 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff2311700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfec0fcef0 0x7fbfec10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff1b10700 1 -- 172.21.15.40:0/4205000592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfec0fcef0 msgr2=0x7fbfec10d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff1b10700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfec0fcef0 0x7fbfec10d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff1b10700 1 -- 172.21.15.40:0/4205000592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc005040 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff1b10700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 0x7fbfec10dea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0093b0 tx=0x7fbfdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbfe2ffd700 1 -- 172.21.15.40:0/4205000592 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfdc00e040 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbfe2ffd700 1 -- 172.21.15.40:0/4205000592 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfdc00bcc0 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec10efc0 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbfe2ffd700 1 -- 172.21.15.40:0/4205000592 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfdc01c630 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.632+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec10f500 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.633+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec04bed0 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.634+0000 7fbfe2ffd700 1 -- 172.21.15.40:0/4205000592 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbfdc013980 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.634+0000 7fbfe2ffd700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbfd806d7e0 0x7fbfd806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.634+0000 7fbfe2ffd700 1 -- 172.21.15.40:0/4205000592 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbfdc08e280 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.634+0000 7fbff2311700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbfd806d7e0 0x7fbfd806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.635+0000 7fbff2311700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbfd806d7e0 0x7fbfd806fc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0fcc30 tx=0x7fbfe8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:34.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.639+0000 7fbfe2ffd700 1 -- 172.21.15.40:0/4205000592 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbfdc05b680 con 0x7fbfec0fe0e0 2024-02-16T19:30:34.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.859+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfec10f8f0 con 0x7fbfd806d7e0 2024-02-16T19:30:34.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.860+0000 7fbfe2ffd700 1 -- 172.21.15.40:0/4205000592 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbfec10f8f0 con 0x7fbfd806d7e0 2024-02-16T19:30:34.861 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:30:34.861 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:30:34.861 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:30:34.861 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "10/21 daemons upgraded", 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:30:34.862 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:30:34.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbfd806d7e0 msgr2=0x7fbfd806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbfd806d7e0 0x7fbfd806fc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0fcc30 tx=0x7fbfe8009040 comp rx=0 tx=0).stop 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 msgr2=0x7fbfec10dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 0x7fbfec10dea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0093b0 tx=0x7fbfdc009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 shutdown_connections 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfec0fcef0 0x7fbfec10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbfd806d7e0 0x7fbfd806fc90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 --2- 172.21.15.40:0/4205000592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0fe0e0 0x7fbfec10dea0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 >> 172.21.15.40:0/4205000592 conn(0x7fbfec0f8620 msgr2=0x7fbfec1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 shutdown_connections 2024-02-16T19:30:34.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.862+0000 7fbff4575700 1 -- 172.21.15.40:0/4205000592 wait complete. 2024-02-16T19:30:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 -- 172.21.15.40:0/1882577808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd981000c0 msgr2=0x7efd98102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 --2- 172.21.15.40:0/1882577808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd981000c0 0x7efd98102520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd80004940 tx=0x7efd80009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 -- 172.21.15.40:0/1882577808 shutdown_connections 2024-02-16T19:30:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 --2- 172.21.15.40:0/1882577808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd981000c0 0x7efd98102520 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 --2- 172.21.15.40:0/1882577808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd98066c70 0x7efd980ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 -- 172.21.15.40:0/1882577808 >> 172.21.15.40:0/1882577808 conn(0x7efd980f7520 msgr2=0x7efd980f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 -- 172.21.15.40:0/1882577808 shutdown_connections 2024-02-16T19:30:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 -- 172.21.15.40:0/1882577808 wait complete. 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 Processor -- start 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.965+0000 7efd9d67b700 1 -- start start 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd9d67b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd98066c70 0x7efd9810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd9d67b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd981000c0 0x7efd9810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd9d67b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9810e470 con 0x7efd98066c70 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd9d67b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9810ed10 con 0x7efd981000c0 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd96ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd98066c70 0x7efd9810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd96ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd98066c70 0x7efd9810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52724/0 (socket says 172.21.15.40:52724) 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd96ffd700 1 -- 172.21.15.40:0/4089331652 learned_addr learned my addr 172.21.15.40:0/4089331652 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd967fc700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd981000c0 0x7efd9810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd96ffd700 1 -- 172.21.15.40:0/4089331652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd981000c0 msgr2=0x7efd9810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd96ffd700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd981000c0 0x7efd9810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd96ffd700 1 -- 172.21.15.40:0/4089331652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80005040 con 0x7efd98066c70 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd96ffd700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd98066c70 0x7efd9810d930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd88001d80 tx=0x7efd88009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd8f7fe700 1 -- 172.21.15.40:0/4089331652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd88007d10 con 0x7efd98066c70 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd8f7fe700 1 -- 172.21.15.40:0/4089331652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd88001e80 con 0x7efd98066c70 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9810eff0 con 0x7efd98066c70 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd8f7fe700 1 -- 172.21.15.40:0/4089331652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd88014640 con 0x7efd98066c70 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.966+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9810f530 con 0x7efd98066c70 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.967+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd98107a50 con 0x7efd98066c70 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.968+0000 7efd8f7fe700 1 -- 172.21.15.40:0/4089331652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efd8800e960 con 0x7efd98066c70 2024-02-16T19:30:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.968+0000 7efd8f7fe700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8406d7e0 0x7efd8406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:30:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.968+0000 7efd8f7fe700 1 -- 172.21.15.40:0/4089331652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efd8808cb90 con 0x7efd98066c70 2024-02-16T19:30:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.968+0000 7efd967fc700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8406d7e0 0x7efd8406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:30:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.969+0000 7efd967fc700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8406d7e0 0x7efd8406fc90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd80004890 tx=0x7efd80009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:30:34.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:34.973+0000 7efd8f7fe700 1 -- 172.21.15.40:0/4089331652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd8805a040 con 0x7efd98066c70 2024-02-16T19:30:35.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.261+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efd9804bed0 con 0x7efd98066c70 2024-02-16T19:30:35.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.261+0000 7efd8f7fe700 1 -- 172.21.15.40:0/4089331652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7efd88019020 con 0x7efd98066c70 2024-02-16T19:30:35.263 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8406d7e0 msgr2=0x7efd8406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8406d7e0 0x7efd8406fc90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd80004890 tx=0x7efd80009e30 comp rx=0 tx=0).stop 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd98066c70 msgr2=0x7efd9810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd98066c70 0x7efd9810d930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd88001d80 tx=0x7efd88009ec0 comp rx=0 tx=0).stop 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 shutdown_connections 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8406d7e0 0x7efd8406fc90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd98066c70 0x7efd9810d930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 --2- 172.21.15.40:0/4089331652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd981000c0 0x7efd9810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.263+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 >> 172.21.15.40:0/4089331652 conn(0x7efd980f7520 msgr2=0x7efd980f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.264+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 shutdown_connections 2024-02-16T19:30:35.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:30:35.264+0000 7efd9d67b700 1 -- 172.21.15.40:0/4089331652 wait complete. 2024-02-16T19:30:35.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:35 smithi163 ceph-mon[81637]: pgmap v129: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:35.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:35 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2354945816' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:30:35.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:35 smithi163 ceph-mon[81637]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:35 smithi040 ceph-mon[94210]: pgmap v129: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:35 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2354945816' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:30:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:35 smithi040 ceph-mon[94210]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:30:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:36 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/4089331652' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:30:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:36 smithi163 ceph-mon[81637]: Upgrade: Updating osd.4 2024-02-16T19:30:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:36 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:36 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:30:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:36 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:36.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:36 smithi163 ceph-mon[81637]: Deploying daemon osd.4 on smithi163 2024-02-16T19:30:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:36 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/4089331652' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:30:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:36 smithi040 ceph-mon[94210]: Upgrade: Updating osd.4 2024-02-16T19:30:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:36 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:36 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-16T19:30:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:36 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:36.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:36 smithi040 ceph-mon[94210]: Deploying daemon osd.4 on smithi163 2024-02-16T19:30:37.883 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:37 smithi163 ceph-mon[81637]: pgmap v130: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:38.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:37 smithi040 ceph-mon[94210]: pgmap v130: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:39.168 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:38 smithi163 systemd[1]: Stopping Ceph osd.4 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:30:39.487 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:39 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[48494]: 2024-02-16T19:30:39.176+0000 7fd75ecae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:30:39.487 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:39 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[48494]: 2024-02-16T19:30:39.176+0000 7fd75ecae700 -1 osd.4 69 *** Got signal Terminated *** 2024-02-16T19:30:39.487 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:39 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[48494]: 2024-02-16T19:30:39.176+0000 7fd75ecae700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:30:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:39 smithi163 ceph-mon[81637]: pgmap v131: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:39.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:39 smithi163 ceph-mon[81637]: osd.4 marked itself down and dead 2024-02-16T19:30:40.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:39 smithi040 ceph-mon[94210]: pgmap v131: 65 pgs: 65 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:40.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:39 smithi040 ceph-mon[94210]: osd.4 marked itself down and dead 2024-02-16T19:30:40.436 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:40 smithi163 podman[95482]: 2024-02-16 19:30:40.214899195 +0000 UTC m=+1.157276553 container died 7e2bfb0602a6764163c9739e3bc53caf5f3af9ecd4281d50def4d643e2d50540 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=quincy-b88fae6, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-16T19:30:40.701 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:40 smithi163 podman[95482]: 2024-02-16 19:30:40.534995893 +0000 UTC m=+1.477373247 container cleanup 7e2bfb0602a6764163c9739e3bc53caf5f3af9ecd4281d50def4d643e2d50540 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-b88fae6) 2024-02-16T19:30:40.702 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:40 smithi163 bash[95482]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4 2024-02-16T19:30:40.987 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:40 smithi163 podman[95511]: 2024-02-16 19:30:40.727773871 +0000 UTC m=+0.511697349 container remove 7e2bfb0602a6764163c9739e3bc53caf5f3af9ecd4281d50def4d643e2d50540 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=quincy-b88fae6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:30:40.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:40 smithi163 ceph-mon[81637]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:30:40.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:40 smithi163 ceph-mon[81637]: failed to encode map e70 with expected crc 2024-02-16T19:30:40.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:40 smithi163 ceph-mon[81637]: failed to encode map e70 with expected crc 2024-02-16T19:30:40.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:40 smithi163 ceph-mon[81637]: osdmap e70: 6 total, 5 up, 6 in 2024-02-16T19:30:40.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:40 smithi163 ceph-mon[81637]: osdmap e71: 6 total, 5 up, 6 in 2024-02-16T19:30:41.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:40 smithi040 ceph-mon[94210]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:30:41.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:40 smithi040 ceph-mon[94210]: failed to encode map e70 with expected crc 2024-02-16T19:30:41.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:40 smithi040 ceph-mon[94210]: failed to encode map e70 with expected crc 2024-02-16T19:30:41.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:40 smithi040 ceph-mon[94210]: osdmap e70: 6 total, 5 up, 6 in 2024-02-16T19:30:41.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:40 smithi040 ceph-mon[94210]: osdmap e71: 6 total, 5 up, 6 in 2024-02-16T19:30:41.406 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:41 smithi163 podman[95603]: 2024-02-16 19:30:41.052454598 +0000 UTC m=+0.018722675 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:30:41.406 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:41 smithi163 podman[95603]: 2024-02-16 19:30:41.153688434 +0000 UTC m=+0.119956511 container create ba234e4fdef1a52d80baad6ea8e149b0f7e18aa4db58213e79aabdf59dce5e8f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-16T19:30:41.406 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:41 smithi163 podman[95603]: 2024-02-16 19:30:41.406173776 +0000 UTC m=+0.372441852 container init ba234e4fdef1a52d80baad6ea8e149b0f7e18aa4db58213e79aabdf59dce5e8f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-16T19:30:41.703 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:41 smithi163 podman[95603]: 2024-02-16 19:30:41.40936049 +0000 UTC m=+0.375628575 container start ba234e4fdef1a52d80baad6ea8e149b0f7e18aa4db58213e79aabdf59dce5e8f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-16T19:30:41.703 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:41 smithi163 podman[95603]: 2024-02-16 19:30:41.50631072 +0000 UTC m=+0.472578802 container attach ba234e4fdef1a52d80baad6ea8e149b0f7e18aa4db58213e79aabdf59dce5e8f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T19:30:41.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:41 smithi163 ceph-mon[81637]: pgmap v133: 65 pgs: 6 stale+active+clean, 59 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:41.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:41 smithi163 ceph-mon[81637]: failed to encode map e71 with expected crc 2024-02-16T19:30:41.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:41 smithi163 ceph-mon[81637]: failed to encode map e71 with expected crc 2024-02-16T19:30:41.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:41 smithi163 ceph-mon[81637]: failed to encode map e71 with expected crc 2024-02-16T19:30:41.988 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:41 smithi163 podman[95603]: 2024-02-16 19:30:41.707926251 +0000 UTC m=+0.674194327 container died ba234e4fdef1a52d80baad6ea8e149b0f7e18aa4db58213e79aabdf59dce5e8f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:30:42.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:41 smithi040 ceph-mon[94210]: pgmap v133: 65 pgs: 6 stale+active+clean, 59 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:42.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:41 smithi040 ceph-mon[94210]: failed to encode map e71 with expected crc 2024-02-16T19:30:42.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:41 smithi040 ceph-mon[94210]: failed to encode map e71 with expected crc 2024-02-16T19:30:42.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:41 smithi040 ceph-mon[94210]: failed to encode map e71 with expected crc 2024-02-16T19:30:42.734 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:42 smithi163 podman[95629]: 2024-02-16 19:30:42.301309733 +0000 UTC m=+0.583196329 container remove ba234e4fdef1a52d80baad6ea8e149b0f7e18aa4db58213e79aabdf59dce5e8f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:30:42.734 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:42 smithi163 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.4.service: Deactivated successfully. 2024-02-16T19:30:42.734 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:42 smithi163 systemd[1]: Stopped Ceph osd.4 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:30:42.734 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:42 smithi163 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.4.service: Consumed 58.236s CPU time. 2024-02-16T19:30:42.734 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:42 smithi163 systemd[1]: Starting Ceph osd.4 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:30:42.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:42 smithi163 ceph-mon[81637]: Health check failed: Degraded data redundancy: 12/360 objects degraded (3.333%), 4 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:42.987 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:42 smithi163 podman[95731]: 2024-02-16 19:30:42.636088074 +0000 UTC m=+0.018280886 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:30:42.988 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:42 smithi163 podman[95731]: 2024-02-16 19:30:42.769228042 +0000 UTC m=+0.151420896 container create 76f5248ef213f244034eee88302a8db42c59ebd8a26a05cfc7caea90f7f8f816 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-16T19:30:43.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:42 smithi040 ceph-mon[94210]: Health check failed: Degraded data redundancy: 12/360 objects degraded (3.333%), 4 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:43.483 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 podman[95731]: 2024-02-16 19:30:43.033952124 +0000 UTC m=+0.416144936 container init 76f5248ef213f244034eee88302a8db42c59ebd8a26a05cfc7caea90f7f8f816 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-16T19:30:43.484 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 podman[95731]: 2024-02-16 19:30:43.037248031 +0000 UTC m=+0.419440849 container start 76f5248ef213f244034eee88302a8db42c59ebd8a26a05cfc7caea90f7f8f816 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-02-16T19:30:43.484 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 podman[95731]: 2024-02-16 19:30:43.121923743 +0000 UTC m=+0.504116563 container attach 76f5248ef213f244034eee88302a8db42c59ebd8a26a05cfc7caea90f7f8f816 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-16T19:30:43.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate[95745]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate[95745]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 bash[95731]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 bash[95731]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 bash[95731]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 bash[95731]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 bash[95731]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 bash[95731]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 bash[95731]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate[95745]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate[95745]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate[95745]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate[95745]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate[95745]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-16T19:30:43.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:43 smithi163 podman[95731]: 2024-02-16 19:30:43.509682868 +0000 UTC m=+0.891875684 container died 76f5248ef213f244034eee88302a8db42c59ebd8a26a05cfc7caea90f7f8f816 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-16T19:30:44.100 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:43 smithi163 ceph-mon[81637]: pgmap v135: 65 pgs: 4 active+undersized+degraded, 4 stale+active+clean, 6 active+undersized, 51 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 12/360 objects degraded (3.333%) 2024-02-16T19:30:44.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:43 smithi040 ceph-mon[94210]: pgmap v135: 65 pgs: 4 active+undersized+degraded, 4 stale+active+clean, 6 active+undersized, 51 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 12/360 objects degraded (3.333%) 2024-02-16T19:30:44.433 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:44 smithi163 podman[95879]: 2024-02-16 19:30:44.100215671 +0000 UTC m=+0.585297591 container remove 76f5248ef213f244034eee88302a8db42c59ebd8a26a05cfc7caea90f7f8f816 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-16T19:30:44.433 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:44 smithi163 podman[95938]: 2024-02-16 19:30:44.214427576 +0000 UTC m=+0.017994202 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:30:44.433 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:44 smithi163 podman[95938]: 2024-02-16 19:30:44.334398872 +0000 UTC m=+0.137965541 container create 0c47887cb850ea42f823fce44902fb0f2fbeaddd26137305ca13ad7013d23520 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-16T19:30:44.987 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:44 smithi163 podman[95938]: 2024-02-16 19:30:44.570117288 +0000 UTC m=+0.373683915 container init 0c47887cb850ea42f823fce44902fb0f2fbeaddd26137305ca13ad7013d23520 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-16T19:30:44.987 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:44 smithi163 podman[95938]: 2024-02-16 19:30:44.573173344 +0000 UTC m=+0.376739971 container start 0c47887cb850ea42f823fce44902fb0f2fbeaddd26137305ca13ad7013d23520 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-16T19:30:44.988 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:44 smithi163 ceph-osd[95958]: -- 172.21.15.163:0/710550045 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f9313691e0 con 0x55f93133ac00 2024-02-16T19:30:44.988 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:44 smithi163 bash[95938]: 0c47887cb850ea42f823fce44902fb0f2fbeaddd26137305ca13ad7013d23520 2024-02-16T19:30:44.988 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:44 smithi163 systemd[1]: Started Ceph osd.4 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:30:45.487 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:45 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:30:45.121+0000 7f9badc4a7c0 -1 Falling back to public interface 2024-02-16T19:30:45.851 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:45 smithi163 ceph-mon[81637]: pgmap v136: 65 pgs: 12 active+undersized+degraded, 2 stale+active+clean, 14 active+undersized, 37 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 47/360 objects degraded (13.056%) 2024-02-16T19:30:45.851 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:45 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:45.851 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:45 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:45.851 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:45 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:46.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:45 smithi040 ceph-mon[94210]: pgmap v136: 65 pgs: 12 active+undersized+degraded, 2 stale+active+clean, 14 active+undersized, 37 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 47/360 objects degraded (13.056%) 2024-02-16T19:30:46.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:45 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:46.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:45 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:46.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:45 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:47.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:47 smithi163 ceph-mon[81637]: pgmap v137: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-02-16T19:30:47.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:47.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:48.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:47 smithi040 ceph-mon[94210]: pgmap v137: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-02-16T19:30:48.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:48.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:30:49.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:49.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:49.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:48 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 61/360 objects degraded (16.944%), 16 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:49.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:49.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:49.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:48 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 61/360 objects degraded (16.944%), 16 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:49.892 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:49 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:30:49.700+0000 7f9badc4a7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-02-16T19:30:50.174 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:49 smithi163 ceph-mon[81637]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-02-16T19:30:50.174 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:50.174 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:49 smithi040 ceph-mon[94210]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-02-16T19:30:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:50.487 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:50 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:30:50.267+0000 7f9badc4a7c0 -1 osd.4 69 log_to_monitors true 2024-02-16T19:30:51.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:51 smithi163 ceph-mon[81637]: from='osd.4 [v2:172.21.15.163:6808/2382516830,v1:172.21.15.163:6809/2382516830]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:30:51.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:51 smithi163 ceph-mon[81637]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:30:51.232 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:30:51 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:30:51.015+0000 7f9b9f0a2700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:30:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:50 smithi040 ceph-mon[94210]: from='osd.4 [v2:172.21.15.163:6808/2382516830,v1:172.21.15.163:6809/2382516830]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:30:51.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:50 smithi040 ceph-mon[94210]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-16T19:30:52.169 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:52 smithi163 ceph-mon[81637]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-02-16T19:30:52.170 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:52 smithi163 ceph-mon[81637]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T19:30:52.170 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:52 smithi163 ceph-mon[81637]: failed to encode map e72 with expected crc 2024-02-16T19:30:52.170 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:52 smithi163 ceph-mon[81637]: osdmap e72: 6 total, 5 up, 6 in 2024-02-16T19:30:52.170 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:52 smithi163 ceph-mon[81637]: from='osd.4 [v2:172.21.15.163:6808/2382516830,v1:172.21.15.163:6809/2382516830]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:30:52.170 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:52 smithi163 ceph-mon[81637]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:30:52.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:52 smithi040 ceph-mon[94210]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-02-16T19:30:52.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:52 smithi040 ceph-mon[94210]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-16T19:30:52.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:52 smithi040 ceph-mon[94210]: failed to encode map e72 with expected crc 2024-02-16T19:30:52.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:52 smithi040 ceph-mon[94210]: osdmap e72: 6 total, 5 up, 6 in 2024-02-16T19:30:52.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:52 smithi040 ceph-mon[94210]: from='osd.4 [v2:172.21.15.163:6808/2382516830,v1:172.21.15.163:6809/2382516830]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:30:52.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:52 smithi040 ceph-mon[94210]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:30:53.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:30:53.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: failed to encode map e73 with expected crc 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: osd.4 [v2:172.21.15.163:6808/2382516830,v1:172.21.15.163:6809/2382516830] boot 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: osdmap e73: 6 total, 6 up, 6 in 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:53.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:30:53.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:30:53.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: failed to encode map e73 with expected crc 2024-02-16T19:30:53.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: osd.4 [v2:172.21.15.163:6808/2382516830,v1:172.21.15.163:6809/2382516830] boot 2024-02-16T19:30:53.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: osdmap e73: 6 total, 6 up, 6 in 2024-02-16T19:30:53.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:30:53.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:30:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:54 smithi040 ceph-mon[94210]: pgmap v142: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-02-16T19:30:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:54 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:30:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:54 smithi040 ceph-mon[94210]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T19:30:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:54 smithi040 ceph-mon[94210]: failed to encode map e74 with expected crc 2024-02-16T19:30:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:54 smithi040 ceph-mon[94210]: osdmap e74: 6 total, 6 up, 6 in 2024-02-16T19:30:54.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:54 smithi040 ceph-mon[94210]: failed to encode map e74 with expected crc 2024-02-16T19:30:54.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:54 smithi040 ceph-mon[94210]: failed to encode map e74 with expected crc 2024-02-16T19:30:54.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:54 smithi040 ceph-mon[94210]: failed to encode map e74 with expected crc 2024-02-16T19:30:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:54 smithi163 ceph-mon[81637]: pgmap v142: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-02-16T19:30:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:54 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:30:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:54 smithi163 ceph-mon[81637]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-16T19:30:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:54 smithi163 ceph-mon[81637]: failed to encode map e74 with expected crc 2024-02-16T19:30:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:54 smithi163 ceph-mon[81637]: osdmap e74: 6 total, 6 up, 6 in 2024-02-16T19:30:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:54 smithi163 ceph-mon[81637]: failed to encode map e74 with expected crc 2024-02-16T19:30:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:54 smithi163 ceph-mon[81637]: failed to encode map e74 with expected crc 2024-02-16T19:30:54.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:54 smithi163 ceph-mon[81637]: failed to encode map e74 with expected crc 2024-02-16T19:30:55.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:55 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 31/360 objects degraded (8.611%), 10 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:55 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 31/360 objects degraded (8.611%), 10 pgs degraded (PG_DEGRADED) 2024-02-16T19:30:56.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:56 smithi040 ceph-mon[94210]: pgmap v144: 65 pgs: 10 peering, 10 active+undersized+degraded, 10 active+undersized, 35 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31/360 objects degraded (8.611%) 2024-02-16T19:30:56.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:56 smithi163 ceph-mon[81637]: pgmap v144: 65 pgs: 10 peering, 10 active+undersized+degraded, 10 active+undersized, 35 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31/360 objects degraded (8.611%) 2024-02-16T19:30:57.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:57 smithi040 ceph-mon[94210]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/360 objects degraded (8.611%), 10 pgs degraded) 2024-02-16T19:30:57.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:57 smithi040 ceph-mon[94210]: Cluster is now healthy 2024-02-16T19:30:57.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:57 smithi163 ceph-mon[81637]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/360 objects degraded (8.611%), 10 pgs degraded) 2024-02-16T19:30:57.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:57 smithi163 ceph-mon[81637]: Cluster is now healthy 2024-02-16T19:30:58.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:30:58 smithi040 ceph-mon[94210]: pgmap v145: 65 pgs: 10 peering, 55 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:30:58.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:30:58 smithi163 ceph-mon[81637]: pgmap v145: 65 pgs: 10 peering, 55 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:00.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:00 smithi040 ceph-mon[94210]: pgmap v146: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:00.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:00 smithi163 ceph-mon[81637]: pgmap v146: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:02.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:02 smithi040 ceph-mon[94210]: pgmap v147: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:02 smithi163 ceph-mon[81637]: pgmap v147: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:03 smithi163 ceph-mon[81637]: pgmap v148: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:03 smithi040 ceph-mon[94210]: pgmap v148: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.382+0000 7ff52598e700 1 -- 172.21.15.40:0/1190278879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520071eb0 msgr2=0x7ff520070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.382+0000 7ff52598e700 1 --2- 172.21.15.40:0/1190278879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520071eb0 0x7ff520070320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff510009ce0 tx=0x7ff510004d10 comp rx=0 tx=0).stop 2024-02-16T19:31:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.382+0000 7ff52598e700 1 -- 172.21.15.40:0/1190278879 shutdown_connections 2024-02-16T19:31:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.382+0000 7ff52598e700 1 --2- 172.21.15.40:0/1190278879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520070860 0x7ff520070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.382+0000 7ff52598e700 1 --2- 172.21.15.40:0/1190278879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520071eb0 0x7ff520070320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.382+0000 7ff52598e700 1 -- 172.21.15.40:0/1190278879 >> 172.21.15.40:0/1190278879 conn(0x7ff5200f8620 msgr2=0x7ff5200faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.382+0000 7ff52598e700 1 -- 172.21.15.40:0/1190278879 shutdown_connections 2024-02-16T19:31:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.382+0000 7ff52598e700 1 -- 172.21.15.40:0/1190278879 wait complete. 2024-02-16T19:31:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff52598e700 1 Processor -- start 2024-02-16T19:31:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff52598e700 1 -- start start 2024-02-16T19:31:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff52598e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520070860 0x7ff52010fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff52598e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520071eb0 0x7ff520110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff52598e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520111040 con 0x7ff520070860 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff52598e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201111b0 con 0x7ff520071eb0 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff51effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520070860 0x7ff52010fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff51effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520070860 0x7ff52010fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36612/0 (socket says 172.21.15.40:36612) 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff51effd700 1 -- 172.21.15.40:0/4119475403 learned_addr learned my addr 172.21.15.40:0/4119475403 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff51e7fc700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520071eb0 0x7ff520110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff51effd700 1 -- 172.21.15.40:0/4119475403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520071eb0 msgr2=0x7ff520110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff51effd700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520071eb0 0x7ff520110280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.383+0000 7ff51effd700 1 -- 172.21.15.40:0/4119475403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff510009940 con 0x7ff520070860 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.384+0000 7ff51effd700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520070860 0x7ff52010fd40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff510015040 tx=0x7ff510004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.384+0000 7ff5177fe700 1 -- 172.21.15.40:0/4119475403 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51000e040 con 0x7ff520070860 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.384+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5201113d0 con 0x7ff520070860 2024-02-16T19:31:05.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.384+0000 7ff5177fe700 1 -- 172.21.15.40:0/4119475403 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5100128e0 con 0x7ff520070860 2024-02-16T19:31:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.384+0000 7ff5177fe700 1 -- 172.21.15.40:0/4119475403 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51001c760 con 0x7ff520070860 2024-02-16T19:31:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.384+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520111910 con 0x7ff520070860 2024-02-16T19:31:05.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.385+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5200fc2a0 con 0x7ff520070860 2024-02-16T19:31:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.385+0000 7ff5177fe700 1 -- 172.21.15.40:0/4119475403 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff510012a50 con 0x7ff520070860 2024-02-16T19:31:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.386+0000 7ff5177fe700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff50c06d7e0 0x7ff50c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.386+0000 7ff5177fe700 1 -- 172.21.15.40:0/4119475403 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff51008eb30 con 0x7ff520070860 2024-02-16T19:31:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.386+0000 7ff51e7fc700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff50c06d7e0 0x7ff50c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:05.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.386+0000 7ff51e7fc700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff50c06d7e0 0x7ff50c06fc90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff508004f40 tx=0x7ff50800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:05.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.391+0000 7ff5177fe700 1 -- 172.21.15.40:0/4119475403 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff510058380 con 0x7ff520070860 2024-02-16T19:31:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.614+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff520111d70 con 0x7ff50c06d7e0 2024-02-16T19:31:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.615+0000 7ff5177fe700 1 -- 172.21.15.40:0/4119475403 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff520111d70 con 0x7ff50c06d7e0 2024-02-16T19:31:05.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff50c06d7e0 msgr2=0x7ff50c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:05.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff50c06d7e0 0x7ff50c06fc90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff508004f40 tx=0x7ff50800b040 comp rx=0 tx=0).stop 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520070860 msgr2=0x7ff52010fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520070860 0x7ff52010fd40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff510015040 tx=0x7ff510004d10 comp rx=0 tx=0).stop 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 shutdown_connections 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff50c06d7e0 0x7ff50c06fc90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff520070860 0x7ff52010fd40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 --2- 172.21.15.40:0/4119475403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520071eb0 0x7ff520110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 >> 172.21.15.40:0/4119475403 conn(0x7ff5200f8620 msgr2=0x7ff520102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 shutdown_connections 2024-02-16T19:31:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.617+0000 7ff52598e700 1 -- 172.21.15.40:0/4119475403 wait complete. 2024-02-16T19:31:05.629 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:31:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 -- 172.21.15.40:0/3349499599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fb220 msgr2=0x7fbe1c0fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 --2- 172.21.15.40:0/3349499599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fb220 0x7fbe1c0fb630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009ce0 tx=0x7fbe0c004d10 comp rx=0 tx=0).stop 2024-02-16T19:31:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 -- 172.21.15.40:0/3349499599 shutdown_connections 2024-02-16T19:31:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 --2- 172.21.15.40:0/3349499599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe1c0fe2b0 0x7fbe1c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 --2- 172.21.15.40:0/3349499599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fb220 0x7fbe1c0fb630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 -- 172.21.15.40:0/3349499599 >> 172.21.15.40:0/3349499599 conn(0x7fbe1c0f6ca0 msgr2=0x7fbe1c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 -- 172.21.15.40:0/3349499599 shutdown_connections 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 -- 172.21.15.40:0/3349499599 wait complete. 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 Processor -- start 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.721+0000 7fbe21357700 1 -- start start 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe21357700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe1c0fb220 0x7fbe1c0fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe21357700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fe2b0 0x7fbe1c0fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe21357700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0fc7c0 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe21357700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0fc930 con 0x7fbe1c0fb220 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe1a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fe2b0 0x7fbe1c0fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe1a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fe2b0 0x7fbe1c0fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36632/0 (socket says 172.21.15.40:36632) 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe1a7fc700 1 -- 172.21.15.40:0/918508023 learned_addr learned my addr 172.21.15.40:0/918508023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe1affd700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe1c0fb220 0x7fbe1c0fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe1a7fc700 1 -- 172.21.15.40:0/918508023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe1c0fb220 msgr2=0x7fbe1c0fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:05.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe1a7fc700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe1c0fb220 0x7fbe1c0fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe1a7fc700 1 -- 172.21.15.40:0/918508023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c009940 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe1a7fc700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fe2b0 0x7fbe1c0fc1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c8c0 tx=0x7fbe040044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe137fe700 1 -- 172.21.15.40:0/918508023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0400a760 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c0fcc10 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe137fe700 1 -- 172.21.15.40:0/918508023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe0400ad90 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.722+0000 7fbe137fe700 1 -- 172.21.15.40:0/918508023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe040113a0 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.723+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c115be0 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.723+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c04bed0 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.724+0000 7fbe137fe700 1 -- 172.21.15.40:0/918508023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbe04012400 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.724+0000 7fbe137fe700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbe0806d7e0 0x7fbe0806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.724+0000 7fbe137fe700 1 -- 172.21.15.40:0/918508023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbe0408cf30 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.724+0000 7fbe1affd700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbe0806d7e0 0x7fbe0806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.725+0000 7fbe1affd700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbe0806d7e0 0x7fbe0806fc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c015040 tx=0x7fbe0c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:05.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.729+0000 7fbe137fe700 1 -- 172.21.15.40:0/918508023 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe0405a290 con 0x7fbe1c0fe2b0 2024-02-16T19:31:05.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.952+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe1c102c00 con 0x7fbe0806d7e0 2024-02-16T19:31:05.953 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:05 smithi040 ceph-mon[94210]: pgmap v149: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:05.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.953+0000 7fbe137fe700 1 -- 172.21.15.40:0/918508023 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbe1c102c00 con 0x7fbe0806d7e0 2024-02-16T19:31:05.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbe0806d7e0 msgr2=0x7fbe0806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:05.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbe0806d7e0 0x7fbe0806fc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c015040 tx=0x7fbe0c004d10 comp rx=0 tx=0).stop 2024-02-16T19:31:05.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fe2b0 msgr2=0x7fbe1c0fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:05.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fe2b0 0x7fbe1c0fc1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c8c0 tx=0x7fbe040044c0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 shutdown_connections 2024-02-16T19:31:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe1c0fb220 0x7fbe1c0fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbe0806d7e0 0x7fbe0806fc90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 --2- 172.21.15.40:0/918508023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe1c0fe2b0 0x7fbe1c0fc1c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 >> 172.21.15.40:0/918508023 conn(0x7fbe1c0f6ca0 msgr2=0x7fbe1c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 shutdown_connections 2024-02-16T19:31:05.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:05.955+0000 7fbe21357700 1 -- 172.21.15.40:0/918508023 wait complete. 2024-02-16T19:31:05.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:05 smithi163 ceph-mon[81637]: pgmap v149: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.057+0000 7fa7e21a2700 1 -- 172.21.15.40:0/4054826528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 msgr2=0x7fa7dc102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.057+0000 7fa7e21a2700 1 --2- 172.21.15.40:0/4054826528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 0x7fa7dc102520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0004940 tx=0x7fa7d0009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.057+0000 7fa7e21a2700 1 -- 172.21.15.40:0/4054826528 shutdown_connections 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.057+0000 7fa7e21a2700 1 --2- 172.21.15.40:0/4054826528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 0x7fa7dc102520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.057+0000 7fa7e21a2700 1 --2- 172.21.15.40:0/4054826528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7dc066c30 0x7fa7dc0ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.057+0000 7fa7e21a2700 1 -- 172.21.15.40:0/4054826528 >> 172.21.15.40:0/4054826528 conn(0x7fa7dc0f7520 msgr2=0x7fa7dc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.057+0000 7fa7e21a2700 1 -- 172.21.15.40:0/4054826528 shutdown_connections 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.057+0000 7fa7e21a2700 1 -- 172.21.15.40:0/4054826528 wait complete. 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7e21a2700 1 Processor -- start 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7e21a2700 1 -- start start 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7e21a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7dc066c30 0x7fa7dc10d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7e21a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 0x7fa7dc10de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7e21a2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc10e420 con 0x7fa7dc100100 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7e21a2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc10ecc0 con 0x7fa7dc066c30 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7daffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 0x7fa7dc10de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7daffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 0x7fa7dc10de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36648/0 (socket says 172.21.15.40:36648) 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.058+0000 7fa7daffd700 1 -- 172.21.15.40:0/246812399 learned_addr learned my addr 172.21.15.40:0/246812399 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7db7fe700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7dc066c30 0x7fa7dc10d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7daffd700 1 -- 172.21.15.40:0/246812399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7dc066c30 msgr2=0x7fa7dc10d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7daffd700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7dc066c30 0x7fa7dc10d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7daffd700 1 -- 172.21.15.40:0/246812399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d0005040 con 0x7fa7dc100100 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7daffd700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 0x7fa7dc10de20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0000c00 tx=0x7fa7d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7c3fff700 1 -- 172.21.15.40:0/246812399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d0021070 con 0x7fa7dc100100 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7c3fff700 1 -- 172.21.15.40:0/246812399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7d001c400 con 0x7fa7dc100100 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc10ef40 con 0x7fa7dc100100 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7c3fff700 1 -- 172.21.15.40:0/246812399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d000e070 con 0x7fa7dc100100 2024-02-16T19:31:06.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.059+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc10f420 con 0x7fa7dc100100 2024-02-16T19:31:06.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.060+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc04bed0 con 0x7fa7dc100100 2024-02-16T19:31:06.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.061+0000 7fa7c3fff700 1 -- 172.21.15.40:0/246812399 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa7d001c9e0 con 0x7fa7dc100100 2024-02-16T19:31:06.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.061+0000 7fa7c3fff700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa7c406d7e0 0x7fa7c406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.061+0000 7fa7c3fff700 1 -- 172.21.15.40:0/246812399 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa7d008e0d0 con 0x7fa7dc100100 2024-02-16T19:31:06.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.061+0000 7fa7db7fe700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa7c406d7e0 0x7fa7c406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.062+0000 7fa7db7fe700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa7c406d7e0 0x7fa7c406fc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc004600 tx=0x7fa7cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:06.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.066+0000 7fa7c3fff700 1 -- 172.21.15.40:0/246812399 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7d005b4d0 con 0x7fa7dc100100 2024-02-16T19:31:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.277+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7dc10f920 con 0x7fa7c406d7e0 2024-02-16T19:31:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.287+0000 7fa7c3fff700 1 -- 172.21.15.40:0/246812399 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa7dc10f920 con 0x7fa7c406d7e0 2024-02-16T19:31:06.288 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (5m) 85s ago 14m 25.3M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (3m) 85s ago 14m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (3m) 18s ago 13m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (5m) 85s ago 14m 81.7M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (8m) 85s ago 8m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (9m) 85s ago 9m 13.8M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (8m) 18s ago 8m 17.6M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (9m) 18s ago 9m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (5m) 85s ago 16m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (4m) 18s ago 13m 450M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (4m) 85s ago 16m 47.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (3m) 18s ago 13m 35.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (5m) 85s ago 14m 15.7M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (5m) 18s ago 13m 15.6M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:31:06.289 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (2m) 85s ago 12m 96.1M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:31:06.290 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (2m) 85s ago 12m 99.2M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:31:06.290 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (91s) 85s ago 11m 12.4M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:31:06.290 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (55s) 18s ago 11m 95.7M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:31:06.290 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (22s) 18s ago 11m 12.6M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:31:06.290 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (10m) 18s ago 10m 238M 2581M 17.2.7-513-gb88fae66 870a0944ccb8 6eebba4ea30d 2024-02-16T19:31:06.290 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (4m) 85s ago 14m 53.8M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa7c406d7e0 msgr2=0x7fa7c406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa7c406d7e0 0x7fa7c406fc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc004600 tx=0x7fa7cc00b040 comp rx=0 tx=0).stop 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 msgr2=0x7fa7dc10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 0x7fa7dc10de20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0000c00 tx=0x7fa7d0009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 shutdown_connections 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7dc066c30 0x7fa7dc10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa7c406d7e0 0x7fa7c406fc90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 --2- 172.21.15.40:0/246812399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7dc100100 0x7fa7dc10de20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 >> 172.21.15.40:0/246812399 conn(0x7fa7dc0f7520 msgr2=0x7fa7dc0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.289+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 shutdown_connections 2024-02-16T19:31:06.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.290+0000 7fa7e21a2700 1 -- 172.21.15.40:0/246812399 wait complete. 2024-02-16T19:31:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 -- 172.21.15.40:0/4107945188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fcfb0 msgr2=0x7fca340fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 --2- 172.21.15.40:0/4107945188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fcfb0 0x7fca340fd420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca1c004940 tx=0x7fca1c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 -- 172.21.15.40:0/4107945188 shutdown_connections 2024-02-16T19:31:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 --2- 172.21.15.40:0/4107945188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fcfb0 0x7fca340fd420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 --2- 172.21.15.40:0/4107945188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca340fbe40 0x7fca340fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 -- 172.21.15.40:0/4107945188 >> 172.21.15.40:0/4107945188 conn(0x7fca340f74e0 msgr2=0x7fca340f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 -- 172.21.15.40:0/4107945188 shutdown_connections 2024-02-16T19:31:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 -- 172.21.15.40:0/4107945188 wait complete. 2024-02-16T19:31:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 Processor -- start 2024-02-16T19:31:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.391+0000 7fca3892f700 1 -- start start 2024-02-16T19:31:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3892f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fbe40 0x7fca340fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3892f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca340fcfb0 0x7fca340fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3892f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340fe980 con 0x7fca340fbe40 2024-02-16T19:31:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3892f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340feaf0 con 0x7fca340fcfb0 2024-02-16T19:31:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fbe40 0x7fca340fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fbe40 0x7fca340fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36666/0 (socket says 172.21.15.40:36666) 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3259c700 1 -- 172.21.15.40:0/581994871 learned_addr learned my addr 172.21.15.40:0/581994871 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca31d9b700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca340fcfb0 0x7fca340fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3259c700 1 -- 172.21.15.40:0/581994871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca340fcfb0 msgr2=0x7fca340fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3259c700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca340fcfb0 0x7fca340fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3259c700 1 -- 172.21.15.40:0/581994871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c005040 con 0x7fca340fbe40 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca31d9b700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca340fcfb0 0x7fca340fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3259c700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fbe40 0x7fca340fff00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c830 tx=0x7fca2c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca2affd700 1 -- 172.21.15.40:0/581994871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca2c00a760 con 0x7fca340fbe40 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca2affd700 1 -- 172.21.15.40:0/581994871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca2c00ad90 con 0x7fca340fbe40 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca340fedd0 con 0x7fca340fbe40 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.392+0000 7fca2affd700 1 -- 172.21.15.40:0/581994871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca2c0113a0 con 0x7fca340fbe40 2024-02-16T19:31:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.393+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca34115990 con 0x7fca340fbe40 2024-02-16T19:31:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.394+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3404bed0 con 0x7fca340fbe40 2024-02-16T19:31:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.394+0000 7fca2affd700 1 -- 172.21.15.40:0/581994871 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fca2c012400 con 0x7fca340fbe40 2024-02-16T19:31:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.394+0000 7fca2affd700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fca2006d7e0 0x7fca2006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.395+0000 7fca2affd700 1 -- 172.21.15.40:0/581994871 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fca2c08cf20 con 0x7fca340fbe40 2024-02-16T19:31:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.395+0000 7fca31d9b700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fca2006d7e0 0x7fca2006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.395+0000 7fca31d9b700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fca2006d7e0 0x7fca2006fc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca1c004560 tx=0x7fca1c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:06.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.399+0000 7fca2affd700 1 -- 172.21.15.40:0/581994871 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca2c0c0d20 con 0x7fca340fbe40 2024-02-16T19:31:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.694+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fca34115e20 con 0x7fca340fbe40 2024-02-16T19:31:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.695+0000 7fca2affd700 1 -- 172.21.15.40:0/581994871 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fca2c05a320 con 0x7fca340fbe40 2024-02-16T19:31:06.696 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:31:06.696 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:31:06.696 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:31:06.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:31:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:31:06.698 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 5, 2024-02-16T19:31:06.698 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 9 2024-02-16T19:31:06.698 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:31:06.698 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:31:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fca2006d7e0 msgr2=0x7fca2006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fca2006d7e0 0x7fca2006fc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca1c004560 tx=0x7fca1c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fbe40 msgr2=0x7fca340fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fbe40 0x7fca340fff00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c830 tx=0x7fca2c004470 comp rx=0 tx=0).stop 2024-02-16T19:31:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 shutdown_connections 2024-02-16T19:31:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fca2006d7e0 0x7fca2006fc90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca340fbe40 0x7fca340fff00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 --2- 172.21.15.40:0/581994871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca340fcfb0 0x7fca340fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 >> 172.21.15.40:0/581994871 conn(0x7fca340f74e0 msgr2=0x7fca340656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 shutdown_connections 2024-02-16T19:31:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.697+0000 7fca3892f700 1 -- 172.21.15.40:0/581994871 wait complete. 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.798+0000 7efc3866c700 1 -- 172.21.15.40:0/1456966671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fbe40 msgr2=0x7efc300fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.798+0000 7efc3866c700 1 --2- 172.21.15.40:0/1456966671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fbe40 0x7efc300fc250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efc2c004940 tx=0x7efc2c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 -- 172.21.15.40:0/1456966671 shutdown_connections 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 --2- 172.21.15.40:0/1456966671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc300fcfb0 0x7efc300fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 --2- 172.21.15.40:0/1456966671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fbe40 0x7efc300fc250 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 -- 172.21.15.40:0/1456966671 >> 172.21.15.40:0/1456966671 conn(0x7efc300f7520 msgr2=0x7efc300f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 -- 172.21.15.40:0/1456966671 shutdown_connections 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 -- 172.21.15.40:0/1456966671 wait complete. 2024-02-16T19:31:06.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 Processor -- start 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 -- start start 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc300fbe40 0x7efc3010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fcfb0 0x7efc3010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3010e500 con 0x7efc300fcfb0 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc3866c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3010ed40 con 0x7efc300fbe40 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc35c07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fcfb0 0x7efc3010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.799+0000 7efc35c07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fcfb0 0x7efc3010de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36686/0 (socket says 172.21.15.40:36686) 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc35c07700 1 -- 172.21.15.40:0/2341516027 learned_addr learned my addr 172.21.15.40:0/2341516027 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc36408700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc300fbe40 0x7efc3010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc35c07700 1 -- 172.21.15.40:0/2341516027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc300fbe40 msgr2=0x7efc3010d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc35c07700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc300fbe40 0x7efc3010d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:06.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc35c07700 1 -- 172.21.15.40:0/2341516027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c005040 con 0x7efc300fcfb0 2024-02-16T19:31:06.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc36408700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc300fbe40 0x7efc3010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:31:06.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc35c07700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fcfb0 0x7efc3010de70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc2000b790 tx=0x7efc20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:06.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc26ffd700 1 -- 172.21.15.40:0/2341516027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2000f770 con 0x7efc300fcfb0 2024-02-16T19:31:06.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc26ffd700 1 -- 172.21.15.40:0/2341516027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc2000fda0 con 0x7efc300fcfb0 2024-02-16T19:31:06.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3010f020 con 0x7efc300fcfb0 2024-02-16T19:31:06.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc26ffd700 1 -- 172.21.15.40:0/2341516027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2000d310 con 0x7efc300fcfb0 2024-02-16T19:31:06.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.800+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3010f560 con 0x7efc300fcfb0 2024-02-16T19:31:06.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.801+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3004bed0 con 0x7efc300fcfb0 2024-02-16T19:31:06.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.802+0000 7efc26ffd700 1 -- 172.21.15.40:0/2341516027 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efc20010420 con 0x7efc300fcfb0 2024-02-16T19:31:06.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.802+0000 7efc26ffd700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efc1c06d7e0 0x7efc1c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:06.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.802+0000 7efc26ffd700 1 -- 172.21.15.40:0/2341516027 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efc2008cf20 con 0x7efc300fcfb0 2024-02-16T19:31:06.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.802+0000 7efc36408700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efc1c06d7e0 0x7efc1c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:06.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.803+0000 7efc36408700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efc1c06d7e0 0x7efc1c06fc90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc2c004560 tx=0x7efc2c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:06.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:06.807+0000 7efc26ffd700 1 -- 172.21.15.40:0/2341516027 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc200c0d20 con 0x7efc300fcfb0 2024-02-16T19:31:06.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:06 smithi163 ceph-mon[81637]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:06.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:06 smithi163 ceph-mon[81637]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.053+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efc3010fab0 con 0x7efc300fcfb0 2024-02-16T19:31:07.055 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:06 smithi040 ceph-mon[94210]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:07.055 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:06 smithi040 ceph-mon[94210]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:07.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.053+0000 7efc26ffd700 1 -- 172.21.15.40:0/2341516027 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7efc2005a320 con 0x7efc300fcfb0 2024-02-16T19:31:07.055 INFO:teuthology.orchestra.run.smithi040.stdout:e20 2024-02-16T19:31:07.055 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:31:07.055 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:31:07.055 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 20 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:27:43.925056+0000 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:31:07.056 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:31:07.057 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:31:07.058 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:31:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.055+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efc1c06d7e0 msgr2=0x7efc1c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.055+0000 7efc3866c700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efc1c06d7e0 0x7efc1c06fc90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc2c004560 tx=0x7efc2c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.055+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fcfb0 msgr2=0x7efc3010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.055+0000 7efc3866c700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fcfb0 0x7efc3010de70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc2000b790 tx=0x7efc20009250 comp rx=0 tx=0).stop 2024-02-16T19:31:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.056+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 shutdown_connections 2024-02-16T19:31:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.056+0000 7efc3866c700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc300fbe40 0x7efc3010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.056+0000 7efc3866c700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efc1c06d7e0 0x7efc1c06fc90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.056+0000 7efc3866c700 1 --2- 172.21.15.40:0/2341516027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc300fcfb0 0x7efc3010de70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.056+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 >> 172.21.15.40:0/2341516027 conn(0x7efc300f7520 msgr2=0x7efc30065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.056+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 shutdown_connections 2024-02-16T19:31:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.056+0000 7efc3866c700 1 -- 172.21.15.40:0/2341516027 wait complete. 2024-02-16T19:31:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-02-16T19:31:07.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.159+0000 7f334c0ba700 1 -- 172.21.15.40:0/2730223392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fbe40 msgr2=0x7f33440fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.159+0000 7f334c0ba700 1 --2- 172.21.15.40:0/2730223392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fbe40 0x7f33440fc250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f333c004940 tx=0x7f333c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:07.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.159+0000 7f334c0ba700 1 -- 172.21.15.40:0/2730223392 shutdown_connections 2024-02-16T19:31:07.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.159+0000 7f334c0ba700 1 --2- 172.21.15.40:0/2730223392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33440fcfb0 0x7f33440fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.159+0000 7f334c0ba700 1 --2- 172.21.15.40:0/2730223392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fbe40 0x7f33440fc250 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.159+0000 7f334c0ba700 1 -- 172.21.15.40:0/2730223392 >> 172.21.15.40:0/2730223392 conn(0x7f33440f7500 msgr2=0x7f33440f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:07.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.159+0000 7f334c0ba700 1 -- 172.21.15.40:0/2730223392 shutdown_connections 2024-02-16T19:31:07.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.159+0000 7f334c0ba700 1 -- 172.21.15.40:0/2730223392 wait complete. 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f334c0ba700 1 Processor -- start 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f334c0ba700 1 -- start start 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f334c0ba700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33440fbe40 0x7f33440fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f334c0ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fcfb0 0x7f33440fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f334c0ba700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33440fe980 con 0x7f33440fcfb0 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f334c0ba700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33440feaf0 con 0x7f33440fbe40 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f3349655700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fcfb0 0x7f33440fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f3349655700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fcfb0 0x7f33440fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36696/0 (socket says 172.21.15.40:36696) 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f3349655700 1 -- 172.21.15.40:0/998403717 learned_addr learned my addr 172.21.15.40:0/998403717 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:07.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f3349e56700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33440fbe40 0x7f33440fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f3349655700 1 -- 172.21.15.40:0/998403717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33440fbe40 msgr2=0x7f33440fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f3349655700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33440fbe40 0x7f33440fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.160+0000 7f3349655700 1 -- 172.21.15.40:0/998403717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c005040 con 0x7f33440fcfb0 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.161+0000 7f3349655700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fcfb0 0x7f33440fe380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3338002a50 tx=0x7f33380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.161+0000 7f33427fc700 1 -- 172.21.15.40:0/998403717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3338007cc0 con 0x7f33440fcfb0 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.161+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33440fedd0 con 0x7f33440fcfb0 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.161+0000 7f33427fc700 1 -- 172.21.15.40:0/998403717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f333800f040 con 0x7f33440fcfb0 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.161+0000 7f33427fc700 1 -- 172.21.15.40:0/998403717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333800da30 con 0x7f33440fcfb0 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.161+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3344115970 con 0x7f33440fcfb0 2024-02-16T19:31:07.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.162+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334404bed0 con 0x7f33440fcfb0 2024-02-16T19:31:07.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.162+0000 7f33427fc700 1 -- 172.21.15.40:0/998403717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f333800db90 con 0x7f33440fcfb0 2024-02-16T19:31:07.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.163+0000 7f33427fc700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f332c06d7f0 0x7f332c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:07.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.163+0000 7f33427fc700 1 -- 172.21.15.40:0/998403717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f333808cc70 con 0x7f33440fcfb0 2024-02-16T19:31:07.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.163+0000 7f3349e56700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f332c06d7f0 0x7f332c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:07.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.163+0000 7f3349e56700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f332c06d7f0 0x7f332c06fca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f333c00b3b0 tx=0x7f333c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:07.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.167+0000 7f33427fc700 1 -- 172.21.15.40:0/998403717 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3338059ff0 con 0x7f33440fcfb0 2024-02-16T19:31:07.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.390+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3344066660 con 0x7f332c06d7f0 2024-02-16T19:31:07.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.391+0000 7f33427fc700 1 -- 172.21.15.40:0/998403717 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f3344066660 con 0x7f332c06d7f0 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "11/21 daemons upgraded", 2024-02-16T19:31:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:31:07.394 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:31:07.394 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:31:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f332c06d7f0 msgr2=0x7f332c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f332c06d7f0 0x7f332c06fca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f333c00b3b0 tx=0x7f333c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fcfb0 msgr2=0x7f33440fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fcfb0 0x7f33440fe380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3338002a50 tx=0x7f33380046b0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 shutdown_connections 2024-02-16T19:31:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33440fbe40 0x7f33440fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f332c06d7f0 0x7f332c06fca0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 --2- 172.21.15.40:0/998403717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440fcfb0 0x7f33440fe380 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.393+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 >> 172.21.15.40:0/998403717 conn(0x7f33440f7500 msgr2=0x7f3344065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.394+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 shutdown_connections 2024-02-16T19:31:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.394+0000 7f334c0ba700 1 -- 172.21.15.40:0/998403717 wait complete. 2024-02-16T19:31:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 -- 172.21.15.40:0/1336969487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 msgr2=0x7f44480fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 --2- 172.21.15.40:0/1336969487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 0x7f44480fd420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f443c004940 tx=0x7f443c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 -- 172.21.15.40:0/1336969487 shutdown_connections 2024-02-16T19:31:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 --2- 172.21.15.40:0/1336969487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 0x7f44480fd420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 --2- 172.21.15.40:0/1336969487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44480fbe40 0x7f44480fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 -- 172.21.15.40:0/1336969487 >> 172.21.15.40:0/1336969487 conn(0x7f44480f7520 msgr2=0x7f44480f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 -- 172.21.15.40:0/1336969487 shutdown_connections 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 -- 172.21.15.40:0/1336969487 wait complete. 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.497+0000 7f444dfc8700 1 Processor -- start 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f444dfc8700 1 -- start start 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f444dfc8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44480fbe40 0x7f444810fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f444dfc8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 0x7f4448110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f444dfc8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4448110680 con 0x7f44480fcfb0 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f444dfc8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4448110f20 con 0x7f44480fbe40 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f4446ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 0x7f4448110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f4446ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 0x7f4448110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45164/0 (socket says 172.21.15.40:45164) 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f4446ffd700 1 -- 172.21.15.40:0/1027254485 learned_addr learned my addr 172.21.15.40:0/1027254485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f44477fe700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44480fbe40 0x7f444810fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f4446ffd700 1 -- 172.21.15.40:0/1027254485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44480fbe40 msgr2=0x7f444810fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f4446ffd700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44480fbe40 0x7f444810fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f4446ffd700 1 -- 172.21.15.40:0/1027254485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c005040 con 0x7f44480fcfb0 2024-02-16T19:31:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f4446ffd700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 0x7f4448110080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f443c009fd0 tx=0x7f443c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.498+0000 7f442ffff700 1 -- 172.21.15.40:0/1027254485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f443c00e040 con 0x7f44480fcfb0 2024-02-16T19:31:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.499+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44481111a0 con 0x7f44480fcfb0 2024-02-16T19:31:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.499+0000 7f442ffff700 1 -- 172.21.15.40:0/1027254485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f443c00bcc0 con 0x7f44480fcfb0 2024-02-16T19:31:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.499+0000 7f442ffff700 1 -- 172.21.15.40:0/1027254485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f443c01c630 con 0x7f44480fcfb0 2024-02-16T19:31:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.499+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44481116e0 con 0x7f44480fcfb0 2024-02-16T19:31:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.499+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444804bed0 con 0x7f44480fcfb0 2024-02-16T19:31:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.500+0000 7f442ffff700 1 -- 172.21.15.40:0/1027254485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f443c013420 con 0x7f44480fcfb0 2024-02-16T19:31:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.500+0000 7f442ffff700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f443006d7f0 0x7f443006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.500+0000 7f442ffff700 1 -- 172.21.15.40:0/1027254485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f443c08e390 con 0x7f44480fcfb0 2024-02-16T19:31:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.500+0000 7f44477fe700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f443006d7f0 0x7f443006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.501+0000 7f44477fe700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f443006d7f0 0x7f443006fca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4438004f40 tx=0x7f443800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:07.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.505+0000 7f442ffff700 1 -- 172.21.15.40:0/1027254485 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f443c05b6f0 con 0x7f44480fcfb0 2024-02-16T19:31:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.793+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4448104930 con 0x7f44480fcfb0 2024-02-16T19:31:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.793+0000 7f442ffff700 1 -- 172.21.15.40:0/1027254485 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f443c05b280 con 0x7f44480fcfb0 2024-02-16T19:31:07.795 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f443006d7f0 msgr2=0x7f443006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f443006d7f0 0x7f443006fca0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4438004f40 tx=0x7f443800b040 comp rx=0 tx=0).stop 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 msgr2=0x7f4448110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 0x7f4448110080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f443c009fd0 tx=0x7f443c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 shutdown_connections 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44480fbe40 0x7f444810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f443006d7f0 0x7f443006fca0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 --2- 172.21.15.40:0/1027254485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44480fcfb0 0x7f4448110080 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 >> 172.21.15.40:0/1027254485 conn(0x7f44480f7520 msgr2=0x7f4448065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 shutdown_connections 2024-02-16T19:31:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:07.795+0000 7f444dfc8700 1 -- 172.21.15.40:0/1027254485 wait complete. 2024-02-16T19:31:07.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:07 smithi163 ceph-mon[81637]: pgmap v150: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:07.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:07 smithi163 ceph-mon[81637]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:07.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:07 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/581994871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:07.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:07 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2341516027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:31:07.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:08.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:07 smithi040 ceph-mon[94210]: pgmap v150: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:08.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:07 smithi040 ceph-mon[94210]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:08.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:07 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/581994871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:08.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:07 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2341516027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:31:08.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:08 smithi163 ceph-mon[81637]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:08 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:31:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:08 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:31:08.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:08 smithi163 ceph-mon[81637]: Upgrade: osd.5 is safe to restart 2024-02-16T19:31:08.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:08 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1027254485' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:31:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:08 smithi040 ceph-mon[94210]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:08 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:31:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:08 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-16T19:31:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:08 smithi040 ceph-mon[94210]: Upgrade: osd.5 is safe to restart 2024-02-16T19:31:09.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:08 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1027254485' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:31:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:09 smithi163 ceph-mon[81637]: pgmap v151: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:09 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:09 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:31:09.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:09 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:09 smithi040 ceph-mon[94210]: pgmap v151: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:09 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:09 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-16T19:31:10.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:09 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:10.949 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:10 smithi163 ceph-mon[81637]: Upgrade: Updating osd.5 2024-02-16T19:31:10.949 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:10 smithi163 ceph-mon[81637]: Deploying daemon osd.5 on smithi163 2024-02-16T19:31:11.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:10 smithi040 ceph-mon[94210]: Upgrade: Updating osd.5 2024-02-16T19:31:11.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:10 smithi040 ceph-mon[94210]: Deploying daemon osd.5 on smithi163 2024-02-16T19:31:11.981 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:11 smithi163 ceph-mon[81637]: pgmap v152: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:12.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:11 smithi040 ceph-mon[94210]: pgmap v152: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:12.903 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:12 smithi163 systemd[1]: Stopping Ceph osd.5 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:31:13.237 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:12 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[55137]: 2024-02-16T19:31:12.901+0000 7f45af933700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:31:13.237 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:12 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[55137]: 2024-02-16T19:31:12.901+0000 7f45af933700 -1 osd.5 74 *** Got signal Terminated *** 2024-02-16T19:31:13.237 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:12 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[55137]: 2024-02-16T19:31:12.901+0000 7f45af933700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:31:13.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:13 smithi163 ceph-mon[81637]: pgmap v153: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:13.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:13 smithi163 ceph-mon[81637]: osd.5 marked itself down and dead 2024-02-16T19:31:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:13 smithi040 ceph-mon[94210]: pgmap v153: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:13 smithi040 ceph-mon[94210]: osd.5 marked itself down and dead 2024-02-16T19:31:14.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:14 smithi163 podman[100802]: 2024-02-16 19:31:14.190168137 +0000 UTC m=+1.412506236 container died 6eebba4ea30d26c22b0fd7b960b82b20c21f54ee81e5c298953cf29081ca0d97 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=quincy-b88fae6, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:31:14.853 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:14 smithi163 ceph-mon[81637]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:31:14.854 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:14 smithi163 ceph-mon[81637]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-16T19:31:14.854 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:14 smithi163 ceph-mon[81637]: osdmap e75: 6 total, 5 up, 6 in 2024-02-16T19:31:14.854 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:14 smithi163 podman[100802]: 2024-02-16 19:31:14.618692922 +0000 UTC m=+1.841031020 container cleanup 6eebba4ea30d26c22b0fd7b960b82b20c21f54ee81e5c298953cf29081ca0d97 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=quincy-b88fae6, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-16T19:31:14.854 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:14 smithi163 bash[100802]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5 2024-02-16T19:31:15.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:14 smithi040 ceph-mon[94210]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-16T19:31:15.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:14 smithi040 ceph-mon[94210]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-16T19:31:15.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:14 smithi040 ceph-mon[94210]: osdmap e75: 6 total, 5 up, 6 in 2024-02-16T19:31:15.237 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:14 smithi163 podman[100821]: 2024-02-16 19:31:14.853192071 +0000 UTC m=+0.658472452 container remove 6eebba4ea30d26c22b0fd7b960b82b20c21f54ee81e5c298953cf29081ca0d97 (image=quay.ceph.io/ceph-ci/ceph@sha256:bd04e78e52e49fa75ab32885b8568e6cedc3b0022775783fd0d608060059f38c, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=quincy-b88fae6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-02-16T19:31:15.607 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:15 smithi163 podman[100914]: 2024-02-16 19:31:15.279138359 +0000 UTC m=+0.112671485 container create e10a63f614014544e65c8399d3407daaa98a2d17f133637e0774f085b3963fb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:31:15.607 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:15 smithi163 podman[100914]: 2024-02-16 19:31:15.18561223 +0000 UTC m=+0.019145371 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:31:15.607 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:15 smithi163 podman[100914]: 2024-02-16 19:31:15.523337987 +0000 UTC m=+0.356871113 container init e10a63f614014544e65c8399d3407daaa98a2d17f133637e0774f085b3963fb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:31:15.607 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:15 smithi163 podman[100914]: 2024-02-16 19:31:15.526623035 +0000 UTC m=+0.360156171 container start e10a63f614014544e65c8399d3407daaa98a2d17f133637e0774f085b3963fb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-16T19:31:15.607 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:15 smithi163 podman[100914]: 2024-02-16 19:31:15.60661134 +0000 UTC m=+0.440144467 container attach e10a63f614014544e65c8399d3407daaa98a2d17f133637e0774f085b3963fb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-16T19:31:15.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:15 smithi163 ceph-mon[81637]: pgmap v155: 65 pgs: 13 stale+active+clean, 52 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:15.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:15 smithi163 ceph-mon[81637]: osdmap e76: 6 total, 5 up, 6 in 2024-02-16T19:31:15.988 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:15 smithi163 podman[100914]: 2024-02-16 19:31:15.817878735 +0000 UTC m=+0.651411863 container died e10a63f614014544e65c8399d3407daaa98a2d17f133637e0774f085b3963fb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True) 2024-02-16T19:31:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:15 smithi040 ceph-mon[94210]: pgmap v155: 65 pgs: 13 stale+active+clean, 52 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-02-16T19:31:16.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:15 smithi040 ceph-mon[94210]: osdmap e76: 6 total, 5 up, 6 in 2024-02-16T19:31:16.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:16 smithi163 podman[100940]: 2024-02-16 19:31:16.401826749 +0000 UTC m=+0.578225577 container remove e10a63f614014544e65c8399d3407daaa98a2d17f133637e0774f085b3963fb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-16T19:31:16.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:16 smithi163 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.5.service: Deactivated successfully. 2024-02-16T19:31:16.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:16 smithi163 systemd[1]: Stopped Ceph osd.5 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:31:16.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:16 smithi163 systemd[1]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.5.service: Consumed 1min 630ms CPU time. 2024-02-16T19:31:16.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:16 smithi163 systemd[1]: Starting Ceph osd.5 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:31:17.045 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:16 smithi040 ceph-mon[94210]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-16T19:31:17.045 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:16 smithi040 ceph-mon[94210]: Health check failed: Degraded data redundancy: 24/360 objects degraded (6.667%), 8 pgs degraded (PG_DEGRADED) 2024-02-16T19:31:17.097 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:16 smithi163 ceph-mon[81637]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-16T19:31:17.097 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:16 smithi163 ceph-mon[81637]: Health check failed: Degraded data redundancy: 24/360 objects degraded (6.667%), 8 pgs degraded (PG_DEGRADED) 2024-02-16T19:31:17.097 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:16 smithi163 podman[101041]: 2024-02-16 19:31:16.734117406 +0000 UTC m=+0.018752477 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:31:17.097 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:16 smithi163 podman[101041]: 2024-02-16 19:31:16.877915783 +0000 UTC m=+0.162550856 container create d5111df4464ca50039c3b8e328f2323f1a069d9fa5bbcf0791febed13d7ee662 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5) 2024-02-16T19:31:17.097 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 podman[101041]: 2024-02-16 19:31:17.096891856 +0000 UTC m=+0.381526926 container init d5111df4464ca50039c3b8e328f2323f1a069d9fa5bbcf0791febed13d7ee662 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:31:17.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 podman[101041]: 2024-02-16 19:31:17.100217984 +0000 UTC m=+0.384853055 container start d5111df4464ca50039c3b8e328f2323f1a069d9fa5bbcf0791febed13d7ee662 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0) 2024-02-16T19:31:17.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 podman[101041]: 2024-02-16 19:31:17.171939534 +0000 UTC m=+0.456574605 container attach d5111df4464ca50039c3b8e328f2323f1a069d9fa5bbcf0791febed13d7ee662 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-16T19:31:17.881 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-mon[81637]: pgmap v157: 65 pgs: 8 active+undersized+degraded, 3 peering, 3 stale+active+clean, 7 active+undersized, 44 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24/360 objects degraded (6.667%) 2024-02-16T19:31:17.881 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:17.881 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate[101056]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate[101056]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 bash[101041]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 bash[101041]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 bash[101041]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 bash[101041]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 bash[101041]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 bash[101041]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 bash[101041]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate[101056]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate[101056]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate[101056]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate[101056]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-16T19:31:17.882 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate[101056]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-16T19:31:17.883 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:17 smithi163 podman[101041]: 2024-02-16 19:31:17.566570693 +0000 UTC m=+0.851205755 container died d5111df4464ca50039c3b8e328f2323f1a069d9fa5bbcf0791febed13d7ee662 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-16T19:31:18.147 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:17 smithi040 ceph-mon[94210]: pgmap v157: 65 pgs: 8 active+undersized+degraded, 3 peering, 3 stale+active+clean, 7 active+undersized, 44 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24/360 objects degraded (6.667%) 2024-02-16T19:31:18.147 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:18.147 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:18.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:18 smithi163 podman[101191]: 2024-02-16 19:31:18.159200175 +0000 UTC m=+0.587432648 container remove d5111df4464ca50039c3b8e328f2323f1a069d9fa5bbcf0791febed13d7ee662 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-16T19:31:18.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:18 smithi163 podman[101255]: 2024-02-16 19:31:18.274225531 +0000 UTC m=+0.017602031 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:31:18.865 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:18 smithi163 podman[101255]: 2024-02-16 19:31:18.55182702 +0000 UTC m=+0.295203511 container create 807b119a828c7e11884364b2fbee089a31d9f3bc5d4ee2fd9e5e6c80bf8c4d56 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:31:18.865 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:18 smithi163 podman[101255]: 2024-02-16 19:31:18.754346758 +0000 UTC m=+0.497723245 container init 807b119a828c7e11884364b2fbee089a31d9f3bc5d4ee2fd9e5e6c80bf8c4d56 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:31:18.865 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:18 smithi163 podman[101255]: 2024-02-16 19:31:18.757641819 +0000 UTC m=+0.501018306 container start 807b119a828c7e11884364b2fbee089a31d9f3bc5d4ee2fd9e5e6c80bf8c4d56 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD) 2024-02-16T19:31:18.866 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:18 smithi163 ceph-osd[101274]: -- 172.21.15.163:0/2040182066 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560a5505d1e0 con 0x560a5502ec00 2024-02-16T19:31:18.866 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:18 smithi163 bash[101255]: 807b119a828c7e11884364b2fbee089a31d9f3bc5d4ee2fd9e5e6c80bf8c4d56 2024-02-16T19:31:19.237 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:18 smithi163 systemd[1]: Started Ceph osd.5 for 8c23bd6c-ccff-11ee-95ba-87774f69a715. 2024-02-16T19:31:19.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:19 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:31:19.310+0000 7f663827c7c0 -1 Falling back to public interface 2024-02-16T19:31:20.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:19 smithi163 ceph-mon[81637]: pgmap v158: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:31:20.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:20.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:20.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:20.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:19 smithi040 ceph-mon[94210]: pgmap v158: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:31:20.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:20.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:20.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:20.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:20 smithi163 ceph-mon[81637]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-16T19:31:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:20 smithi040 ceph-mon[94210]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-16T19:31:22.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:21 smithi163 ceph-mon[81637]: pgmap v159: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:31:22.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:21 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:22.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:21 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:21 smithi040 ceph-mon[94210]: pgmap v159: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:31:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:21 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:22.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:21 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:23.846 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:23 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:31:23.550+0000 7f663827c7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-02-16T19:31:24.237 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:23 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:31:23.974+0000 7f663827c7c0 -1 osd.5 74 log_to_monitors true 2024-02-16T19:31:24.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:24 smithi163 ceph-mon[81637]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:31:24.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:24 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:24.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:24 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:24.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:24 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 56/360 objects degraded (15.556%), 14 pgs degraded (PG_DEGRADED) 2024-02-16T19:31:24.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:24 smithi163 ceph-mon[81637]: from='osd.5 [v2:172.21.15.163:6816/10307935,v1:172.21.15.163:6817/10307935]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:31:24.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:24 smithi163 ceph-mon[81637]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:31:24.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:24 smithi040 ceph-mon[94210]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:31:24.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:24 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:24.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:24 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:24.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:24 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 56/360 objects degraded (15.556%), 14 pgs degraded (PG_DEGRADED) 2024-02-16T19:31:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:24 smithi040 ceph-mon[94210]: from='osd.5 [v2:172.21.15.163:6816/10307935,v1:172.21.15.163:6817/10307935]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:31:24.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:24 smithi040 ceph-mon[94210]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-16T19:31:24.987 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:31:24 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:31:24.859+0000 7f66256cc700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-16T19:31:25.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:25 smithi040 ceph-mon[94210]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T19:31:25.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:25 smithi040 ceph-mon[94210]: osdmap e77: 6 total, 5 up, 6 in 2024-02-16T19:31:25.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:25 smithi040 ceph-mon[94210]: from='osd.5 [v2:172.21.15.163:6816/10307935,v1:172.21.15.163:6817/10307935]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:31:25.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:25 smithi040 ceph-mon[94210]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:31:25.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:25 smithi040 ceph-mon[94210]: from='osd.5 ' entity='osd.5' 2024-02-16T19:31:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:25 smithi163 ceph-mon[81637]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-16T19:31:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:25 smithi163 ceph-mon[81637]: osdmap e77: 6 total, 5 up, 6 in 2024-02-16T19:31:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:25 smithi163 ceph-mon[81637]: from='osd.5 [v2:172.21.15.163:6816/10307935,v1:172.21.15.163:6817/10307935]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:31:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:25 smithi163 ceph-mon[81637]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-02-16T19:31:25.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:25 smithi163 ceph-mon[81637]: from='osd.5 ' entity='osd.5' 2024-02-16T19:31:26.211 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:26 smithi163 ceph-mon[81637]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:31:26.211 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:26 smithi163 ceph-mon[81637]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:31:26.211 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:26 smithi163 ceph-mon[81637]: osd.5 [v2:172.21.15.163:6816/10307935,v1:172.21.15.163:6817/10307935] boot 2024-02-16T19:31:26.211 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:26 smithi163 ceph-mon[81637]: osdmap e78: 6 total, 6 up, 6 in 2024-02-16T19:31:26.212 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:26 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:31:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:26 smithi040 ceph-mon[94210]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-02-16T19:31:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:26 smithi040 ceph-mon[94210]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-16T19:31:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:26 smithi040 ceph-mon[94210]: osd.5 [v2:172.21.15.163:6816/10307935,v1:172.21.15.163:6817/10307935] boot 2024-02-16T19:31:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:26 smithi040 ceph-mon[94210]: osdmap e78: 6 total, 6 up, 6 in 2024-02-16T19:31:26.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:26 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-16T19:31:27.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: pgmap v164: 65 pgs: 7 peering, 10 active+undersized+degraded, 11 active+undersized, 37 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/360 objects degraded (12.778%) 2024-02-16T19:31:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:27.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all osd 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-16T19:31:27.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: Upgrade: Setting require_osd_release to 19 squid 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: osdmap e79: 6 total, 6 up, 6 in 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: Upgrade: Scaling down filesystem cephfs 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.897 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: pgmap v164: 65 pgs: 7 peering, 10 active+undersized+degraded, 11 active+undersized, 37 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/360 objects degraded (12.778%) 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:27.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all osd 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-16T19:31:27.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: Upgrade: Setting require_osd_release to 19 squid 2024-02-16T19:31:27.990 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-16T19:31:27.990 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-16T19:31:27.990 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-16T19:31:27.990 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: osdmap e79: 6 total, 6 up, 6 in 2024-02-16T19:31:27.990 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: Upgrade: Scaling down filesystem cephfs 2024-02-16T19:31:27.990 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:27.990 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-16T19:31:29.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-16T19:31:29.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: stopping daemon mds.cephfs.smithi163.rggdhh 2024-02-16T19:31:29.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:31:29.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:stopping} 2 up:standby 2024-02-16T19:31:29.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-16T19:31:29.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:28 smithi040 ceph-mon[94210]: Health check update: Degraded data redundancy: 46/360 objects degraded (12.778%), 10 pgs degraded (PG_DEGRADED) 2024-02-16T19:31:29.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-16T19:31:29.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: stopping daemon mds.cephfs.smithi163.rggdhh 2024-02-16T19:31:29.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:active} 2 up:standby 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: fsmap cephfs:2 {0=cephfs.smithi040.bmpjpk=up:active,1=cephfs.smithi163.rggdhh=up:stopping} 2 up:standby 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-16T19:31:29.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:28 smithi163 ceph-mon[81637]: Health check update: Degraded data redundancy: 46/360 objects degraded (12.778%), 10 pgs degraded (PG_DEGRADED) 2024-02-16T19:31:30.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:29 smithi040 ceph-mon[94210]: pgmap v166: 65 pgs: 7 peering, 5 active+undersized+degraded, 9 active+undersized, 44 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18/360 objects degraded (5.000%) 2024-02-16T19:31:30.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:29 smithi163 ceph-mon[81637]: pgmap v166: 65 pgs: 7 peering, 5 active+undersized+degraded, 9 active+undersized, 44 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18/360 objects degraded (5.000%) 2024-02-16T19:31:31.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:30 smithi163 ceph-mon[81637]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/360 objects degraded (5.000%), 5 pgs degraded) 2024-02-16T19:31:31.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:30 smithi163 ceph-mon[81637]: Cluster is now healthy 2024-02-16T19:31:31.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:30 smithi040 ceph-mon[94210]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/360 objects degraded (5.000%), 5 pgs degraded) 2024-02-16T19:31:31.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:30 smithi040 ceph-mon[94210]: Cluster is now healthy 2024-02-16T19:31:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:31 smithi163 ceph-mon[81637]: pgmap v167: 65 pgs: 7 peering, 58 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 165 B/s wr, 0 op/s 2024-02-16T19:31:32.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:31 smithi040 ceph-mon[94210]: pgmap v167: 65 pgs: 7 peering, 58 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 165 B/s wr, 0 op/s 2024-02-16T19:31:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:33 smithi163 ceph-mon[81637]: pgmap v168: 65 pgs: 65 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 127 B/s wr, 0 op/s 2024-02-16T19:31:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:33.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:33 smithi040 ceph-mon[94210]: pgmap v168: 65 pgs: 65 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 127 B/s wr, 0 op/s 2024-02-16T19:31:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:35 smithi040 ceph-mon[94210]: pgmap v169: 65 pgs: 65 active+clean; 303 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-16T19:31:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:35 smithi163 ceph-mon[81637]: pgmap v169: 65 pgs: 65 active+clean; 303 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-16T19:31:37.540 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:37 smithi163 ceph-mon[81637]: pgmap v170: 65 pgs: 65 active+clean; 287 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-16T19:31:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:37 smithi040 ceph-mon[94210]: pgmap v170: 65 pgs: 65 active+clean; 287 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-16T19:31:37.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.913+0000 7f5700b43700 1 -- 172.21.15.40:0/3713063055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fcef0 msgr2=0x7f56fc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.913+0000 7f5700b43700 1 --2- 172.21.15.40:0/3713063055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fcef0 0x7f56fc0fd320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009ce0 tx=0x7f56f4004d10 comp rx=0 tx=0).stop 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.913+0000 7f5700b43700 1 -- 172.21.15.40:0/3713063055 shutdown_connections 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.913+0000 7f5700b43700 1 --2- 172.21.15.40:0/3713063055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56fc0fe0e0 0x7f56fc0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.913+0000 7f5700b43700 1 --2- 172.21.15.40:0/3713063055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fcef0 0x7f56fc0fd320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.913+0000 7f5700b43700 1 -- 172.21.15.40:0/3713063055 >> 172.21.15.40:0/3713063055 conn(0x7f56fc0f8620 msgr2=0x7f56fc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.913+0000 7f5700b43700 1 -- 172.21.15.40:0/3713063055 shutdown_connections 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f5700b43700 1 -- 172.21.15.40:0/3713063055 wait complete. 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f5700b43700 1 Processor -- start 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f5700b43700 1 -- start start 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f5700b43700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56fc0fcef0 0x7f56fc10fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:37.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f5700b43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fe0e0 0x7f56fc110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f5700b43700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc111040 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f5700b43700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1111b0 con 0x7f56fc0fcef0 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f56f9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fe0e0 0x7f56fc110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f56f9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fe0e0 0x7f56fc110280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45822/0 (socket says 172.21.15.40:45822) 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f56f9d9b700 1 -- 172.21.15.40:0/2187148524 learned_addr learned my addr 172.21.15.40:0/2187148524 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.914+0000 7f56fa59c700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56fc0fcef0 0x7f56fc10fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f56f9d9b700 1 -- 172.21.15.40:0/2187148524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56fc0fcef0 msgr2=0x7f56fc10fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f56f9d9b700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56fc0fcef0 0x7f56fc10fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f56f9d9b700 1 -- 172.21.15.40:0/2187148524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f4009940 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f56f9d9b700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fe0e0 0x7f56fc110280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f56e4002a80 tx=0x7f56e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f56f2ffd700 1 -- 172.21.15.40:0/2187148524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e4007cc0 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc111430 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f56f2ffd700 1 -- 172.21.15.40:0/2187148524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56e400f040 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f56f2ffd700 1 -- 172.21.15.40:0/2187148524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e400dac0 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.915+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc111970 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.916+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc102210 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.917+0000 7f56f2ffd700 1 -- 172.21.15.40:0/2187148524 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f56e400dc20 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.917+0000 7f56f2ffd700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f56e806d7f0 0x7f56e806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.918+0000 7f56f2ffd700 1 -- 172.21.15.40:0/2187148524 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f56e408cbb0 con 0x7f56fc0fe0e0 2024-02-16T19:31:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.918+0000 7f56fa59c700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f56e806d7f0 0x7f56e806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:37.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.918+0000 7f56fa59c700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f56e806d7f0 0x7f56e806fca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56fc10fab0 tx=0x7f56f4004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:37.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:37.922+0000 7f56f2ffd700 1 -- 172.21.15.40:0/2187148524 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56e4056360 con 0x7f56fc0fe0e0 2024-02-16T19:31:38.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.150+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56fc111c30 con 0x7f56e806d7f0 2024-02-16T19:31:38.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.151+0000 7f56f2ffd700 1 -- 172.21.15.40:0/2187148524 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f56fc111c30 con 0x7f56e806d7f0 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f56e806d7f0 msgr2=0x7f56e806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f56e806d7f0 0x7f56e806fca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56fc10fab0 tx=0x7f56f4004d10 comp rx=0 tx=0).stop 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fe0e0 msgr2=0x7f56fc110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fe0e0 0x7f56fc110280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f56e4002a80 tx=0x7f56e40046b0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 shutdown_connections 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56fc0fcef0 0x7f56fc10fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f56e806d7f0 0x7f56e806fca0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 --2- 172.21.15.40:0/2187148524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56fc0fe0e0 0x7f56fc110280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.153+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 >> 172.21.15.40:0/2187148524 conn(0x7f56fc0f8620 msgr2=0x7f56fc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.154+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 shutdown_connections 2024-02-16T19:31:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.154+0000 7f5700b43700 1 -- 172.21.15.40:0/2187148524 wait complete. 2024-02-16T19:31:38.165 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:31:38.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.258+0000 7fd63ff93700 1 -- 172.21.15.40:0/1118756717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 msgr2=0x7fd6380fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.258+0000 7fd63ff93700 1 --2- 172.21.15.40:0/1118756717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 0x7fd6380fc250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd62c004940 tx=0x7fd62c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:38.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.258+0000 7fd63ff93700 1 -- 172.21.15.40:0/1118756717 shutdown_connections 2024-02-16T19:31:38.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.258+0000 7fd63ff93700 1 --2- 172.21.15.40:0/1118756717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6380fcfb0 0x7fd6380fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.258+0000 7fd63ff93700 1 --2- 172.21.15.40:0/1118756717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 0x7fd6380fc250 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.258+0000 7fd63ff93700 1 -- 172.21.15.40:0/1118756717 >> 172.21.15.40:0/1118756717 conn(0x7fd6380f74e0 msgr2=0x7fd6380f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.258+0000 7fd63ff93700 1 -- 172.21.15.40:0/1118756717 shutdown_connections 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.258+0000 7fd63ff93700 1 -- 172.21.15.40:0/1118756717 wait complete. 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63ff93700 1 Processor -- start 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63ff93700 1 -- start start 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63ff93700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 0x7fd63810d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63ff93700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6380fcfb0 0x7fd63810ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63ff93700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63810e3f0 con 0x7fd6380fbe40 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63ff93700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63810ec90 con 0x7fd6380fcfb0 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63dd2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 0x7fd63810d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63dd2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 0x7fd63810d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45842/0 (socket says 172.21.15.40:45842) 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63dd2f700 1 -- 172.21.15.40:0/204938581 learned_addr learned my addr 172.21.15.40:0/204938581 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63d52e700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6380fcfb0 0x7fd63810ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63dd2f700 1 -- 172.21.15.40:0/204938581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6380fcfb0 msgr2=0x7fd63810ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63dd2f700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6380fcfb0 0x7fd63810ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63dd2f700 1 -- 172.21.15.40:0/204938581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c005040 con 0x7fd6380fbe40 2024-02-16T19:31:38.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd63dd2f700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 0x7fd63810d8b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0093b0 tx=0x7fd62c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:38.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.259+0000 7fd62a7fc700 1 -- 172.21.15.40:0/204938581 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd62c00e040 con 0x7fd6380fbe40 2024-02-16T19:31:38.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.260+0000 7fd62a7fc700 1 -- 172.21.15.40:0/204938581 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd62c00bcc0 con 0x7fd6380fbe40 2024-02-16T19:31:38.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.260+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63810ef10 con 0x7fd6380fbe40 2024-02-16T19:31:38.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.260+0000 7fd62a7fc700 1 -- 172.21.15.40:0/204938581 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd62c01c630 con 0x7fd6380fbe40 2024-02-16T19:31:38.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.260+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63810f450 con 0x7fd6380fbe40 2024-02-16T19:31:38.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.261+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63804bed0 con 0x7fd6380fbe40 2024-02-16T19:31:38.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.261+0000 7fd62a7fc700 1 -- 172.21.15.40:0/204938581 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd62c013980 con 0x7fd6380fbe40 2024-02-16T19:31:38.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.261+0000 7fd62a7fc700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd62406d7e0 0x7fd62406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.261+0000 7fd62a7fc700 1 -- 172.21.15.40:0/204938581 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd62c08e190 con 0x7fd6380fbe40 2024-02-16T19:31:38.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.262+0000 7fd63d52e700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd62406d7e0 0x7fd62406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.262+0000 7fd63d52e700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd62406d7e0 0x7fd62406fc90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd634009fd0 tx=0x7fd63400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:38.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.266+0000 7fd62a7fc700 1 -- 172.21.15.40:0/204938581 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd62c05b630 con 0x7fd6380fbe40 2024-02-16T19:31:38.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.489+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd638065dd0 con 0x7fd62406d7e0 2024-02-16T19:31:38.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-16T19:31:38.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:38 smithi040 ceph-mon[94210]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.490+0000 7fd62a7fc700 1 -- 172.21.15.40:0/204938581 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd638065dd0 con 0x7fd62406d7e0 2024-02-16T19:31:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.492+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd62406d7e0 msgr2=0x7fd62406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.492+0000 7fd63ff93700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd62406d7e0 0x7fd62406fc90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd634009fd0 tx=0x7fd63400b040 comp rx=0 tx=0).stop 2024-02-16T19:31:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.492+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 msgr2=0x7fd63810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.492+0000 7fd63ff93700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 0x7fd63810d8b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0093b0 tx=0x7fd62c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.493+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 shutdown_connections 2024-02-16T19:31:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.493+0000 7fd63ff93700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd62406d7e0 0x7fd62406fc90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.493+0000 7fd63ff93700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6380fbe40 0x7fd63810d8b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.493+0000 7fd63ff93700 1 --2- 172.21.15.40:0/204938581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6380fcfb0 0x7fd63810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.493+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 >> 172.21.15.40:0/204938581 conn(0x7fd6380f74e0 msgr2=0x7fd6380656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.493+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 shutdown_connections 2024-02-16T19:31:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.493+0000 7fd63ff93700 1 -- 172.21.15.40:0/204938581 wait complete. 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.595+0000 7f3a5e319700 1 -- 172.21.15.40:0/592203101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 msgr2=0x7f3a580fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.595+0000 7f3a5e319700 1 --2- 172.21.15.40:0/592203101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 0x7f3a580fe550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c004940 tx=0x7f3a4c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.595+0000 7f3a5e319700 1 -- 172.21.15.40:0/592203101 shutdown_connections 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.595+0000 7f3a5e319700 1 --2- 172.21.15.40:0/592203101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 0x7f3a580fe550 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.595+0000 7f3a5e319700 1 --2- 172.21.15.40:0/592203101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a580fcef0 0x7f3a580fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.595+0000 7f3a5e319700 1 -- 172.21.15.40:0/592203101 >> 172.21.15.40:0/592203101 conn(0x7f3a580f8620 msgr2=0x7f3a580faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.595+0000 7f3a5e319700 1 -- 172.21.15.40:0/592203101 shutdown_connections 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.595+0000 7f3a5e319700 1 -- 172.21.15.40:0/592203101 wait complete. 2024-02-16T19:31:38.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a5e319700 1 Processor -- start 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a5e319700 1 -- start start 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a5e319700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a580fcef0 0x7f3a5810d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a5e319700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 0x7f3a5810de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a5e319700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5810e410 con 0x7f3a580fe0e0 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a5e319700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5810ecb0 con 0x7f3a580fcef0 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 0x7f3a5810de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 0x7f3a5810de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45862/0 (socket says 172.21.15.40:45862) 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a577fe700 1 -- 172.21.15.40:0/3584551439 learned_addr learned my addr 172.21.15.40:0/3584551439 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.596+0000 7f3a57fff700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a580fcef0 0x7f3a5810d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a577fe700 1 -- 172.21.15.40:0/3584551439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a580fcef0 msgr2=0x7f3a5810d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a577fe700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a580fcef0 0x7f3a5810d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a577fe700 1 -- 172.21.15.40:0/3584551439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c005040 con 0x7f3a580fe0e0 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a577fe700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 0x7f3a5810de10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c000c00 tx=0x7f3a4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a54ff9700 1 -- 172.21.15.40:0/3584551439 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4c00e070 con 0x7f3a580fe0e0 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a54ff9700 1 -- 172.21.15.40:0/3584551439 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a4c009430 con 0x7f3a580fe0e0 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5810ef30 con 0x7f3a580fe0e0 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a54ff9700 1 -- 172.21.15.40:0/3584551439 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4c00b410 con 0x7f3a580fe0e0 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.597+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5810f410 con 0x7f3a580fe0e0 2024-02-16T19:31:38.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.598+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5804bed0 con 0x7f3a580fe0e0 2024-02-16T19:31:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.598+0000 7f3a54ff9700 1 -- 172.21.15.40:0/3584551439 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3a4c01c820 con 0x7f3a580fe0e0 2024-02-16T19:31:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.599+0000 7f3a54ff9700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3a4006d7f0 0x7f3a4006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.599+0000 7f3a54ff9700 1 -- 172.21.15.40:0/3584551439 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3a4c08e3f0 con 0x7f3a580fe0e0 2024-02-16T19:31:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.599+0000 7f3a57fff700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3a4006d7f0 0x7f3a4006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.599+0000 7f3a57fff700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3a4006d7f0 0x7f3a4006fca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009b80 tx=0x7f3a4800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.603+0000 7f3a54ff9700 1 -- 172.21.15.40:0/3584551439 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a4c05b890 con 0x7f3a580fe0e0 2024-02-16T19:31:38.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:38.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:38.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-16T19:31:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:38 smithi163 ceph-mon[81637]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:38.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.817+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a5810f8b0 con 0x7f3a4006d7f0 2024-02-16T19:31:38.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.829+0000 7f3a54ff9700 1 -- 172.21.15.40:0/3584551439 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3a5810f8b0 con 0x7f3a4006d7f0 2024-02-16T19:31:38.830 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:31:38.830 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (6m) 118s ago 15m 25.3M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:31:38.830 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (3m) 118s ago 15m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (3m) 16s ago 14m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (5m) 118s ago 14m 81.7M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (9m) 118s ago 9m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (9m) 118s ago 9m 13.8M - 17.2.7-513-gb88fae66 870a0944ccb8 c02ae069eafc 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (9m) 16s ago 9m 17.8M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (9m) 16s ago 9m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (5m) 118s ago 16m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (4m) 16s ago 14m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (4m) 118s ago 16m 47.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (4m) 16s ago 14m 36.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (6m) 118s ago 15m 15.7M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (5m) 16s ago 14m 15.6M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:31:38.831 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (3m) 118s ago 13m 96.1M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:31:38.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (2m) 118s ago 12m 99.2M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:31:38.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (2m) 118s ago 12m 12.4M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:31:38.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (88s) 16s ago 11m 97.7M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:31:38.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (54s) 16s ago 11m 93.3M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:31:38.832 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (20s) 16s ago 11m 12.4M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 807b119a828c 2024-02-16T19:31:38.832 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (5m) 118s ago 14m 53.8M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3a4006d7f0 msgr2=0x7f3a4006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3a4006d7f0 0x7f3a4006fca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009b80 tx=0x7f3a4800b040 comp rx=0 tx=0).stop 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 msgr2=0x7f3a5810de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 0x7f3a5810de10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c000c00 tx=0x7f3a4c009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 shutdown_connections 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a580fcef0 0x7f3a5810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f3a4006d7f0 0x7f3a4006fca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 --2- 172.21.15.40:0/3584551439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a580fe0e0 0x7f3a5810de10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 >> 172.21.15.40:0/3584551439 conn(0x7f3a580f8620 msgr2=0x7f3a581013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:38.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.831+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 shutdown_connections 2024-02-16T19:31:38.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.832+0000 7f3a5e319700 1 -- 172.21.15.40:0/3584551439 wait complete. 2024-02-16T19:31:38.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.936+0000 7f985b867700 1 -- 172.21.15.40:0/337848526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406f430 msgr2=0x7f9854108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.936+0000 7f985b867700 1 --2- 172.21.15.40:0/337848526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406f430 0x7f9854108fb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9844004a50 tx=0x7f984400af40 comp rx=0 tx=0).stop 2024-02-16T19:31:38.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.936+0000 7f985b867700 1 -- 172.21.15.40:0/337848526 shutdown_connections 2024-02-16T19:31:38.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.936+0000 7f985b867700 1 --2- 172.21.15.40:0/337848526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406f430 0x7f9854108fb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.936+0000 7f985b867700 1 --2- 172.21.15.40:0/337848526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f985406ea50 0x7f985406ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.936+0000 7f985b867700 1 -- 172.21.15.40:0/337848526 >> 172.21.15.40:0/337848526 conn(0x7f9854069e70 msgr2=0x7f985406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.937+0000 7f985b867700 1 -- 172.21.15.40:0/337848526 shutdown_connections 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.937+0000 7f985b867700 1 -- 172.21.15.40:0/337848526 wait complete. 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f985b867700 1 Processor -- start 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f985b867700 1 -- start start 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f985b867700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406ea50 0x7f9854113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f985b867700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9854114510 0x7f98541153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f985b867700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541149d0 con 0x7f985406ea50 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f985b867700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9854114b10 con 0x7f9854114510 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f9859603700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406ea50 0x7f9854113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f9859603700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406ea50 0x7f9854113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45882/0 (socket says 172.21.15.40:45882) 2024-02-16T19:31:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.938+0000 7f9859603700 1 -- 172.21.15.40:0/1209721468 learned_addr learned my addr 172.21.15.40:0/1209721468 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:38.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.939+0000 7f9858e02700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9854114510 0x7f98541153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.939+0000 7f9859603700 1 -- 172.21.15.40:0/1209721468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9854114510 msgr2=0x7f98541153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:38.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.939+0000 7f9859603700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9854114510 0x7f98541153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:38.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.939+0000 7f9859603700 1 -- 172.21.15.40:0/1209721468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9844005040 con 0x7f985406ea50 2024-02-16T19:31:38.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.939+0000 7f9858e02700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9854114510 0x7f98541153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:31:38.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.939+0000 7f9859603700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406ea50 0x7f9854113fd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f985000eb40 tx=0x7f985000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.941+0000 7f9849ffb700 1 -- 172.21.15.40:0/1209721468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9850007cc0 con 0x7f985406ea50 2024-02-16T19:31:38.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.941+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98541159b0 con 0x7f985406ea50 2024-02-16T19:31:38.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.941+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9854115ec0 con 0x7f985406ea50 2024-02-16T19:31:38.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.941+0000 7f9849ffb700 1 -- 172.21.15.40:0/1209721468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f985000f040 con 0x7f985406ea50 2024-02-16T19:31:38.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.941+0000 7f9849ffb700 1 -- 172.21.15.40:0/1209721468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9850009620 con 0x7f985406ea50 2024-02-16T19:31:38.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.943+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98380052e0 con 0x7f985406ea50 2024-02-16T19:31:38.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.944+0000 7f9849ffb700 1 -- 172.21.15.40:0/1209721468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9850009780 con 0x7f985406ea50 2024-02-16T19:31:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.945+0000 7f9849ffb700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f984006d980 0x7f984006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.945+0000 7f9849ffb700 1 -- 172.21.15.40:0/1209721468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f985008cae0 con 0x7f985406ea50 2024-02-16T19:31:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.946+0000 7f9858e02700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f984006d980 0x7f984006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:38.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.947+0000 7f9858e02700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f984006d980 0x7f984006fe30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f984400c3b0 tx=0x7f984400af40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:38.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:38.953+0000 7f9849ffb700 1 -- 172.21.15.40:0/1209721468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9850059f70 con 0x7f985406ea50 2024-02-16T19:31:39.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.251+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9838005da0 con 0x7f985406ea50 2024-02-16T19:31:39.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.252+0000 7f9849ffb700 1 -- 172.21.15.40:0/1209721468 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f9850059b00 con 0x7f985406ea50 2024-02-16T19:31:39.253 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:31:39.253 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 4, 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 10 2024-02-16T19:31:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:31:39.255 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:31:39.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f984006d980 msgr2=0x7f984006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f984006d980 0x7f984006fe30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f984400c3b0 tx=0x7f984400af40 comp rx=0 tx=0).stop 2024-02-16T19:31:39.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406ea50 msgr2=0x7f9854113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406ea50 0x7f9854113fd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f985000eb40 tx=0x7f985000b040 comp rx=0 tx=0).stop 2024-02-16T19:31:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 shutdown_connections 2024-02-16T19:31:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f984006d980 0x7f984006fe30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985406ea50 0x7f9854113fd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 --2- 172.21.15.40:0/1209721468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9854114510 0x7f98541153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 >> 172.21.15.40:0/1209721468 conn(0x7f9854069e70 msgr2=0x7f9854107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 shutdown_connections 2024-02-16T19:31:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.254+0000 7f985b867700 1 -- 172.21.15.40:0/1209721468 wait complete. 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.358+0000 7f195adf6700 1 -- 172.21.15.40:0/2282389138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 msgr2=0x7f19540fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.358+0000 7f195adf6700 1 --2- 172.21.15.40:0/2282389138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 0x7f19540fc250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1948004940 tx=0x7f1948009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.358+0000 7f195adf6700 1 -- 172.21.15.40:0/2282389138 shutdown_connections 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.358+0000 7f195adf6700 1 --2- 172.21.15.40:0/2282389138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19540fcfb0 0x7f19540fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.358+0000 7f195adf6700 1 --2- 172.21.15.40:0/2282389138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 0x7f19540fc250 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.358+0000 7f195adf6700 1 -- 172.21.15.40:0/2282389138 >> 172.21.15.40:0/2282389138 conn(0x7f19540f7520 msgr2=0x7f19540f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.359+0000 7f195adf6700 1 -- 172.21.15.40:0/2282389138 shutdown_connections 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.359+0000 7f195adf6700 1 -- 172.21.15.40:0/2282389138 wait complete. 2024-02-16T19:31:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.359+0000 7f195adf6700 1 Processor -- start 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.359+0000 7f195adf6700 1 -- start start 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.359+0000 7f195adf6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 0x7f1954110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.359+0000 7f195adf6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19540fcfb0 0x7f1954112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.359+0000 7f195adf6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954066d10 con 0x7f19540fbe40 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.359+0000 7f195adf6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954066e50 con 0x7f19540fcfb0 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f1958b92700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 0x7f1954110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f1958b92700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 0x7f1954110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45894/0 (socket says 172.21.15.40:45894) 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f1958b92700 1 -- 172.21.15.40:0/3085678313 learned_addr learned my addr 172.21.15.40:0/3085678313 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f1953fff700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19540fcfb0 0x7f1954112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f1958b92700 1 -- 172.21.15.40:0/3085678313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19540fcfb0 msgr2=0x7f1954112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f1958b92700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19540fcfb0 0x7f1954112b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f1958b92700 1 -- 172.21.15.40:0/3085678313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1948005040 con 0x7f19540fbe40 2024-02-16T19:31:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f1958b92700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 0x7f1954110640 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19480093b0 tx=0x7f1948009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f19517fa700 1 -- 172.21.15.40:0/3085678313 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f194800e040 con 0x7f19540fbe40 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f19517fa700 1 -- 172.21.15.40:0/3085678313 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f194800bcc0 con 0x7f19540fbe40 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1954113190 con 0x7f19540fbe40 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f19517fa700 1 -- 172.21.15.40:0/3085678313 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f194801c630 con 0x7f19540fbe40 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.360+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1954113680 con 0x7f19540fbe40 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.361+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195404bed0 con 0x7f19540fbe40 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.362+0000 7f19517fa700 1 -- 172.21.15.40:0/3085678313 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1948013980 con 0x7f19540fbe40 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.362+0000 7f19517fa700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f194406d7e0 0x7f194406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.362+0000 7f19517fa700 1 -- 172.21.15.40:0/3085678313 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f194808e190 con 0x7f19540fbe40 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.362+0000 7f1953fff700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f194406d7e0 0x7f194406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.363+0000 7f1953fff700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f194406d7e0 0x7f194406fc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1940004f40 tx=0x7f194000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:39.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.367+0000 7f19517fa700 1 -- 172.21.15.40:0/3085678313 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1948057b00 con 0x7f19540fbe40 2024-02-16T19:31:39.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.619+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f19541138f0 con 0x7f19540fbe40 2024-02-16T19:31:39.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.619+0000 7f19517fa700 1 -- 172.21.15.40:0/3085678313 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1860 (secure 0 0 0) 0x7f19541138f0 con 0x7f19540fbe40 2024-02-16T19:31:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:e22 2024-02-16T19:31:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:31:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:31:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:31:39.621 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:31:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:31:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:31:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 22 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:31:27.873379+0000 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:31:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 1 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486,1=24263} 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{1:24263} state up:stopping seq 4 join_fscid=1 addr [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:31:39.623 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:31:39.624 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:31:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:31:39.624 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:31:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/4253297290,v1:172.21.15.40:6827/4253297290] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:31:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.621+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f194406d7e0 msgr2=0x7f194406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.621+0000 7f195adf6700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f194406d7e0 0x7f194406fc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1940004f40 tx=0x7f194000b040 comp rx=0 tx=0).stop 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.621+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 msgr2=0x7f1954110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.621+0000 7f195adf6700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 0x7f1954110640 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19480093b0 tx=0x7f1948009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.622+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 shutdown_connections 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.622+0000 7f195adf6700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f194406d7e0 0x7f194406fc90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.622+0000 7f195adf6700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19540fbe40 0x7f1954110640 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.622+0000 7f195adf6700 1 --2- 172.21.15.40:0/3085678313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19540fcfb0 0x7f1954112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.622+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 >> 172.21.15.40:0/3085678313 conn(0x7f19540f7520 msgr2=0x7f1954100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.622+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 shutdown_connections 2024-02-16T19:31:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.622+0000 7f195adf6700 1 -- 172.21.15.40:0/3085678313 wait complete. 2024-02-16T19:31:39.626 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-02-16T19:31:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.725+0000 7fab6d423700 1 -- 172.21.15.40:0/2848434283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 msgr2=0x7fab680fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.725+0000 7fab6d423700 1 --2- 172.21.15.40:0/2848434283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 0x7fab680fc250 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fab60004940 tx=0x7fab60009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.725+0000 7fab6d423700 1 -- 172.21.15.40:0/2848434283 shutdown_connections 2024-02-16T19:31:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.725+0000 7fab6d423700 1 --2- 172.21.15.40:0/2848434283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab680fcfb0 0x7fab680fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.725+0000 7fab6d423700 1 --2- 172.21.15.40:0/2848434283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 0x7fab680fc250 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.725+0000 7fab6d423700 1 -- 172.21.15.40:0/2848434283 >> 172.21.15.40:0/2848434283 conn(0x7fab680f74e0 msgr2=0x7fab680f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab6d423700 1 -- 172.21.15.40:0/2848434283 shutdown_connections 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab6d423700 1 -- 172.21.15.40:0/2848434283 wait complete. 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab6d423700 1 Processor -- start 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab6d423700 1 -- start start 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab6d423700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 0x7fab6810fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab6d423700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab680fcfb0 0x7fab68110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab6d423700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab68110700 con 0x7fab680fbe40 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab6d423700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab68110f40 con 0x7fab680fcfb0 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab66ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 0x7fab6810fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.726+0000 7fab66ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 0x7fab6810fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45908/0 (socket says 172.21.15.40:45908) 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab66ffd700 1 -- 172.21.15.40:0/1291797818 learned_addr learned my addr 172.21.15.40:0/1291797818 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab667fc700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab680fcfb0 0x7fab68110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab66ffd700 1 -- 172.21.15.40:0/1291797818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab680fcfb0 msgr2=0x7fab68110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab66ffd700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab680fcfb0 0x7fab68110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab66ffd700 1 -- 172.21.15.40:0/1291797818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab60005040 con 0x7fab680fbe40 2024-02-16T19:31:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab66ffd700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 0x7fab6810fb30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab600093b0 tx=0x7fab60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab5f7fe700 1 -- 172.21.15.40:0/1291797818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6000e040 con 0x7fab680fbe40 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab5f7fe700 1 -- 172.21.15.40:0/1291797818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab6000bcc0 con 0x7fab680fbe40 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab5f7fe700 1 -- 172.21.15.40:0/1291797818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6001c630 con 0x7fab680fbe40 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab681111c0 con 0x7fab680fbe40 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.727+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab681116b0 con 0x7fab680fbe40 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.728+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6804bed0 con 0x7fab680fbe40 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.729+0000 7fab5f7fe700 1 -- 172.21.15.40:0/1291797818 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fab60013980 con 0x7fab680fbe40 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.729+0000 7fab5f7fe700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab5406d7e0 0x7fab5406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.729+0000 7fab5f7fe700 1 -- 172.21.15.40:0/1291797818 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fab6008e290 con 0x7fab680fbe40 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.729+0000 7fab667fc700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab5406d7e0 0x7fab5406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.730+0000 7fab667fc700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab5406d7e0 0x7fab5406fc90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab50009bf0 tx=0x7fab500091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.734+0000 7fab5f7fe700 1 -- 172.21.15.40:0/1291797818 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab6005b690 con 0x7fab680fbe40 2024-02-16T19:31:39.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.954+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab68065dd0 con 0x7fab5406d7e0 2024-02-16T19:31:39.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.955+0000 7fab5f7fe700 1 -- 172.21.15.40:0/1291797818 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fab68065dd0 con 0x7fab5406d7e0 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "osd", 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "12/21 daemons upgraded", 2024-02-16T19:31:39.957 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading osd daemons", 2024-02-16T19:31:39.958 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:31:39.958 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:31:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab5406d7e0 msgr2=0x7fab5406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab5406d7e0 0x7fab5406fc90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab50009bf0 tx=0x7fab500091a0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 msgr2=0x7fab6810fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 0x7fab6810fb30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab600093b0 tx=0x7fab60009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 shutdown_connections 2024-02-16T19:31:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fab5406d7e0 0x7fab5406fc90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab680fbe40 0x7fab6810fb30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 --2- 172.21.15.40:0/1291797818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab680fcfb0 0x7fab68110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 >> 172.21.15.40:0/1291797818 conn(0x7fab680f74e0 msgr2=0x7fab680656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 shutdown_connections 2024-02-16T19:31:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:39.958+0000 7fab6d423700 1 -- 172.21.15.40:0/1291797818 wait complete. 2024-02-16T19:31:40.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.060+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3590247199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fe0e0 msgr2=0x7fe1340fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:40.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.060+0000 7fe13aa5c700 1 --2- 172.21.15.40:0/3590247199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fe0e0 0x7fe1340fe550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe128004940 tx=0x7fe128009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:40.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.060+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3590247199 shutdown_connections 2024-02-16T19:31:40.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.060+0000 7fe13aa5c700 1 --2- 172.21.15.40:0/3590247199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fe0e0 0x7fe1340fe550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:40.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.060+0000 7fe13aa5c700 1 --2- 172.21.15.40:0/3590247199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1340fcef0 0x7fe1340fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:40.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.060+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3590247199 >> 172.21.15.40:0/3590247199 conn(0x7fe1340f8620 msgr2=0x7fe1340faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:40.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.060+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3590247199 shutdown_connections 2024-02-16T19:31:40.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.060+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3590247199 wait complete. 2024-02-16T19:31:40.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe13aa5c700 1 Processor -- start 2024-02-16T19:31:40.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe13aa5c700 1 -- start start 2024-02-16T19:31:40.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe13aa5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fcef0 0x7fe13410fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:40.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe13aa5c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1340fe0e0 0x7fe134110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:40.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe13aa5c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe134110600 con 0x7fe1340fcef0 2024-02-16T19:31:40.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe13aa5c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe134110ea0 con 0x7fe1340fe0e0 2024-02-16T19:31:40.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe133fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fcef0 0x7fe13410fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:40.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe133fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fcef0 0x7fe13410fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45922/0 (socket says 172.21.15.40:45922) 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe133fff700 1 -- 172.21.15.40:0/3384156473 learned_addr learned my addr 172.21.15.40:0/3384156473 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.061+0000 7fe1337fe700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1340fe0e0 0x7fe134110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe133fff700 1 -- 172.21.15.40:0/3384156473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1340fe0e0 msgr2=0x7fe134110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe133fff700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1340fe0e0 0x7fe134110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe133fff700 1 -- 172.21.15.40:0/3384156473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128005040 con 0x7fe1340fcef0 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe133fff700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fcef0 0x7fe13410fac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe12400eb70 tx=0x7fe12400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe130ff9700 1 -- 172.21.15.40:0/3384156473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe124007cc0 con 0x7fe1340fcef0 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe130ff9700 1 -- 172.21.15.40:0/3384156473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe12400f040 con 0x7fe1340fcef0 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe134111120 con 0x7fe1340fcef0 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe130ff9700 1 -- 172.21.15.40:0/3384156473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12400cb20 con 0x7fe1340fcef0 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.062+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe134111660 con 0x7fe1340fcef0 2024-02-16T19:31:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.063+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe134102210 con 0x7fe1340fcef0 2024-02-16T19:31:40.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.064+0000 7fe130ff9700 1 -- 172.21.15.40:0/3384156473 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe12400cc80 con 0x7fe1340fcef0 2024-02-16T19:31:40.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.064+0000 7fe130ff9700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe11c06d7e0 0x7fe11c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:31:40.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.064+0000 7fe130ff9700 1 -- 172.21.15.40:0/3384156473 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe12408c820 con 0x7fe1340fcef0 2024-02-16T19:31:40.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.064+0000 7fe1337fe700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe11c06d7e0 0x7fe11c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:31:40.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.064+0000 7fe1337fe700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe11c06d7e0 0x7fe11c06fc90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe128009b30 tx=0x7fe128009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:31:40.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.069+0000 7fe130ff9700 1 -- 172.21.15.40:0/3384156473 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe124059cd0 con 0x7fe1340fcef0 2024-02-16T19:31:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.358+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe134111990 con 0x7fe1340fcef0 2024-02-16T19:31:40.360 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:40 smithi040 ceph-mon[94210]: pgmap v171: 65 pgs: 65 active+clean; 279 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.1 KiB/s wr, 6 op/s 2024-02-16T19:31:40.360 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:40 smithi040 ceph-mon[94210]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:40.360 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:40 smithi040 ceph-mon[94210]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:40.360 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:40 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1209721468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:40.360 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:40 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3085678313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:31:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.359+0000 7fe130ff9700 1 -- 172.21.15.40:0/3384156473 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe124059860 con 0x7fe1340fcef0 2024-02-16T19:31:40.361 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:31:40.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe11c06d7e0 msgr2=0x7fe11c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:40.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe11c06d7e0 0x7fe11c06fc90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe128009b30 tx=0x7fe128009e30 comp rx=0 tx=0).stop 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fcef0 msgr2=0x7fe13410fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fcef0 0x7fe13410fac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe12400eb70 tx=0x7fe12400b040 comp rx=0 tx=0).stop 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 shutdown_connections 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe11c06d7e0 0x7fe11c06fc90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1340fcef0 0x7fe13410fac0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 --2- 172.21.15.40:0/3384156473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1340fe0e0 0x7fe134110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 >> 172.21.15.40:0/3384156473 conn(0x7fe1340f8620 msgr2=0x7fe1340ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 shutdown_connections 2024-02-16T19:31:40.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:31:40.361+0000 7fe13aa5c700 1 -- 172.21.15.40:0/3384156473 wait complete. 2024-02-16T19:31:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:40 smithi163 ceph-mon[81637]: pgmap v171: 65 pgs: 65 active+clean; 279 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.1 KiB/s wr, 6 op/s 2024-02-16T19:31:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:40 smithi163 ceph-mon[81637]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:40.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:40 smithi163 ceph-mon[81637]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:40.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:40 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1209721468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:40.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:40 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3085678313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:31:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:41 smithi040 ceph-mon[94210]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:41 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3384156473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:31:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:41 smithi163 ceph-mon[81637]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:31:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:41 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3384156473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:31:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:42 smithi040 ceph-mon[94210]: pgmap v172: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-16T19:31:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:42 smithi163 ceph-mon[81637]: pgmap v172: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-16T19:31:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:44 smithi040 ceph-mon[94210]: pgmap v173: 65 pgs: 65 active+clean; 211 MiB data, 900 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-16T19:31:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:44 smithi163 ceph-mon[81637]: pgmap v173: 65 pgs: 65 active+clean; 211 MiB data, 900 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 7 op/s 2024-02-16T19:31:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:46 smithi040 ceph-mon[94210]: pgmap v174: 65 pgs: 65 active+clean; 171 MiB data, 768 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-16T19:31:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:46 smithi163 ceph-mon[81637]: pgmap v174: 65 pgs: 65 active+clean; 171 MiB data, 768 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.4 KiB/s wr, 8 op/s 2024-02-16T19:31:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:48 smithi040 ceph-mon[94210]: pgmap v175: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 597 B/s wr, 6 op/s 2024-02-16T19:31:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:48 smithi163 ceph-mon[81637]: pgmap v175: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 597 B/s wr, 6 op/s 2024-02-16T19:31:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:31:49.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:49 smithi163 ceph-mon[81637]: daemon mds.cephfs.smithi163.rggdhh finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-02-16T19:31:49.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:49.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:49.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:49.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:49.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:49.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:49.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:49 smithi040 ceph-mon[94210]: daemon mds.cephfs.smithi163.rggdhh finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-02-16T19:31:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:50 smithi040 ceph-mon[94210]: pgmap v176: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 511 B/s wr, 4 op/s 2024-02-16T19:31:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:50 smithi040 ceph-mon[94210]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-16T19:31:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:50 smithi040 ceph-mon[94210]: mds.1 [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] down:stopped 2024-02-16T19:31:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:50 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi040.bmpjpk=up:active} 2 up:standby 2024-02-16T19:31:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:50 smithi163 ceph-mon[81637]: pgmap v176: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 511 B/s wr, 4 op/s 2024-02-16T19:31:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:50 smithi163 ceph-mon[81637]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-16T19:31:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:50 smithi163 ceph-mon[81637]: mds.1 [v2:172.21.15.163:6824/1432724463,v1:172.21.15.163:6825/1432724463] down:stopped 2024-02-16T19:31:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:50 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi040.bmpjpk=up:active} 2 up:standby 2024-02-16T19:31:52.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:51 smithi040 ceph-mon[94210]: pgmap v177: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-02-16T19:31:52.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:51 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.163:6824/2272386057,v1:172.21.15.163:6825/2272386057] up:boot 2024-02-16T19:31:52.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:51 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi040.bmpjpk=up:active} 3 up:standby 2024-02-16T19:31:52.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:31:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:51 smithi163 ceph-mon[81637]: pgmap v177: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-02-16T19:31:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:51 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.163:6824/2272386057,v1:172.21.15.163:6825/2272386057] up:boot 2024-02-16T19:31:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:51 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi040.bmpjpk=up:active} 3 up:standby 2024-02-16T19:31:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:31:54.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:53 smithi040 ceph-mon[94210]: pgmap v178: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 2 op/s 2024-02-16T19:31:54.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:53 smithi163 ceph-mon[81637]: pgmap v178: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 2 op/s 2024-02-16T19:31:56.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:55 smithi040 ceph-mon[94210]: pgmap v179: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 1 op/s 2024-02-16T19:31:56.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:55 smithi163 ceph-mon[81637]: pgmap v179: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 1 op/s 2024-02-16T19:31:58.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:57 smithi040 ceph-mon[94210]: pgmap v180: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:31:58.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:57 smithi163 ceph-mon[81637]: pgmap v180: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:31:59.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: pgmap v181: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi040.ezkppl"]}]: dispatch 2024-02-16T19:31:59.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:31:59 smithi040 ceph-mon[94210]: Upgrade: It appears safe to stop mds.cephfs.smithi040.ezkppl 2024-02-16T19:32:00.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: pgmap v181: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:00.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi040.ezkppl"]}]: dispatch 2024-02-16T19:32:00.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:31:59 smithi163 ceph-mon[81637]: Upgrade: It appears safe to stop mds.cephfs.smithi040.ezkppl 2024-02-16T19:32:01.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:01 smithi040 ceph-mon[94210]: pgmap v182: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:01.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:01 smithi040 ceph-mon[94210]: Upgrade: Updating mds.cephfs.smithi040.ezkppl 2024-02-16T19:32:01.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:01 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:01.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:01 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.ezkppl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:32:01.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:01 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:01.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:01 smithi040 ceph-mon[94210]: Deploying daemon mds.cephfs.smithi040.ezkppl on smithi040 2024-02-16T19:32:01.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:01 smithi163 ceph-mon[81637]: pgmap v182: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:01.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:01 smithi163 ceph-mon[81637]: Upgrade: Updating mds.cephfs.smithi040.ezkppl 2024-02-16T19:32:01.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:01 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:01.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:01 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.ezkppl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:32:01.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:01 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:01.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:01 smithi163 ceph-mon[81637]: Deploying daemon mds.cephfs.smithi040.ezkppl on smithi040 2024-02-16T19:32:02.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:02.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:02 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:03.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:03 smithi040 ceph-mon[94210]: pgmap v183: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:03.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:03 smithi163 ceph-mon[81637]: pgmap v183: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:04.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:04 smithi040 ceph-mon[94210]: osdmap e80: 6 total, 6 up, 6 in 2024-02-16T19:32:04.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:04 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi040.bmpjpk=up:active} 2 up:standby 2024-02-16T19:32:04.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:04 smithi163 ceph-mon[81637]: osdmap e80: 6 total, 6 up, 6 in 2024-02-16T19:32:04.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:04 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi040.bmpjpk=up:active} 2 up:standby 2024-02-16T19:32:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:05 smithi040 ceph-mon[94210]: pgmap v185: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:05.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:05 smithi163 ceph-mon[81637]: pgmap v185: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:07.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:07 smithi040 ceph-mon[94210]: pgmap v186: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:07.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:07 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] up:boot 2024-02-16T19:32:07.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:07 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi040.bmpjpk=up:active} 3 up:standby 2024-02-16T19:32:07.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:32:07.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:07.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:07.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:07.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:07 smithi163 ceph-mon[81637]: pgmap v186: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:07.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:07 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] up:boot 2024-02-16T19:32:07.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:07 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi040.bmpjpk=up:active} 3 up:standby 2024-02-16T19:32:07.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:32:07.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:07.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:07.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:09.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:09 smithi040 ceph-mon[94210]: pgmap v187: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:09 smithi163 ceph-mon[81637]: pgmap v187: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:10.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.468+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/1366520663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fe0e0 msgr2=0x7fe9dc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.468+0000 7fe9e1d7b700 1 --2- 172.21.15.40:0/1366520663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fe0e0 0x7fe9dc0fe550 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004940 tx=0x7fe9d0009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.468+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/1366520663 shutdown_connections 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.468+0000 7fe9e1d7b700 1 --2- 172.21.15.40:0/1366520663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fe0e0 0x7fe9dc0fe550 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.468+0000 7fe9e1d7b700 1 --2- 172.21.15.40:0/1366520663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9dc0fcef0 0x7fe9dc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.468+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/1366520663 >> 172.21.15.40:0/1366520663 conn(0x7fe9dc0f8620 msgr2=0x7fe9dc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.468+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/1366520663 shutdown_connections 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.468+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/1366520663 wait complete. 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9e1d7b700 1 Processor -- start 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9e1d7b700 1 -- start start 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9e1d7b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fcef0 0x7fe9dc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9e1d7b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9dc0fe0e0 0x7fe9dc0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:10.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9e1d7b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0708e0 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9e1d7b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc070a50 con 0x7fe9dc0fe0e0 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fcef0 0x7fe9dc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fcef0 0x7fe9dc071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43068/0 (socket says 172.21.15.40:43068) 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9db7fe700 1 -- 172.21.15.40:0/2784325228 learned_addr learned my addr 172.21.15.40:0/2784325228 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9daffd700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9dc0fe0e0 0x7fe9dc0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9db7fe700 1 -- 172.21.15.40:0/2784325228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9dc0fe0e0 msgr2=0x7fe9dc0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.469+0000 7fe9db7fe700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9dc0fe0e0 0x7fe9dc0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.470+0000 7fe9db7fe700 1 -- 172.21.15.40:0/2784325228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d0005040 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.470+0000 7fe9daffd700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9dc0fe0e0 0x7fe9dc0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.470+0000 7fe9db7fe700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fcef0 0x7fe9dc071e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc002a50 tx=0x7fe9cc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.470+0000 7fe9c3fff700 1 -- 172.21.15.40:0/2784325228 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9cc007cc0 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.470+0000 7fe9c3fff700 1 -- 172.21.15.40:0/2784325228 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9cc00f040 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.470+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc070d30 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.470+0000 7fe9c3fff700 1 -- 172.21.15.40:0/2784325228 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9cc00da40 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.470+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc1159c0 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.471+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc04bed0 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.471+0000 7fe9c3fff700 1 -- 172.21.15.40:0/2784325228 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe9cc0097c0 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.472+0000 7fe9c3fff700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe9c406d7e0 0x7fe9c406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:10.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.472+0000 7fe9c3fff700 1 -- 172.21.15.40:0/2784325228 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe9cc08cb20 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.472+0000 7fe9daffd700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe9c406d7e0 0x7fe9c406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:10.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.472+0000 7fe9daffd700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe9c406d7e0 0x7fe9c406fc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004560 tx=0x7fe9d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:10.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.477+0000 7fe9c3fff700 1 -- 172.21.15.40:0/2784325228 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9cc059f20 con 0x7fe9dc0fcef0 2024-02-16T19:32:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.699+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9dc102ac0 con 0x7fe9c406d7e0 2024-02-16T19:32:10.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.701+0000 7fe9c3fff700 1 -- 172.21.15.40:0/2784325228 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe9dc102ac0 con 0x7fe9c406d7e0 2024-02-16T19:32:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe9c406d7e0 msgr2=0x7fe9c406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe9c406d7e0 0x7fe9c406fc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0004560 tx=0x7fe9d0009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fcef0 msgr2=0x7fe9dc071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fcef0 0x7fe9dc071e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc002a50 tx=0x7fe9cc004600 comp rx=0 tx=0).stop 2024-02-16T19:32:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 shutdown_connections 2024-02-16T19:32:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe9c406d7e0 0x7fe9c406fc90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9dc0fcef0 0x7fe9dc071e60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 --2- 172.21.15.40:0/2784325228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9dc0fe0e0 0x7fe9dc0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 >> 172.21.15.40:0/2784325228 conn(0x7fe9dc0f8620 msgr2=0x7fe9dc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 shutdown_connections 2024-02-16T19:32:10.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.703+0000 7fe9e1d7b700 1 -- 172.21.15.40:0/2784325228 wait complete. 2024-02-16T19:32:10.714 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:32:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.807+0000 7f1249581700 1 -- 172.21.15.40:0/245857739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440663e0 msgr2=0x7f12440ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.807+0000 7f1249581700 1 --2- 172.21.15.40:0/245857739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440663e0 0x7f12440ffaf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1234004940 tx=0x7f1234009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.807+0000 7f1249581700 1 -- 172.21.15.40:0/245857739 shutdown_connections 2024-02-16T19:32:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.807+0000 7f1249581700 1 --2- 172.21.15.40:0/245857739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12441000c0 0x7f1244102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.807+0000 7f1249581700 1 --2- 172.21.15.40:0/245857739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440663e0 0x7f12440ffaf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.807+0000 7f1249581700 1 -- 172.21.15.40:0/245857739 >> 172.21.15.40:0/245857739 conn(0x7f12440f74e0 msgr2=0x7f12440f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.808+0000 7f1249581700 1 -- 172.21.15.40:0/245857739 shutdown_connections 2024-02-16T19:32:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.808+0000 7f1249581700 1 -- 172.21.15.40:0/245857739 wait complete. 2024-02-16T19:32:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.808+0000 7f1249581700 1 Processor -- start 2024-02-16T19:32:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.808+0000 7f1249581700 1 -- start start 2024-02-16T19:32:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f1249581700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12440663e0 0x7f124410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f1249581700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441000c0 0x7f124410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f1249581700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124410e550 con 0x7f12441000c0 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f1249581700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124410ed90 con 0x7f12440663e0 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f12427fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441000c0 0x7f124410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f12427fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441000c0 0x7f124410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43092/0 (socket says 172.21.15.40:43092) 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f12427fc700 1 -- 172.21.15.40:0/3454285441 learned_addr learned my addr 172.21.15.40:0/3454285441 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f1242ffd700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12440663e0 0x7f124410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f12427fc700 1 -- 172.21.15.40:0/3454285441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12440663e0 msgr2=0x7f124410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f12427fc700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12440663e0 0x7f124410d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.809+0000 7f12427fc700 1 -- 172.21.15.40:0/3454285441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1234005040 con 0x7f12441000c0 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.810+0000 7f12427fc700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441000c0 0x7f124410dec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f123800b790 tx=0x7f1238009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.810+0000 7f122b7fe700 1 -- 172.21.15.40:0/3454285441 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123800f6f0 con 0x7f12441000c0 2024-02-16T19:32:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.810+0000 7f122b7fe700 1 -- 172.21.15.40:0/3454285441 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f123800fd20 con 0x7f12441000c0 2024-02-16T19:32:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.810+0000 7f122b7fe700 1 -- 172.21.15.40:0/3454285441 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12380103a0 con 0x7f12441000c0 2024-02-16T19:32:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.810+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124410f070 con 0x7f12441000c0 2024-02-16T19:32:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.810+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124410f5b0 con 0x7f12441000c0 2024-02-16T19:32:10.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.811+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1244107af0 con 0x7f12441000c0 2024-02-16T19:32:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.812+0000 7f122b7fe700 1 -- 172.21.15.40:0/3454285441 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1238011400 con 0x7f12441000c0 2024-02-16T19:32:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.812+0000 7f122b7fe700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f122c06d7e0 0x7f122c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.812+0000 7f122b7fe700 1 -- 172.21.15.40:0/3454285441 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f123808cbc0 con 0x7f12441000c0 2024-02-16T19:32:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.812+0000 7f1242ffd700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f122c06d7e0 0x7f122c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.813+0000 7f1242ffd700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f122c06d7e0 0x7f122c06fc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1234004f90 tx=0x7f1234009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:10.817+0000 7f122b7fe700 1 -- 172.21.15.40:0/3454285441 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f123805a060 con 0x7f12441000c0 2024-02-16T19:32:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.091+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f124410f8f0 con 0x7f122c06d7e0 2024-02-16T19:32:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.092+0000 7f122b7fe700 1 -- 172.21.15.40:0/3454285441 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f124410f8f0 con 0x7f122c06d7e0 2024-02-16T19:32:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f122c06d7e0 msgr2=0x7f122c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f122c06d7e0 0x7f122c06fc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1234004f90 tx=0x7f1234009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441000c0 msgr2=0x7f124410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441000c0 0x7f124410dec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f123800b790 tx=0x7f1238009220 comp rx=0 tx=0).stop 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 shutdown_connections 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12440663e0 0x7f124410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f122c06d7e0 0x7f122c06fc90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 --2- 172.21.15.40:0/3454285441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441000c0 0x7f124410dec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 >> 172.21.15.40:0/3454285441 conn(0x7f12440f74e0 msgr2=0x7f12440f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 shutdown_connections 2024-02-16T19:32:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.094+0000 7f1249581700 1 -- 172.21.15.40:0/3454285441 wait complete. 2024-02-16T19:32:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 -- 172.21.15.40:0/1630022051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 msgr2=0x7f71500fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 --2- 172.21.15.40:0/1630022051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 0x7f71500fe550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7144004940 tx=0x7f7144009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 -- 172.21.15.40:0/1630022051 shutdown_connections 2024-02-16T19:32:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 --2- 172.21.15.40:0/1630022051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 0x7f71500fe550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 --2- 172.21.15.40:0/1630022051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71500fcef0 0x7f71500fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 -- 172.21.15.40:0/1630022051 >> 172.21.15.40:0/1630022051 conn(0x7f71500f8620 msgr2=0x7f71500faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 -- 172.21.15.40:0/1630022051 shutdown_connections 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 -- 172.21.15.40:0/1630022051 wait complete. 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 Processor -- start 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.197+0000 7f71555e7700 1 -- start start 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f71555e7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71500fcef0 0x7f7150071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f71555e7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 0x7f71500702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f71555e7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7150070970 con 0x7f71500fe0e0 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f71555e7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7150070ae0 con 0x7f71500fcef0 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f714e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 0x7f71500702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f714e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 0x7f71500702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43112/0 (socket says 172.21.15.40:43112) 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f714e7fc700 1 -- 172.21.15.40:0/2938513161 learned_addr learned my addr 172.21.15.40:0/2938513161 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f714effd700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71500fcef0 0x7f7150071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f714e7fc700 1 -- 172.21.15.40:0/2938513161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71500fcef0 msgr2=0x7f7150071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f714e7fc700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71500fcef0 0x7f7150071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f714e7fc700 1 -- 172.21.15.40:0/2938513161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7144005040 con 0x7f71500fe0e0 2024-02-16T19:32:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f714e7fc700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 0x7f71500702e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f71440093b0 tx=0x7f7144009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f71377fe700 1 -- 172.21.15.40:0/2938513161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714400e040 con 0x7f71500fe0e0 2024-02-16T19:32:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f71377fe700 1 -- 172.21.15.40:0/2938513161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f714400bcc0 con 0x7f71500fe0e0 2024-02-16T19:32:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.198+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7150070d60 con 0x7f71500fe0e0 2024-02-16T19:32:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.199+0000 7f71377fe700 1 -- 172.21.15.40:0/2938513161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714401c630 con 0x7f71500fe0e0 2024-02-16T19:32:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.199+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71501159c0 con 0x7f71500fe0e0 2024-02-16T19:32:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.199+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715004bed0 con 0x7f71500fe0e0 2024-02-16T19:32:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.200+0000 7f71377fe700 1 -- 172.21.15.40:0/2938513161 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f71440139f0 con 0x7f71500fe0e0 2024-02-16T19:32:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.200+0000 7f71377fe700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f713806d7e0 0x7f713806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.201+0000 7f71377fe700 1 -- 172.21.15.40:0/2938513161 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f714408e3d0 con 0x7f71500fe0e0 2024-02-16T19:32:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.201+0000 7f714effd700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f713806d7e0 0x7f713806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.201+0000 7f714effd700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f713806d7e0 0x7f713806fc90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7140006fd0 tx=0x7f7140008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.205+0000 7f71377fe700 1 -- 172.21.15.40:0/2938513161 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f714405b730 con 0x7f71500fe0e0 2024-02-16T19:32:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.422+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7150102ac0 con 0x7f713806d7e0 2024-02-16T19:32:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.432+0000 7f71377fe700 1 -- 172.21.15.40:0/2938513161 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7150102ac0 con 0x7f713806d7e0 2024-02-16T19:32:11.433 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:32:11.433 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (6m) 2m ago 15m 25.3M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:32:11.433 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (4m) 2m ago 15m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:32:11.433 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (4m) 49s ago 14m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:32:11.433 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (6m) 2m ago 15m 81.7M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:32:11.433 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (10m) 2m ago 10m 126M - 17.2.7-513-gb88fae66 870a0944ccb8 3c8179f31791 2024-02-16T19:32:11.433 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 starting - - - - 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (10m) 49s ago 10m 17.8M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (10m) 49s ago 10m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (6m) 2m ago 17m 500M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (5m) 49s ago 14m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (5m) 2m ago 17m 47.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (4m) 49s ago 14m 36.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (7m) 2m ago 15m 15.7M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (6m) 49s ago 14m 15.6M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (3m) 2m ago 13m 96.1M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (3m) 2m ago 13m 99.2M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (2m) 2m ago 12m 12.4M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (2m) 49s ago 12m 97.7M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:32:11.434 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (87s) 49s ago 12m 93.3M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:32:11.435 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (53s) 49s ago 11m 12.4M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 807b119a828c 2024-02-16T19:32:11.435 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (5m) 2m ago 15m 53.8M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:32:11.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f713806d7e0 msgr2=0x7f713806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f713806d7e0 0x7f713806fc90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7140006fd0 tx=0x7f7140008040 comp rx=0 tx=0).stop 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 msgr2=0x7f71500702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 0x7f71500702e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f71440093b0 tx=0x7f7144009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 shutdown_connections 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71500fcef0 0x7f7150071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f713806d7e0 0x7f713806fc90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 --2- 172.21.15.40:0/2938513161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71500fe0e0 0x7f71500702e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 >> 172.21.15.40:0/2938513161 conn(0x7f71500f8620 msgr2=0x7f71501013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 shutdown_connections 2024-02-16T19:32:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.434+0000 7f71555e7700 1 -- 172.21.15.40:0/2938513161 wait complete. 2024-02-16T19:32:11.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.536+0000 7f86b69ee700 1 -- 172.21.15.40:0/3843293177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b0066c70 msgr2=0x7f86b00ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.536+0000 7f86b69ee700 1 --2- 172.21.15.40:0/3843293177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b0066c70 0x7f86b00ffaf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86a0004940 tx=0x7f86a0009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.536+0000 7f86b69ee700 1 -- 172.21.15.40:0/3843293177 shutdown_connections 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.536+0000 7f86b69ee700 1 --2- 172.21.15.40:0/3843293177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86b01000c0 0x7f86b0102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.536+0000 7f86b69ee700 1 --2- 172.21.15.40:0/3843293177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b0066c70 0x7f86b00ffaf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.536+0000 7f86b69ee700 1 -- 172.21.15.40:0/3843293177 >> 172.21.15.40:0/3843293177 conn(0x7f86b00f7520 msgr2=0x7f86b00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.537+0000 7f86b69ee700 1 -- 172.21.15.40:0/3843293177 shutdown_connections 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.537+0000 7f86b69ee700 1 -- 172.21.15.40:0/3843293177 wait complete. 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.537+0000 7f86b69ee700 1 Processor -- start 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.537+0000 7f86b69ee700 1 -- start start 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.537+0000 7f86b69ee700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86b0066c70 0x7f86b010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.537+0000 7f86b69ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b01000c0 0x7f86b010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.537+0000 7f86b69ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b010e4c0 con 0x7f86b01000c0 2024-02-16T19:32:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.537+0000 7f86b69ee700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b010ed60 con 0x7f86b0066c70 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86af7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b01000c0 0x7f86b010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86af7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b01000c0 0x7f86b010dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43122/0 (socket says 172.21.15.40:43122) 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86af7fe700 1 -- 172.21.15.40:0/2689111859 learned_addr learned my addr 172.21.15.40:0/2689111859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86affff700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86b0066c70 0x7f86b010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86af7fe700 1 -- 172.21.15.40:0/2689111859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86b0066c70 msgr2=0x7f86b010d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86af7fe700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86b0066c70 0x7f86b010d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86af7fe700 1 -- 172.21.15.40:0/2689111859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a0005040 con 0x7f86b01000c0 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86af7fe700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b01000c0 0x7f86b010dec0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8698002a50 tx=0x7f8698004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86acff9700 1 -- 172.21.15.40:0/2689111859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8698007cc0 con 0x7f86b01000c0 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86acff9700 1 -- 172.21.15.40:0/2689111859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f869800f040 con 0x7f86b01000c0 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b010f040 con 0x7f86b01000c0 2024-02-16T19:32:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86acff9700 1 -- 172.21.15.40:0/2689111859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f869800d9c0 con 0x7f86b01000c0 2024-02-16T19:32:11.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.538+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b010f580 con 0x7f86b01000c0 2024-02-16T19:32:11.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.539+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b0107a70 con 0x7f86b01000c0 2024-02-16T19:32:11.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.540+0000 7f86acff9700 1 -- 172.21.15.40:0/2689111859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8698009730 con 0x7f86b01000c0 2024-02-16T19:32:11.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.540+0000 7f86acff9700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f869c06d7e0 0x7f869c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.540+0000 7f86acff9700 1 -- 172.21.15.40:0/2689111859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f869808ccc0 con 0x7f86b01000c0 2024-02-16T19:32:11.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.540+0000 7f86affff700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f869c06d7e0 0x7f869c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.541+0000 7f86affff700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f869c06d7e0 0x7f869c06fc90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b3b0 tx=0x7f86a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:11.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.545+0000 7f86acff9700 1 -- 172.21.15.40:0/2689111859 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8698092020 con 0x7f86b01000c0 2024-02-16T19:32:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.853+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f86b0105a30 con 0x7f86b01000c0 2024-02-16T19:32:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.854+0000 7f86acff9700 1 -- 172.21.15.40:0/2689111859 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f8698059d10 con 0x7f86b01000c0 2024-02-16T19:32:11.855 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:32:11.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:32:11.855 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 3, 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:32:11.856 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:32:11.857 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 3, 2024-02-16T19:32:11.857 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 11 2024-02-16T19:32:11.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:32:11.857 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:32:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f869c06d7e0 msgr2=0x7f869c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f869c06d7e0 0x7f869c06fc90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b3b0 tx=0x7f86a0009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:11.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b01000c0 msgr2=0x7f86b010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b01000c0 0x7f86b010dec0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8698002a50 tx=0x7f8698004600 comp rx=0 tx=0).stop 2024-02-16T19:32:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 shutdown_connections 2024-02-16T19:32:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86b0066c70 0x7f86b010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f869c06d7e0 0x7f869c06fc90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 --2- 172.21.15.40:0/2689111859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86b01000c0 0x7f86b010dec0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 >> 172.21.15.40:0/2689111859 conn(0x7f86b00f7520 msgr2=0x7f86b00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 shutdown_connections 2024-02-16T19:32:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.856+0000 7f86b69ee700 1 -- 172.21.15.40:0/2689111859 wait complete. 2024-02-16T19:32:11.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:11 smithi040 ceph-mon[94210]: pgmap v188: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:11.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:11 smithi040 ceph-mon[94210]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:11.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:11 smithi040 ceph-mon[94210]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:11.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 -- 172.21.15.40:0/4145935351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fce10 msgr2=0x7f65240fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 --2- 172.21.15.40:0/4145935351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fce10 0x7f65240fd260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6510004940 tx=0x7f6510009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:11.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 -- 172.21.15.40:0/4145935351 shutdown_connections 2024-02-16T19:32:11.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 --2- 172.21.15.40:0/4145935351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fce10 0x7f65240fd260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 --2- 172.21.15.40:0/4145935351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65240fc430 0x7f65240fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 -- 172.21.15.40:0/4145935351 >> 172.21.15.40:0/4145935351 conn(0x7f65240f6ca0 msgr2=0x7f65240f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:11.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 -- 172.21.15.40:0/4145935351 shutdown_connections 2024-02-16T19:32:11.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 -- 172.21.15.40:0/4145935351 wait complete. 2024-02-16T19:32:11.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 Processor -- start 2024-02-16T19:32:11.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.962+0000 7f652aec7700 1 -- start start 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f652aec7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fc430 0x7f6524110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f652aec7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65240fce10 0x7f6524112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f652aec7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65240fbe70 con 0x7f65240fc430 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f652aec7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65240fbfb0 con 0x7f65240fce10 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6528c63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fc430 0x7f6524110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6528c63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fc430 0x7f6524110710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43142/0 (socket says 172.21.15.40:43142) 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6528c63700 1 -- 172.21.15.40:0/3537529631 learned_addr learned my addr 172.21.15.40:0/3537529631 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6523fff700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65240fce10 0x7f6524112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6528c63700 1 -- 172.21.15.40:0/3537529631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65240fce10 msgr2=0x7f6524112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6528c63700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65240fce10 0x7f6524112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6528c63700 1 -- 172.21.15.40:0/3537529631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510005040 con 0x7f65240fc430 2024-02-16T19:32:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6523fff700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65240fce10 0x7f6524112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:32:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f6528c63700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fc430 0x7f6524110710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6518002a80 tx=0x7f6518004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.963+0000 7f65217fa700 1 -- 172.21.15.40:0/3537529631 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6518007cc0 con 0x7f65240fc430 2024-02-16T19:32:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.964+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6524113260 con 0x7f65240fc430 2024-02-16T19:32:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.964+0000 7f65217fa700 1 -- 172.21.15.40:0/3537529631 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f651800f040 con 0x7f65240fc430 2024-02-16T19:32:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.964+0000 7f65217fa700 1 -- 172.21.15.40:0/3537529631 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f651800dad0 con 0x7f65240fc430 2024-02-16T19:32:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.964+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65241137a0 con 0x7f65240fc430 2024-02-16T19:32:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.964+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652404bed0 con 0x7f65240fc430 2024-02-16T19:32:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.965+0000 7f65217fa700 1 -- 172.21.15.40:0/3537529631 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f651800dc50 con 0x7f65240fc430 2024-02-16T19:32:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.966+0000 7f65217fa700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f651406d7e0 0x7f651406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.966+0000 7f65217fa700 1 -- 172.21.15.40:0/3537529631 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f651808cc60 con 0x7f65240fc430 2024-02-16T19:32:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.966+0000 7f6523fff700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f651406d7e0 0x7f651406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.966+0000 7f6523fff700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f651406d7e0 0x7f651406fc90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6510004560 tx=0x7f6510009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:11.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:11.970+0000 7f65217fa700 1 -- 172.21.15.40:0/3537529631 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65180564b0 con 0x7f65240fc430 2024-02-16T19:32:11.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:11 smithi163 ceph-mon[81637]: pgmap v188: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:11.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:11 smithi163 ceph-mon[81637]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:11.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:11 smithi163 ceph-mon[81637]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:12.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.223+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6524113a20 con 0x7f65240fc430 2024-02-16T19:32:12.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.224+0000 7f65217fa700 1 -- 172.21.15.40:0/3537529631 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1849 (secure 0 0 0) 0x7f6518059b70 con 0x7f65240fc430 2024-02-16T19:32:12.225 INFO:teuthology.orchestra.run.smithi040.stdout:e26 2024-02-16T19:32:12.225 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:32:12.225 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:32:12.225 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 23 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:31:48.931474+0000 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:32:12.226 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 46 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 1 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:in 0 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=14486} 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:32:12.227 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 1 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.40:6828/2154720438,v1:172.21.15.40:6829/2154720438] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:32:12.228 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:32:12.229 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:32:12.229 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{-1:44285} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6824/2272386057,v1:172.21.15.163:6825/2272386057] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f651406d7e0 msgr2=0x7f651406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f651406d7e0 0x7f651406fc90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6510004560 tx=0x7f6510009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fc430 msgr2=0x7f6524110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fc430 0x7f6524110710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6518002a80 tx=0x7f6518004600 comp rx=0 tx=0).stop 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 shutdown_connections 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f651406d7e0 0x7f651406fc90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65240fc430 0x7f6524110710 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 --2- 172.21.15.40:0/3537529631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65240fce10 0x7f6524112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 >> 172.21.15.40:0/3537529631 conn(0x7f65240f6ca0 msgr2=0x7f6524100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:12.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 shutdown_connections 2024-02-16T19:32:12.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.226+0000 7f652aec7700 1 -- 172.21.15.40:0/3537529631 wait complete. 2024-02-16T19:32:12.231 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 26 2024-02-16T19:32:12.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.338+0000 7f871995f700 1 -- 172.21.15.40:0/1590700026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 msgr2=0x7f87140ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:12.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.339+0000 7f871995f700 1 --2- 172.21.15.40:0/1590700026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 0x7f87140ffaf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8704004940 tx=0x7f8704009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:12.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.339+0000 7f871995f700 1 -- 172.21.15.40:0/1590700026 shutdown_connections 2024-02-16T19:32:12.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.339+0000 7f871995f700 1 --2- 172.21.15.40:0/1590700026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87141000c0 0x7f8714102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.339+0000 7f871995f700 1 --2- 172.21.15.40:0/1590700026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 0x7f87140ffaf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.339+0000 7f871995f700 1 -- 172.21.15.40:0/1590700026 >> 172.21.15.40:0/1590700026 conn(0x7f87140f7500 msgr2=0x7f87140f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:12.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.339+0000 7f871995f700 1 -- 172.21.15.40:0/1590700026 shutdown_connections 2024-02-16T19:32:12.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.339+0000 7f871995f700 1 -- 172.21.15.40:0/1590700026 wait complete. 2024-02-16T19:32:12.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.339+0000 7f871995f700 1 Processor -- start 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f871995f700 1 -- start start 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f871995f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 0x7f871410d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f871995f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87141000c0 0x7f871410de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f871995f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871410e440 con 0x7f8714066c70 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f871995f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871410ece0 con 0x7f87141000c0 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f8712ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 0x7f871410d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f8712ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 0x7f871410d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43158/0 (socket says 172.21.15.40:43158) 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f8712ffd700 1 -- 172.21.15.40:0/847314941 learned_addr learned my addr 172.21.15.40:0/847314941 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f87127fc700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87141000c0 0x7f871410de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f8712ffd700 1 -- 172.21.15.40:0/847314941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87141000c0 msgr2=0x7f871410de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f8712ffd700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87141000c0 0x7f871410de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f8712ffd700 1 -- 172.21.15.40:0/847314941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8704005040 con 0x7f8714066c70 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.340+0000 7f87127fc700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87141000c0 0x7f871410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:32:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.341+0000 7f8712ffd700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 0x7f871410d900 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8704000c00 tx=0x7f8704009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:12.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.341+0000 7f86fb7fe700 1 -- 172.21.15.40:0/847314941 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8704021070 con 0x7f8714066c70 2024-02-16T19:32:12.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.341+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871410ef60 con 0x7f8714066c70 2024-02-16T19:32:12.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.341+0000 7f86fb7fe700 1 -- 172.21.15.40:0/847314941 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f870401c400 con 0x7f8714066c70 2024-02-16T19:32:12.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.341+0000 7f86fb7fe700 1 -- 172.21.15.40:0/847314941 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f870400e070 con 0x7f8714066c70 2024-02-16T19:32:12.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.341+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871410f440 con 0x7f8714066c70 2024-02-16T19:32:12.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.342+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871404bed0 con 0x7f8714066c70 2024-02-16T19:32:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.343+0000 7f86fb7fe700 1 -- 172.21.15.40:0/847314941 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f870401ca50 con 0x7f8714066c70 2024-02-16T19:32:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.344+0000 7f86fb7fe700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f86fc06d7f0 0x7f86fc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.344+0000 7f86fb7fe700 1 -- 172.21.15.40:0/847314941 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f870408e070 con 0x7f8714066c70 2024-02-16T19:32:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.344+0000 7f87127fc700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f86fc06d7f0 0x7f86fc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.344+0000 7f87127fc700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f86fc06d7f0 0x7f86fc06fca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8708009b80 tx=0x7f870800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:12.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.351+0000 7f86fb7fe700 1 -- 172.21.15.40:0/847314941 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f870405b470 con 0x7f8714066c70 2024-02-16T19:32:12.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.579+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8714100d20 con 0x7f86fc06d7f0 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.581+0000 7f86fb7fe700 1 -- 172.21.15.40:0/847314941 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8714100d20 con 0x7f86fc06d7f0 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "osd", 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:32:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:32:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:32:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "13/21 daemons upgraded", 2024-02-16T19:32:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T19:32:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:32:12.583 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:32:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f86fc06d7f0 msgr2=0x7f86fc06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f86fc06d7f0 0x7f86fc06fca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8708009b80 tx=0x7f870800b040 comp rx=0 tx=0).stop 2024-02-16T19:32:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 msgr2=0x7f871410d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 0x7f871410d900 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8704000c00 tx=0x7f8704009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 shutdown_connections 2024-02-16T19:32:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f86fc06d7f0 0x7f86fc06fca0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8714066c70 0x7f871410d900 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 --2- 172.21.15.40:0/847314941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87141000c0 0x7f871410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 >> 172.21.15.40:0/847314941 conn(0x7f87140f7500 msgr2=0x7f87140f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:12.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 shutdown_connections 2024-02-16T19:32:12.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.584+0000 7f871995f700 1 -- 172.21.15.40:0/847314941 wait complete. 2024-02-16T19:32:12.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.689+0000 7f876124f700 1 -- 172.21.15.40:0/2433330881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 msgr2=0x7f875c0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:12.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.689+0000 7f876124f700 1 --2- 172.21.15.40:0/2433330881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 0x7f875c0fc250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f874c004940 tx=0x7f874c009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:12.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.689+0000 7f876124f700 1 -- 172.21.15.40:0/2433330881 shutdown_connections 2024-02-16T19:32:12.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.689+0000 7f876124f700 1 --2- 172.21.15.40:0/2433330881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f875c0fcfb0 0x7f875c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.689+0000 7f876124f700 1 --2- 172.21.15.40:0/2433330881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 0x7f875c0fc250 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.689+0000 7f876124f700 1 -- 172.21.15.40:0/2433330881 >> 172.21.15.40:0/2433330881 conn(0x7f875c0f7520 msgr2=0x7f875c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:12.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.689+0000 7f876124f700 1 -- 172.21.15.40:0/2433330881 shutdown_connections 2024-02-16T19:32:12.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.689+0000 7f876124f700 1 -- 172.21.15.40:0/2433330881 wait complete. 2024-02-16T19:32:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f876124f700 1 Processor -- start 2024-02-16T19:32:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f876124f700 1 -- start start 2024-02-16T19:32:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f876124f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 0x7f875c110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f876124f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f875c0fcfb0 0x7f875c112b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f876124f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c066d50 con 0x7f875c0fbe40 2024-02-16T19:32:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f876124f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c066e90 con 0x7f875c0fcfb0 2024-02-16T19:32:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f875ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 0x7f875c110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f875ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 0x7f875c110600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43174/0 (socket says 172.21.15.40:43174) 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.690+0000 7f875ad9d700 1 -- 172.21.15.40:0/2267510387 learned_addr learned my addr 172.21.15.40:0/2267510387 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f875a59c700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f875c0fcfb0 0x7f875c112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f875ad9d700 1 -- 172.21.15.40:0/2267510387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f875c0fcfb0 msgr2=0x7f875c112b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f875ad9d700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f875c0fcfb0 0x7f875c112b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f875ad9d700 1 -- 172.21.15.40:0/2267510387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c005040 con 0x7f875c0fbe40 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f875ad9d700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 0x7f875c110600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f874c0093e0 tx=0x7f874c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f87437fe700 1 -- 172.21.15.40:0/2267510387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f874c00e040 con 0x7f875c0fbe40 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f87437fe700 1 -- 172.21.15.40:0/2267510387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f874c00bcc0 con 0x7f875c0fbe40 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875c113150 con 0x7f875c0fbe40 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f87437fe700 1 -- 172.21.15.40:0/2267510387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f874c01c630 con 0x7f875c0fbe40 2024-02-16T19:32:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.691+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875c113690 con 0x7f875c0fbe40 2024-02-16T19:32:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.692+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875c04bed0 con 0x7f875c0fbe40 2024-02-16T19:32:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.693+0000 7f87437fe700 1 -- 172.21.15.40:0/2267510387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f874c013980 con 0x7f875c0fbe40 2024-02-16T19:32:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.694+0000 7f87437fe700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f874406d7e0 0x7f874406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.694+0000 7f87437fe700 1 -- 172.21.15.40:0/2267510387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f874c08e250 con 0x7f875c0fbe40 2024-02-16T19:32:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.694+0000 7f875a59c700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f874406d7e0 0x7f874406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.694+0000 7f875a59c700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f874406d7e0 0x7f874406fc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8750007fd0 tx=0x7f8750009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:12.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.698+0000 7f87437fe700 1 -- 172.21.15.40:0/2267510387 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f874c057b20 con 0x7f875c0fbe40 2024-02-16T19:32:12.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:12 smithi040 ceph-mon[94210]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:12.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:12.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:12.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:12 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2689111859' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:12.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:12 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3537529631' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:32:12.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.998+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f875c113a90 con 0x7f875c0fbe40 2024-02-16T19:32:12.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:12.998+0000 7f87437fe700 1 -- 172.21.15.40:0/2267510387 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f874c05b1e0 con 0x7f875c0fbe40 2024-02-16T19:32:12.999 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f874406d7e0 msgr2=0x7f874406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f874406d7e0 0x7f874406fc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8750007fd0 tx=0x7f8750009040 comp rx=0 tx=0).stop 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 msgr2=0x7f875c110600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 0x7f875c110600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f874c0093e0 tx=0x7f874c009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 shutdown_connections 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f874406d7e0 0x7f874406fc90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f875c0fbe40 0x7f875c110600 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 --2- 172.21.15.40:0/2267510387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f875c0fcfb0 0x7f875c112b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 >> 172.21.15.40:0/2267510387 conn(0x7f875c0f7520 msgr2=0x7f875c100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 shutdown_connections 2024-02-16T19:32:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:13.000+0000 7f876124f700 1 -- 172.21.15.40:0/2267510387 wait complete. 2024-02-16T19:32:13.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:12 smithi163 ceph-mon[81637]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:13.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:12 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:13.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:12 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:13.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:12 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2689111859' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:13.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:12 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3537529631' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:32:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:13 smithi040 ceph-mon[94210]: pgmap v189: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:13 smithi040 ceph-mon[94210]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:13 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:14.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:13 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2267510387' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:32:14.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:13 smithi163 ceph-mon[81637]: pgmap v189: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:14.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:13 smithi163 ceph-mon[81637]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:14.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:14.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:13 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:14.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:13 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2267510387' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:32:15.947 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:15 smithi040 ceph-mon[94210]: pgmap v190: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:16.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:15 smithi163 ceph-mon[81637]: pgmap v190: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: pgmap v191: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: Detected new or changed devices on smithi040 2024-02-16T19:32:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi040.bmpjpk"]}]: dispatch 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: Upgrade: It appears safe to stop mds.cephfs.smithi040.bmpjpk 2024-02-16T19:32:17.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: pgmap v191: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: Detected new or changed devices on smithi040 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi040.bmpjpk"]}]: dispatch 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: Upgrade: It appears safe to stop mds.cephfs.smithi040.bmpjpk 2024-02-16T19:32:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:19.363 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:19 smithi040 ceph-mon[94210]: Upgrade: Updating mds.cephfs.smithi040.bmpjpk 2024-02-16T19:32:19.363 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:19.363 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.bmpjpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:32:19.363 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:19.363 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:19 smithi040 ceph-mon[94210]: Deploying daemon mds.cephfs.smithi040.bmpjpk on smithi040 2024-02-16T19:32:19.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:19 smithi163 ceph-mon[81637]: Upgrade: Updating mds.cephfs.smithi040.bmpjpk 2024-02-16T19:32:19.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:19.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.bmpjpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:32:19.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:19.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:19 smithi163 ceph-mon[81637]: Deploying daemon mds.cephfs.smithi040.bmpjpk on smithi040 2024-02-16T19:32:20.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:20 smithi040 ceph-mon[94210]: pgmap v192: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:20.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:20 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[94205]: 2024-02-16T19:32:20.151+0000 7fd6fd9b1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:32:20.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:20 smithi163 ceph-mon[81637]: pgmap v192: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:21 smithi040 ceph-mon[94210]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:32:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:21 smithi040 ceph-mon[94210]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:32:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:21 smithi040 ceph-mon[94210]: osdmap e81: 6 total, 6 up, 6 in 2024-02-16T19:32:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:21 smithi040 ceph-mon[94210]: Standby daemon mds.cephfs.smithi163.qovvde assigned to filesystem cephfs as rank 0 2024-02-16T19:32:21.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:21 smithi040 ceph-mon[94210]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:32:21.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:21 smithi040 ceph-mon[94210]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:32:21.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:21 smithi040 ceph-mon[94210]: fsmap cephfs:1/1 {0=cephfs.smithi163.qovvde=up:replay} 2 up:standby 2024-02-16T19:32:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:21 smithi163 ceph-mon[81637]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:32:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:21 smithi163 ceph-mon[81637]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:32:21.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:21 smithi163 ceph-mon[81637]: osdmap e81: 6 total, 6 up, 6 in 2024-02-16T19:32:21.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:21 smithi163 ceph-mon[81637]: Standby daemon mds.cephfs.smithi163.qovvde assigned to filesystem cephfs as rank 0 2024-02-16T19:32:21.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:21 smithi163 ceph-mon[81637]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:32:21.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:21 smithi163 ceph-mon[81637]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:32:21.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:21 smithi163 ceph-mon[81637]: fsmap cephfs:1/1 {0=cephfs.smithi163.qovvde=up:replay} 2 up:standby 2024-02-16T19:32:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:22 smithi163 ceph-mon[81637]: pgmap v194: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:22.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:22 smithi040 ceph-mon[94210]: pgmap v194: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:24.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:24 smithi163 ceph-mon[81637]: pgmap v195: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-02-16T19:32:24.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:24 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:32:24.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:24 smithi040 ceph-mon[94210]: pgmap v195: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-02-16T19:32:24.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:24 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:32:26.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:26 smithi163 ceph-mon[81637]: pgmap v196: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 8.0 MiB/s rd, 3 op/s 2024-02-16T19:32:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:26 smithi040 ceph-mon[94210]: pgmap v196: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 8.0 MiB/s rd, 3 op/s 2024-02-16T19:32:27.455 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:27 smithi040 ceph-mon[94210]: pgmap v197: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-16T19:32:27.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:27 smithi163 ceph-mon[81637]: pgmap v197: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-16T19:32:28.466 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:28 smithi040 ceph-mon[94210]: reconnect by client.24309 v1:192.168.0.1:0/3707514684 after 0 2024-02-16T19:32:28.466 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:28 smithi040 ceph-mon[94210]: reconnect by client.14536 v1:192.168.0.1:0/3779740512 after 0 2024-02-16T19:32:28.466 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:28 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:reconnect 2024-02-16T19:32:28.466 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:28 smithi040 ceph-mon[94210]: fsmap cephfs:1/1 {0=cephfs.smithi163.qovvde=up:reconnect} 2 up:standby 2024-02-16T19:32:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:28 smithi163 ceph-mon[81637]: reconnect by client.24309 v1:192.168.0.1:0/3707514684 after 0 2024-02-16T19:32:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:28 smithi163 ceph-mon[81637]: reconnect by client.14536 v1:192.168.0.1:0/3779740512 after 0 2024-02-16T19:32:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:28 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:reconnect 2024-02-16T19:32:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:28 smithi163 ceph-mon[81637]: fsmap cephfs:1/1 {0=cephfs.smithi163.qovvde=up:reconnect} 2 up:standby 2024-02-16T19:32:29.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:29 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:rejoin 2024-02-16T19:32:29.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:29 smithi163 ceph-mon[81637]: fsmap cephfs:1/1 {0=cephfs.smithi163.qovvde=up:rejoin} 2 up:standby 2024-02-16T19:32:29.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:29 smithi163 ceph-mon[81637]: daemon mds.cephfs.smithi163.qovvde is now active in filesystem cephfs as rank 0 2024-02-16T19:32:29.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:29 smithi163 ceph-mon[81637]: pgmap v198: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-02-16T19:32:29.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:29.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:29.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:29 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:rejoin 2024-02-16T19:32:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:29 smithi040 ceph-mon[94210]: fsmap cephfs:1/1 {0=cephfs.smithi163.qovvde=up:rejoin} 2 up:standby 2024-02-16T19:32:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:29 smithi040 ceph-mon[94210]: daemon mds.cephfs.smithi163.qovvde is now active in filesystem cephfs as rank 0 2024-02-16T19:32:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:29 smithi040 ceph-mon[94210]: pgmap v198: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-02-16T19:32:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:30 smithi163 ceph-mon[81637]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:32:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:30 smithi163 ceph-mon[81637]: Cluster is now healthy 2024-02-16T19:32:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:30 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:active 2024-02-16T19:32:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:30 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] up:boot 2024-02-16T19:32:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:30 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi163.qovvde=up:active} 3 up:standby 2024-02-16T19:32:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:30 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:32:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:30 smithi040 ceph-mon[94210]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:32:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:30 smithi040 ceph-mon[94210]: Cluster is now healthy 2024-02-16T19:32:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:30 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] up:active 2024-02-16T19:32:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:30 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] up:boot 2024-02-16T19:32:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:30 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi163.qovvde=up:active} 3 up:standby 2024-02-16T19:32:30.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:30 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:32:31.467 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:31 smithi040 ceph-mon[94210]: pgmap v199: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 101 B/s wr, 6 op/s 2024-02-16T19:32:31.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:31 smithi163 ceph-mon[81637]: pgmap v199: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 101 B/s wr, 6 op/s 2024-02-16T19:32:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:33 smithi163 ceph-mon[81637]: pgmap v200: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 341 B/s wr, 7 op/s 2024-02-16T19:32:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:33 smithi040 ceph-mon[94210]: pgmap v200: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 341 B/s wr, 7 op/s 2024-02-16T19:32:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:34.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:34 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:34.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:34 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:35 smithi040 ceph-mon[94210]: pgmap v201: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-02-16T19:32:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:35 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:35 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:35 smithi163 ceph-mon[81637]: pgmap v201: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-02-16T19:32:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:35 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:35 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:37.616 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:37 smithi040 ceph-mon[94210]: pgmap v202: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-02-16T19:32:37.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:37 smithi163 ceph-mon[81637]: pgmap v202: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-02-16T19:32:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:39.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:39.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:39.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:39.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:39.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:39.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:39 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi163.rggdhh"]}]: dispatch 2024-02-16T19:32:39.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:39.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:39.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:39.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:39 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi163.rggdhh"]}]: dispatch 2024-02-16T19:32:40.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:40 smithi163 ceph-mon[81637]: Upgrade: It appears safe to stop mds.cephfs.smithi163.rggdhh 2024-02-16T19:32:40.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:40 smithi163 ceph-mon[81637]: pgmap v203: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-02-16T19:32:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:40 smithi040 ceph-mon[94210]: Upgrade: It appears safe to stop mds.cephfs.smithi163.rggdhh 2024-02-16T19:32:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:40 smithi040 ceph-mon[94210]: pgmap v203: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-02-16T19:32:41.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:41 smithi163 ceph-mon[81637]: Upgrade: Updating mds.cephfs.smithi163.rggdhh 2024-02-16T19:32:41.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:41.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.rggdhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:32:41.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:41 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:41.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:41 smithi163 ceph-mon[81637]: Deploying daemon mds.cephfs.smithi163.rggdhh on smithi163 2024-02-16T19:32:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:41 smithi040 ceph-mon[94210]: Upgrade: Updating mds.cephfs.smithi163.rggdhh 2024-02-16T19:32:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.rggdhh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:32:41.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:41 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:41.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:41 smithi040 ceph-mon[94210]: Deploying daemon mds.cephfs.smithi163.rggdhh on smithi163 2024-02-16T19:32:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:42 smithi040 ceph-mon[94210]: pgmap v204: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 557 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-02-16T19:32:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:42 smithi163 ceph-mon[81637]: pgmap v204: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 557 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 -- 172.21.15.40:0/2458924481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fe0e0 msgr2=0x7fafcc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 --2- 172.21.15.40:0/2458924481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fe0e0 0x7fafcc0fe550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafbc004940 tx=0x7fafbc009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 -- 172.21.15.40:0/2458924481 shutdown_connections 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 --2- 172.21.15.40:0/2458924481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fe0e0 0x7fafcc0fe550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 --2- 172.21.15.40:0/2458924481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafcc0fcef0 0x7fafcc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 -- 172.21.15.40:0/2458924481 >> 172.21.15.40:0/2458924481 conn(0x7fafcc0f8620 msgr2=0x7fafcc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 -- 172.21.15.40:0/2458924481 shutdown_connections 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 -- 172.21.15.40:0/2458924481 wait complete. 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 Processor -- start 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.121+0000 7fafd3f56700 1 -- start start 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd3f56700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fcef0 0x7fafcc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd3f56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafcc0fe0e0 0x7fafcc0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd3f56700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0708e0 con 0x7fafcc0fcef0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd3f56700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc070a50 con 0x7fafcc0fe0e0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd1cf2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fcef0 0x7fafcc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd1cf2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fcef0 0x7fafcc071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36170/0 (socket says 172.21.15.40:36170) 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd1cf2700 1 -- 172.21.15.40:0/3070950167 learned_addr learned my addr 172.21.15.40:0/3070950167 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd14f1700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafcc0fe0e0 0x7fafcc0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd1cf2700 1 -- 172.21.15.40:0/3070950167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafcc0fe0e0 msgr2=0x7fafcc0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd1cf2700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafcc0fe0e0 0x7fafcc0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd1cf2700 1 -- 172.21.15.40:0/3070950167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc005040 con 0x7fafcc0fcef0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd1cf2700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fcef0 0x7fafcc071e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fafc8002a80 tx=0x7fafc80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafc27fc700 1 -- 172.21.15.40:0/3070950167 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc8007cc0 con 0x7fafcc0fcef0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafc27fc700 1 -- 172.21.15.40:0/3070950167 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafc800f040 con 0x7fafcc0fcef0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc070d30 con 0x7fafcc0fcef0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafc27fc700 1 -- 172.21.15.40:0/3070950167 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc800db30 con 0x7fafcc0fcef0 2024-02-16T19:32:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.122+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc115920 con 0x7fafcc0fcef0 2024-02-16T19:32:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.123+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc04bed0 con 0x7fafcc0fcef0 2024-02-16T19:32:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.124+0000 7fafc27fc700 1 -- 172.21.15.40:0/3070950167 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fafc800dc90 con 0x7fafcc0fcef0 2024-02-16T19:32:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.125+0000 7fafc27fc700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fafb806d7e0 0x7fafb806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.125+0000 7fafd14f1700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fafb806d7e0 0x7fafb806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.125+0000 7fafc27fc700 1 -- 172.21.15.40:0/3070950167 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fafc808c720 con 0x7fafcc0fcef0 2024-02-16T19:32:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.125+0000 7fafd14f1700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fafb806d7e0 0x7fafb806fc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fafbc004f90 tx=0x7fafbc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.130+0000 7fafc27fc700 1 -- 172.21.15.40:0/3070950167 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafc8059a40 con 0x7fafcc0fcef0 2024-02-16T19:32:43.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.352+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafcc102ac0 con 0x7fafb806d7e0 2024-02-16T19:32:43.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.354+0000 7fafc27fc700 1 -- 172.21.15.40:0/3070950167 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fafcc102ac0 con 0x7fafb806d7e0 2024-02-16T19:32:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fafb806d7e0 msgr2=0x7fafb806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fafb806d7e0 0x7fafb806fc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fafbc004f90 tx=0x7fafbc009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fcef0 msgr2=0x7fafcc071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fcef0 0x7fafcc071e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fafc8002a80 tx=0x7fafc80046b0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 shutdown_connections 2024-02-16T19:32:43.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fafb806d7e0 0x7fafb806fc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafcc0fcef0 0x7fafcc071e60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 --2- 172.21.15.40:0/3070950167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafcc0fe0e0 0x7fafcc0702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 >> 172.21.15.40:0/3070950167 conn(0x7fafcc0f8620 msgr2=0x7fafcc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 shutdown_connections 2024-02-16T19:32:43.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.356+0000 7fafd3f56700 1 -- 172.21.15.40:0/3070950167 wait complete. 2024-02-16T19:32:43.368 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:32:43.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.459+0000 7f0038165700 1 -- 172.21.15.40:0/83853878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fcef0 msgr2=0x7f00300fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.459+0000 7f0038165700 1 --2- 172.21.15.40:0/83853878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fcef0 0x7f00300fd320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f002c004940 tx=0x7f002c009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:43.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 -- 172.21.15.40:0/83853878 shutdown_connections 2024-02-16T19:32:43.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 --2- 172.21.15.40:0/83853878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00300fe0e0 0x7f00300fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 --2- 172.21.15.40:0/83853878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fcef0 0x7f00300fd320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 -- 172.21.15.40:0/83853878 >> 172.21.15.40:0/83853878 conn(0x7f00300f8620 msgr2=0x7f00300faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:43.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 -- 172.21.15.40:0/83853878 shutdown_connections 2024-02-16T19:32:43.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 -- 172.21.15.40:0/83853878 wait complete. 2024-02-16T19:32:43.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 Processor -- start 2024-02-16T19:32:43.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 -- start start 2024-02-16T19:32:43.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00300fcef0 0x7f003010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fe0e0 0x7f003010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003010e4b0 con 0x7f00300fe0e0 2024-02-16T19:32:43.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0038165700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003010ed50 con 0x7f00300fcef0 2024-02-16T19:32:43.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.460+0000 7f0035700700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fe0e0 0x7f003010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0035700700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fe0e0 0x7f003010deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36196/0 (socket says 172.21.15.40:36196) 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0035700700 1 -- 172.21.15.40:0/1461628524 learned_addr learned my addr 172.21.15.40:0/1461628524 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0035f01700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00300fcef0 0x7f003010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0035700700 1 -- 172.21.15.40:0/1461628524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00300fcef0 msgr2=0x7f003010d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0035700700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00300fcef0 0x7f003010d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0035700700 1 -- 172.21.15.40:0/1461628524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c005040 con 0x7f00300fe0e0 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0035700700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fe0e0 0x7f003010deb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0020002a50 tx=0x7f00200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f00267fc700 1 -- 172.21.15.40:0/1461628524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0020007cc0 con 0x7f00300fe0e0 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003010f030 con 0x7f00300fe0e0 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f00267fc700 1 -- 172.21.15.40:0/1461628524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f002000f040 con 0x7f00300fe0e0 2024-02-16T19:32:43.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f00267fc700 1 -- 172.21.15.40:0/1461628524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f002000da30 con 0x7f00300fe0e0 2024-02-16T19:32:43.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.461+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003010f570 con 0x7f00300fe0e0 2024-02-16T19:32:43.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.462+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003004bed0 con 0x7f00300fe0e0 2024-02-16T19:32:43.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.463+0000 7f00267fc700 1 -- 172.21.15.40:0/1461628524 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f002000dbb0 con 0x7f00300fe0e0 2024-02-16T19:32:43.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.463+0000 7f00267fc700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f001c06d7f0 0x7f001c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.463+0000 7f00267fc700 1 -- 172.21.15.40:0/1461628524 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f002008ccd0 con 0x7f00300fe0e0 2024-02-16T19:32:43.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.463+0000 7f0035f01700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f001c06d7f0 0x7f001c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.464+0000 7f0035f01700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f001c06d7f0 0x7f001c06fca0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f002c00b3b0 tx=0x7f002c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:43.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.468+0000 7f00267fc700 1 -- 172.21.15.40:0/1461628524 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0020059ff0 con 0x7f00300fe0e0 2024-02-16T19:32:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.694+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f003010f9c0 con 0x7f001c06d7f0 2024-02-16T19:32:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.695+0000 7f00267fc700 1 -- 172.21.15.40:0/1461628524 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f003010f9c0 con 0x7f001c06d7f0 2024-02-16T19:32:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f001c06d7f0 msgr2=0x7f001c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f001c06d7f0 0x7f001c06fca0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f002c00b3b0 tx=0x7f002c009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fe0e0 msgr2=0x7f003010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fe0e0 0x7f003010deb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0020002a50 tx=0x7f00200046b0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 shutdown_connections 2024-02-16T19:32:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00300fcef0 0x7f003010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f001c06d7f0 0x7f001c06fca0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 --2- 172.21.15.40:0/1461628524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300fe0e0 0x7f003010deb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 >> 172.21.15.40:0/1461628524 conn(0x7f00300f8620 msgr2=0x7f00301013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 shutdown_connections 2024-02-16T19:32:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.697+0000 7f0038165700 1 -- 172.21.15.40:0/1461628524 wait complete. 2024-02-16T19:32:43.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.801+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1103943635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fcef0 msgr2=0x7f8c780fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.801+0000 7f8c7faaf700 1 --2- 172.21.15.40:0/1103943635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fcef0 0x7f8c780fd320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8c68004940 tx=0x7f8c68009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.801+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1103943635 shutdown_connections 2024-02-16T19:32:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.801+0000 7f8c7faaf700 1 --2- 172.21.15.40:0/1103943635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c780fe0e0 0x7f8c780fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.801+0000 7f8c7faaf700 1 --2- 172.21.15.40:0/1103943635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fcef0 0x7f8c780fd320 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.801+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1103943635 >> 172.21.15.40:0/1103943635 conn(0x7f8c780f8620 msgr2=0x7f8c780faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.801+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1103943635 shutdown_connections 2024-02-16T19:32:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.801+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1103943635 wait complete. 2024-02-16T19:32:43.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7faaf700 1 Processor -- start 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7faaf700 1 -- start start 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7faaf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c780fcef0 0x7f8c78071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7faaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fe0e0 0x7f8c780702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7faaf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780708e0 con 0x7f8c780fe0e0 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7faaf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78070a50 con 0x7f8c780fcef0 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7d04a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fe0e0 0x7f8c780702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7d04a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fe0e0 0x7f8c780702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36202/0 (socket says 172.21.15.40:36202) 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7d04a700 1 -- 172.21.15.40:0/1498240535 learned_addr learned my addr 172.21.15.40:0/1498240535 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7d84b700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c780fcef0 0x7f8c78071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7d04a700 1 -- 172.21.15.40:0/1498240535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c780fcef0 msgr2=0x7f8c78071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7d04a700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c780fcef0 0x7f8c78071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.802+0000 7f8c7d04a700 1 -- 172.21.15.40:0/1498240535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68005040 con 0x7f8c780fe0e0 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.803+0000 7f8c7d04a700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fe0e0 0x7f8c780702e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c830 tx=0x7f8c74004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.803+0000 7f8c6e7fc700 1 -- 172.21.15.40:0/1498240535 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c740106f0 con 0x7f8c780fe0e0 2024-02-16T19:32:43.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.803+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c78070d30 con 0x7f8c780fe0e0 2024-02-16T19:32:43.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.803+0000 7f8c6e7fc700 1 -- 172.21.15.40:0/1498240535 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c74010d20 con 0x7f8c780fe0e0 2024-02-16T19:32:43.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.803+0000 7f8c6e7fc700 1 -- 172.21.15.40:0/1498240535 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c740113a0 con 0x7f8c780fe0e0 2024-02-16T19:32:43.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.803+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c781159a0 con 0x7f8c780fe0e0 2024-02-16T19:32:43.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.804+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7804bed0 con 0x7f8c780fe0e0 2024-02-16T19:32:43.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.804+0000 7f8c6e7fc700 1 -- 172.21.15.40:0/1498240535 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8c74012400 con 0x7f8c780fe0e0 2024-02-16T19:32:43.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.805+0000 7f8c6e7fc700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c6406d7e0 0x7f8c6406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:43.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.805+0000 7f8c6e7fc700 1 -- 172.21.15.40:0/1498240535 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8c7408ce90 con 0x7f8c780fe0e0 2024-02-16T19:32:43.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.805+0000 7f8c7d84b700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c6406d7e0 0x7f8c6406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:43.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.805+0000 7f8c7d84b700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c6406d7e0 0x7f8c6406fc90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800b840 tx=0x7f8c68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:43.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:43.810+0000 7f8c6e7fc700 1 -- 172.21.15.40:0/1498240535 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c7405a230 con 0x7f8c780fe0e0 2024-02-16T19:32:44.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.022+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c78102ac0 con 0x7f8c6406d7e0 2024-02-16T19:32:44.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.032+0000 7f8c6e7fc700 1 -- 172.21.15.40:0/1498240535 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8c78102ac0 con 0x7f8c6406d7e0 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (7m) 10s ago 16m 25.1M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (4m) 10s ago 16m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (4m) 82s ago 15m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (6m) 10s ago 16m 85.7M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (15s) 10s ago 10m 12.2M - 19.0.0-1450-gf78a58c0 f081f847070a 084709bde4d3 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (37s) 10s ago 10m 12.3M - 19.0.0-1450-gf78a58c0 f081f847070a 79213f1b751f 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (10m) 82s ago 10m 17.8M - 17.2.7-513-gb88fae66 870a0944ccb8 1c013ec4419c 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (10m) 82s ago 10m 121M - 17.2.7-513-gb88fae66 870a0944ccb8 ef389170ffad 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (6m) 10s ago 17m 507M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (6m) 82s ago 15m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (5m) 10s ago 17m 54.2M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (5m) 82s ago 15m 36.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:32:44.034 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (7m) 10s ago 16m 15.7M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:32:44.035 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (6m) 82s ago 15m 15.6M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:32:44.035 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (4m) 10s ago 14m 107M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:32:44.035 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (3m) 10s ago 13m 113M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:32:44.035 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (3m) 10s ago 13m 88.7M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:32:44.035 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (2m) 82s ago 13m 97.7M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:32:44.035 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (119s) 82s ago 12m 93.3M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:32:44.035 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (85s) 82s ago 12m 12.4M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 807b119a828c 2024-02-16T19:32:44.035 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (6m) 10s ago 15m 60.9M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c6406d7e0 msgr2=0x7f8c6406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c6406d7e0 0x7f8c6406fc90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800b840 tx=0x7f8c68009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fe0e0 msgr2=0x7f8c780702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fe0e0 0x7f8c780702e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c830 tx=0x7f8c74004470 comp rx=0 tx=0).stop 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 shutdown_connections 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c780fcef0 0x7f8c78071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8c6406d7e0 0x7f8c6406fc90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 --2- 172.21.15.40:0/1498240535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c780fe0e0 0x7f8c780702e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.034+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 >> 172.21.15.40:0/1498240535 conn(0x7f8c780f8620 msgr2=0x7f8c781013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.035+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 shutdown_connections 2024-02-16T19:32:44.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.035+0000 7f8c7faaf700 1 -- 172.21.15.40:0/1498240535 wait complete. 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.136+0000 7f708b897700 1 -- 172.21.15.40:0/3104798955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 msgr2=0x7f70840fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.136+0000 7f708b897700 1 --2- 172.21.15.40:0/3104798955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 0x7f70840fc250 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f707c004940 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.136+0000 7f708b897700 1 -- 172.21.15.40:0/3104798955 shutdown_connections 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.136+0000 7f708b897700 1 --2- 172.21.15.40:0/3104798955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70840fcfb0 0x7f70840fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.136+0000 7f708b897700 1 --2- 172.21.15.40:0/3104798955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 0x7f70840fc250 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.136+0000 7f708b897700 1 -- 172.21.15.40:0/3104798955 >> 172.21.15.40:0/3104798955 conn(0x7f70840f74e0 msgr2=0x7f70840f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.136+0000 7f708b897700 1 -- 172.21.15.40:0/3104798955 shutdown_connections 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.136+0000 7f708b897700 1 -- 172.21.15.40:0/3104798955 wait complete. 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f708b897700 1 Processor -- start 2024-02-16T19:32:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f708b897700 1 -- start start 2024-02-16T19:32:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f708b897700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 0x7f708410c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f708b897700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70840fcfb0 0x7f708410cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f708b897700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708410d380 con 0x7f70840fbe40 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f708b897700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708410dc20 con 0x7f70840fcfb0 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f7089633700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 0x7f708410c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f7089633700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 0x7f708410c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36216/0 (socket says 172.21.15.40:36216) 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f7089633700 1 -- 172.21.15.40:0/3123171843 learned_addr learned my addr 172.21.15.40:0/3123171843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f7088e32700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70840fcfb0 0x7f708410cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f7089633700 1 -- 172.21.15.40:0/3123171843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70840fcfb0 msgr2=0x7f708410cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f7089633700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70840fcfb0 0x7f708410cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.137+0000 7f7089633700 1 -- 172.21.15.40:0/3123171843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c005040 con 0x7f70840fbe40 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.138+0000 7f7089633700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 0x7f708410c840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f707c0093b0 tx=0x7f707c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.138+0000 7f7081ffb700 1 -- 172.21.15.40:0/3123171843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707c00e040 con 0x7f70840fbe40 2024-02-16T19:32:44.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.138+0000 7f7081ffb700 1 -- 172.21.15.40:0/3123171843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f707c00bcc0 con 0x7f70840fbe40 2024-02-16T19:32:44.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.138+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708410dea0 con 0x7f70840fbe40 2024-02-16T19:32:44.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.138+0000 7f7081ffb700 1 -- 172.21.15.40:0/3123171843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707c01c630 con 0x7f70840fbe40 2024-02-16T19:32:44.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.138+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708410e390 con 0x7f70840fbe40 2024-02-16T19:32:44.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.139+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708404bed0 con 0x7f70840fbe40 2024-02-16T19:32:44.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.139+0000 7f7081ffb700 1 -- 172.21.15.40:0/3123171843 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f707c0139f0 con 0x7f70840fbe40 2024-02-16T19:32:44.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.140+0000 7f7081ffb700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f706c06d7e0 0x7f706c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.140+0000 7f7081ffb700 1 -- 172.21.15.40:0/3123171843 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f707c08e390 con 0x7f70840fbe40 2024-02-16T19:32:44.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.140+0000 7f7088e32700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f706c06d7e0 0x7f706c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.140+0000 7f7088e32700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f706c06d7e0 0x7f706c06fc90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7078004d60 tx=0x7f707800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.144+0000 7f7081ffb700 1 -- 172.21.15.40:0/3123171843 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f707c05b730 con 0x7f70840fbe40 2024-02-16T19:32:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:44 smithi040 ceph-mon[94210]: pgmap v205: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-02-16T19:32:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:44 smithi040 ceph-mon[94210]: osdmap e82: 6 total, 6 up, 6 in 2024-02-16T19:32:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:44 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi163.qovvde=up:active} 2 up:standby 2024-02-16T19:32:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.434+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7084104930 con 0x7f70840fbe40 2024-02-16T19:32:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.434+0000 7f7081ffb700 1 -- 172.21.15.40:0/3123171843 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f707c05b2c0 con 0x7f70840fbe40 2024-02-16T19:32:44.435 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:32:44.435 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:32:44.435 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:32:44.436 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 17.2.7-513-gb88fae66 (b88fae66e6ad896087939a6bde358f026d85f561) quincy (stable)": 1, 2024-02-16T19:32:44.437 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 12 2024-02-16T19:32:44.437 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:32:44.437 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:32:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f706c06d7e0 msgr2=0x7f706c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f706c06d7e0 0x7f706c06fc90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7078004d60 tx=0x7f707800b040 comp rx=0 tx=0).stop 2024-02-16T19:32:44.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 msgr2=0x7f708410c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 0x7f708410c840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f707c0093b0 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:44.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 shutdown_connections 2024-02-16T19:32:44.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f706c06d7e0 0x7f706c06fc90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70840fbe40 0x7f708410c840 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 --2- 172.21.15.40:0/3123171843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70840fcfb0 0x7f708410cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 >> 172.21.15.40:0/3123171843 conn(0x7f70840f74e0 msgr2=0x7f7084100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:44.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 shutdown_connections 2024-02-16T19:32:44.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.436+0000 7f708b897700 1 -- 172.21.15.40:0/3123171843 wait complete. 2024-02-16T19:32:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:44 smithi163 ceph-mon[81637]: pgmap v205: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-02-16T19:32:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:44 smithi163 ceph-mon[81637]: osdmap e82: 6 total, 6 up, 6 in 2024-02-16T19:32:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:44 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi163.qovvde=up:active} 2 up:standby 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.538+0000 7f180007c700 1 -- 172.21.15.40:0/428496724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 msgr2=0x7f17f80fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.538+0000 7f180007c700 1 --2- 172.21.15.40:0/428496724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 0x7f17f80fc250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17f4004940 tx=0x7f17f4009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.538+0000 7f180007c700 1 -- 172.21.15.40:0/428496724 shutdown_connections 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.538+0000 7f180007c700 1 --2- 172.21.15.40:0/428496724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f80fcfb0 0x7f17f80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.538+0000 7f180007c700 1 --2- 172.21.15.40:0/428496724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 0x7f17f80fc250 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.538+0000 7f180007c700 1 -- 172.21.15.40:0/428496724 >> 172.21.15.40:0/428496724 conn(0x7f17f80f74e0 msgr2=0x7f17f80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f180007c700 1 -- 172.21.15.40:0/428496724 shutdown_connections 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f180007c700 1 -- 172.21.15.40:0/428496724 wait complete. 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f180007c700 1 Processor -- start 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f180007c700 1 -- start start 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f180007c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 0x7f17f810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f180007c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f80fcfb0 0x7f17f810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f180007c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f810e480 con 0x7f17f80fbe40 2024-02-16T19:32:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f180007c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f810ed20 con 0x7f17f80fcfb0 2024-02-16T19:32:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f17fde18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 0x7f17f810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f17fde18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 0x7f17f810d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36238/0 (socket says 172.21.15.40:36238) 2024-02-16T19:32:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f17fde18700 1 -- 172.21.15.40:0/1425723625 learned_addr learned my addr 172.21.15.40:0/1425723625 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.539+0000 7f17fd617700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f80fcfb0 0x7f17f810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f17fde18700 1 -- 172.21.15.40:0/1425723625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f80fcfb0 msgr2=0x7f17f810de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f17fde18700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f80fcfb0 0x7f17f810de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f17fde18700 1 -- 172.21.15.40:0/1425723625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f4005040 con 0x7f17f80fbe40 2024-02-16T19:32:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f17fde18700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 0x7f17f810d940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009fd0 tx=0x7f17f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f17ee7fc700 1 -- 172.21.15.40:0/1425723625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17f400e040 con 0x7f17f80fbe40 2024-02-16T19:32:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f17ee7fc700 1 -- 172.21.15.40:0/1425723625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17f400bcc0 con 0x7f17f80fbe40 2024-02-16T19:32:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f810efa0 con 0x7f17f80fbe40 2024-02-16T19:32:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f17ee7fc700 1 -- 172.21.15.40:0/1425723625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17f401c630 con 0x7f17f80fbe40 2024-02-16T19:32:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.540+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f810f4e0 con 0x7f17f80fbe40 2024-02-16T19:32:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.541+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f804bed0 con 0x7f17f80fbe40 2024-02-16T19:32:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.542+0000 7f17ee7fc700 1 -- 172.21.15.40:0/1425723625 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f17f4013420 con 0x7f17f80fbe40 2024-02-16T19:32:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.542+0000 7f17ee7fc700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f17e406d7f0 0x7f17e406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.542+0000 7f17ee7fc700 1 -- 172.21.15.40:0/1425723625 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f17f408e390 con 0x7f17f80fbe40 2024-02-16T19:32:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.542+0000 7f17fd617700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f17e406d7f0 0x7f17e406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.543+0000 7f17fd617700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f17e406d7f0 0x7f17e406fca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f17e8004f40 tx=0x7f17e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.547+0000 7f17ee7fc700 1 -- 172.21.15.40:0/1425723625 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17f405b730 con 0x7f17f80fbe40 2024-02-16T19:32:44.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.796+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f17f810fae0 con 0x7f17f80fbe40 2024-02-16T19:32:44.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.796+0000 7f17ee7fc700 1 -- 172.21.15.40:0/1425723625 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1673 (secure 0 0 0) 0x7f17f405b2c0 con 0x7f17f80fbe40 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:e32 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 31 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:32:28.219148+0000 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:32:44.798 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 81 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 1 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:in 0 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=24275} 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 1 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:32:44.799 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:32:44.800 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:32:44.800 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:32:44.800 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{0:24275} state up:active seq 157 join_fscid=1 addr [v2:172.21.15.163:6826/1325230089,v1:172.21.15.163:6827/1325230089] compat {c=[1],r=[1],i=[7ff]}] 2024-02-16T19:32:44.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:32:44.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:32:44.800 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:32:44.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:32:44.801 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:32:44.801 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.798+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f17e406d7f0 msgr2=0x7f17e406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.798+0000 7f180007c700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f17e406d7f0 0x7f17e406fca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f17e8004f40 tx=0x7f17e800b040 comp rx=0 tx=0).stop 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.798+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 msgr2=0x7f17f810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.798+0000 7f180007c700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 0x7f17f810d940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009fd0 tx=0x7f17f4009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.799+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 shutdown_connections 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.799+0000 7f180007c700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f17e406d7f0 0x7f17e406fca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.799+0000 7f180007c700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17f80fbe40 0x7f17f810d940 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.799+0000 7f180007c700 1 --2- 172.21.15.40:0/1425723625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17f80fcfb0 0x7f17f810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.799+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 >> 172.21.15.40:0/1425723625 conn(0x7f17f80f74e0 msgr2=0x7f17f80656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.799+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 shutdown_connections 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.799+0000 7f180007c700 1 -- 172.21.15.40:0/1425723625 wait complete. 2024-02-16T19:32:44.802 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 32 2024-02-16T19:32:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.902+0000 7f63abf82700 1 -- 172.21.15.40:0/4046007602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 msgr2=0x7f63a40fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.902+0000 7f63abf82700 1 --2- 172.21.15.40:0/4046007602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 0x7f63a40fe550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6394004940 tx=0x7f6394009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.902+0000 7f63abf82700 1 -- 172.21.15.40:0/4046007602 shutdown_connections 2024-02-16T19:32:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.902+0000 7f63abf82700 1 --2- 172.21.15.40:0/4046007602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 0x7f63a40fe550 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.902+0000 7f63abf82700 1 --2- 172.21.15.40:0/4046007602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63a40fcef0 0x7f63a40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.902+0000 7f63abf82700 1 -- 172.21.15.40:0/4046007602 >> 172.21.15.40:0/4046007602 conn(0x7f63a40f8620 msgr2=0x7f63a40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63abf82700 1 -- 172.21.15.40:0/4046007602 shutdown_connections 2024-02-16T19:32:44.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63abf82700 1 -- 172.21.15.40:0/4046007602 wait complete. 2024-02-16T19:32:44.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63abf82700 1 Processor -- start 2024-02-16T19:32:44.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63abf82700 1 -- start start 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63abf82700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63a40fcef0 0x7f63a410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63abf82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 0x7f63a410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63abf82700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a410e560 con 0x7f63a40fe0e0 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63abf82700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a410eda0 con 0x7f63a40fcef0 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63a951d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 0x7f63a410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63a951d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 0x7f63a410ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36258/0 (socket says 172.21.15.40:36258) 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.903+0000 7f63a951d700 1 -- 172.21.15.40:0/2960701198 learned_addr learned my addr 172.21.15.40:0/2960701198 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f63a951d700 1 -- 172.21.15.40:0/2960701198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63a40fcef0 msgr2=0x7f63a410d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f63a9d1e700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63a40fcef0 0x7f63a410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f63a951d700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63a40fcef0 0x7f63a410d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f63a951d700 1 -- 172.21.15.40:0/2960701198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394005040 con 0x7f63a40fe0e0 2024-02-16T19:32:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f63a951d700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 0x7f63a410ded0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f639400b3b0 tx=0x7f6394009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f639a7fc700 1 -- 172.21.15.40:0/2960701198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639400e040 con 0x7f63a40fe0e0 2024-02-16T19:32:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f639a7fc700 1 -- 172.21.15.40:0/2960701198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f639400bd90 con 0x7f63a40fe0e0 2024-02-16T19:32:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f639a7fc700 1 -- 172.21.15.40:0/2960701198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639401c630 con 0x7f63a40fe0e0 2024-02-16T19:32:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a410f020 con 0x7f63a40fe0e0 2024-02-16T19:32:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.904+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a410f560 con 0x7f63a40fe0e0 2024-02-16T19:32:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.905+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a404bed0 con 0x7f63a40fe0e0 2024-02-16T19:32:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.906+0000 7f639a7fc700 1 -- 172.21.15.40:0/2960701198 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f63940139f0 con 0x7f63a40fe0e0 2024-02-16T19:32:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.906+0000 7f639a7fc700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f639006d7f0 0x7f639006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.906+0000 7f639a7fc700 1 -- 172.21.15.40:0/2960701198 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f639408e450 con 0x7f63a40fe0e0 2024-02-16T19:32:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.906+0000 7f63a9d1e700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f639006d7f0 0x7f639006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.907+0000 7f63a9d1e700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f639006d7f0 0x7f639006fca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f63a0006fd0 tx=0x7f63a0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:44.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:44.911+0000 7f639a7fc700 1 -- 172.21.15.40:0/2960701198 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f639405b7f0 con 0x7f63a40fe0e0 2024-02-16T19:32:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.137+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a4102ac0 con 0x7f639006d7f0 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.139+0000 7f639a7fc700 1 -- 172.21.15.40:0/2960701198 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f63a4102ac0 con 0x7f639006d7f0 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "osd", 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:32:45.140 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "14/21 daemons upgraded", 2024-02-16T19:32:45.141 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T19:32:45.141 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:32:45.141 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f639006d7f0 msgr2=0x7f639006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f639006d7f0 0x7f639006fca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f63a0006fd0 tx=0x7f63a0008040 comp rx=0 tx=0).stop 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 msgr2=0x7f63a410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 0x7f63a410ded0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f639400b3b0 tx=0x7f6394009e30 comp rx=0 tx=0).stop 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 shutdown_connections 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63a40fcef0 0x7f63a410d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f639006d7f0 0x7f639006fca0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 --2- 172.21.15.40:0/2960701198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63a40fe0e0 0x7f63a410ded0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 >> 172.21.15.40:0/2960701198 conn(0x7f63a40f8620 msgr2=0x7f63a41013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 shutdown_connections 2024-02-16T19:32:45.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.141+0000 7f63abf82700 1 -- 172.21.15.40:0/2960701198 wait complete. 2024-02-16T19:32:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3338312896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fcef0 msgr2=0x7ff3c00fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 --2- 172.21.15.40:0/3338312896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fcef0 0x7ff3c00fd320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009ce0 tx=0x7ff3b0004d10 comp rx=0 tx=0).stop 2024-02-16T19:32:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3338312896 shutdown_connections 2024-02-16T19:32:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 --2- 172.21.15.40:0/3338312896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c00fe0e0 0x7ff3c00fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 --2- 172.21.15.40:0/3338312896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fcef0 0x7ff3c00fd320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3338312896 >> 172.21.15.40:0/3338312896 conn(0x7ff3c00f8620 msgr2=0x7ff3c00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3338312896 shutdown_connections 2024-02-16T19:32:45.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3338312896 wait complete. 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 Processor -- start 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.243+0000 7ff3c4b51700 1 -- start start 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3c4b51700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c00fcef0 0x7ff3c010dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3c4b51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fe0e0 0x7ff3c010e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3c4b51700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c010e610 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3c4b51700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c010eeb0 con 0x7ff3c00fcef0 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3bdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fe0e0 0x7ff3c010e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3bdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fe0e0 0x7ff3c010e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36272/0 (socket says 172.21.15.40:36272) 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3bdd9b700 1 -- 172.21.15.40:0/3943386677 learned_addr learned my addr 172.21.15.40:0/3943386677 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3be59c700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c00fcef0 0x7ff3c010dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3bdd9b700 1 -- 172.21.15.40:0/3943386677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c00fcef0 msgr2=0x7ff3c010dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3bdd9b700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c00fcef0 0x7ff3c010dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3bdd9b700 1 -- 172.21.15.40:0/3943386677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0009940 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3bdd9b700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fe0e0 0x7ff3c010e010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8002a80 tx=0x7ff3a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3b6ffd700 1 -- 172.21.15.40:0/3943386677 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3a8007cc0 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3b6ffd700 1 -- 172.21.15.40:0/3943386677 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3a800f040 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c010f190 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.244+0000 7ff3b6ffd700 1 -- 172.21.15.40:0/3943386677 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3a800dac0 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.245+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c010f6d0 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.245+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c004bed0 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.246+0000 7ff3b6ffd700 1 -- 172.21.15.40:0/3943386677 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff3a800dc20 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.246+0000 7ff3b6ffd700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3ac06d7f0 0x7ff3ac06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:32:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.246+0000 7ff3b6ffd700 1 -- 172.21.15.40:0/3943386677 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff3a808cd30 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.246+0000 7ff3be59c700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3ac06d7f0 0x7ff3ac06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:32:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.247+0000 7ff3be59c700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3ac06d7f0 0x7ff3ac06fca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009cb0 tx=0x7ff3b0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:32:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.251+0000 7ff3b6ffd700 1 -- 172.21.15.40:0/3943386677 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff3a805a050 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:45 smithi040 ceph-mon[94210]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:45 smithi040 ceph-mon[94210]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:45 smithi040 ceph-mon[94210]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:45 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3123171843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:45.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:45 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1425723625' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:32:45.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:45 smithi163 ceph-mon[81637]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:45.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:45 smithi163 ceph-mon[81637]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:45.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:45 smithi163 ceph-mon[81637]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:45.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:45 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3123171843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:45.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:45 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1425723625' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:32:45.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.536+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff3c0105a30 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.536+0000 7ff3b6ffd700 1 -- 172.21.15.40:0/3943386677 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff3a8059be0 con 0x7ff3c00fe0e0 2024-02-16T19:32:45.538 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.538+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3ac06d7f0 msgr2=0x7ff3ac06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.538+0000 7ff3c4b51700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3ac06d7f0 0x7ff3ac06fca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009cb0 tx=0x7ff3b0004d10 comp rx=0 tx=0).stop 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.538+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fe0e0 msgr2=0x7ff3c010e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.538+0000 7ff3c4b51700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fe0e0 0x7ff3c010e010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8002a80 tx=0x7ff3a80046b0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.539+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 shutdown_connections 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.539+0000 7ff3c4b51700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c00fcef0 0x7ff3c010dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.539+0000 7ff3c4b51700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff3ac06d7f0 0x7ff3ac06fca0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.539+0000 7ff3c4b51700 1 --2- 172.21.15.40:0/3943386677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00fe0e0 0x7ff3c010e010 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.539+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 >> 172.21.15.40:0/3943386677 conn(0x7ff3c00f8620 msgr2=0x7ff3c01013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:32:45.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.539+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 shutdown_connections 2024-02-16T19:32:45.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:32:45.539+0000 7ff3c4b51700 1 -- 172.21.15.40:0/3943386677 wait complete. 2024-02-16T19:32:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:46 smithi040 ceph-mon[94210]: pgmap v207: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:46 smithi040 ceph-mon[94210]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:46 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3943386677' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:32:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:46 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:32:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:46 smithi163 ceph-mon[81637]: pgmap v207: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:46 smithi163 ceph-mon[81637]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:32:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:46 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3943386677' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:32:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:46 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:32:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:48 smithi040 ceph-mon[94210]: pgmap v208: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:48.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:48 smithi163 ceph-mon[81637]: pgmap v208: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:48.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:32:48.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:48.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:48.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:49.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:49 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.163:6824/1456075822,v1:172.21.15.163:6825/1456075822] up:boot 2024-02-16T19:32:49.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:49 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi163.qovvde=up:active} 3 up:standby 2024-02-16T19:32:49.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:32:49.394 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:49 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.163:6824/1456075822,v1:172.21.15.163:6825/1456075822] up:boot 2024-02-16T19:32:49.418 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:49 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi163.qovvde=up:active} 3 up:standby 2024-02-16T19:32:49.418 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:32:50.356 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:50 smithi163 ceph-mon[81637]: pgmap v209: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:50.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:50 smithi040 ceph-mon[94210]: pgmap v209: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:51.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:51 smithi040 ceph-mon[94210]: pgmap v210: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:51.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:51.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:51 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:51.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:51 smithi163 ceph-mon[81637]: pgmap v210: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:51.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:51.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:51 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:52.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:52 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:52.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:52 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:53.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:52 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:53.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:52 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:53 smithi163 ceph-mon[81637]: pgmap v211: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:54.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:53 smithi040 ceph-mon[94210]: pgmap v211: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: pgmap v212: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:56.115 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:55 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi163.qovvde"]}]: dispatch 2024-02-16T19:32:56.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: pgmap v212: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:56.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:56.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:56.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:56.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:32:56.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:56.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:32:56.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:56.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:56.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:56.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:32:56.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:55 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi163.qovvde"]}]: dispatch 2024-02-16T19:32:57.095 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:56 smithi163 ceph-mon[81637]: Detected new or changed devices on smithi163 2024-02-16T19:32:57.095 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:56 smithi163 ceph-mon[81637]: Upgrade: It appears safe to stop mds.cephfs.smithi163.qovvde 2024-02-16T19:32:57.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:56 smithi040 ceph-mon[94210]: Detected new or changed devices on smithi163 2024-02-16T19:32:57.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:56 smithi040 ceph-mon[94210]: Upgrade: It appears safe to stop mds.cephfs.smithi163.qovvde 2024-02-16T19:32:58.392 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:58 smithi163 ceph-mon[81637]: pgmap v213: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:58.393 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:58 smithi163 ceph-mon[81637]: Upgrade: Updating mds.cephfs.smithi163.qovvde 2024-02-16T19:32:58.393 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:58 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:58.393 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:58 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.qovvde", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:32:58.393 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:58 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:58.393 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:32:58 smithi163 ceph-mon[81637]: Deploying daemon mds.cephfs.smithi163.qovvde on smithi163 2024-02-16T19:32:58.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:58 smithi040 ceph-mon[94210]: pgmap v213: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:32:58.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:58 smithi040 ceph-mon[94210]: Upgrade: Updating mds.cephfs.smithi163.qovvde 2024-02-16T19:32:58.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:58 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:32:58.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:58 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi163.qovvde", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-16T19:32:58.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:58 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:32:58.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:58 smithi040 ceph-mon[94210]: Deploying daemon mds.cephfs.smithi163.qovvde on smithi163 2024-02-16T19:32:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:32:59 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[94205]: 2024-02-16T19:32:59.314+0000 7fd6fd9b1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:33:00.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:00 smithi163 ceph-mon[81637]: pgmap v214: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:00.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:00 smithi163 ceph-mon[81637]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:33:00.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:00 smithi163 ceph-mon[81637]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:33:00.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:00 smithi163 ceph-mon[81637]: osdmap e83: 6 total, 6 up, 6 in 2024-02-16T19:33:00.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:00 smithi163 ceph-mon[81637]: Standby daemon mds.cephfs.smithi040.ezkppl assigned to filesystem cephfs as rank 0 2024-02-16T19:33:00.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:00 smithi163 ceph-mon[81637]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:33:00.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:00 smithi163 ceph-mon[81637]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:33:00.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:00 smithi163 ceph-mon[81637]: fsmap cephfs:1/1 {0=cephfs.smithi040.ezkppl=up:replay} 2 up:standby 2024-02-16T19:33:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:00 smithi040 ceph-mon[94210]: pgmap v214: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:00 smithi040 ceph-mon[94210]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-16T19:33:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:00 smithi040 ceph-mon[94210]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-16T19:33:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:00 smithi040 ceph-mon[94210]: osdmap e83: 6 total, 6 up, 6 in 2024-02-16T19:33:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:00 smithi040 ceph-mon[94210]: Standby daemon mds.cephfs.smithi040.ezkppl assigned to filesystem cephfs as rank 0 2024-02-16T19:33:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:00 smithi040 ceph-mon[94210]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-16T19:33:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:00 smithi040 ceph-mon[94210]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-16T19:33:00.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:00 smithi040 ceph-mon[94210]: fsmap cephfs:1/1 {0=cephfs.smithi040.ezkppl=up:replay} 2 up:standby 2024-02-16T19:33:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:02 smithi163 ceph-mon[81637]: pgmap v216: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-02-16T19:33:02.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:02 smithi040 ceph-mon[94210]: pgmap v216: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-02-16T19:33:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:03 smithi040 ceph-mon[94210]: pgmap v217: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-02-16T19:33:03.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:33:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:03 smithi163 ceph-mon[81637]: pgmap v217: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-02-16T19:33:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:33:05.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:05 smithi040 ceph-mon[94210]: pgmap v218: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-02-16T19:33:05.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:05 smithi163 ceph-mon[81637]: pgmap v218: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-02-16T19:33:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:07 smithi040 ceph-mon[94210]: pgmap v219: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-02-16T19:33:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:07 smithi040 ceph-mon[94210]: reconnect by client.14536 v1:192.168.0.1:0/3779740512 after 0 2024-02-16T19:33:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:07 smithi040 ceph-mon[94210]: reconnect by client.24309 v1:192.168.0.1:0/3707514684 after 0 2024-02-16T19:33:07.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:07 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] up:reconnect 2024-02-16T19:33:07.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:07 smithi040 ceph-mon[94210]: fsmap cephfs:1/1 {0=cephfs.smithi040.ezkppl=up:reconnect} 2 up:standby 2024-02-16T19:33:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:07 smithi163 ceph-mon[81637]: pgmap v219: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-02-16T19:33:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:07 smithi163 ceph-mon[81637]: reconnect by client.14536 v1:192.168.0.1:0/3779740512 after 0 2024-02-16T19:33:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:07 smithi163 ceph-mon[81637]: reconnect by client.24309 v1:192.168.0.1:0/3707514684 after 0 2024-02-16T19:33:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:07 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] up:reconnect 2024-02-16T19:33:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:07 smithi163 ceph-mon[81637]: fsmap cephfs:1/1 {0=cephfs.smithi040.ezkppl=up:reconnect} 2 up:standby 2024-02-16T19:33:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:08 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] up:rejoin 2024-02-16T19:33:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:08 smithi040 ceph-mon[94210]: fsmap cephfs:1/1 {0=cephfs.smithi040.ezkppl=up:rejoin} 2 up:standby 2024-02-16T19:33:08.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:08 smithi040 ceph-mon[94210]: daemon mds.cephfs.smithi040.ezkppl is now active in filesystem cephfs as rank 0 2024-02-16T19:33:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:08 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] up:rejoin 2024-02-16T19:33:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:08 smithi163 ceph-mon[81637]: fsmap cephfs:1/1 {0=cephfs.smithi040.ezkppl=up:rejoin} 2 up:standby 2024-02-16T19:33:08.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:08 smithi163 ceph-mon[81637]: daemon mds.cephfs.smithi040.ezkppl is now active in filesystem cephfs as rank 0 2024-02-16T19:33:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:09 smithi040 ceph-mon[94210]: pgmap v220: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 102 B/s wr, 5 op/s 2024-02-16T19:33:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:09 smithi040 ceph-mon[94210]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:33:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:09 smithi040 ceph-mon[94210]: Cluster is now healthy 2024-02-16T19:33:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:09 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] up:active 2024-02-16T19:33:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:09 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:active} 2 up:standby 2024-02-16T19:33:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:09 smithi163 ceph-mon[81637]: pgmap v220: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 102 B/s wr, 5 op/s 2024-02-16T19:33:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:09 smithi163 ceph-mon[81637]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-16T19:33:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:09 smithi163 ceph-mon[81637]: Cluster is now healthy 2024-02-16T19:33:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:09 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] up:active 2024-02-16T19:33:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:09 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:active} 2 up:standby 2024-02-16T19:33:12.172 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:12 smithi163 ceph-mon[81637]: pgmap v221: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 187 B/s wr, 5 op/s 2024-02-16T19:33:12.172 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:12 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:12.172 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:12 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:12.172 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:12 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:12.172 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:12 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.163:6826/2051774557,v1:172.21.15.163:6827/2051774557] up:boot 2024-02-16T19:33:12.173 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:12 smithi163 ceph-mon[81637]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:active} 3 up:standby 2024-02-16T19:33:12.173 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:12 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:33:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:12 smithi040 ceph-mon[94210]: pgmap v221: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 187 B/s wr, 5 op/s 2024-02-16T19:33:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:12 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.163:6826/2051774557,v1:172.21.15.163:6827/2051774557] up:boot 2024-02-16T19:33:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:12 smithi040 ceph-mon[94210]: fsmap cephfs:1 {0=cephfs.smithi040.ezkppl=up:active} 3 up:standby 2024-02-16T19:33:12.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:12 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:33:14.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:14 smithi040 ceph-mon[94210]: pgmap v222: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:33:14.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:14 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:14.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:14 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:14 smithi163 ceph-mon[81637]: pgmap v222: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:33:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:14 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:14 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2025055598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fbcd0 msgr2=0x7f7e94102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 --2- 172.21.15.40:0/2025055598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fbcd0 0x7f7e94102500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e88004940 tx=0x7f7e88009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2025055598 shutdown_connections 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 --2- 172.21.15.40:0/2025055598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fbcd0 0x7f7e94102500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 --2- 172.21.15.40:0/2025055598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940fb380 0x7f7e940fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2025055598 >> 172.21.15.40:0/2025055598 conn(0x7f7e940f6d20 msgr2=0x7f7e940f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2025055598 shutdown_connections 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2025055598 wait complete. 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 Processor -- start 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.657+0000 7f7e98d5a700 1 -- start start 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e98d5a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fb380 0x7f7e940fde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e98d5a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940fbcd0 0x7f7e940fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e98d5a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e940fc910 con 0x7f7e940fb380 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e98d5a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e940fca80 con 0x7f7e940fbcd0 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fb380 0x7f7e940fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fb380 0x7f7e940fde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58300/0 (socket says 172.21.15.40:58300) 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e9259c700 1 -- 172.21.15.40:0/2219622921 learned_addr learned my addr 172.21.15.40:0/2219622921 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e91d9b700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940fbcd0 0x7f7e940fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e9259c700 1 -- 172.21.15.40:0/2219622921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940fbcd0 msgr2=0x7f7e940fc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e9259c700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940fbcd0 0x7f7e940fc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e9259c700 1 -- 172.21.15.40:0/2219622921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e88005040 con 0x7f7e940fb380 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e9259c700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fb380 0x7f7e940fde90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e84002a50 tx=0x7f7e840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.658+0000 7f7e82ffd700 1 -- 172.21.15.40:0/2219622921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e84007cc0 con 0x7f7e940fb380 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.659+0000 7f7e82ffd700 1 -- 172.21.15.40:0/2219622921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e8400f040 con 0x7f7e940fb380 2024-02-16T19:33:15.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.659+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e940fcd60 con 0x7f7e940fb380 2024-02-16T19:33:15.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.659+0000 7f7e82ffd700 1 -- 172.21.15.40:0/2219622921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e84009640 con 0x7f7e940fb380 2024-02-16T19:33:15.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.659+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e94115cf0 con 0x7f7e940fb380 2024-02-16T19:33:15.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.660+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e94107a80 con 0x7f7e940fb380 2024-02-16T19:33:15.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.660+0000 7f7e82ffd700 1 -- 172.21.15.40:0/2219622921 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7e840097a0 con 0x7f7e940fb380 2024-02-16T19:33:15.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.661+0000 7f7e82ffd700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7e7c06d7e0 0x7f7e7c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:15.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.661+0000 7f7e91d9b700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7e7c06d7e0 0x7f7e7c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:15.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.661+0000 7f7e82ffd700 1 -- 172.21.15.40:0/2219622921 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7e8408cd40 con 0x7f7e940fb380 2024-02-16T19:33:15.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.661+0000 7f7e91d9b700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7e7c06d7e0 0x7f7e7c06fc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e88004f90 tx=0x7f7e88009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:15.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.666+0000 7f7e82ffd700 1 -- 172.21.15.40:0/2219622921 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e840564e0 con 0x7f7e940fb380 2024-02-16T19:33:15.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.888+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e94105ab0 con 0x7f7e7c06d7e0 2024-02-16T19:33:15.889 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:15 smithi040 ceph-mon[94210]: pgmap v223: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-02-16T19:33:15.889 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:15 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:15.889 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:15 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:15.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.889+0000 7f7e82ffd700 1 -- 172.21.15.40:0/2219622921 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7e94105ab0 con 0x7f7e7c06d7e0 2024-02-16T19:33:15.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7e7c06d7e0 msgr2=0x7f7e7c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:15.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7e7c06d7e0 0x7f7e7c06fc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e88004f90 tx=0x7f7e88009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:15.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fb380 msgr2=0x7f7e940fde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:15.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fb380 0x7f7e940fde90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e84002a50 tx=0x7f7e840046b0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 shutdown_connections 2024-02-16T19:33:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7e7c06d7e0 0x7f7e7c06fc90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e940fb380 0x7f7e940fde90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 --2- 172.21.15.40:0/2219622921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e940fbcd0 0x7f7e940fc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 >> 172.21.15.40:0/2219622921 conn(0x7f7e940f6d20 msgr2=0x7f7e94100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 shutdown_connections 2024-02-16T19:33:15.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.891+0000 7f7e98d5a700 1 -- 172.21.15.40:0/2219622921 wait complete. 2024-02-16T19:33:15.903 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:33:15.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:15 smithi163 ceph-mon[81637]: pgmap v223: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-02-16T19:33:15.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:15 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:15.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:15 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.995+0000 7fe3899e1700 1 -- 172.21.15.40:0/1706257703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3841000e0 msgr2=0x7fe384102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.995+0000 7fe3899e1700 1 --2- 172.21.15.40:0/1706257703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3841000e0 0x7fe384102520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe36c004940 tx=0x7fe36c009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.995+0000 7fe3899e1700 1 -- 172.21.15.40:0/1706257703 shutdown_connections 2024-02-16T19:33:15.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.995+0000 7fe3899e1700 1 --2- 172.21.15.40:0/1706257703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3841000e0 0x7fe384102520 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.995+0000 7fe3899e1700 1 --2- 172.21.15.40:0/1706257703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe384066c50 0x7fe3840ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.995+0000 7fe3899e1700 1 -- 172.21.15.40:0/1706257703 >> 172.21.15.40:0/1706257703 conn(0x7fe3840f7520 msgr2=0x7fe3840f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:15.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.995+0000 7fe3899e1700 1 -- 172.21.15.40:0/1706257703 shutdown_connections 2024-02-16T19:33:15.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.995+0000 7fe3899e1700 1 -- 172.21.15.40:0/1706257703 wait complete. 2024-02-16T19:33:15.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe3899e1700 1 Processor -- start 2024-02-16T19:33:15.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe3899e1700 1 -- start start 2024-02-16T19:33:15.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe3899e1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe384066c50 0x7fe38410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:15.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe3899e1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3841000e0 0x7fe38410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe3899e1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38410e4a0 con 0x7fe384066c50 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe3899e1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38410ed40 con 0x7fe3841000e0 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe382ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe384066c50 0x7fe38410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe382ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe384066c50 0x7fe38410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58316/0 (socket says 172.21.15.40:58316) 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe382ffd700 1 -- 172.21.15.40:0/1813759974 learned_addr learned my addr 172.21.15.40:0/1813759974 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.996+0000 7fe3827fc700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3841000e0 0x7fe38410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe382ffd700 1 -- 172.21.15.40:0/1813759974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3841000e0 msgr2=0x7fe38410dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe382ffd700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3841000e0 0x7fe38410dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe382ffd700 1 -- 172.21.15.40:0/1813759974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c005040 con 0x7fe384066c50 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe382ffd700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe384066c50 0x7fe38410d960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe37400cb10 tx=0x7fe374007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe37b7fe700 1 -- 172.21.15.40:0/1813759974 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe374004200 con 0x7fe384066c50 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe37b7fe700 1 -- 172.21.15.40:0/1813759974 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe37400acc0 con 0x7fe384066c50 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38410f020 con 0x7fe384066c50 2024-02-16T19:33:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe37b7fe700 1 -- 172.21.15.40:0/1813759974 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe374003d90 con 0x7fe384066c50 2024-02-16T19:33:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.997+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38410f560 con 0x7fe384066c50 2024-02-16T19:33:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.998+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe384107a50 con 0x7fe384066c50 2024-02-16T19:33:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.998+0000 7fe37b7fe700 1 -- 172.21.15.40:0/1813759974 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe374003430 con 0x7fe384066c50 2024-02-16T19:33:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.999+0000 7fe37b7fe700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe37006d7e0 0x7fe37006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.999+0000 7fe37b7fe700 1 -- 172.21.15.40:0/1813759974 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe37408d4b0 con 0x7fe384066c50 2024-02-16T19:33:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.999+0000 7fe3827fc700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe37006d7e0 0x7fe37006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:15.999+0000 7fe3827fc700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe37006d7e0 0x7fe37006fc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00b3b0 tx=0x7fe36c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.003+0000 7fe37b7fe700 1 -- 172.21.15.40:0/1813759974 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe37405a800 con 0x7fe384066c50 2024-02-16T19:33:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.224+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe38410f8a0 con 0x7fe37006d7e0 2024-02-16T19:33:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.225+0000 7fe37b7fe700 1 -- 172.21.15.40:0/1813759974 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe38410f8a0 con 0x7fe37006d7e0 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe37006d7e0 msgr2=0x7fe37006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe37006d7e0 0x7fe37006fc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00b3b0 tx=0x7fe36c009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe384066c50 msgr2=0x7fe38410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe384066c50 0x7fe38410d960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe37400cb10 tx=0x7fe374007f90 comp rx=0 tx=0).stop 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 shutdown_connections 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe37006d7e0 0x7fe37006fc90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe384066c50 0x7fe38410d960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 --2- 172.21.15.40:0/1813759974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3841000e0 0x7fe38410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 >> 172.21.15.40:0/1813759974 conn(0x7fe3840f7520 msgr2=0x7fe3840f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.227+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 shutdown_connections 2024-02-16T19:33:16.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.228+0000 7fe3899e1700 1 -- 172.21.15.40:0/1813759974 wait complete. 2024-02-16T19:33:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 -- 172.21.15.40:0/4266256708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 msgr2=0x7f42940fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 --2- 172.21.15.40:0/4266256708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 0x7f42940fd420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4288004940 tx=0x7f4288009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 -- 172.21.15.40:0/4266256708 shutdown_connections 2024-02-16T19:33:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 --2- 172.21.15.40:0/4266256708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 0x7f42940fd420 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 --2- 172.21.15.40:0/4266256708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42940fbe40 0x7f42940fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 -- 172.21.15.40:0/4266256708 >> 172.21.15.40:0/4266256708 conn(0x7f42940f7520 msgr2=0x7f42940f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 -- 172.21.15.40:0/4266256708 shutdown_connections 2024-02-16T19:33:16.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 -- 172.21.15.40:0/4266256708 wait complete. 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 Processor -- start 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.332+0000 7f429a048700 1 -- start start 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f429a048700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42940fbe40 0x7f429410c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f429a048700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 0x7f429410cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f429a048700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429410d380 con 0x7f42940fcfb0 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f429a048700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429410dc20 con 0x7f42940fbe40 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f4292ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 0x7f429410cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f42937fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42940fbe40 0x7f429410c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f4292ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 0x7f429410cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58334/0 (socket says 172.21.15.40:58334) 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f4292ffd700 1 -- 172.21.15.40:0/2990672716 learned_addr learned my addr 172.21.15.40:0/2990672716 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f4292ffd700 1 -- 172.21.15.40:0/2990672716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42940fbe40 msgr2=0x7f429410c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f4292ffd700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42940fbe40 0x7f429410c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f4292ffd700 1 -- 172.21.15.40:0/2990672716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288005040 con 0x7f42940fcfb0 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f4292ffd700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 0x7f429410cd80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f428800b3b0 tx=0x7f4288009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:16.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f427bfff700 1 -- 172.21.15.40:0/2990672716 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428800e040 con 0x7f42940fcfb0 2024-02-16T19:33:16.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f427bfff700 1 -- 172.21.15.40:0/2990672716 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f428801e8f0 con 0x7f42940fcfb0 2024-02-16T19:33:16.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429410dea0 con 0x7f42940fcfb0 2024-02-16T19:33:16.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f427bfff700 1 -- 172.21.15.40:0/2990672716 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4288013870 con 0x7f42940fcfb0 2024-02-16T19:33:16.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.333+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429410e380 con 0x7f42940fcfb0 2024-02-16T19:33:16.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.334+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429404bed0 con 0x7f42940fcfb0 2024-02-16T19:33:16.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.335+0000 7f427bfff700 1 -- 172.21.15.40:0/2990672716 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f428801ea60 con 0x7f42940fcfb0 2024-02-16T19:33:16.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.335+0000 7f427bfff700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f427c06d7f0 0x7f427c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:16.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.335+0000 7f427bfff700 1 -- 172.21.15.40:0/2990672716 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4288090f50 con 0x7f42940fcfb0 2024-02-16T19:33:16.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.335+0000 7f42937fe700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f427c06d7f0 0x7f427c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:16.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.336+0000 7f42937fe700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f427c06d7f0 0x7f427c06fca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4284004f40 tx=0x7f428400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:16.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.340+0000 7f427bfff700 1 -- 172.21.15.40:0/2990672716 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42880c4d20 con 0x7f42940fcfb0 2024-02-16T19:33:16.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.553+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f429410e7b0 con 0x7f427c06d7f0 2024-02-16T19:33:16.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.563+0000 7f427bfff700 1 -- 172.21.15.40:0/2990672716 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f429410e7b0 con 0x7f427c06d7f0 2024-02-16T19:33:16.564 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (7m) 43s ago 16m 25.1M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (5m) 43s ago 17m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (5m) 3s ago 15m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (7m) 43s ago 16m 85.7M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (48s) 43s ago 11m 12.2M - 19.0.0-1450-gf78a58c0 f081f847070a 084709bde4d3 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (70s) 43s ago 11m 12.3M - 19.0.0-1450-gf78a58c0 f081f847070a 79213f1b751f 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (5s) 3s ago 11m 13.8M - 19.0.0-1450-gf78a58c0 f081f847070a 8358b866ac44 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (29s) 3s ago 11m 12.7M - 19.0.0-1450-gf78a58c0 f081f847070a 33502c65114c 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (7m) 43s ago 18m 507M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (6m) 3s ago 15m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (6m) 43s ago 18m 54.2M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (6m) 3s ago 15m 40.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (8m) 43s ago 17m 15.7M - 1.5.0 0da6a335fe13 d3d72b4b738b 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (7m) 3s ago 15m 15.6M - 1.5.0 0da6a335fe13 0cdb2243447b 2024-02-16T19:33:16.565 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (4m) 43s ago 14m 107M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:33:16.566 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (4m) 43s ago 14m 113M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:33:16.566 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (3m) 43s ago 14m 88.7M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:33:16.566 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (3m) 3s ago 13m 112M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:33:16.566 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (2m) 3s ago 13m 99.9M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:33:16.566 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (118s) 3s ago 12m 103M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 807b119a828c 2024-02-16T19:33:16.566 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (6m) 43s ago 16m 60.9M - 2.43.0 a07b618ecd1d 5fec20be5fa2 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.565+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f427c06d7f0 msgr2=0x7f427c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.565+0000 7f429a048700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f427c06d7f0 0x7f427c06fca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4284004f40 tx=0x7f428400b040 comp rx=0 tx=0).stop 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.565+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 msgr2=0x7f429410cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.565+0000 7f429a048700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 0x7f429410cd80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f428800b3b0 tx=0x7f4288009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.565+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 shutdown_connections 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.565+0000 7f429a048700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42940fbe40 0x7f429410c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.565+0000 7f429a048700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f427c06d7f0 0x7f427c06fca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.566+0000 7f429a048700 1 --2- 172.21.15.40:0/2990672716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42940fcfb0 0x7f429410cd80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.566+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 >> 172.21.15.40:0/2990672716 conn(0x7f42940f7520 msgr2=0x7f4294100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.566+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 shutdown_connections 2024-02-16T19:33:16.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.566+0000 7f429a048700 1 -- 172.21.15.40:0/2990672716 wait complete. 2024-02-16T19:33:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 -- 172.21.15.40:0/3659808322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fcef0 msgr2=0x7f9a1c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 --2- 172.21.15.40:0/3659808322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fcef0 0x7f9a1c0fd320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a14004940 tx=0x7f9a14009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 -- 172.21.15.40:0/3659808322 shutdown_connections 2024-02-16T19:33:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 --2- 172.21.15.40:0/3659808322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a1c0fe0e0 0x7f9a1c0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 --2- 172.21.15.40:0/3659808322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fcef0 0x7f9a1c0fd320 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 -- 172.21.15.40:0/3659808322 >> 172.21.15.40:0/3659808322 conn(0x7f9a1c0f8620 msgr2=0x7f9a1c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:16.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 -- 172.21.15.40:0/3659808322 shutdown_connections 2024-02-16T19:33:16.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 -- 172.21.15.40:0/3659808322 wait complete. 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.668+0000 7f9a20f57700 1 Processor -- start 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a20f57700 1 -- start start 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a20f57700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a1c0fcef0 0x7f9a1c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a20f57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fe0e0 0x7f9a1c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a20f57700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c10e4b0 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a20f57700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c10ed50 con 0x7f9a1c0fcef0 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a19d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fe0e0 0x7f9a1c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a19d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fe0e0 0x7f9a1c10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58356/0 (socket says 172.21.15.40:58356) 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a19d9b700 1 -- 172.21.15.40:0/3877632355 learned_addr learned my addr 172.21.15.40:0/3877632355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:16.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a1a59c700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a1c0fcef0 0x7f9a1c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a19d9b700 1 -- 172.21.15.40:0/3877632355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a1c0fcef0 msgr2=0x7f9a1c10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a19d9b700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a1c0fcef0 0x7f9a1c10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a19d9b700 1 -- 172.21.15.40:0/3877632355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a14005040 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a19d9b700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fe0e0 0x7f9a1c10deb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a04002a50 tx=0x7f9a040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a12ffd700 1 -- 172.21.15.40:0/3877632355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a04007cc0 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a12ffd700 1 -- 172.21.15.40:0/3877632355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a0400f040 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c10f030 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.669+0000 7f9a12ffd700 1 -- 172.21.15.40:0/3877632355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a0400da30 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.670+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c10f570 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.670+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c04bed0 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.671+0000 7f9a12ffd700 1 -- 172.21.15.40:0/3877632355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9a0400dbb0 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.671+0000 7f9a12ffd700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a0806d840 0x7f9a0806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.671+0000 7f9a12ffd700 1 -- 172.21.15.40:0/3877632355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9a0408cb80 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.672+0000 7f9a1a59c700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a0806d840 0x7f9a0806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:16.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.672+0000 7f9a1a59c700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a0806d840 0x7f9a0806fcf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400b840 tx=0x7f9a14009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:16.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.676+0000 7f9a12ffd700 1 -- 172.21.15.40:0/3877632355 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a04059db0 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:16 smithi040 ceph-mon[94210]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.964+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9a1c10f960 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.964+0000 7f9a12ffd700 1 -- 172.21.15.40:0/3877632355 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9a04059940 con 0x7f9a1c0fe0e0 2024-02-16T19:33:16.965 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:33:16.966 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a0806d840 msgr2=0x7f9a0806fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a0806d840 0x7f9a0806fcf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400b840 tx=0x7f9a14009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fe0e0 msgr2=0x7f9a1c10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fe0e0 0x7f9a1c10deb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a04002a50 tx=0x7f9a040046b0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 shutdown_connections 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a1c0fcef0 0x7f9a1c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a0806d840 0x7f9a0806fcf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 --2- 172.21.15.40:0/3877632355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1c0fe0e0 0x7f9a1c10deb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.966+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 >> 172.21.15.40:0/3877632355 conn(0x7f9a1c0f8620 msgr2=0x7f9a1c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.967+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 shutdown_connections 2024-02-16T19:33:16.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:16.967+0000 7f9a20f57700 1 -- 172.21.15.40:0/3877632355 wait complete. 2024-02-16T19:33:16.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:16 smithi163 ceph-mon[81637]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.070+0000 7f77aa2f4700 1 -- 172.21.15.40:0/1791239847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41003a0 msgr2=0x7f77a4102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.070+0000 7f77aa2f4700 1 --2- 172.21.15.40:0/1791239847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41003a0 0x7f77a4102780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7798004940 tx=0x7f7798009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.070+0000 7f77aa2f4700 1 -- 172.21.15.40:0/1791239847 shutdown_connections 2024-02-16T19:33:17.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.070+0000 7f77aa2f4700 1 --2- 172.21.15.40:0/1791239847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a41003a0 0x7f77a4102780 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.070+0000 7f77aa2f4700 1 --2- 172.21.15.40:0/1791239847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77a40fda80 0x7f77a40ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.070+0000 7f77aa2f4700 1 -- 172.21.15.40:0/1791239847 >> 172.21.15.40:0/1791239847 conn(0x7f77a40f74e0 msgr2=0x7f77a40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.070+0000 7f77aa2f4700 1 -- 172.21.15.40:0/1791239847 shutdown_connections 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.070+0000 7f77aa2f4700 1 -- 172.21.15.40:0/1791239847 wait complete. 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77aa2f4700 1 Processor -- start 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77aa2f4700 1 -- start start 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77aa2f4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a40fda80 0x7f77a410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77aa2f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77a41003a0 0x7f77a410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77aa2f4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a410e480 con 0x7f77a40fda80 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77aa2f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a410ed20 con 0x7f77a41003a0 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a40fda80 0x7f77a410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a40fda80 0x7f77a410d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58384/0 (socket says 172.21.15.40:58384) 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77a3fff700 1 -- 172.21.15.40:0/2598039427 learned_addr learned my addr 172.21.15.40:0/2598039427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77a37fe700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77a41003a0 0x7f77a410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77a3fff700 1 -- 172.21.15.40:0/2598039427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77a41003a0 msgr2=0x7f77a410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77a3fff700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77a41003a0 0x7f77a410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.071+0000 7f77a3fff700 1 -- 172.21.15.40:0/2598039427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798005040 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.072+0000 7f77a3fff700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a40fda80 0x7f77a410d940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7794002a50 tx=0x7f77940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.072+0000 7f77a0ff9700 1 -- 172.21.15.40:0/2598039427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7794007cc0 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.072+0000 7f77a0ff9700 1 -- 172.21.15.40:0/2598039427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f779400f040 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.072+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a410f000 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.072+0000 7f77a0ff9700 1 -- 172.21.15.40:0/2598039427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f779400da30 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.072+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a410f540 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.073+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a404bed0 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.073+0000 7f77a0ff9700 1 -- 172.21.15.40:0/2598039427 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f779400dbb0 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.074+0000 7f77a0ff9700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f778c06d7e0 0x7f778c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.074+0000 7f77a0ff9700 1 -- 172.21.15.40:0/2598039427 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f779408cd20 con 0x7f77a40fda80 2024-02-16T19:33:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.074+0000 7f77a37fe700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f778c06d7e0 0x7f778c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.074+0000 7f77a37fe700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f778c06d7e0 0x7f778c06fc90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f779800b3b0 tx=0x7f7798009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:17.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.078+0000 7f77a0ff9700 1 -- 172.21.15.40:0/2598039427 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7794059ff0 con 0x7f77a40fda80 2024-02-16T19:33:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.327+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f77a410f920 con 0x7f77a40fda80 2024-02-16T19:33:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.327+0000 7f77a0ff9700 1 -- 172.21.15.40:0/2598039427 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f7794059b80 con 0x7f77a40fda80 2024-02-16T19:33:17.328 INFO:teuthology.orchestra.run.smithi040.stdout:e39 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 38 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:33:07.341718+0000 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:33:17.329 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 83 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 1 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:in 0 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=34404} 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 1 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:33:17.330 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{0:34404} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6824/1456075822,v1:172.21.15.163:6825/1456075822] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:33:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6826/2051774557,v1:172.21.15.163:6827/2051774557] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:33:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f778c06d7e0 msgr2=0x7f778c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f778c06d7e0 0x7f778c06fc90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f779800b3b0 tx=0x7f7798009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a40fda80 msgr2=0x7f77a410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a40fda80 0x7f77a410d940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7794002a50 tx=0x7f77940046b0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 shutdown_connections 2024-02-16T19:33:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f778c06d7e0 0x7f778c06fc90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a40fda80 0x7f77a410d940 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 --2- 172.21.15.40:0/2598039427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77a41003a0 0x7f77a410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 >> 172.21.15.40:0/2598039427 conn(0x7f77a40f74e0 msgr2=0x7f77a40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 shutdown_connections 2024-02-16T19:33:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.329+0000 7f77aa2f4700 1 -- 172.21.15.40:0/2598039427 wait complete. 2024-02-16T19:33:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 39 2024-02-16T19:33:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.432+0000 7fe1d1f62700 1 -- 172.21.15.40:0/634648738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fbe40 msgr2=0x7fe1cc0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.432+0000 7fe1d1f62700 1 --2- 172.21.15.40:0/634648738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fbe40 0x7fe1cc0fc250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc004940 tx=0x7fe1bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.432+0000 7fe1d1f62700 1 -- 172.21.15.40:0/634648738 shutdown_connections 2024-02-16T19:33:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.432+0000 7fe1d1f62700 1 --2- 172.21.15.40:0/634648738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1cc0fcfb0 0x7fe1cc0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.432+0000 7fe1d1f62700 1 --2- 172.21.15.40:0/634648738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fbe40 0x7fe1cc0fc250 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.432+0000 7fe1d1f62700 1 -- 172.21.15.40:0/634648738 >> 172.21.15.40:0/634648738 conn(0x7fe1cc0f7520 msgr2=0x7fe1cc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.432+0000 7fe1d1f62700 1 -- 172.21.15.40:0/634648738 shutdown_connections 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.432+0000 7fe1d1f62700 1 -- 172.21.15.40:0/634648738 wait complete. 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1d1f62700 1 Processor -- start 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1d1f62700 1 -- start start 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1d1f62700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1cc0fbe40 0x7fe1cc0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1d1f62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fcfb0 0x7fe1cc0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1d1f62700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0fe980 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1d1f62700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0feaf0 con 0x7fe1cc0fbe40 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1caffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fcfb0 0x7fe1cc0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1caffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fcfb0 0x7fe1cc0fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58410/0 (socket says 172.21.15.40:58410) 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1caffd700 1 -- 172.21.15.40:0/1339018978 learned_addr learned my addr 172.21.15.40:0/1339018978 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.433+0000 7fe1cb7fe700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1cc0fbe40 0x7fe1cc0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1caffd700 1 -- 172.21.15.40:0/1339018978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1cc0fbe40 msgr2=0x7fe1cc0fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1caffd700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1cc0fbe40 0x7fe1cc0fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1caffd700 1 -- 172.21.15.40:0/1339018978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc005040 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1caffd700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fcfb0 0x7fe1cc0fe380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0002a80 tx=0x7fe1c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1b3fff700 1 -- 172.21.15.40:0/1339018978 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c0007cc0 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1b3fff700 1 -- 172.21.15.40:0/1339018978 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1c000f040 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc0fedd0 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1b3fff700 1 -- 172.21.15.40:0/1339018978 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c000dac0 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.434+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc1158d0 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.435+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc04bed0 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.435+0000 7fe1b3fff700 1 -- 172.21.15.40:0/1339018978 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe1c000dc20 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.436+0000 7fe1b3fff700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe1b406d7e0 0x7fe1b406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.436+0000 7fe1cb7fe700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe1b406d7e0 0x7fe1b406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.436+0000 7fe1b3fff700 1 -- 172.21.15.40:0/1339018978 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe1c008cd80 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.436+0000 7fe1cb7fe700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe1b406d7e0 0x7fe1b406fc90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc004890 tx=0x7fe1bc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.441+0000 7fe1b3fff700 1 -- 172.21.15.40:0/1339018978 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1c005a050 con 0x7fe1cc0fcfb0 2024-02-16T19:33:17.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.662+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1cc066660 con 0x7fe1b406d7e0 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.664+0000 7fe1b3fff700 1 -- 172.21.15.40:0/1339018978 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe1cc066660 con 0x7fe1b406d7e0 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "osd", 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "mds", 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:33:17.665 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "16/21 daemons upgraded", 2024-02-16T19:33:17.666 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading mds daemons", 2024-02-16T19:33:17.666 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:33:17.666 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe1b406d7e0 msgr2=0x7fe1b406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe1b406d7e0 0x7fe1b406fc90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc004890 tx=0x7fe1bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fcfb0 msgr2=0x7fe1cc0fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fcfb0 0x7fe1cc0fe380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0002a80 tx=0x7fe1c00046b0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 shutdown_connections 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1cc0fbe40 0x7fe1cc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe1b406d7e0 0x7fe1b406fc90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 --2- 172.21.15.40:0/1339018978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1cc0fcfb0 0x7fe1cc0fe380 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 >> 172.21.15.40:0/1339018978 conn(0x7fe1cc0f7520 msgr2=0x7fe1cc065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 shutdown_connections 2024-02-16T19:33:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.666+0000 7fe1d1f62700 1 -- 172.21.15.40:0/1339018978 wait complete. 2024-02-16T19:33:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:17 smithi163 ceph-mon[81637]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:17 smithi163 ceph-mon[81637]: pgmap v224: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-02-16T19:33:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:17 smithi163 ceph-mon[81637]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:17 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3877632355' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:17.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:17 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2598039427' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:33:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1584310996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa780fb2c0 msgr2=0x7faa780fb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 --2- 172.21.15.40:0/1584310996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa780fb2c0 0x7faa780fb6d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faa70009ce0 tx=0x7faa70004d10 comp rx=0 tx=0).stop 2024-02-16T19:33:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1584310996 shutdown_connections 2024-02-16T19:33:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 --2- 172.21.15.40:0/1584310996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa780fbca0 0x7faa780fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 --2- 172.21.15.40:0/1584310996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa780fb2c0 0x7faa780fb6d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1584310996 >> 172.21.15.40:0/1584310996 conn(0x7faa780f6ca0 msgr2=0x7faa780f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1584310996 shutdown_connections 2024-02-16T19:33:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1584310996 wait complete. 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 Processor -- start 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.769+0000 7faa7f1c0700 1 -- start start 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa7f1c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa780fb2c0 0x7faa7810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa7f1c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7810e340 0x7faa7810f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa7f1c0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7810e800 con 0x7faa7810e340 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa7f1c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7810e970 con 0x7faa780fb2c0 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7810e340 0x7faa7810f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7810e340 0x7faa7810f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41144/0 (socket says 172.21.15.40:41144) 2024-02-16T19:33:17.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa77fff700 1 -- 172.21.15.40:0/1742192540 learned_addr learned my addr 172.21.15.40:0/1742192540 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa7cf5c700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa780fb2c0 0x7faa7810de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa77fff700 1 -- 172.21.15.40:0/1742192540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa780fb2c0 msgr2=0x7faa7810de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa77fff700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa780fb2c0 0x7faa7810de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa77fff700 1 -- 172.21.15.40:0/1742192540 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa70009940 con 0x7faa7810e340 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa77fff700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7810e340 0x7faa7810f100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faa68002a50 tx=0x7faa680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa757fa700 1 -- 172.21.15.40:0/1742192540 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa68007cc0 con 0x7faa7810e340 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa757fa700 1 -- 172.21.15.40:0/1742192540 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa6800f040 con 0x7faa7810e340 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa757fa700 1 -- 172.21.15.40:0/1742192540 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6800da30 con 0x7faa7810e340 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.770+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7810f7e0 con 0x7faa7810e340 2024-02-16T19:33:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.771+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7810fcd0 con 0x7faa7810e340 2024-02-16T19:33:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.772+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa780fa940 con 0x7faa7810e340 2024-02-16T19:33:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.772+0000 7faa757fa700 1 -- 172.21.15.40:0/1742192540 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7faa6800db90 con 0x7faa7810e340 2024-02-16T19:33:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.772+0000 7faa757fa700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa6406d7e0 0x7faa6406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.772+0000 7faa757fa700 1 -- 172.21.15.40:0/1742192540 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faa6808cd20 con 0x7faa7810e340 2024-02-16T19:33:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.772+0000 7faa7cf5c700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa6406d7e0 0x7faa6406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.773+0000 7faa7cf5c700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa6406d7e0 0x7faa6406fc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faa70003400 tx=0x7faa70004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:17.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:17.777+0000 7faa757fa700 1 -- 172.21.15.40:0/1742192540 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa680564c0 con 0x7faa7810e340 2024-02-16T19:33:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:17 smithi040 ceph-mon[94210]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:17 smithi040 ceph-mon[94210]: pgmap v224: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-02-16T19:33:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:17 smithi040 ceph-mon[94210]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:17 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3877632355' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:17.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:17 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2598039427' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:33:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.067+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faa780fbca0 con 0x7faa7810e340 2024-02-16T19:33:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.067+0000 7faa757fa700 1 -- 172.21.15.40:0/1742192540 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7faa68059b80 con 0x7faa7810e340 2024-02-16T19:33:18.069 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.069+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa6406d7e0 msgr2=0x7faa6406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.069+0000 7faa7f1c0700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa6406d7e0 0x7faa6406fc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faa70003400 tx=0x7faa70004d10 comp rx=0 tx=0).stop 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.069+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7810e340 msgr2=0x7faa7810f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.069+0000 7faa7f1c0700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7810e340 0x7faa7810f100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faa68002a50 tx=0x7faa680046b0 comp rx=0 tx=0).stop 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.070+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 shutdown_connections 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.070+0000 7faa7f1c0700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa780fb2c0 0x7faa7810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.070+0000 7faa7f1c0700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa6406d7e0 0x7faa6406fc90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.070+0000 7faa7f1c0700 1 --2- 172.21.15.40:0/1742192540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7810e340 0x7faa7810f100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.070+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 >> 172.21.15.40:0/1742192540 conn(0x7faa780f6ca0 msgr2=0x7faa78101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.070+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 shutdown_connections 2024-02-16T19:33:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:18.070+0000 7faa7f1c0700 1 -- 172.21.15.40:0/1742192540 wait complete. 2024-02-16T19:33:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1742192540' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:33:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:18.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.bmpjpk"}]': finished 2024-02-16T19:33:18.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:33:18.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.ezkppl"}]': finished 2024-02-16T19:33:18.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:33:18.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi163.qovvde"}]': finished 2024-02-16T19:33:18.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:33:18.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi163.rggdhh"}]': finished 2024-02-16T19:33:18.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-16T19:33:18.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:18.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1742192540' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:33:18.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.bmpjpk"}]: dispatch 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.bmpjpk"}]': finished 2024-02-16T19:33:18.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.ezkppl"}]: dispatch 2024-02-16T19:33:18.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.ezkppl"}]': finished 2024-02-16T19:33:18.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi163.qovvde"}]: dispatch 2024-02-16T19:33:18.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi163.qovvde"}]': finished 2024-02-16T19:33:18.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi163.rggdhh"}]: dispatch 2024-02-16T19:33:18.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi163.rggdhh"}]': finished 2024-02-16T19:33:18.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-16T19:33:19.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: pgmap v225: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all mds 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: daemon mds.cephfs.smithi040.bmpjpk assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi040.bmpjpk=up:starting} 2 up:standby 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: daemon mds.cephfs.smithi040.bmpjpk is now active in filesystem cephfs as rank 1 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all rgw 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.895 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all ceph-exporter 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all iscsi 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all nfs 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: Upgrade: Setting container_image for all nvmeof 2024-02-16T19:33:19.896 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:19 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: pgmap v225: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:33:19.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all mds 2024-02-16T19:33:19.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: daemon mds.cephfs.smithi040.bmpjpk assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi040.bmpjpk=up:starting} 2 up:standby 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: daemon mds.cephfs.smithi040.bmpjpk is now active in filesystem cephfs as rank 1 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all rgw 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all rbd-mirror 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all ceph-exporter 2024-02-16T19:33:19.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all iscsi 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all nfs 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: Upgrade: Setting container_image for all nvmeof 2024-02-16T19:33:19.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:19 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:20.980 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:20 smithi040 ceph-mon[94210]: mds.? [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] up:active 2024-02-16T19:33:20.981 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:20 smithi040 ceph-mon[94210]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi040.bmpjpk=up:active} 2 up:standby 2024-02-16T19:33:20.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:20 smithi163 ceph-mon[81637]: mds.? [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] up:active 2024-02-16T19:33:20.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:20 smithi163 ceph-mon[81637]: fsmap cephfs:2 {0=cephfs.smithi040.ezkppl=up:active,1=cephfs.smithi040.bmpjpk=up:active} 2 up:standby 2024-02-16T19:33:21.805 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:21 smithi040 ceph-mon[94210]: pgmap v226: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 220 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:33:21.805 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:21 smithi040 ceph-mon[94210]: Upgrade: Updating node-exporter.smithi040 (1/2) 2024-02-16T19:33:21.805 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:21 smithi040 ceph-mon[94210]: Deploying daemon node-exporter.smithi040 on smithi040 2024-02-16T19:33:21.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:21 smithi163 ceph-mon[81637]: pgmap v226: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 220 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-02-16T19:33:21.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:21 smithi163 ceph-mon[81637]: Upgrade: Updating node-exporter.smithi040 (1/2) 2024-02-16T19:33:21.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:21 smithi163 ceph-mon[81637]: Deploying daemon node-exporter.smithi040 on smithi040 2024-02-16T19:33:24.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:24 smithi163 ceph-mon[81637]: pgmap v227: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 4.6 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-02-16T19:33:24.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:24 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:24.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:24 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:24.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:24 smithi040 ceph-mon[94210]: pgmap v227: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 4.6 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-02-16T19:33:24.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:24 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:24.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:24 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:26.446 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:26 smithi163 ceph-mon[81637]: pgmap v228: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-02-16T19:33:26.446 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:26 smithi163 ceph-mon[81637]: Upgrade: Updating node-exporter.smithi163 (2/2) 2024-02-16T19:33:26.446 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:26 smithi163 ceph-mon[81637]: Deploying daemon node-exporter.smithi163 on smithi163 2024-02-16T19:33:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:26 smithi040 ceph-mon[94210]: pgmap v228: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-02-16T19:33:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:26 smithi040 ceph-mon[94210]: Upgrade: Updating node-exporter.smithi163 (2/2) 2024-02-16T19:33:26.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:26 smithi040 ceph-mon[94210]: Deploying daemon node-exporter.smithi163 on smithi163 2024-02-16T19:33:28.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:28 smithi163 ceph-mon[81637]: pgmap v229: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:33:28.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:28.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:28.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:28 smithi040 ceph-mon[94210]: pgmap v229: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:33:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:28.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:30.445 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:30 smithi040 ceph-mon[94210]: pgmap v230: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:33:30.446 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:30 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:30.446 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:30 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:30 smithi163 ceph-mon[81637]: pgmap v230: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:33:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:30 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:30 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:31 smithi040 ceph-mon[94210]: pgmap v231: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:33:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:31 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:31 smithi163 ceph-mon[81637]: pgmap v231: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-16T19:33:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:31 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:32.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:33.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:34.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:33 smithi040 ceph-mon[94210]: pgmap v232: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-16T19:33:34.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:34.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:34.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:33 smithi163 ceph-mon[81637]: pgmap v232: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 85 B/s wr, 5 op/s 2024-02-16T19:33:34.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:34.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:35 smithi163 ceph-mon[81637]: pgmap v233: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 4.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-02-16T19:33:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:35 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:35 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:35.791 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:35 smithi040 ceph-mon[94210]: pgmap v233: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 4.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-02-16T19:33:35.792 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:35 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:35.792 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:35 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:37.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: pgmap v234: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-02-16T19:33:37.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:37.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.525 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.526 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.526 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.526 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.526 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:37 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: pgmap v234: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:37.989 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:37 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:39.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:39 smithi040 ceph-mon[94210]: pgmap v235: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:39.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:39 smithi040 ceph-mon[94210]: Upgrade: Updating prometheus.smithi040 2024-02-16T19:33:39.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:39 smithi040 ceph-mon[94210]: Deploying daemon prometheus.smithi040 on smithi040 2024-02-16T19:33:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:39 smithi163 ceph-mon[81637]: pgmap v235: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:39 smithi163 ceph-mon[81637]: Upgrade: Updating prometheus.smithi040 2024-02-16T19:33:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:39 smithi163 ceph-mon[81637]: Deploying daemon prometheus.smithi040 on smithi040 2024-02-16T19:33:41.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:41 smithi163 ceph-mon[81637]: pgmap v236: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:42.105 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:41 smithi040 ceph-mon[94210]: pgmap v236: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:43.960 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:43 smithi040 ceph-mon[94210]: pgmap v237: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:43.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:43 smithi163 ceph-mon[81637]: pgmap v237: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:45.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:45 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:45.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:45 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:45.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:45 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:45.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:45 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:45.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:45 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:45.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:45 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:46.361 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:46 smithi040 ceph-mon[94210]: pgmap v238: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:46 smithi163 ceph-mon[81637]: pgmap v238: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:48.138 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:48 smithi040 ceph-mon[94210]: pgmap v239: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:48.138 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:48.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 -- 172.21.15.40:0/3141809433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fb220 msgr2=0x7fe13c0fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 --2- 172.21.15.40:0/3141809433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fb220 0x7fe13c0fb630 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe130004940 tx=0x7fe130009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:48.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 -- 172.21.15.40:0/3141809433 shutdown_connections 2024-02-16T19:33:48.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 --2- 172.21.15.40:0/3141809433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe13c0fe2b0 0x7fe13c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 --2- 172.21.15.40:0/3141809433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fb220 0x7fe13c0fb630 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 -- 172.21.15.40:0/3141809433 >> 172.21.15.40:0/3141809433 conn(0x7fe13c0f6ca0 msgr2=0x7fe13c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 -- 172.21.15.40:0/3141809433 shutdown_connections 2024-02-16T19:33:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 -- 172.21.15.40:0/3141809433 wait complete. 2024-02-16T19:33:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 Processor -- start 2024-02-16T19:33:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 -- start start 2024-02-16T19:33:48.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe13c0fb220 0x7fe13c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fe2b0 0x7fe13c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.177+0000 7fe14300a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c10e5d0 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe14300a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c10ee10 con 0x7fe13c0fb220 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe13bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fe2b0 0x7fe13c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe13bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fe2b0 0x7fe13c10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58544/0 (socket says 172.21.15.40:58544) 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe13bfff700 1 -- 172.21.15.40:0/1354082275 learned_addr learned my addr 172.21.15.40:0/1354082275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe140da6700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe13c0fb220 0x7fe13c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe13bfff700 1 -- 172.21.15.40:0/1354082275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe13c0fb220 msgr2=0x7fe13c10da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe13bfff700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe13c0fb220 0x7fe13c10da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe13bfff700 1 -- 172.21.15.40:0/1354082275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130005040 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe13bfff700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fe2b0 0x7fe13c10df40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe12800b790 tx=0x7fe128009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe1397fa700 1 -- 172.21.15.40:0/1354082275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12800f770 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe1397fa700 1 -- 172.21.15.40:0/1354082275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe12800fda0 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe1397fa700 1 -- 172.21.15.40:0/1354082275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12800d310 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c10f0f0 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.178+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c10f630 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.179+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c04bed0 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.180+0000 7fe1397fa700 1 -- 172.21.15.40:0/1354082275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe128010420 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.180+0000 7fe1397fa700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe12c06d7e0 0x7fe12c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.181+0000 7fe1397fa700 1 -- 172.21.15.40:0/1354082275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe12808cef0 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.181+0000 7fe140da6700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe12c06d7e0 0x7fe12c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.181+0000 7fe140da6700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe12c06d7e0 0x7fe12c06fc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe130004f90 tx=0x7fe130009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:48.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.185+0000 7fe1397fa700 1 -- 172.21.15.40:0/1354082275 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe12805a2e0 con 0x7fe13c0fe2b0 2024-02-16T19:33:48.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.417+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe13c10fc20 con 0x7fe12c06d7e0 2024-02-16T19:33:48.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.419+0000 7fe1397fa700 1 -- 172.21.15.40:0/1354082275 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fe13c10fc20 con 0x7fe12c06d7e0 2024-02-16T19:33:48.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe12c06d7e0 msgr2=0x7fe12c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe12c06d7e0 0x7fe12c06fc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe130004f90 tx=0x7fe130009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:48.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fe2b0 msgr2=0x7fe13c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fe2b0 0x7fe13c10df40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe12800b790 tx=0x7fe128009250 comp rx=0 tx=0).stop 2024-02-16T19:33:48.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 shutdown_connections 2024-02-16T19:33:48.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe13c0fb220 0x7fe13c10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe12c06d7e0 0x7fe12c06fc90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 --2- 172.21.15.40:0/1354082275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe13c0fe2b0 0x7fe13c10df40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 >> 172.21.15.40:0/1354082275 conn(0x7fe13c0f6ca0 msgr2=0x7fe13c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:48.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 shutdown_connections 2024-02-16T19:33:48.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.421+0000 7fe14300a700 1 -- 172.21.15.40:0/1354082275 wait complete. 2024-02-16T19:33:48.434 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:33:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:48 smithi163 ceph-mon[81637]: pgmap v239: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:33:48.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.546+0000 7f25c72d4700 1 -- 172.21.15.40:0/3284184699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 msgr2=0x7f25c0070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.546+0000 7f25c72d4700 1 --2- 172.21.15.40:0/3284184699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 0x7f25c0070cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f25bc004940 tx=0x7f25bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:48.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.546+0000 7f25c72d4700 1 -- 172.21.15.40:0/3284184699 shutdown_connections 2024-02-16T19:33:48.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.546+0000 7f25c72d4700 1 --2- 172.21.15.40:0/3284184699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 0x7f25c0070cb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.546+0000 7f25c72d4700 1 --2- 172.21.15.40:0/3284184699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c0071e90 0x7f25c0070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.546+0000 7f25c72d4700 1 -- 172.21.15.40:0/3284184699 >> 172.21.15.40:0/3284184699 conn(0x7f25c00f8620 msgr2=0x7f25c00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.546+0000 7f25c72d4700 1 -- 172.21.15.40:0/3284184699 shutdown_connections 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c72d4700 1 -- 172.21.15.40:0/3284184699 wait complete. 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c72d4700 1 Processor -- start 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c72d4700 1 -- start start 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c72d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 0x7f25c010fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c72d4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c0071e90 0x7f25c01100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c72d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01106f0 con 0x7f25c0070840 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c72d4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0110f90 con 0x7f25c0071e90 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c5070700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 0x7f25c010fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c5070700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 0x7f25c010fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58574/0 (socket says 172.21.15.40:58574) 2024-02-16T19:33:48.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.547+0000 7f25c5070700 1 -- 172.21.15.40:0/2367088088 learned_addr learned my addr 172.21.15.40:0/2367088088 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:48.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25c486f700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c0071e90 0x7f25c01100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25c5070700 1 -- 172.21.15.40:0/2367088088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c0071e90 msgr2=0x7f25c01100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25c5070700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c0071e90 0x7f25c01100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25c5070700 1 -- 172.21.15.40:0/2367088088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc005040 con 0x7f25c0070840 2024-02-16T19:33:48.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25c5070700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 0x7f25c010fbb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25b4002a50 tx=0x7f25b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:48.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25b1ffb700 1 -- 172.21.15.40:0/2367088088 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b4007cc0 con 0x7f25c0070840 2024-02-16T19:33:48.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25b1ffb700 1 -- 172.21.15.40:0/2367088088 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25b400f040 con 0x7f25c0070840 2024-02-16T19:33:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25b1ffb700 1 -- 172.21.15.40:0/2367088088 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b400da30 con 0x7f25c0070840 2024-02-16T19:33:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c0111270 con 0x7f25c0070840 2024-02-16T19:33:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.548+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c01117b0 con 0x7f25c0070840 2024-02-16T19:33:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.550+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c004bed0 con 0x7f25c0070840 2024-02-16T19:33:48.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.550+0000 7f25b1ffb700 1 -- 172.21.15.40:0/2367088088 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f25b400db90 con 0x7f25c0070840 2024-02-16T19:33:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.551+0000 7f25b1ffb700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f25ac06d7e0 0x7f25ac06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.551+0000 7f25b1ffb700 1 -- 172.21.15.40:0/2367088088 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f25b408cd20 con 0x7f25c0070840 2024-02-16T19:33:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.551+0000 7f25c486f700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f25ac06d7e0 0x7f25ac06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.551+0000 7f25c486f700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f25ac06d7e0 0x7f25ac06fc90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25bc004f90 tx=0x7f25bc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:48.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.557+0000 7f25b1ffb700 1 -- 172.21.15.40:0/2367088088 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25b4059ff0 con 0x7f25c0070840 2024-02-16T19:33:48.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.783+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25c0111bd0 con 0x7f25ac06d7e0 2024-02-16T19:33:48.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.785+0000 7f25b1ffb700 1 -- 172.21.15.40:0/2367088088 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f25c0111bd0 con 0x7f25ac06d7e0 2024-02-16T19:33:48.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f25ac06d7e0 msgr2=0x7f25ac06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f25ac06d7e0 0x7f25ac06fc90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25bc004f90 tx=0x7f25bc009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 msgr2=0x7f25c010fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 0x7f25c010fbb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25b4002a50 tx=0x7f25b40046b0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 shutdown_connections 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f25ac06d7e0 0x7f25ac06fc90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25c0070840 0x7f25c010fbb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 --2- 172.21.15.40:0/2367088088 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c0071e90 0x7f25c01100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 >> 172.21.15.40:0/2367088088 conn(0x7f25c00f8620 msgr2=0x7f25c0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 shutdown_connections 2024-02-16T19:33:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.787+0000 7f25c72d4700 1 -- 172.21.15.40:0/2367088088 wait complete. 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.892+0000 7f999cca1700 1 -- 172.21.15.40:0/386102472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 msgr2=0x7f99980fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.892+0000 7f999cca1700 1 --2- 172.21.15.40:0/386102472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 0x7f99980fd420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9988004940 tx=0x7f9988009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.892+0000 7f999cca1700 1 -- 172.21.15.40:0/386102472 shutdown_connections 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.892+0000 7f999cca1700 1 --2- 172.21.15.40:0/386102472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 0x7f99980fd420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.892+0000 7f999cca1700 1 --2- 172.21.15.40:0/386102472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980fbe40 0x7f99980fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.892+0000 7f999cca1700 1 -- 172.21.15.40:0/386102472 >> 172.21.15.40:0/386102472 conn(0x7f99980f74e0 msgr2=0x7f99980f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.892+0000 7f999cca1700 1 -- 172.21.15.40:0/386102472 shutdown_connections 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.892+0000 7f999cca1700 1 -- 172.21.15.40:0/386102472 wait complete. 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f999cca1700 1 Processor -- start 2024-02-16T19:33:48.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f999cca1700 1 -- start start 2024-02-16T19:33:48.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f999cca1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980fbe40 0x7f999810fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f999cca1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 0x7f999810fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f999cca1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99981105f0 con 0x7f99980fcfb0 2024-02-16T19:33:48.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f999cca1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998110e90 con 0x7f99980fbe40 2024-02-16T19:33:48.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f9995d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 0x7f999810fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f9995d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 0x7f999810fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58596/0 (socket says 172.21.15.40:58596) 2024-02-16T19:33:48.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f9995d9b700 1 -- 172.21.15.40:0/380929857 learned_addr learned my addr 172.21.15.40:0/380929857 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:48.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f999659c700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980fbe40 0x7f999810fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f9995d9b700 1 -- 172.21.15.40:0/380929857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980fbe40 msgr2=0x7f999810fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:48.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f9995d9b700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980fbe40 0x7f999810fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f9995d9b700 1 -- 172.21.15.40:0/380929857 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988005040 con 0x7f99980fcfb0 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.893+0000 7f9995d9b700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 0x7f999810fff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99880093b0 tx=0x7f9988009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.894+0000 7f998effd700 1 -- 172.21.15.40:0/380929857 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998800e040 con 0x7f99980fcfb0 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.894+0000 7f998effd700 1 -- 172.21.15.40:0/380929857 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f998800bcc0 con 0x7f99980fcfb0 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.894+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9998111110 con 0x7f99980fcfb0 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.894+0000 7f998effd700 1 -- 172.21.15.40:0/380929857 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f998801c630 con 0x7f99980fcfb0 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.894+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9998111600 con 0x7f99980fcfb0 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.895+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999804bed0 con 0x7f99980fcfb0 2024-02-16T19:33:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.895+0000 7f998effd700 1 -- 172.21.15.40:0/380929857 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9988013420 con 0x7f99980fcfb0 2024-02-16T19:33:48.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.896+0000 7f998effd700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f998406d7e0 0x7f998406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:48.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.896+0000 7f998effd700 1 -- 172.21.15.40:0/380929857 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f998808e420 con 0x7f99980fcfb0 2024-02-16T19:33:48.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.896+0000 7f999659c700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f998406d7e0 0x7f998406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:48.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.896+0000 7f999659c700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f998406d7e0 0x7f998406fc90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9980009fd0 tx=0x7f9980009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:48.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:48.901+0000 7f998effd700 1 -- 172.21.15.40:0/380929857 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f998805b710 con 0x7f99980fcfb0 2024-02-16T19:33:49.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.119+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9998111a70 con 0x7f998406d7e0 2024-02-16T19:33:49.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.130+0000 7f998effd700 1 -- 172.21.15.40:0/380929857 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9998111a70 con 0x7f998406d7e0 2024-02-16T19:33:49.131 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:33:49.131 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (8m) 0s ago 17m 25.1M - 0.25.0 c8568f914cd2 d3aacf297346 2024-02-16T19:33:49.131 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (5m) 0s ago 17m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (5m) 19s ago 16m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (7m) 0s ago 17m 87.6M - 9.4.7 954c08fa6188 029f4a4498c2 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (80s) 0s ago 11m 15.4M - 19.0.0-1450-gf78a58c0 f081f847070a 084709bde4d3 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (102s) 0s ago 11m 72.1M - 19.0.0-1450-gf78a58c0 f081f847070a 79213f1b751f 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (38s) 19s ago 11m 13.9M - 19.0.0-1450-gf78a58c0 f081f847070a 8358b866ac44 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (62s) 19s ago 11m 12.7M - 19.0.0-1450-gf78a58c0 f081f847070a 33502c65114c 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (7m) 0s ago 18m 511M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (7m) 19s ago 16m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (6m) 0s ago 18m 54.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (6m) 19s ago 16m 40.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (26s) 0s ago 17m 8510k - 1.5.0 0da6a335fe13 e7da1cc993ec 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (22s) 19s ago 16m 7994k - 1.5.0 0da6a335fe13 b6ef06e45eb0 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (5m) 0s ago 15m 108M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (4m) 0s ago 15m 115M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:33:49.132 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (4m) 0s ago 14m 89.8M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:33:49.133 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (3m) 19s ago 14m 111M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:33:49.133 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (3m) 19s ago 13m 100M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:33:49.133 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (2m) 19s ago 13m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 807b119a828c 2024-02-16T19:33:49.133 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (5s) 0s ago 16m 40.2M - 2.43.0 a07b618ecd1d 8a7a0a27c8fb 2024-02-16T19:33:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f998406d7e0 msgr2=0x7f998406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f998406d7e0 0x7f998406fc90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9980009fd0 tx=0x7f9980009f60 comp rx=0 tx=0).stop 2024-02-16T19:33:49.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 msgr2=0x7f999810fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 0x7f999810fff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99880093b0 tx=0x7f9988009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 shutdown_connections 2024-02-16T19:33:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980fbe40 0x7f999810fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f998406d7e0 0x7f998406fc90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 --2- 172.21.15.40:0/380929857 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99980fcfb0 0x7f999810fff0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 >> 172.21.15.40:0/380929857 conn(0x7f99980f74e0 msgr2=0x7f99980656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 shutdown_connections 2024-02-16T19:33:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.132+0000 7f999cca1700 1 -- 172.21.15.40:0/380929857 wait complete. 2024-02-16T19:33:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 -- 172.21.15.40:0/3714138386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fd460 msgr2=0x7fb0840fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 --2- 172.21.15.40:0/3714138386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fd460 0x7fb0840fd870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb078004940 tx=0x7fb078009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 -- 172.21.15.40:0/3714138386 shutdown_connections 2024-02-16T19:33:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 --2- 172.21.15.40:0/3714138386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0840fc1c0 0x7fb0840fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 --2- 172.21.15.40:0/3714138386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fd460 0x7fb0840fd870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 -- 172.21.15.40:0/3714138386 >> 172.21.15.40:0/3714138386 conn(0x7fb0840f6ca0 msgr2=0x7fb0840f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 -- 172.21.15.40:0/3714138386 shutdown_connections 2024-02-16T19:33:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 -- 172.21.15.40:0/3714138386 wait complete. 2024-02-16T19:33:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 Processor -- start 2024-02-16T19:33:49.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.255+0000 7fb088ae8700 1 -- start start 2024-02-16T19:33:49.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb088ae8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fc1c0 0x7fb08410fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb088ae8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0840fd460 0x7fb084110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb088ae8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb084110750 con 0x7fb0840fc1c0 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb088ae8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb084110ff0 con 0x7fb0840fd460 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb08259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fc1c0 0x7fb08410fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb08259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fc1c0 0x7fb08410fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58618/0 (socket says 172.21.15.40:58618) 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb08259c700 1 -- 172.21.15.40:0/1210439307 learned_addr learned my addr 172.21.15.40:0/1210439307 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb081d9b700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0840fd460 0x7fb084110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb08259c700 1 -- 172.21.15.40:0/1210439307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0840fd460 msgr2=0x7fb084110150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb08259c700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0840fd460 0x7fb084110150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb08259c700 1 -- 172.21.15.40:0/1210439307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078005040 con 0x7fb0840fc1c0 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb081d9b700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0840fd460 0x7fb084110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb08259c700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fc1c0 0x7fb08410fc10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb078009c90 tx=0x7fb078009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb072ffd700 1 -- 172.21.15.40:0/1210439307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07800e040 con 0x7fb0840fc1c0 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.256+0000 7fb072ffd700 1 -- 172.21.15.40:0/1210439307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb078004290 con 0x7fb0840fc1c0 2024-02-16T19:33:49.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.257+0000 7fb072ffd700 1 -- 172.21.15.40:0/1210439307 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07801c3f0 con 0x7fb0840fc1c0 2024-02-16T19:33:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.257+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb084111270 con 0x7fb0840fc1c0 2024-02-16T19:33:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.257+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb084111760 con 0x7fb0840fc1c0 2024-02-16T19:33:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.258+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08404bed0 con 0x7fb0840fc1c0 2024-02-16T19:33:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.258+0000 7fb072ffd700 1 -- 172.21.15.40:0/1210439307 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb07800bb90 con 0x7fb0840fc1c0 2024-02-16T19:33:49.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.259+0000 7fb072ffd700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb06c06d7e0 0x7fb06c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.259+0000 7fb072ffd700 1 -- 172.21.15.40:0/1210439307 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb07808e300 con 0x7fb0840fc1c0 2024-02-16T19:33:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.259+0000 7fb081d9b700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb06c06d7e0 0x7fb06c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:49.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.259+0000 7fb081d9b700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb06c06d7e0 0x7fb06c06fc90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb074007fd0 tx=0x7fb074009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:49.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.263+0000 7fb072ffd700 1 -- 172.21.15.40:0/1210439307 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb07805b650 con 0x7fb0840fc1c0 2024-02-16T19:33:49.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.561+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb084111d90 con 0x7fb0840fc1c0 2024-02-16T19:33:49.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.561+0000 7fb072ffd700 1 -- 172.21.15.40:0/1210439307 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb07805b1e0 con 0x7fb0840fc1c0 2024-02-16T19:33:49.562 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:33:49.563 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T19:33:49.564 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:33:49.564 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:33:49.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb06c06d7e0 msgr2=0x7fb06c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb06c06d7e0 0x7fb06c06fc90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb074007fd0 tx=0x7fb074009040 comp rx=0 tx=0).stop 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fc1c0 msgr2=0x7fb08410fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fc1c0 0x7fb08410fc10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb078009c90 tx=0x7fb078009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 shutdown_connections 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb06c06d7e0 0x7fb06c06fc90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0840fc1c0 0x7fb08410fc10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 --2- 172.21.15.40:0/1210439307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0840fd460 0x7fb084110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 >> 172.21.15.40:0/1210439307 conn(0x7fb0840f6ca0 msgr2=0x7fb0840fb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 shutdown_connections 2024-02-16T19:33:49.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.563+0000 7fb088ae8700 1 -- 172.21.15.40:0/1210439307 wait complete. 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.669+0000 7f476da9d700 1 -- 172.21.15.40:0/2391333702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fe0e0 msgr2=0x7f47680fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.669+0000 7f476da9d700 1 --2- 172.21.15.40:0/2391333702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fe0e0 0x7f47680fe550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4758004940 tx=0x7f4758009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.669+0000 7f476da9d700 1 -- 172.21.15.40:0/2391333702 shutdown_connections 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.669+0000 7f476da9d700 1 --2- 172.21.15.40:0/2391333702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fe0e0 0x7f47680fe550 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.669+0000 7f476da9d700 1 --2- 172.21.15.40:0/2391333702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47680fcef0 0x7f47680fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.669+0000 7f476da9d700 1 -- 172.21.15.40:0/2391333702 >> 172.21.15.40:0/2391333702 conn(0x7f47680f8620 msgr2=0x7f47680faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.669+0000 7f476da9d700 1 -- 172.21.15.40:0/2391333702 shutdown_connections 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.669+0000 7f476da9d700 1 -- 172.21.15.40:0/2391333702 wait complete. 2024-02-16T19:33:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.670+0000 7f476da9d700 1 Processor -- start 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.670+0000 7f476da9d700 1 -- start start 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.670+0000 7f476da9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fcef0 0x7f4768071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.670+0000 7f476da9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47680fe0e0 0x7f47680702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.670+0000 7f476da9d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681115d0 con 0x7f47680fcef0 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.670+0000 7f476da9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4768111740 con 0x7f47680fe0e0 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.670+0000 7f47677fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fcef0 0x7f4768071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f47677fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fcef0 0x7f4768071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58650/0 (socket says 172.21.15.40:58650) 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f47677fe700 1 -- 172.21.15.40:0/1711709959 learned_addr learned my addr 172.21.15.40:0/1711709959 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f4766ffd700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47680fe0e0 0x7f47680702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f47677fe700 1 -- 172.21.15.40:0/1711709959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47680fe0e0 msgr2=0x7f47680702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f47677fe700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47680fe0e0 0x7f47680702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f47677fe700 1 -- 172.21.15.40:0/1711709959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4758005040 con 0x7f47680fcef0 2024-02-16T19:33:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f4766ffd700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47680fe0e0 0x7f47680702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:33:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f47677fe700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fcef0 0x7f4768071e60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f475000eb70 tx=0x7f475000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f475ffff700 1 -- 172.21.15.40:0/1711709959 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4750007cc0 con 0x7f47680fcef0 2024-02-16T19:33:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f475ffff700 1 -- 172.21.15.40:0/1711709959 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f475000f040 con 0x7f47680fcef0 2024-02-16T19:33:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47680708e0 con 0x7f47680fcef0 2024-02-16T19:33:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f475ffff700 1 -- 172.21.15.40:0/1711709959 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f475000cbd0 con 0x7f47680fcef0 2024-02-16T19:33:49.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.671+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4768070e20 con 0x7f47680fcef0 2024-02-16T19:33:49.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.672+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476804bed0 con 0x7f47680fcef0 2024-02-16T19:33:49.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.673+0000 7f475ffff700 1 -- 172.21.15.40:0/1711709959 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f475000cd30 con 0x7f47680fcef0 2024-02-16T19:33:49.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.673+0000 7f475ffff700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f475406d7f0 0x7f475406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:49.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.674+0000 7f475ffff700 1 -- 172.21.15.40:0/1711709959 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f475008ca40 con 0x7f47680fcef0 2024-02-16T19:33:49.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.674+0000 7f4766ffd700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f475406d7f0 0x7f475406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:49.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.674+0000 7f4766ffd700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f475406d7f0 0x7f475406fca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f475800b0d0 tx=0x7f4758009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:49.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.679+0000 7f475ffff700 1 -- 172.21.15.40:0/1711709959 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4750059dc0 con 0x7f47680fcef0 2024-02-16T19:33:49.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:49 smithi040 ceph-mon[94210]: pgmap v240: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:49.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:49 smithi040 ceph-mon[94210]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:49.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:49.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:49 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:49.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:49 smithi040 ceph-mon[94210]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:49.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:49 smithi040 ceph-mon[94210]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:49.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:49 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1210439307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.946+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f476811a110 con 0x7f47680fcef0 2024-02-16T19:33:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.946+0000 7f475ffff700 1 -- 172.21.15.40:0/1711709959 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 41 v41) v1 ==== 76+0+1880 (secure 0 0 0) 0x7f4750059950 con 0x7f47680fcef0 2024-02-16T19:33:49.947 INFO:teuthology.orchestra.run.smithi040.stdout:e41 2024-02-16T19:33:49.947 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 41 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:33:18.695800+0000 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:33:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 83 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=34404,1=34436} 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:33:49.949 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{0:34404} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{1:34436} state up:active seq 14 join_fscid=1 addr [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6824/1456075822,v1:172.21.15.163:6825/1456075822] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:33:49.950 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6826/2051774557,v1:172.21.15.163:6827/2051774557] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.948+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f475406d7f0 msgr2=0x7f475406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.948+0000 7f476da9d700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f475406d7f0 0x7f475406fca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f475800b0d0 tx=0x7f4758009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.948+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fcef0 msgr2=0x7f4768071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.948+0000 7f476da9d700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fcef0 0x7f4768071e60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f475000eb70 tx=0x7f475000b040 comp rx=0 tx=0).stop 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.949+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 shutdown_connections 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.949+0000 7f476da9d700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f475406d7f0 0x7f475406fca0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.949+0000 7f476da9d700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47680fcef0 0x7f4768071e60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.949+0000 7f476da9d700 1 --2- 172.21.15.40:0/1711709959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47680fe0e0 0x7f47680702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.949+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 >> 172.21.15.40:0/1711709959 conn(0x7f47680f8620 msgr2=0x7f47681013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.949+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 shutdown_connections 2024-02-16T19:33:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:49.949+0000 7f476da9d700 1 -- 172.21.15.40:0/1711709959 wait complete. 2024-02-16T19:33:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2024-02-16T19:33:49.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:49 smithi163 ceph-mon[81637]: pgmap v240: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:49.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:49 smithi163 ceph-mon[81637]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:49.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:49.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:49 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:49.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:49 smithi163 ceph-mon[81637]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:49.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:49 smithi163 ceph-mon[81637]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:49.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:49 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1210439307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 -- 172.21.15.40:0/3935966768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fe0e0 msgr2=0x7fcc640fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 --2- 172.21.15.40:0/3935966768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fe0e0 0x7fcc640fe550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc58004940 tx=0x7fcc58009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 -- 172.21.15.40:0/3935966768 shutdown_connections 2024-02-16T19:33:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 --2- 172.21.15.40:0/3935966768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fe0e0 0x7fcc640fe550 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 --2- 172.21.15.40:0/3935966768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc640fcef0 0x7fcc640fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 -- 172.21.15.40:0/3935966768 >> 172.21.15.40:0/3935966768 conn(0x7fcc640f8620 msgr2=0x7fcc640faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 -- 172.21.15.40:0/3935966768 shutdown_connections 2024-02-16T19:33:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 -- 172.21.15.40:0/3935966768 wait complete. 2024-02-16T19:33:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.055+0000 7fcc69f8f700 1 Processor -- start 2024-02-16T19:33:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc69f8f700 1 -- start start 2024-02-16T19:33:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc69f8f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fcef0 0x7fcc6410d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc69f8f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc640fe0e0 0x7fcc6410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc69f8f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6410e540 con 0x7fcc640fcef0 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc69f8f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6410ed80 con 0x7fcc640fe0e0 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fcef0 0x7fcc6410d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fcef0 0x7fcc6410d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58670/0 (socket says 172.21.15.40:58670) 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc637fe700 1 -- 172.21.15.40:0/1753033282 learned_addr learned my addr 172.21.15.40:0/1753033282 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc62ffd700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc640fe0e0 0x7fcc6410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc637fe700 1 -- 172.21.15.40:0/1753033282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc640fe0e0 msgr2=0x7fcc6410deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc637fe700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc640fe0e0 0x7fcc6410deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc637fe700 1 -- 172.21.15.40:0/1753033282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58005040 con 0x7fcc640fcef0 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.056+0000 7fcc637fe700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fcef0 0x7fcc6410d970 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400b790 tx=0x7fcc54009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:50.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.057+0000 7fcc4bfff700 1 -- 172.21.15.40:0/1753033282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc5400f6f0 con 0x7fcc640fcef0 2024-02-16T19:33:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.057+0000 7fcc4bfff700 1 -- 172.21.15.40:0/1753033282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc5400fd20 con 0x7fcc640fcef0 2024-02-16T19:33:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.057+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6410f060 con 0x7fcc640fcef0 2024-02-16T19:33:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.057+0000 7fcc4bfff700 1 -- 172.21.15.40:0/1753033282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc540103a0 con 0x7fcc640fcef0 2024-02-16T19:33:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.057+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6410f5a0 con 0x7fcc640fcef0 2024-02-16T19:33:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.058+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6404bed0 con 0x7fcc640fcef0 2024-02-16T19:33:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.058+0000 7fcc4bfff700 1 -- 172.21.15.40:0/1753033282 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcc54011400 con 0x7fcc640fcef0 2024-02-16T19:33:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.059+0000 7fcc4bfff700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcc4c06d7e0 0x7fcc4c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.059+0000 7fcc4bfff700 1 -- 172.21.15.40:0/1753033282 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcc5408cd00 con 0x7fcc640fcef0 2024-02-16T19:33:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.059+0000 7fcc62ffd700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcc4c06d7e0 0x7fcc4c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.060+0000 7fcc62ffd700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcc4c06d7e0 0x7fcc4c06fc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcc58004f90 tx=0x7fcc58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.064+0000 7fcc4bfff700 1 -- 172.21.15.40:0/1753033282 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc54090030 con 0x7fcc640fcef0 2024-02-16T19:33:50.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.297+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc64102ac0 con 0x7fcc4c06d7e0 2024-02-16T19:33:50.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.299+0000 7fcc4bfff700 1 -- 172.21.15.40:0/1753033282 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fcc64102ac0 con 0x7fcc4c06d7e0 2024-02-16T19:33:50.300 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:33:50.300 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:33:50.300 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "osd", 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "mds", 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "16/21 daemons upgraded", 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading prometheus daemons", 2024-02-16T19:33:50.301 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:33:50.302 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:33:50.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcc4c06d7e0 msgr2=0x7fcc4c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:50.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcc4c06d7e0 0x7fcc4c06fc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcc58004f90 tx=0x7fcc58009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fcef0 msgr2=0x7fcc6410d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fcef0 0x7fcc6410d970 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400b790 tx=0x7fcc54009220 comp rx=0 tx=0).stop 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 shutdown_connections 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fcc4c06d7e0 0x7fcc4c06fc90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc640fcef0 0x7fcc6410d970 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 --2- 172.21.15.40:0/1753033282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc640fe0e0 0x7fcc6410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 >> 172.21.15.40:0/1753033282 conn(0x7fcc640f8620 msgr2=0x7fcc641013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 shutdown_connections 2024-02-16T19:33:50.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.301+0000 7fcc69f8f700 1 -- 172.21.15.40:0/1753033282 wait complete. 2024-02-16T19:33:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 -- 172.21.15.40:0/3888807574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc1000c0 msgr2=0x7fbafc102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 --2- 172.21.15.40:0/3888807574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc1000c0 0x7fbafc102520 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0004940 tx=0x7fbaf0009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 -- 172.21.15.40:0/3888807574 shutdown_connections 2024-02-16T19:33:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 --2- 172.21.15.40:0/3888807574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc1000c0 0x7fbafc102520 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 --2- 172.21.15.40:0/3888807574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbafc066c70 0x7fbafc0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 -- 172.21.15.40:0/3888807574 >> 172.21.15.40:0/3888807574 conn(0x7fbafc0f7500 msgr2=0x7fbafc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:50.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 -- 172.21.15.40:0/3888807574 shutdown_connections 2024-02-16T19:33:50.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 -- 172.21.15.40:0/3888807574 wait complete. 2024-02-16T19:33:50.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 Processor -- start 2024-02-16T19:33:50.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.405+0000 7fbb03d0a700 1 -- start start 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb03d0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc066c70 0x7fbafc10d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb03d0a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbafc1000c0 0x7fbafc10de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb03d0a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc10e410 con 0x7fbafc066c70 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb03d0a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc10ecb0 con 0x7fbafc1000c0 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb01aa6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc066c70 0x7fbafc10d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb01aa6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc066c70 0x7fbafc10d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58688/0 (socket says 172.21.15.40:58688) 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb01aa6700 1 -- 172.21.15.40:0/1850943949 learned_addr learned my addr 172.21.15.40:0/1850943949 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb012a5700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbafc1000c0 0x7fbafc10de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb01aa6700 1 -- 172.21.15.40:0/1850943949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbafc1000c0 msgr2=0x7fbafc10de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb01aa6700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbafc1000c0 0x7fbafc10de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb01aa6700 1 -- 172.21.15.40:0/1850943949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf0005040 con 0x7fbafc066c70 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb012a5700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbafc1000c0 0x7fbafc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb01aa6700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc066c70 0x7fbafc10d8d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbaec001db0 tx=0x7fbaec009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbafa7fc700 1 -- 172.21.15.40:0/1850943949 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaec00ed50 con 0x7fbafc066c70 2024-02-16T19:33:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbafa7fc700 1 -- 172.21.15.40:0/1850943949 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbaec007df0 con 0x7fbafc066c70 2024-02-16T19:33:50.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc10ef90 con 0x7fbafc066c70 2024-02-16T19:33:50.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.406+0000 7fbafa7fc700 1 -- 172.21.15.40:0/1850943949 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaec014640 con 0x7fbafc066c70 2024-02-16T19:33:50.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.407+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc10f4d0 con 0x7fbafc066c70 2024-02-16T19:33:50.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.408+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc04bed0 con 0x7fbafc066c70 2024-02-16T19:33:50.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.408+0000 7fbafa7fc700 1 -- 172.21.15.40:0/1850943949 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbaec009360 con 0x7fbafc066c70 2024-02-16T19:33:50.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.409+0000 7fbafa7fc700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbae406d7e0 0x7fbae406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:33:50.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.409+0000 7fbafa7fc700 1 -- 172.21.15.40:0/1850943949 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbaec08ce10 con 0x7fbafc066c70 2024-02-16T19:33:50.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.409+0000 7fbb012a5700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbae406d7e0 0x7fbae406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:33:50.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.409+0000 7fbb012a5700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbae406d7e0 0x7fbae406fc90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000b3b0 tx=0x7fbaf0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:33:50.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.414+0000 7fbafa7fc700 1 -- 172.21.15.40:0/1850943949 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbaec05a190 con 0x7fbafc066c70 2024-02-16T19:33:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.708+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbafc105a10 con 0x7fbafc066c70 2024-02-16T19:33:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.709+0000 7fbafa7fc700 1 -- 172.21.15.40:0/1850943949 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbaec059d20 con 0x7fbafc066c70 2024-02-16T19:33:50.710 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:33:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbae406d7e0 msgr2=0x7fbae406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbae406d7e0 0x7fbae406fc90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000b3b0 tx=0x7fbaf0009e30 comp rx=0 tx=0).stop 2024-02-16T19:33:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc066c70 msgr2=0x7fbafc10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:33:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc066c70 0x7fbafc10d8d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbaec001db0 tx=0x7fbaec009ee0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 shutdown_connections 2024-02-16T19:33:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbae406d7e0 0x7fbae406fc90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbafc066c70 0x7fbafc10d8d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 --2- 172.21.15.40:0/1850943949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbafc1000c0 0x7fbafc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:33:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 >> 172.21.15.40:0/1850943949 conn(0x7fbafc0f7500 msgr2=0x7fbafc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:33:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 shutdown_connections 2024-02-16T19:33:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:33:50.711+0000 7fbb03d0a700 1 -- 172.21.15.40:0/1850943949 wait complete. 2024-02-16T19:33:51.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:50 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:51.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:50 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:51.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:50 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1711709959' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:33:51.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:50 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1850943949' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:33:51.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:50 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:51.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:50 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:51.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:50 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1711709959' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:33:51.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:50 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1850943949' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:33:51.936 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:51 smithi040 ceph-mon[94210]: pgmap v241: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:51.936 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:51 smithi040 ceph-mon[94210]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:52.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:51 smithi163 ceph-mon[81637]: pgmap v241: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:52.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:51 smithi163 ceph-mon[81637]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:33:54.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: pgmap v242: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:54.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:54.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:54.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:54.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:54.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:54.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:33:54.145 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.146 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.147 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.147 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.147 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:53 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: pgmap v242: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.239 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.239 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.239 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.239 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.239 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.239 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.239 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:54.239 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:53 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:33:55.031 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:54 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:33:55.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:54 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-16T19:33:55.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:55 smithi040 ceph-mon[94210]: pgmap v243: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:55.930 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:55 smithi040 ceph-mon[94210]: Upgrade: Updating alertmanager.smithi040 2024-02-16T19:33:55.930 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:55 smithi040 ceph-mon[94210]: Deploying daemon alertmanager.smithi040 on smithi040 2024-02-16T19:33:56.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:55 smithi163 ceph-mon[81637]: pgmap v243: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:56.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:55 smithi163 ceph-mon[81637]: Upgrade: Updating alertmanager.smithi040 2024-02-16T19:33:56.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:55 smithi163 ceph-mon[81637]: Deploying daemon alertmanager.smithi040 on smithi040 2024-02-16T19:33:58.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:57 smithi040 ceph-mon[94210]: pgmap v244: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:33:58.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:57 smithi163 ceph-mon[81637]: pgmap v244: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:00.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:33:59 smithi040 ceph-mon[94210]: pgmap v245: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:00.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:33:59 smithi163 ceph-mon[81637]: pgmap v245: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:01.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:01 smithi040 ceph-mon[94210]: pgmap v246: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:01.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:01 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:01.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:01 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:01.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:01 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:01 smithi163 ceph-mon[81637]: pgmap v246: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:01 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:01 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:02.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:01 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:02.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:03.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:02 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:04.082 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:03 smithi040 ceph-mon[94210]: pgmap v247: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:04.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:03 smithi163 ceph-mon[81637]: pgmap v247: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:06.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:05 smithi040 ceph-mon[94210]: pgmap v248: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:06.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:05 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:06.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:05 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:06.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:05 smithi163 ceph-mon[81637]: pgmap v248: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:06.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:05 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:06.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:05 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:07 smithi040 ceph-mon[94210]: pgmap v249: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:07 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:07 smithi163 ceph-mon[81637]: pgmap v249: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:07 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:09.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:09 smithi040 ceph-mon[94210]: pgmap v250: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:09 smithi163 ceph-mon[81637]: pgmap v250: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:11.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:11.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:11.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:11.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:11.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:11.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:34:11.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:11.207 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.208 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.209 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:11 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:11.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:11.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:11 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:12 smithi040 ceph-mon[94210]: pgmap v251: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:12.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:12 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:34:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:12 smithi163 ceph-mon[81637]: pgmap v251: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:12 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-16T19:34:13.342 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:13 smithi040 ceph-mon[94210]: Upgrade: Updating grafana.smithi040 2024-02-16T19:34:13.342 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:13 smithi040 ceph-mon[94210]: Deploying daemon grafana.smithi040 on smithi040 2024-02-16T19:34:13.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:13 smithi163 ceph-mon[81637]: Upgrade: Updating grafana.smithi040 2024-02-16T19:34:13.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:13 smithi163 ceph-mon[81637]: Deploying daemon grafana.smithi040 on smithi040 2024-02-16T19:34:14.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:14 smithi040 ceph-mon[94210]: pgmap v252: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:14 smithi163 ceph-mon[81637]: pgmap v252: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:15.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:15 smithi040 ceph-mon[94210]: pgmap v253: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:15.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:15 smithi163 ceph-mon[81637]: pgmap v253: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:18.377 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:18 smithi040 ceph-mon[94210]: pgmap v254: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:18.377 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:18.377 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:18.377 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:18.377 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:18 smithi163 ceph-mon[81637]: pgmap v254: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:20.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:20 smithi040 ceph-mon[94210]: pgmap v255: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:20.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:20 smithi163 ceph-mon[81637]: pgmap v255: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/2961537381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fe0e0 msgr2=0x7f1cb80fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 --2- 172.21.15.40:0/2961537381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fe0e0 0x7f1cb80fe550 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1cac004940 tx=0x7f1cac009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:20.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/2961537381 shutdown_connections 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 --2- 172.21.15.40:0/2961537381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fe0e0 0x7f1cb80fe550 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 --2- 172.21.15.40:0/2961537381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb80fcef0 0x7f1cb80fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/2961537381 >> 172.21.15.40:0/2961537381 conn(0x7f1cb80f8620 msgr2=0x7f1cb80faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/2961537381 shutdown_connections 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/2961537381 wait complete. 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 Processor -- start 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.828+0000 7f1cbcc7a700 1 -- start start 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cbcc7a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fcef0 0x7f1cb8071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cbcc7a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb80fe0e0 0x7f1cb80702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cbcc7a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb8070970 con 0x7f1cb80fcef0 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cbcc7a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb8070ae0 con 0x7f1cb80fe0e0 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cb659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fcef0 0x7f1cb8071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:20.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cb659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fcef0 0x7f1cb8071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46516/0 (socket says 172.21.15.40:46516) 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cb659c700 1 -- 172.21.15.40:0/1094141912 learned_addr learned my addr 172.21.15.40:0/1094141912 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cb5d9b700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb80fe0e0 0x7f1cb80702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cb659c700 1 -- 172.21.15.40:0/1094141912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb80fe0e0 msgr2=0x7f1cb80702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cb659c700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb80fe0e0 0x7f1cb80702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cb659c700 1 -- 172.21.15.40:0/1094141912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac005040 con 0x7f1cb80fcef0 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cb659c700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fcef0 0x7f1cb8071e60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800b790 tx=0x7f1ca8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1ca6ffd700 1 -- 172.21.15.40:0/1094141912 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca800f6f0 con 0x7f1cb80fcef0 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1ca6ffd700 1 -- 172.21.15.40:0/1094141912 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca800fd20 con 0x7f1cb80fcef0 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb8070dc0 con 0x7f1cb80fcef0 2024-02-16T19:34:20.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1ca6ffd700 1 -- 172.21.15.40:0/1094141912 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca80103a0 con 0x7f1cb80fcef0 2024-02-16T19:34:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.829+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb8071140 con 0x7f1cb80fcef0 2024-02-16T19:34:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.830+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb804bed0 con 0x7f1cb80fcef0 2024-02-16T19:34:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.831+0000 7f1ca6ffd700 1 -- 172.21.15.40:0/1094141912 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1ca8011400 con 0x7f1cb80fcef0 2024-02-16T19:34:20.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.831+0000 7f1ca6ffd700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1ca006d7e0 0x7f1ca006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:20.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.831+0000 7f1ca6ffd700 1 -- 172.21.15.40:0/1094141912 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1ca808cd10 con 0x7f1cb80fcef0 2024-02-16T19:34:20.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.831+0000 7f1cb5d9b700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1ca006d7e0 0x7f1ca006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:20.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.832+0000 7f1cb5d9b700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1ca006d7e0 0x7f1ca006fc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1cac004f90 tx=0x7f1cac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:20.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:20.836+0000 7f1ca6ffd700 1 -- 172.21.15.40:0/1094141912 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ca805a060 con 0x7f1cb80fcef0 2024-02-16T19:34:21.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.062+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cb8102ac0 con 0x7f1ca006d7e0 2024-02-16T19:34:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.063+0000 7f1ca6ffd700 1 -- 172.21.15.40:0/1094141912 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f1cb8102ac0 con 0x7f1ca006d7e0 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1ca006d7e0 msgr2=0x7f1ca006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1ca006d7e0 0x7f1ca006fc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1cac004f90 tx=0x7f1cac009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fcef0 msgr2=0x7f1cb8071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fcef0 0x7f1cb8071e60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800b790 tx=0x7f1ca8009220 comp rx=0 tx=0).stop 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 shutdown_connections 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1ca006d7e0 0x7f1ca006fc90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cb80fcef0 0x7f1cb8071e60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 --2- 172.21.15.40:0/1094141912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb80fe0e0 0x7f1cb80702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 >> 172.21.15.40:0/1094141912 conn(0x7f1cb80f8620 msgr2=0x7f1cb81013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.065+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 shutdown_connections 2024-02-16T19:34:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.066+0000 7f1cbcc7a700 1 -- 172.21.15.40:0/1094141912 wait complete. 2024-02-16T19:34:21.077 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:34:21.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.176+0000 7fc3340e4700 1 -- 172.21.15.40:0/151966846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fbca0 msgr2=0x7fc32c0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.176+0000 7fc3340e4700 1 --2- 172.21.15.40:0/151966846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fbca0 0x7fc32c0fe550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc31c004940 tx=0x7fc31c009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.176+0000 7fc3340e4700 1 -- 172.21.15.40:0/151966846 shutdown_connections 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.176+0000 7fc3340e4700 1 --2- 172.21.15.40:0/151966846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fbca0 0x7fc32c0fe550 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.176+0000 7fc3340e4700 1 --2- 172.21.15.40:0/151966846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fb2c0 0x7fc32c0fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.176+0000 7fc3340e4700 1 -- 172.21.15.40:0/151966846 >> 172.21.15.40:0/151966846 conn(0x7fc32c0f6ca0 msgr2=0x7fc32c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.176+0000 7fc3340e4700 1 -- 172.21.15.40:0/151966846 shutdown_connections 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.176+0000 7fc3340e4700 1 -- 172.21.15.40:0/151966846 wait complete. 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc3340e4700 1 Processor -- start 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc3340e4700 1 -- start start 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc3340e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fb2c0 0x7fc32c0fde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc3340e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fc2a0 0x7fc32c0fc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc3340e4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0fcca0 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc3340e4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0fcde0 con 0x7fc32c0fc2a0 2024-02-16T19:34:21.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc331e80700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fb2c0 0x7fc32c0fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc331e80700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fb2c0 0x7fc32c0fde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46522/0 (socket says 172.21.15.40:46522) 2024-02-16T19:34:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc331e80700 1 -- 172.21.15.40:0/4035524815 learned_addr learned my addr 172.21.15.40:0/4035524815 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.177+0000 7fc33167f700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fc2a0 0x7fc32c0fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc331e80700 1 -- 172.21.15.40:0/4035524815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fc2a0 msgr2=0x7fc32c0fc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc331e80700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fc2a0 0x7fc32c0fc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc331e80700 1 -- 172.21.15.40:0/4035524815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c005040 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc331e80700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fb2c0 0x7fc32c0fde20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc32800eb70 tx=0x7fc32800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc3227fc700 1 -- 172.21.15.40:0/4035524815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc328007cc0 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc3227fc700 1 -- 172.21.15.40:0/4035524815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc32800f040 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c0fd060 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc3227fc700 1 -- 172.21.15.40:0/4035524815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc328009640 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.178+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c115ef0 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.179+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32c04bed0 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.179+0000 7fc3227fc700 1 -- 172.21.15.40:0/4035524815 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc3280097a0 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.180+0000 7fc3227fc700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc31806d7e0 0x7fc31806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.180+0000 7fc3227fc700 1 -- 172.21.15.40:0/4035524815 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc32808cb00 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.180+0000 7fc33167f700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc31806d7e0 0x7fc31806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.181+0000 7fc33167f700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc31806d7e0 0x7fc31806fc90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00b0d0 tx=0x7fc31c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:21.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.185+0000 7fc3227fc700 1 -- 172.21.15.40:0/4035524815 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc328059e50 con 0x7fc32c0fb2c0 2024-02-16T19:34:21.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.428+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc32c0fbdb0 con 0x7fc31806d7e0 2024-02-16T19:34:21.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.430+0000 7fc3227fc700 1 -- 172.21.15.40:0/4035524815 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fc32c0fbdb0 con 0x7fc31806d7e0 2024-02-16T19:34:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc31806d7e0 msgr2=0x7fc31806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc31806d7e0 0x7fc31806fc90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00b0d0 tx=0x7fc31c009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fb2c0 msgr2=0x7fc32c0fde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fb2c0 0x7fc32c0fde20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc32800eb70 tx=0x7fc32800b040 comp rx=0 tx=0).stop 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 shutdown_connections 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fc31806d7e0 0x7fc31806fc90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0fb2c0 0x7fc32c0fde20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 --2- 172.21.15.40:0/4035524815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc32c0fc2a0 0x7fc32c0fc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 >> 172.21.15.40:0/4035524815 conn(0x7fc32c0f6ca0 msgr2=0x7fc32c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 shutdown_connections 2024-02-16T19:34:21.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.432+0000 7fc3340e4700 1 -- 172.21.15.40:0/4035524815 wait complete. 2024-02-16T19:34:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 -- 172.21.15.40:0/3235878544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 msgr2=0x7f90240fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 --2- 172.21.15.40:0/3235878544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 0x7f90240fd320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9014004940 tx=0x7f9014009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 -- 172.21.15.40:0/3235878544 shutdown_connections 2024-02-16T19:34:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 --2- 172.21.15.40:0/3235878544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90240fe0e0 0x7f90240fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 --2- 172.21.15.40:0/3235878544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 0x7f90240fd320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 -- 172.21.15.40:0/3235878544 >> 172.21.15.40:0/3235878544 conn(0x7f90240f8620 msgr2=0x7f90240faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 -- 172.21.15.40:0/3235878544 shutdown_connections 2024-02-16T19:34:21.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 -- 172.21.15.40:0/3235878544 wait complete. 2024-02-16T19:34:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.538+0000 7f9029303700 1 Processor -- start 2024-02-16T19:34:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9029303700 1 -- start start 2024-02-16T19:34:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9029303700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 0x7f9024071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9029303700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90240fe0e0 0x7f90240702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9029303700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90240708e0 con 0x7f90240fcef0 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9029303700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024070a50 con 0x7f90240fe0e0 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9022ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 0x7f9024071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9022ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 0x7f9024071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46538/0 (socket says 172.21.15.40:46538) 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9022ffd700 1 -- 172.21.15.40:0/2720994465 learned_addr learned my addr 172.21.15.40:0/2720994465 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f90227fc700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90240fe0e0 0x7f90240702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9022ffd700 1 -- 172.21.15.40:0/2720994465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90240fe0e0 msgr2=0x7f90240702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9022ffd700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90240fe0e0 0x7f90240702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9022ffd700 1 -- 172.21.15.40:0/2720994465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014005040 con 0x7f90240fcef0 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.539+0000 7f9022ffd700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 0x7f9024071e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f901400b3b0 tx=0x7f9014009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.540+0000 7f900b7fe700 1 -- 172.21.15.40:0/2720994465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901400e040 con 0x7f90240fcef0 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.540+0000 7f900b7fe700 1 -- 172.21.15.40:0/2720994465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f901400bcc0 con 0x7f90240fcef0 2024-02-16T19:34:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.540+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9024070cd0 con 0x7f90240fcef0 2024-02-16T19:34:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.540+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90241159a0 con 0x7f90240fcef0 2024-02-16T19:34:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.540+0000 7f900b7fe700 1 -- 172.21.15.40:0/2720994465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901401c630 con 0x7f90240fcef0 2024-02-16T19:34:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.541+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90100052e0 con 0x7f90240fcef0 2024-02-16T19:34:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.541+0000 7f900b7fe700 1 -- 172.21.15.40:0/2720994465 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9014013980 con 0x7f90240fcef0 2024-02-16T19:34:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.542+0000 7f900b7fe700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f900c06d7e0 0x7f900c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.542+0000 7f900b7fe700 1 -- 172.21.15.40:0/2720994465 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f901408e400 con 0x7f90240fcef0 2024-02-16T19:34:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.542+0000 7f90227fc700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f900c06d7e0 0x7f900c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.542+0000 7f90227fc700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f900c06d7e0 0x7f900c06fc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9018004f40 tx=0x7f901800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.546+0000 7f900b7fe700 1 -- 172.21.15.40:0/2720994465 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f901405b750 con 0x7f90240fcef0 2024-02-16T19:34:21.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.761+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9010000bf0 con 0x7f900c06d7e0 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.774+0000 7f900b7fe700 1 -- 172.21.15.40:0/2720994465 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9010000bf0 con 0x7f900c06d7e0 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (21s) 0s ago 17m 16.3M - 0.25.0 c8568f914cd2 d164871c90a0 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (6m) 0s ago 18m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (6m) 52s ago 17m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (5s) 0s ago 17m 59.8M - 9.4.12 a0ea5e3c46d9 1f69ae11e0c8 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (113s) 0s ago 12m 16.4M - 19.0.0-1450-gf78a58c0 f081f847070a 084709bde4d3 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (2m) 0s ago 12m 72.1M - 19.0.0-1450-gf78a58c0 f081f847070a 79213f1b751f 2024-02-16T19:34:21.775 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (71s) 52s ago 12m 13.9M - 19.0.0-1450-gf78a58c0 f081f847070a 8358b866ac44 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (94s) 52s ago 12m 12.7M - 19.0.0-1450-gf78a58c0 f081f847070a 33502c65114c 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (8m) 0s ago 19m 511M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (7m) 52s ago 16m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (7m) 0s ago 19m 55.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (7m) 52s ago 16m 40.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (59s) 0s ago 18m 8799k - 1.5.0 0da6a335fe13 e7da1cc993ec 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (54s) 52s ago 16m 7994k - 1.5.0 0da6a335fe13 b6ef06e45eb0 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (5m) 0s ago 16m 108M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (5m) 0s ago 15m 115M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (4m) 0s ago 15m 90.0M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (4m) 52s ago 14m 111M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:34:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (3m) 52s ago 14m 100M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:34:21.777 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (3m) 52s ago 13m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 807b119a828c 2024-02-16T19:34:21.777 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (38s) 0s ago 17m 47.1M - 2.43.0 a07b618ecd1d 8a7a0a27c8fb 2024-02-16T19:34:21.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f900c06d7e0 msgr2=0x7f900c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f900c06d7e0 0x7f900c06fc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9018004f40 tx=0x7f901800b040 comp rx=0 tx=0).stop 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 msgr2=0x7f9024071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 0x7f9024071e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f901400b3b0 tx=0x7f9014009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 shutdown_connections 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f900c06d7e0 0x7f900c06fc90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90240fcef0 0x7f9024071e60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 --2- 172.21.15.40:0/2720994465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90240fe0e0 0x7f90240702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 >> 172.21.15.40:0/2720994465 conn(0x7f90240f8620 msgr2=0x7f90241013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 shutdown_connections 2024-02-16T19:34:21.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.776+0000 7f9029303700 1 -- 172.21.15.40:0/2720994465 wait complete. 2024-02-16T19:34:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.880+0000 7fd65208d700 1 -- 172.21.15.40:0/3729218877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 msgr2=0x7fd64c102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.880+0000 7fd65208d700 1 --2- 172.21.15.40:0/3729218877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 0x7fd64c102520 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd640004940 tx=0x7fd640009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.880+0000 7fd65208d700 1 -- 172.21.15.40:0/3729218877 shutdown_connections 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.880+0000 7fd65208d700 1 --2- 172.21.15.40:0/3729218877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 0x7fd64c102520 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.880+0000 7fd65208d700 1 --2- 172.21.15.40:0/3729218877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd64c0663e0 0x7fd64c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.880+0000 7fd65208d700 1 -- 172.21.15.40:0/3729218877 >> 172.21.15.40:0/3729218877 conn(0x7fd64c0f74e0 msgr2=0x7fd64c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.881+0000 7fd65208d700 1 -- 172.21.15.40:0/3729218877 shutdown_connections 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.881+0000 7fd65208d700 1 -- 172.21.15.40:0/3729218877 wait complete. 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.881+0000 7fd65208d700 1 Processor -- start 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.881+0000 7fd65208d700 1 -- start start 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.881+0000 7fd65208d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd64c0663e0 0x7fd64c0fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.881+0000 7fd65208d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 0x7fd64c0fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.881+0000 7fd65208d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0fbff0 con 0x7fd64c1000c0 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.881+0000 7fd65208d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0fc160 con 0x7fd64c0663e0 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd64affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 0x7fd64c0fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd64affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 0x7fd64c0fb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46562/0 (socket says 172.21.15.40:46562) 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd64affd700 1 -- 172.21.15.40:0/3171682409 learned_addr learned my addr 172.21.15.40:0/3171682409 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd64b7fe700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd64c0663e0 0x7fd64c0fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd64affd700 1 -- 172.21.15.40:0/3171682409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd64c0663e0 msgr2=0x7fd64c0fd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd64affd700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd64c0663e0 0x7fd64c0fd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd64affd700 1 -- 172.21.15.40:0/3171682409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd640005040 con 0x7fd64c1000c0 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd64affd700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 0x7fd64c0fb9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd64000b3b0 tx=0x7fd640009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd633fff700 1 -- 172.21.15.40:0/3171682409 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd64000e040 con 0x7fd64c1000c0 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c0fc3e0 con 0x7fd64c1000c0 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd633fff700 1 -- 172.21.15.40:0/3171682409 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd64000bcc0 con 0x7fd64c1000c0 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd633fff700 1 -- 172.21.15.40:0/3171682409 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd64001c630 con 0x7fd64c1000c0 2024-02-16T19:34:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.882+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c1159a0 con 0x7fd64c1000c0 2024-02-16T19:34:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.883+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c04bed0 con 0x7fd64c1000c0 2024-02-16T19:34:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.884+0000 7fd633fff700 1 -- 172.21.15.40:0/3171682409 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd640013420 con 0x7fd64c1000c0 2024-02-16T19:34:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.884+0000 7fd633fff700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd63406d7e0 0x7fd63406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.884+0000 7fd633fff700 1 -- 172.21.15.40:0/3171682409 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd64008e460 con 0x7fd64c1000c0 2024-02-16T19:34:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.884+0000 7fd64b7fe700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd63406d7e0 0x7fd63406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.885+0000 7fd64b7fe700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd63406d7e0 0x7fd63406fc90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0fd300 tx=0x7fd63c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:21.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:21.889+0000 7fd633fff700 1 -- 172.21.15.40:0/3171682409 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd64005b7b0 con 0x7fd64c1000c0 2024-02-16T19:34:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.184+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd64c105a10 con 0x7fd64c1000c0 2024-02-16T19:34:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.184+0000 7fd633fff700 1 -- 172.21.15.40:0/3171682409 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd64005b340 con 0x7fd64c1000c0 2024-02-16T19:34:22.186 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:34:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:34:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:34:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:34:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:34:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:34:22.187 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd63406d7e0 msgr2=0x7fd63406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd63406d7e0 0x7fd63406fc90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0fd300 tx=0x7fd63c00b040 comp rx=0 tx=0).stop 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 msgr2=0x7fd64c0fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 0x7fd64c0fb9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd64000b3b0 tx=0x7fd640009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 shutdown_connections 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd64c0663e0 0x7fd64c0fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd63406d7e0 0x7fd63406fc90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 --2- 172.21.15.40:0/3171682409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd64c1000c0 0x7fd64c0fb9f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 >> 172.21.15.40:0/3171682409 conn(0x7fd64c0f74e0 msgr2=0x7fd64c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 shutdown_connections 2024-02-16T19:34:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.187+0000 7fd65208d700 1 -- 172.21.15.40:0/3171682409 wait complete. 2024-02-16T19:34:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.291+0000 7fa142b72700 1 -- 172.21.15.40:0/3281712687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 msgr2=0x7fa13c0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.291+0000 7fa142b72700 1 --2- 172.21.15.40:0/3281712687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 0x7fa13c0fc250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa130004940 tx=0x7fa130009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.291+0000 7fa142b72700 1 -- 172.21.15.40:0/3281712687 shutdown_connections 2024-02-16T19:34:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.291+0000 7fa142b72700 1 --2- 172.21.15.40:0/3281712687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa13c0fcfb0 0x7fa13c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.291+0000 7fa142b72700 1 --2- 172.21.15.40:0/3281712687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 0x7fa13c0fc250 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.291+0000 7fa142b72700 1 -- 172.21.15.40:0/3281712687 >> 172.21.15.40:0/3281712687 conn(0x7fa13c0f74e0 msgr2=0x7fa13c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:22.293 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:22 smithi040 ceph-mon[94210]: pgmap v256: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:22.293 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:22 smithi040 ceph-mon[94210]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:22.293 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:22 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:22.293 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:22 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.292+0000 7fa142b72700 1 -- 172.21.15.40:0/3281712687 shutdown_connections 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.292+0000 7fa142b72700 1 -- 172.21.15.40:0/3281712687 wait complete. 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.292+0000 7fa142b72700 1 Processor -- start 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.292+0000 7fa142b72700 1 -- start start 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.292+0000 7fa142b72700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 0x7fa13c0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.292+0000 7fa142b72700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa13c0fcfb0 0x7fa13c0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.292+0000 7fa142b72700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0fe980 con 0x7fa13c0fbe40 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.292+0000 7fa142b72700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0feaf0 con 0x7fa13c0fcfb0 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa14090e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 0x7fa13c0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa14090e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 0x7fa13c0fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46576/0 (socket says 172.21.15.40:46576) 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa14090e700 1 -- 172.21.15.40:0/3040147227 learned_addr learned my addr 172.21.15.40:0/3040147227 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa13bfff700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa13c0fcfb0 0x7fa13c0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa14090e700 1 -- 172.21.15.40:0/3040147227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa13c0fcfb0 msgr2=0x7fa13c0fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa14090e700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa13c0fcfb0 0x7fa13c0fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa14090e700 1 -- 172.21.15.40:0/3040147227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa130005040 con 0x7fa13c0fbe40 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa14090e700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 0x7fa13c0fff00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa1300093b0 tx=0x7fa130009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa1397fa700 1 -- 172.21.15.40:0/3040147227 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa13000e040 con 0x7fa13c0fbe40 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa1397fa700 1 -- 172.21.15.40:0/3040147227 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa13000bcc0 con 0x7fa13c0fbe40 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c0fed70 con 0x7fa13c0fbe40 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.293+0000 7fa1397fa700 1 -- 172.21.15.40:0/3040147227 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa13001c630 con 0x7fa13c0fbe40 2024-02-16T19:34:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.294+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c115970 con 0x7fa13c0fbe40 2024-02-16T19:34:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.295+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c04bed0 con 0x7fa13c0fbe40 2024-02-16T19:34:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.296+0000 7fa1397fa700 1 -- 172.21.15.40:0/3040147227 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa130013980 con 0x7fa13c0fbe40 2024-02-16T19:34:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.296+0000 7fa1397fa700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa12c06d7e0 0x7fa12c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.296+0000 7fa1397fa700 1 -- 172.21.15.40:0/3040147227 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa13008e230 con 0x7fa13c0fbe40 2024-02-16T19:34:22.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.296+0000 7fa13bfff700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa12c06d7e0 0x7fa12c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.297+0000 7fa13bfff700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa12c06d7e0 0x7fa12c06fc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa128004f40 tx=0x7fa12800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.301+0000 7fa1397fa700 1 -- 172.21.15.40:0/3040147227 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa13005b620 con 0x7fa13c0fbe40 2024-02-16T19:34:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:22 smithi163 ceph-mon[81637]: pgmap v256: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:22 smithi163 ceph-mon[81637]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:22 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:22 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:22.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.584+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa13c115d00 con 0x7fa13c0fbe40 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.585+0000 7fa1397fa700 1 -- 172.21.15.40:0/3040147227 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 41 v41) v1 ==== 76+0+1880 (secure 0 0 0) 0x7fa13005b1b0 con 0x7fa13c0fbe40 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stdout:e41 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:34:22.586 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 41 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:33:18.695800+0000 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 83 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=34404,1=34436} 2024-02-16T19:34:22.587 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{0:34404} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{1:34436} state up:active seq 14 join_fscid=1 addr [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:34:22.588 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:34:22.589 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:34:22.589 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6824/1456075822,v1:172.21.15.163:6825/1456075822] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:34:22.589 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6826/2051774557,v1:172.21.15.163:6827/2051774557] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa12c06d7e0 msgr2=0x7fa12c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa12c06d7e0 0x7fa12c06fc90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa128004f40 tx=0x7fa12800b040 comp rx=0 tx=0).stop 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 msgr2=0x7fa13c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 0x7fa13c0fff00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa1300093b0 tx=0x7fa130009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 shutdown_connections 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fa12c06d7e0 0x7fa12c06fc90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa13c0fbe40 0x7fa13c0fff00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 --2- 172.21.15.40:0/3040147227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa13c0fcfb0 0x7fa13c0fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 >> 172.21.15.40:0/3040147227 conn(0x7fa13c0f74e0 msgr2=0x7fa13c0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:22.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 shutdown_connections 2024-02-16T19:34:22.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.587+0000 7fa142b72700 1 -- 172.21.15.40:0/3040147227 wait complete. 2024-02-16T19:34:22.591 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2024-02-16T19:34:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 -- 172.21.15.40:0/2225776108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fc7f0 msgr2=0x7f4b500fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 --2- 172.21.15.40:0/2225776108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fc7f0 0x7f4b500fcc40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c004940 tx=0x7f4b4c009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 -- 172.21.15.40:0/2225776108 shutdown_connections 2024-02-16T19:34:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 --2- 172.21.15.40:0/2225776108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fc7f0 0x7f4b500fcc40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 --2- 172.21.15.40:0/2225776108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b500fb5e0 0x7f4b500fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 -- 172.21.15.40:0/2225776108 >> 172.21.15.40:0/2225776108 conn(0x7f4b500f6ca0 msgr2=0x7f4b500f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 -- 172.21.15.40:0/2225776108 shutdown_connections 2024-02-16T19:34:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 -- 172.21.15.40:0/2225776108 wait complete. 2024-02-16T19:34:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 Processor -- start 2024-02-16T19:34:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.699+0000 7f4b57d80700 1 -- start start 2024-02-16T19:34:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b57d80700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fb5e0 0x7f4b5010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b57d80700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b500fc7f0 0x7f4b5010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b57d80700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5010e530 con 0x7f4b500fb5e0 2024-02-16T19:34:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b57d80700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5010edd0 con 0x7f4b500fc7f0 2024-02-16T19:34:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b55b1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fb5e0 0x7f4b5010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b55b1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fb5e0 0x7f4b5010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46602/0 (socket says 172.21.15.40:46602) 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b55b1c700 1 -- 172.21.15.40:0/3571896353 learned_addr learned my addr 172.21.15.40:0/3571896353 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b5531b700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b500fc7f0 0x7f4b5010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b55b1c700 1 -- 172.21.15.40:0/3571896353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b500fc7f0 msgr2=0x7f4b5010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b55b1c700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b500fc7f0 0x7f4b5010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b55b1c700 1 -- 172.21.15.40:0/3571896353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c005040 con 0x7f4b500fb5e0 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b55b1c700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fb5e0 0x7f4b5010d9f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000a710 tx=0x7f4b4000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b467fc700 1 -- 172.21.15.40:0/3571896353 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b4000f040 con 0x7f4b500fb5e0 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b467fc700 1 -- 172.21.15.40:0/3571896353 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b4000dc90 con 0x7f4b500fb5e0 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.700+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5010f0b0 con 0x7f4b500fb5e0 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.701+0000 7f4b467fc700 1 -- 172.21.15.40:0/3571896353 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b40014ea0 con 0x7f4b500fb5e0 2024-02-16T19:34:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.701+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5010f5f0 con 0x7f4b500fb5e0 2024-02-16T19:34:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.702+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b50105880 con 0x7f4b500fb5e0 2024-02-16T19:34:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.702+0000 7f4b467fc700 1 -- 172.21.15.40:0/3571896353 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4b4000d7c0 con 0x7f4b500fb5e0 2024-02-16T19:34:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.703+0000 7f4b467fc700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4b3c06d7e0 0x7f4b3c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.703+0000 7f4b467fc700 1 -- 172.21.15.40:0/3571896353 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4b4008caf0 con 0x7f4b500fb5e0 2024-02-16T19:34:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.703+0000 7f4b5531b700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4b3c06d7e0 0x7f4b3c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.703+0000 7f4b5531b700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4b3c06d7e0 0x7f4b3c06fc90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c004f90 tx=0x7f4b4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:22.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.707+0000 7f4b467fc700 1 -- 172.21.15.40:0/3571896353 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b40059ef0 con 0x7f4b500fb5e0 2024-02-16T19:34:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.932+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b5010f8e0 con 0x7f4b3c06d7e0 2024-02-16T19:34:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.933+0000 7f4b467fc700 1 -- 172.21.15.40:0/3571896353 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f4b5010f8e0 con 0x7f4b3c06d7e0 2024-02-16T19:34:22.934 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:34:22.934 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": true, 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [ 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "osd", 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "mds", 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "mon", 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr" 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": "16/21 daemons upgraded", 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "Currently upgrading grafana daemons", 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:34:22.935 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:34:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4b3c06d7e0 msgr2=0x7f4b3c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4b3c06d7e0 0x7f4b3c06fc90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c004f90 tx=0x7f4b4c009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fb5e0 msgr2=0x7f4b5010d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fb5e0 0x7f4b5010d9f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000a710 tx=0x7f4b4000c040 comp rx=0 tx=0).stop 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 shutdown_connections 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f4b3c06d7e0 0x7f4b3c06fc90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b500fb5e0 0x7f4b5010d9f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 --2- 172.21.15.40:0/3571896353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b500fc7f0 0x7f4b5010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 >> 172.21.15.40:0/3571896353 conn(0x7f4b500f6ca0 msgr2=0x7f4b500f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 shutdown_connections 2024-02-16T19:34:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:22.935+0000 7f4b57d80700 1 -- 172.21.15.40:0/3571896353 wait complete. 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.047+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3577161528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 msgr2=0x7fd6680fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.047+0000 7fd66d6ec700 1 --2- 172.21.15.40:0/3577161528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 0x7fd6680fe550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd650004940 tx=0x7fd650009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.047+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3577161528 shutdown_connections 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.047+0000 7fd66d6ec700 1 --2- 172.21.15.40:0/3577161528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 0x7fd6680fe550 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.047+0000 7fd66d6ec700 1 --2- 172.21.15.40:0/3577161528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6680fcef0 0x7fd6680fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.047+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3577161528 >> 172.21.15.40:0/3577161528 conn(0x7fd6680f8620 msgr2=0x7fd6680faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.047+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3577161528 shutdown_connections 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.047+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3577161528 wait complete. 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd66d6ec700 1 Processor -- start 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd66d6ec700 1 -- start start 2024-02-16T19:34:23.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd66d6ec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6680fcef0 0x7fd668071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd66d6ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 0x7fd6680702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd66d6ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668070970 con 0x7fd6680fe0e0 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd66d6ec700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668070ae0 con 0x7fd6680fcef0 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd65e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 0x7fd6680702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd65e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 0x7fd6680702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46622/0 (socket says 172.21.15.40:46622) 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd65e7fc700 1 -- 172.21.15.40:0/3476518585 learned_addr learned my addr 172.21.15.40:0/3476518585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.048+0000 7fd666ffd700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6680fcef0 0x7fd668071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd65e7fc700 1 -- 172.21.15.40:0/3476518585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6680fcef0 msgr2=0x7fd668071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd65e7fc700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6680fcef0 0x7fd668071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd65e7fc700 1 -- 172.21.15.40:0/3476518585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650005040 con 0x7fd6680fe0e0 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd65e7fc700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 0x7fd6680702e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd6500093b0 tx=0x7fd650009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:23.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd65ffff700 1 -- 172.21.15.40:0/3476518585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd65000e040 con 0x7fd6680fe0e0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd668070d60 con 0x7fd6680fe0e0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd65ffff700 1 -- 172.21.15.40:0/3476518585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd65000bcc0 con 0x7fd6680fe0e0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd65ffff700 1 -- 172.21.15.40:0/3476518585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd65001c630 con 0x7fd6680fe0e0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.049+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6681159e0 con 0x7fd6680fe0e0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.050+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66804bed0 con 0x7fd6680fe0e0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.051+0000 7fd65ffff700 1 -- 172.21.15.40:0/3476518585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd650013980 con 0x7fd6680fe0e0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.051+0000 7fd65ffff700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd65406d7e0 0x7fd65406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.051+0000 7fd65ffff700 1 -- 172.21.15.40:0/3476518585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd65008e2d0 con 0x7fd6680fe0e0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.051+0000 7fd666ffd700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd65406d7e0 0x7fd65406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:23.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.052+0000 7fd666ffd700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd65406d7e0 0x7fd65406fc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd668071bf0 tx=0x7fd6580091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:23.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.056+0000 7fd65ffff700 1 -- 172.21.15.40:0/3476518585 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd65005b600 con 0x7fd6680fe0e0 2024-02-16T19:34:23.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:23 smithi040 ceph-mon[94210]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:23.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:23 smithi040 ceph-mon[94210]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:23.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:23 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3171682409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:23.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:23 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:23.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:23 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:23.217 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:23 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3040147227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:34:23.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.355+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd668115e00 con 0x7fd6680fe0e0 2024-02-16T19:34:23.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.355+0000 7fd65ffff700 1 -- 172.21.15.40:0/3476518585 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd65005b190 con 0x7fd6680fe0e0 2024-02-16T19:34:23.357 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:34:23.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.357+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd65406d7e0 msgr2=0x7fd65406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:23.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.357+0000 7fd66d6ec700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd65406d7e0 0x7fd65406fc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd668071bf0 tx=0x7fd6580091a0 comp rx=0 tx=0).stop 2024-02-16T19:34:23.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.357+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 msgr2=0x7fd6680702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:23.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.357+0000 7fd66d6ec700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 0x7fd6680702e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd6500093b0 tx=0x7fd650009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:23.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.357+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 shutdown_connections 2024-02-16T19:34:23.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.357+0000 7fd66d6ec700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6680fcef0 0x7fd668071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:23.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.358+0000 7fd66d6ec700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fd65406d7e0 0x7fd65406fc90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:23.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.358+0000 7fd66d6ec700 1 --2- 172.21.15.40:0/3476518585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6680fe0e0 0x7fd6680702e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:23.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.358+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 >> 172.21.15.40:0/3476518585 conn(0x7fd6680f8620 msgr2=0x7fd6681013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:23.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.358+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 shutdown_connections 2024-02-16T19:34:23.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:23.358+0000 7fd66d6ec700 1 -- 172.21.15.40:0/3476518585 wait complete. 2024-02-16T19:34:23.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:23 smithi163 ceph-mon[81637]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:23 smithi163 ceph-mon[81637]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:23 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3171682409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:23 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:23 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:23.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:23 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3040147227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:34:24.371 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:24 smithi040 ceph-mon[94210]: pgmap v257: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:24.371 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:24 smithi040 ceph-mon[94210]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:24.371 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:24 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3476518585' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:34:24.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:24 smithi163 ceph-mon[81637]: pgmap v257: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:24.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:24 smithi163 ceph-mon[81637]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:24.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:24 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3476518585' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:34:26.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:26 smithi040 ceph-mon[94210]: pgmap v258: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:26.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:26 smithi163 ceph-mon[81637]: pgmap v258: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.396 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T19:34:27.397 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.398 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:27.399 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:27.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.489 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-16T19:34:27.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-16T19:34:27.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:34:27.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:28 smithi040 ceph-mon[94210]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:34:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:28 smithi040 ceph-mon[94210]: pgmap v259: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:28 smithi040 ceph-mon[94210]: Upgrade: Finalizing container_image settings 2024-02-16T19:34:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:28 smithi040 ceph-mon[94210]: Upgrade: Complete! 2024-02-16T19:34:28.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:28 smithi163 ceph-mon[81637]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-16T19:34:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:28 smithi163 ceph-mon[81637]: pgmap v259: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:28 smithi163 ceph-mon[81637]: Upgrade: Finalizing container_image settings 2024-02-16T19:34:28.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:28 smithi163 ceph-mon[81637]: Upgrade: Complete! 2024-02-16T19:34:28.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:34:30.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:30 smithi040 ceph-mon[94210]: pgmap v260: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:30 smithi163 ceph-mon[81637]: pgmap v260: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:32.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:32 smithi040 ceph-mon[94210]: pgmap v261: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:32.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:32 smithi163 ceph-mon[81637]: pgmap v261: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:33.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:33 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:33.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:33 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:34.395 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:34 smithi040 ceph-mon[94210]: pgmap v262: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:34.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:34 smithi163 ceph-mon[81637]: pgmap v262: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:36.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:36 smithi040 ceph-mon[94210]: pgmap v263: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:36.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:36 smithi163 ceph-mon[81637]: pgmap v263: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:38.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:38 smithi040 ceph-mon[94210]: pgmap v264: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:38.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:38 smithi163 ceph-mon[81637]: pgmap v264: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:40.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:40 smithi040 ceph-mon[94210]: pgmap v265: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:40.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:40 smithi163 ceph-mon[81637]: pgmap v265: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:42.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:42 smithi040 ceph-mon[94210]: pgmap v266: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:42.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:42 smithi163 ceph-mon[81637]: pgmap v266: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:44.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:44 smithi040 ceph-mon[94210]: pgmap v267: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:44.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:44 smithi163 ceph-mon[81637]: pgmap v267: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:46.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:46 smithi040 ceph-mon[94210]: pgmap v268: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:46.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:46 smithi163 ceph-mon[81637]: pgmap v268: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:48 smithi040 ceph-mon[94210]: pgmap v269: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:48.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:48 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:48 smithi163 ceph-mon[81637]: pgmap v269: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:48.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:48 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:34:50.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:50 smithi163 ceph-mon[81637]: pgmap v270: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:50.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:50 smithi040 ceph-mon[94210]: pgmap v270: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:52.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:52 smithi163 ceph-mon[81637]: pgmap v271: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:52.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:52 smithi040 ceph-mon[94210]: pgmap v271: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.476+0000 7ff7ccb65700 1 -- 172.21.15.40:0/4234216425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 msgr2=0x7ff7c80fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.476+0000 7ff7ccb65700 1 --2- 172.21.15.40:0/4234216425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 0x7ff7c80fc290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8004940 tx=0x7ff7b8009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.476+0000 7ff7ccb65700 1 -- 172.21.15.40:0/4234216425 shutdown_connections 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.476+0000 7ff7ccb65700 1 --2- 172.21.15.40:0/4234216425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7c80fd080 0x7ff7c80fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.476+0000 7ff7ccb65700 1 --2- 172.21.15.40:0/4234216425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 0x7ff7c80fc290 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.476+0000 7ff7ccb65700 1 -- 172.21.15.40:0/4234216425 >> 172.21.15.40:0/4234216425 conn(0x7ff7c80f7590 msgr2=0x7ff7c80f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.476+0000 7ff7ccb65700 1 -- 172.21.15.40:0/4234216425 shutdown_connections 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7ccb65700 1 -- 172.21.15.40:0/4234216425 wait complete. 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7ccb65700 1 Processor -- start 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7ccb65700 1 -- start start 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7ccb65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 0x7ff7c810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7ccb65700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7c80fd080 0x7ff7c810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7ccb65700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c810e560 con 0x7ff7c80fbe80 2024-02-16T19:34:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7ccb65700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c810eda0 con 0x7ff7c80fd080 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7c659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 0x7ff7c810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7c659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 0x7ff7c810d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37230/0 (socket says 172.21.15.40:37230) 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.477+0000 7ff7c659c700 1 -- 172.21.15.40:0/1597659546 learned_addr learned my addr 172.21.15.40:0/1597659546 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7c5d9b700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7c80fd080 0x7ff7c810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7c659c700 1 -- 172.21.15.40:0/1597659546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7c80fd080 msgr2=0x7ff7c810ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7c659c700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7c80fd080 0x7ff7c810ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7c659c700 1 -- 172.21.15.40:0/1597659546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8005040 con 0x7ff7c80fbe80 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7c5d9b700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7c80fd080 0x7ff7c810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7c659c700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 0x7ff7c810d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009360 tx=0x7ff7b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7b6ffd700 1 -- 172.21.15.40:0/1597659546 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b800e040 con 0x7ff7c80fbe80 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7b6ffd700 1 -- 172.21.15.40:0/1597659546 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7b8004290 con 0x7ff7c80fbe80 2024-02-16T19:34:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7b6ffd700 1 -- 172.21.15.40:0/1597659546 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b801c630 con 0x7ff7c80fbe80 2024-02-16T19:34:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c810f020 con 0x7ff7c80fbe80 2024-02-16T19:34:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.478+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c810f560 con 0x7ff7c80fbe80 2024-02-16T19:34:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.479+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c804bed0 con 0x7ff7c80fbe80 2024-02-16T19:34:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.480+0000 7ff7b6ffd700 1 -- 172.21.15.40:0/1597659546 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff7b800bcc0 con 0x7ff7c80fbe80 2024-02-16T19:34:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.480+0000 7ff7b6ffd700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff7b006d7f0 0x7ff7b006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.480+0000 7ff7b6ffd700 1 -- 172.21.15.40:0/1597659546 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff7b808e290 con 0x7ff7c80fbe80 2024-02-16T19:34:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.480+0000 7ff7c5d9b700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff7b006d7f0 0x7ff7b006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.481+0000 7ff7c5d9b700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff7b006d7f0 0x7ff7b006fca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006fd0 tx=0x7ff7bc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:53.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.485+0000 7ff7b6ffd700 1 -- 172.21.15.40:0/1597659546 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7b805b680 con 0x7ff7c80fbe80 2024-02-16T19:34:53.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.712+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7c8066680 con 0x7ff7b006d7f0 2024-02-16T19:34:53.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.713+0000 7ff7b6ffd700 1 -- 172.21.15.40:0/1597659546 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff7c8066680 con 0x7ff7b006d7f0 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff7b006d7f0 msgr2=0x7ff7b006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff7b006d7f0 0x7ff7b006fca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006fd0 tx=0x7ff7bc008040 comp rx=0 tx=0).stop 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 msgr2=0x7ff7c810d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 0x7ff7c810d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009360 tx=0x7ff7b8009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 shutdown_connections 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff7b006d7f0 0x7ff7b006fca0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c80fbe80 0x7ff7c810d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 --2- 172.21.15.40:0/1597659546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7c80fd080 0x7ff7c810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.715+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 >> 172.21.15.40:0/1597659546 conn(0x7ff7c80f7590 msgr2=0x7ff7c8065f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:53.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.716+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 shutdown_connections 2024-02-16T19:34:53.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:53.716+0000 7ff7ccb65700 1 -- 172.21.15.40:0/1597659546 wait complete. 2024-02-16T19:34:54.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:54 smithi163 ceph-mon[81637]: pgmap v272: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:54.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:54 smithi040 ceph-mon[94210]: pgmap v272: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:54.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T19:34:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:34:55.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:55 smithi040 ceph-mon[94210]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:55.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:55 smithi163 ceph-mon[81637]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:56.242 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:56 smithi040 ceph-mon[94210]: pgmap v273: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:56.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:56 smithi163 ceph-mon[81637]: pgmap v273: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.584+0000 7fbac1d2c700 1 -- 172.21.15.40:0/2757999806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 msgr2=0x7fbabc103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.584+0000 7fbac1d2c700 1 --2- 172.21.15.40:0/2757999806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 0x7fbabc103320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbaac004940 tx=0x7fbaac009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.584+0000 7fbac1d2c700 1 -- 172.21.15.40:0/2757999806 shutdown_connections 2024-02-16T19:34:56.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.584+0000 7fbac1d2c700 1 --2- 172.21.15.40:0/2757999806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbabc1038f0 0x7fbabc0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:56.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.584+0000 7fbac1d2c700 1 --2- 172.21.15.40:0/2757999806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 0x7fbabc103320 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:56.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.584+0000 7fbac1d2c700 1 -- 172.21.15.40:0/2757999806 >> 172.21.15.40:0/2757999806 conn(0x7fbabc0f8ae0 msgr2=0x7fbabc0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:56.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.585+0000 7fbac1d2c700 1 -- 172.21.15.40:0/2757999806 shutdown_connections 2024-02-16T19:34:56.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.585+0000 7fbac1d2c700 1 -- 172.21.15.40:0/2757999806 wait complete. 2024-02-16T19:34:56.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.585+0000 7fbac1d2c700 1 Processor -- start 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.585+0000 7fbac1d2c700 1 -- start start 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbac1d2c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 0x7fbabc113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbac1d2c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbabc1038f0 0x7fbabc113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbac1d2c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1145f0 con 0x7fbabc102f50 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbac1d2c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc10d920 con 0x7fbabc1038f0 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbabb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 0x7fbabc113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbabb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 0x7fbabc113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37254/0 (socket says 172.21.15.40:37254) 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbabb7fe700 1 -- 172.21.15.40:0/1486894713 learned_addr learned my addr 172.21.15.40:0/1486894713 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbabaffd700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbabc1038f0 0x7fbabc113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbabb7fe700 1 -- 172.21.15.40:0/1486894713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbabc1038f0 msgr2=0x7fbabc113fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbabb7fe700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbabc1038f0 0x7fbabc113fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbabb7fe700 1 -- 172.21.15.40:0/1486894713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac005040 con 0x7fbabc102f50 2024-02-16T19:34:56.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbabb7fe700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 0x7fbabc113a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009100 tx=0x7fbaac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:56.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbaa3fff700 1 -- 172.21.15.40:0/1486894713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaac00e040 con 0x7fbabc102f50 2024-02-16T19:34:56.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc10dba0 con 0x7fbabc102f50 2024-02-16T19:34:56.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbaa3fff700 1 -- 172.21.15.40:0/1486894713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbaac004290 con 0x7fbabc102f50 2024-02-16T19:34:56.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbaa3fff700 1 -- 172.21.15.40:0/1486894713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaac01c630 con 0x7fbabc102f50 2024-02-16T19:34:56.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.586+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc10e0e0 con 0x7fbabc102f50 2024-02-16T19:34:56.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.587+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc100260 con 0x7fbabc102f50 2024-02-16T19:34:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.588+0000 7fbaa3fff700 1 -- 172.21.15.40:0/1486894713 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbaac00bbf0 con 0x7fbabc102f50 2024-02-16T19:34:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.588+0000 7fbaa3fff700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbaa406d7e0 0x7fbaa406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.588+0000 7fbaa3fff700 1 -- 172.21.15.40:0/1486894713 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbaac08e320 con 0x7fbabc102f50 2024-02-16T19:34:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.588+0000 7fbabaffd700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbaa406d7e0 0x7fbaa406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:56.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.589+0000 7fbabaffd700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbaa406d7e0 0x7fbaa406fc90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbabc10f290 tx=0x7fbab0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.593+0000 7fbaa3fff700 1 -- 172.21.15.40:0/1486894713 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbaac05b670 con 0x7fbabc102f50 2024-02-16T19:34:56.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.810+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbabc04bed0 con 0x7fbaa406d7e0 2024-02-16T19:34:56.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.820+0000 7fbaa3fff700 1 -- 172.21.15.40:0/1486894713 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fbabc04bed0 con 0x7fbaa406d7e0 2024-02-16T19:34:56.821 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:34:56.821 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (56s) 35s ago 18m 16.3M - 0.25.0 c8568f914cd2 d164871c90a0 2024-02-16T19:34:56.821 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (6m) 35s ago 18m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:34:56.821 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (6m) 87s ago 17m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (40s) 35s ago 18m 59.8M - 9.4.12 a0ea5e3c46d9 1f69ae11e0c8 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (2m) 35s ago 12m 16.4M - 19.0.0-1450-gf78a58c0 f081f847070a 084709bde4d3 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (2m) 35s ago 12m 72.1M - 19.0.0-1450-gf78a58c0 f081f847070a 79213f1b751f 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (106s) 87s ago 12m 13.9M - 19.0.0-1450-gf78a58c0 f081f847070a 8358b866ac44 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (2m) 87s ago 12m 12.7M - 19.0.0-1450-gf78a58c0 f081f847070a 33502c65114c 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (8m) 35s ago 19m 511M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (8m) 87s ago 17m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (8m) 35s ago 20m 55.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (7m) 87s ago 17m 40.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (94s) 35s ago 18m 8799k - 1.5.0 0da6a335fe13 e7da1cc993ec 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (89s) 87s ago 17m 7994k - 1.5.0 0da6a335fe13 b6ef06e45eb0 2024-02-16T19:34:56.822 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (6m) 35s ago 16m 108M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:34:56.823 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (5m) 35s ago 16m 115M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:34:56.823 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (5m) 35s ago 15m 90.0M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:34:56.823 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (4m) 87s ago 15m 111M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:34:56.823 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (4m) 87s ago 14m 100M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:34:56.823 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (3m) 87s ago 14m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 807b119a828c 2024-02-16T19:34:56.823 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (73s) 35s ago 17m 47.1M - 2.43.0 a07b618ecd1d 8a7a0a27c8fb 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbaa406d7e0 msgr2=0x7fbaa406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbaa406d7e0 0x7fbaa406fc90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbabc10f290 tx=0x7fbab0009040 comp rx=0 tx=0).stop 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 msgr2=0x7fbabc113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 0x7fbabc113a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009100 tx=0x7fbaac009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 shutdown_connections 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbaa406d7e0 0x7fbaa406fc90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbabc102f50 0x7fbabc113a60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 --2- 172.21.15.40:0/1486894713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbabc1038f0 0x7fbabc113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 >> 172.21.15.40:0/1486894713 conn(0x7fbabc0f8ae0 msgr2=0x7fbabc0fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 shutdown_connections 2024-02-16T19:34:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:56.822+0000 7fbac1d2c700 1 -- 172.21.15.40:0/1486894713 wait complete. 2024-02-16T19:34:57.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade status' 2024-02-16T19:34:57.781 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:34:58.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:58 smithi163 ceph-mon[81637]: pgmap v274: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:58.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:34:58 smithi163 ceph-mon[81637]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:58.621 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:58 smithi040 ceph-mon[94210]: pgmap v274: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:34:58.621 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:34:58 smithi040 ceph-mon[94210]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:34:59.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.620+0000 7f5617df4700 1 -- 172.21.15.40:0/2986719015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 msgr2=0x7f56101021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:59.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.620+0000 7f5617df4700 1 --2- 172.21.15.40:0/2986719015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 0x7f56101021b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f560c004940 tx=0x7f560c009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:59.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.620+0000 7f5617df4700 1 -- 172.21.15.40:0/2986719015 shutdown_connections 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.620+0000 7f5617df4700 1 --2- 172.21.15.40:0/2986719015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5610102780 0x7f56100fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.620+0000 7f5617df4700 1 --2- 172.21.15.40:0/2986719015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 0x7f56101021b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.620+0000 7f5617df4700 1 -- 172.21.15.40:0/2986719015 >> 172.21.15.40:0/2986719015 conn(0x7f5610071d50 msgr2=0x7f5610072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.620+0000 7f5617df4700 1 -- 172.21.15.40:0/2986719015 shutdown_connections 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.620+0000 7f5617df4700 1 -- 172.21.15.40:0/2986719015 wait complete. 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5617df4700 1 Processor -- start 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5617df4700 1 -- start start 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5617df4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 0x7f56101139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5617df4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5610102780 0x7f5610113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5617df4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5610114570 con 0x7f5610101de0 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5617df4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561010d8a0 con 0x7f5610102780 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5615b90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 0x7f56101139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5615b90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 0x7f56101139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38014/0 (socket says 172.21.15.40:38014) 2024-02-16T19:34:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5615b90700 1 -- 172.21.15.40:0/121617951 learned_addr learned my addr 172.21.15.40:0/121617951 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:34:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5615b90700 1 -- 172.21.15.40:0/121617951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5610102780 msgr2=0x7f5610113f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5615b90700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5610102780 0x7f5610113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.621+0000 7f5615b90700 1 -- 172.21.15.40:0/121617951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c005040 con 0x7f5610101de0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.622+0000 7f5615b90700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 0x7f56101139e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f560c00b3b0 tx=0x7f560c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.622+0000 7f56027fc700 1 -- 172.21.15.40:0/121617951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f560c00e040 con 0x7f5610101de0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.622+0000 7f56027fc700 1 -- 172.21.15.40:0/121617951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f560c00bbf0 con 0x7f5610101de0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.622+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561010db20 con 0x7f5610101de0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.622+0000 7f56027fc700 1 -- 172.21.15.40:0/121617951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f560c01c630 con 0x7f5610101de0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.622+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561010e060 con 0x7f5610101de0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.623+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5610075ca0 con 0x7f5610101de0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.623+0000 7f56027fc700 1 -- 172.21.15.40:0/121617951 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f560c00bd60 con 0x7f5610101de0 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.624+0000 7f56027fc700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55fc06d7e0 0x7f55fc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:34:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.624+0000 7f56027fc700 1 -- 172.21.15.40:0/121617951 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f560c08e3a0 con 0x7f5610101de0 2024-02-16T19:34:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.624+0000 7f561538f700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55fc06d7e0 0x7f55fc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:34:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.624+0000 7f561538f700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55fc06d7e0 0x7f55fc06fc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f561010f210 tx=0x7f5604007cc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:34:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.629+0000 7f56027fc700 1 -- 172.21.15.40:0/121617951 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f560c05b6f0 con 0x7f5610101de0 2024-02-16T19:34:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.849+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f561004bf70 con 0x7f55fc06d7e0 2024-02-16T19:34:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.850+0000 7f56027fc700 1 -- 172.21.15.40:0/121617951 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f5604004800 con 0x7f55fc06d7e0 2024-02-16T19:34:59.851 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:34:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "target_image": null, 2024-02-16T19:34:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "in_progress": false, 2024-02-16T19:34:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "which": "", 2024-02-16T19:34:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "services_complete": [], 2024-02-16T19:34:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "progress": null, 2024-02-16T19:34:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "message": "", 2024-02-16T19:34:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "is_paused": false 2024-02-16T19:34:59.852 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:34:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55fc06d7e0 msgr2=0x7f55fc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55fc06d7e0 0x7f55fc06fc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f561010f210 tx=0x7f5604007cc0 comp rx=0 tx=0).stop 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 msgr2=0x7f56101139e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 0x7f56101139e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f560c00b3b0 tx=0x7f560c009e30 comp rx=0 tx=0).stop 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 shutdown_connections 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55fc06d7e0 0x7f55fc06fc90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5610101de0 0x7f56101139e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 --2- 172.21.15.40:0/121617951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5610102780 0x7f5610113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 >> 172.21.15.40:0/121617951 conn(0x7f5610071d50 msgr2=0x7f56101051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.852+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 shutdown_connections 2024-02-16T19:34:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:34:59.853+0000 7f5617df4700 1 -- 172.21.15.40:0/121617951 wait complete. 2024-02-16T19:35:00.438 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:00 smithi040 ceph-mon[94210]: pgmap v275: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:00.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph health detail' 2024-02-16T19:35:00.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:00 smithi163 ceph-mon[81637]: pgmap v275: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:00.726 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:35:01.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:01 smithi163 ceph-mon[81637]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:01.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:01 smithi040 ceph-mon[94210]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:35:02.377 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:02 smithi040 ceph-mon[94210]: pgmap v276: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.375+0000 7f9a8790a700 1 -- 172.21.15.40:0/1323792317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fdfe0 msgr2=0x7f9a800fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.375+0000 7f9a8790a700 1 --2- 172.21.15.40:0/1323792317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fdfe0 0x7f9a800fe3b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9a74004970 tx=0x7f9a74009e30 comp rx=0 tx=0).stop 2024-02-16T19:35:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.375+0000 7f9a8790a700 1 -- 172.21.15.40:0/1323792317 shutdown_connections 2024-02-16T19:35:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.375+0000 7f9a8790a700 1 --2- 172.21.15.40:0/1323792317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a800fe8f0 0x7f9a80107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.375+0000 7f9a8790a700 1 --2- 172.21.15.40:0/1323792317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fdfe0 0x7f9a800fe3b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.375+0000 7f9a8790a700 1 -- 172.21.15.40:0/1323792317 >> 172.21.15.40:0/1323792317 conn(0x7f9a80071d90 msgr2=0x7f9a80072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.375+0000 7f9a8790a700 1 -- 172.21.15.40:0/1323792317 shutdown_connections 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.375+0000 7f9a8790a700 1 -- 172.21.15.40:0/1323792317 wait complete. 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a8790a700 1 Processor -- start 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a8790a700 1 -- start start 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a8790a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a800fdfe0 0x7f9a80115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a8790a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fe8f0 0x7f9a80116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a8790a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80116790 con 0x7f9a800fe8f0 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a8790a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8010fae0 con 0x7f9a800fdfe0 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a84ea5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fe8f0 0x7f9a80116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a84ea5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fe8f0 0x7f9a80116140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38034/0 (socket says 172.21.15.40:38034) 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.376+0000 7f9a84ea5700 1 -- 172.21.15.40:0/1444102505 learned_addr learned my addr 172.21.15.40:0/1444102505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a856a6700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a800fdfe0 0x7f9a80115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a84ea5700 1 -- 172.21.15.40:0/1444102505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a800fdfe0 msgr2=0x7f9a80115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a84ea5700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a800fdfe0 0x7f9a80115c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a84ea5700 1 -- 172.21.15.40:0/1444102505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74005040 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a856a6700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a800fdfe0 0x7f9a80115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a84ea5700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fe8f0 0x7f9a80116140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00c830 tx=0x7f9a7c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a71ffb700 1 -- 172.21.15.40:0/1444102505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a7c00a760 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a71ffb700 1 -- 172.21.15.40:0/1444102505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a7c00ad90 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a71ffb700 1 -- 172.21.15.40:0/1444102505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a7c0113a0 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8010fdc0 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.377+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a80110300 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.378+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a80106b20 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.378+0000 7f9a71ffb700 1 -- 172.21.15.40:0/1444102505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9a7c012400 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.379+0000 7f9a71ffb700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a6c06d7e0 0x7f9a6c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.379+0000 7f9a71ffb700 1 -- 172.21.15.40:0/1444102505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9a7c08cfc0 con 0x7f9a800fe8f0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.379+0000 7f9a856a6700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a6c06d7e0 0x7f9a6c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.379+0000 7f9a856a6700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a6c06d7e0 0x7f9a6c06fc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9a74004f50 tx=0x7f9a74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:02.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.385+0000 7f9a71ffb700 1 -- 172.21.15.40:0/1444102505 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a7c0c0d20 con 0x7f9a800fe8f0 2024-02-16T19:35:02.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:02 smithi163 ceph-mon[81637]: pgmap v276: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:02.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.671+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9a8004bed0 con 0x7f9a800fe8f0 2024-02-16T19:35:02.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.671+0000 7f9a71ffb700 1 -- 172.21.15.40:0/1444102505 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9a7c0567c0 con 0x7f9a800fe8f0 2024-02-16T19:35:02.673 INFO:teuthology.orchestra.run.smithi040.stdout:HEALTH_OK 2024-02-16T19:35:02.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a6c06d7e0 msgr2=0x7f9a6c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:02.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a6c06d7e0 0x7f9a6c06fc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9a74004f50 tx=0x7f9a74009e30 comp rx=0 tx=0).stop 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fe8f0 msgr2=0x7f9a80116140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fe8f0 0x7f9a80116140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00c830 tx=0x7f9a7c004470 comp rx=0 tx=0).stop 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 shutdown_connections 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a800fdfe0 0x7f9a80115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9a6c06d7e0 0x7f9a6c06fc90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 --2- 172.21.15.40:0/1444102505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a800fe8f0 0x7f9a80116140 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 >> 172.21.15.40:0/1444102505 conn(0x7f9a80071d90 msgr2=0x7f9a800fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 shutdown_connections 2024-02-16T19:35:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:02.674+0000 7f9a8790a700 1 -- 172.21.15.40:0/1444102505 wait complete. 2024-02-16T19:35:03.337 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:03 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:03.337 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:03 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1444102505' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:35:03.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-16T19:35:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:03 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:03.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:03 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1444102505' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-16T19:35:03.631 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:35:04.377 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:04 smithi040 ceph-mon[94210]: pgmap v277: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:04.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:04 smithi163 ceph-mon[81637]: pgmap v277: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.316+0000 7f7ac527e700 1 -- 172.21.15.40:0/3998838761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fdf90 msgr2=0x7f7ac00fe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.316+0000 7f7ac527e700 1 --2- 172.21.15.40:0/3998838761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fdf90 0x7f7ac00fe360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009ce0 tx=0x7f7ab8004d10 comp rx=0 tx=0).stop 2024-02-16T19:35:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.316+0000 7f7ac527e700 1 -- 172.21.15.40:0/3998838761 shutdown_connections 2024-02-16T19:35:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.316+0000 7f7ac527e700 1 --2- 172.21.15.40:0/3998838761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac00fe930 0x7f7ac0107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.316+0000 7f7ac527e700 1 --2- 172.21.15.40:0/3998838761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fdf90 0x7f7ac00fe360 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.316+0000 7f7ac527e700 1 -- 172.21.15.40:0/3998838761 >> 172.21.15.40:0/3998838761 conn(0x7f7ac00f7a10 msgr2=0x7f7ac00f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.316+0000 7f7ac527e700 1 -- 172.21.15.40:0/3998838761 shutdown_connections 2024-02-16T19:35:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.316+0000 7f7ac527e700 1 -- 172.21.15.40:0/3998838761 wait complete. 2024-02-16T19:35:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7ac527e700 1 Processor -- start 2024-02-16T19:35:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7ac527e700 1 -- start start 2024-02-16T19:35:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7ac527e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac00fdf90 0x7f7ac0113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7ac527e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fe930 0x7f7ac0113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7ac527e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac0114640 con 0x7f7ac00fe930 2024-02-16T19:35:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7ac527e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac010d970 con 0x7f7ac00fdf90 2024-02-16T19:35:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7abe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fe930 0x7f7ac0113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7abe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fe930 0x7f7ac0113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38048/0 (socket says 172.21.15.40:38048) 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7abe7fc700 1 -- 172.21.15.40:0/2313511278 learned_addr learned my addr 172.21.15.40:0/2313511278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7abeffd700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac00fdf90 0x7f7ac0113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7abe7fc700 1 -- 172.21.15.40:0/2313511278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac00fdf90 msgr2=0x7f7ac0113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7abe7fc700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac00fdf90 0x7f7ac0113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.317+0000 7f7abe7fc700 1 -- 172.21.15.40:0/2313511278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8009940 con 0x7f7ac00fe930 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.318+0000 7f7abe7fc700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fe930 0x7f7ac0113ff0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c830 tx=0x7f7aa8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.318+0000 7f7ab77fe700 1 -- 172.21.15.40:0/2313511278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aa80106f0 con 0x7f7ac00fe930 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.318+0000 7f7ab77fe700 1 -- 172.21.15.40:0/2313511278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7aa8010d20 con 0x7f7ac00fe930 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.318+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac010dc50 con 0x7f7ac00fe930 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.318+0000 7f7ab77fe700 1 -- 172.21.15.40:0/2313511278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aa80113a0 con 0x7f7ac00fe930 2024-02-16T19:35:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.318+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac010e190 con 0x7f7ac00fe930 2024-02-16T19:35:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.319+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac01046c0 con 0x7f7ac00fe930 2024-02-16T19:35:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.319+0000 7f7ab77fe700 1 -- 172.21.15.40:0/2313511278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7aa8012400 con 0x7f7ac00fe930 2024-02-16T19:35:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.320+0000 7f7ab77fe700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7aac06d7e0 0x7f7aac06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:35:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.320+0000 7f7ab77fe700 1 -- 172.21.15.40:0/2313511278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7aa808cee0 con 0x7f7ac00fe930 2024-02-16T19:35:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.320+0000 7f7abeffd700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7aac06d7e0 0x7f7aac06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:35:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.321+0000 7f7abeffd700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7aac06d7e0 0x7f7aac06fc90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009f90 tx=0x7f7ab8004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:35:05.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.325+0000 7f7ab77fe700 1 -- 172.21.15.40:0/2313511278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7aa805a230 con 0x7f7ac00fe930 2024-02-16T19:35:05.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.611+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7ac010f100 con 0x7f7ac00fe930 2024-02-16T19:35:05.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.612+0000 7f7ab77fe700 1 -- 172.21.15.40:0/2313511278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7ac010f100 con 0x7f7ac00fe930 2024-02-16T19:35:05.613 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:35:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:35:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:35:05.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:35:05.614 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:35:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7aac06d7e0 msgr2=0x7f7aac06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7aac06d7e0 0x7f7aac06fc90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009f90 tx=0x7f7ab8004d10 comp rx=0 tx=0).stop 2024-02-16T19:35:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fe930 msgr2=0x7f7ac0113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:35:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fe930 0x7f7ac0113ff0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c830 tx=0x7f7aa8004470 comp rx=0 tx=0).stop 2024-02-16T19:35:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 shutdown_connections 2024-02-16T19:35:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac00fdf90 0x7f7ac0113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7aac06d7e0 0x7f7aac06fc90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 --2- 172.21.15.40:0/2313511278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac00fe930 0x7f7ac0113ff0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:35:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 >> 172.21.15.40:0/2313511278 conn(0x7f7ac00f7a10 msgr2=0x7f7ac0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:35:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 shutdown_connections 2024-02-16T19:35:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:35:05.614+0000 7f7ac527e700 1 -- 172.21.15.40:0/2313511278 wait complete. 2024-02-16T19:35:06.234 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:06 smithi040 ceph-mon[94210]: pgmap v278: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:06.235 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:06 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2313511278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:06.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-16T19:35:06.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:06 smithi163 ceph-mon[81637]: pgmap v278: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:06.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:06 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2313511278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:35:06.523 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:35:08.138 INFO:teuthology.orchestra.run.smithi040.stdout:wait for servicemap items w/ changing names to refresh 2024-02-16T19:35:08.394 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:08 smithi040 ceph-mon[94210]: pgmap v279: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:08.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:08 smithi163 ceph-mon[81637]: pgmap v279: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:08.818 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'sleep 60' 2024-02-16T19:35:09.083 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:35:10.467 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:10 smithi040 ceph-mon[94210]: pgmap v280: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:10.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:10 smithi163 ceph-mon[81637]: pgmap v280: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:12.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:12 smithi163 ceph-mon[81637]: pgmap v281: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:12.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:12 smithi040 ceph-mon[94210]: pgmap v281: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:14.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:14 smithi163 ceph-mon[81637]: pgmap v282: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:14.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:14 smithi040 ceph-mon[94210]: pgmap v282: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:16.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:16 smithi163 ceph-mon[81637]: pgmap v283: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:16.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:16 smithi040 ceph-mon[94210]: pgmap v283: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:18 smithi163 ceph-mon[81637]: pgmap v284: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:18.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:18 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:18.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:18 smithi040 ceph-mon[94210]: pgmap v284: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:18.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:18 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:19.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:19 smithi040 ceph-mon[94210]: pgmap v285: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:19 smithi163 ceph-mon[81637]: pgmap v285: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:21.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:21 smithi040 ceph-mon[94210]: pgmap v286: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:21.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:21 smithi163 ceph-mon[81637]: pgmap v286: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:23.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:23 smithi040 ceph-mon[94210]: pgmap v287: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:23 smithi163 ceph-mon[81637]: pgmap v287: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:25.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:25 smithi040 ceph-mon[94210]: pgmap v288: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:25 smithi163 ceph-mon[81637]: pgmap v288: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:27.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:27 smithi040 ceph-mon[94210]: pgmap v289: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:27.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:35:27.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:35:27.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:35:27.645 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:35:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:27 smithi163 ceph-mon[81637]: pgmap v289: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:35:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:35:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:35:27.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:35:29.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:29 smithi040 ceph-mon[94210]: pgmap v290: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:29 smithi163 ceph-mon[81637]: pgmap v290: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:31 smithi040 ceph-mon[94210]: pgmap v291: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:31 smithi163 ceph-mon[81637]: pgmap v291: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:33 smithi040 ceph-mon[94210]: pgmap v292: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:33 smithi163 ceph-mon[81637]: pgmap v292: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:35.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:35 smithi040 ceph-mon[94210]: pgmap v293: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:35 smithi163 ceph-mon[81637]: pgmap v293: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:37.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:37 smithi040 ceph-mon[94210]: pgmap v294: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:37.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:37 smithi163 ceph-mon[81637]: pgmap v294: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:39 smithi040 ceph-mon[94210]: pgmap v295: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:39 smithi163 ceph-mon[81637]: pgmap v295: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:41.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:41 smithi040 ceph-mon[94210]: pgmap v296: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:41.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:41 smithi163 ceph-mon[81637]: pgmap v296: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:43.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:43 smithi040 ceph-mon[94210]: pgmap v297: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:43.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:43 smithi163 ceph-mon[81637]: pgmap v297: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:45.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:45 smithi040 ceph-mon[94210]: pgmap v298: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:45.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:45 smithi163 ceph-mon[81637]: pgmap v298: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:47 smithi040 ceph-mon[94210]: pgmap v299: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:47.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:47 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:47 smithi163 ceph-mon[81637]: pgmap v299: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:47.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:47 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:35:49.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:49 smithi040 ceph-mon[94210]: pgmap v300: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:49.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:49 smithi163 ceph-mon[81637]: pgmap v300: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:51.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:51 smithi040 ceph-mon[94210]: pgmap v301: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:51.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:51 smithi163 ceph-mon[81637]: pgmap v301: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:53.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:53 smithi040 ceph-mon[94210]: pgmap v302: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:53.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:53 smithi163 ceph-mon[81637]: pgmap v302: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:55.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:55 smithi040 ceph-mon[94210]: pgmap v303: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:55.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:55 smithi163 ceph-mon[81637]: pgmap v303: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:57.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:57 smithi040 ceph-mon[94210]: pgmap v304: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:57.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:57 smithi163 ceph-mon[81637]: pgmap v304: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:59.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:35:59 smithi040 ceph-mon[94210]: pgmap v305: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:35:59.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:35:59 smithi163 ceph-mon[81637]: pgmap v305: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:01.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:01 smithi163 ceph-mon[81637]: pgmap v306: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:01.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:01 smithi040 ceph-mon[94210]: pgmap v306: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:02.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:02 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:02.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:02 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:03.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:03 smithi163 ceph-mon[81637]: pgmap v307: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:03.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:03 smithi040 ceph-mon[94210]: pgmap v307: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:05.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:05 smithi163 ceph-mon[81637]: pgmap v308: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:05.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:05 smithi040 ceph-mon[94210]: pgmap v308: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:07.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:07 smithi040 ceph-mon[94210]: pgmap v309: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:07.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:07 smithi163 ceph-mon[81637]: pgmap v309: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:09.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:09 smithi163 ceph-mon[81637]: pgmap v310: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:09.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:09 smithi040 ceph-mon[94210]: pgmap v310: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:11.321 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-16T19:36:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:36:11.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:11 smithi040 ceph-mon[94210]: pgmap v311: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:11.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:11 smithi163 ceph-mon[81637]: pgmap v311: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/4092379358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 msgr2=0x7f8e68102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 --2- 172.21.15.40:0/4092379358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 0x7f8e68102210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e58004940 tx=0x7f8e58009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/4092379358 shutdown_connections 2024-02-16T19:36:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 --2- 172.21.15.40:0/4092379358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e681027e0 0x7f8e680fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 --2- 172.21.15.40:0/4092379358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 0x7f8e68102210 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/4092379358 >> 172.21.15.40:0/4092379358 conn(0x7f8e68071d90 msgr2=0x7f8e68072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/4092379358 shutdown_connections 2024-02-16T19:36:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/4092379358 wait complete. 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.302+0000 7f8e6e8e9700 1 Processor -- start 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e6e8e9700 1 -- start start 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e6e8e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 0x7f8e68112970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e6e8e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e681027e0 0x7f8e68112eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e6e8e9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68113590 con 0x7f8e68101e40 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e6e8e9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6810c830 con 0x7f8e681027e0 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 0x7f8e68112970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 0x7f8e68112970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51976/0 (socket says 172.21.15.40:51976) 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e67fff700 1 -- 172.21.15.40:0/598352014 learned_addr learned my addr 172.21.15.40:0/598352014 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e67fff700 1 -- 172.21.15.40:0/598352014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e681027e0 msgr2=0x7f8e68112eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e67fff700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e681027e0 0x7f8e68112eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e67fff700 1 -- 172.21.15.40:0/598352014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58005040 con 0x7f8e68101e40 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.303+0000 7f8e67fff700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 0x7f8e68112970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800b3b0 tx=0x7f8e58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:13.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.304+0000 7f8e64ff9700 1 -- 172.21.15.40:0/598352014 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5800e040 con 0x7f8e68101e40 2024-02-16T19:36:13.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.304+0000 7f8e64ff9700 1 -- 172.21.15.40:0/598352014 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e5800bbf0 con 0x7f8e68101e40 2024-02-16T19:36:13.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.304+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6810cab0 con 0x7f8e68101e40 2024-02-16T19:36:13.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.304+0000 7f8e64ff9700 1 -- 172.21.15.40:0/598352014 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5801c630 con 0x7f8e68101e40 2024-02-16T19:36:13.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.304+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6810cff0 con 0x7f8e68101e40 2024-02-16T19:36:13.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.305+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e680738b0 con 0x7f8e68101e40 2024-02-16T19:36:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.305+0000 7f8e64ff9700 1 -- 172.21.15.40:0/598352014 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8e5800bd60 con 0x7f8e68101e40 2024-02-16T19:36:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.305+0000 7f8e64ff9700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8e5406d7e0 0x7f8e5406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.306+0000 7f8e64ff9700 1 -- 172.21.15.40:0/598352014 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8e5808e2a0 con 0x7f8e68101e40 2024-02-16T19:36:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.306+0000 7f8e677fe700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8e5406d7e0 0x7f8e5406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.306+0000 7f8e677fe700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8e5406d7e0 0x7f8e5406fc90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e50006fd0 tx=0x7f8e50006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:13.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.310+0000 7f8e64ff9700 1 -- 172.21.15.40:0/598352014 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8e5805b690 con 0x7f8e68101e40 2024-02-16T19:36:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.526+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 --> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e6804bed0 con 0x7f8e5406d7e0 2024-02-16T19:36:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.536+0000 7f8e64ff9700 1 -- 172.21.15.40:0/598352014 <== mgr.34104 v2:172.21.15.40:6800/1911827957 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8e6804bed0 con 0x7f8e5406d7e0 2024-02-16T19:36:13.537 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-16T19:36:13.537 INFO:teuthology.orchestra.run.smithi040.stdout:alertmanager.smithi040 smithi040 *:9093,9094 running (2m) 112s ago 19m 16.3M - 0.25.0 c8568f914cd2 d164871c90a0 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi040 smithi040 running (8m) 112s ago 20m 6665k - 19.0.0-1450-gf78a58c0 f081f847070a 9e90d9b2ad66 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:crash.smithi163 smithi163 running (8m) 2m ago 18m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a 04cd6ad04333 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:grafana.smithi040 smithi040 *:3000 running (117s) 112s ago 19m 59.8M - 9.4.12 a0ea5e3c46d9 1f69ae11e0c8 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.bmpjpk smithi040 running (3m) 112s ago 14m 16.4M - 19.0.0-1450-gf78a58c0 f081f847070a 084709bde4d3 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi040.ezkppl smithi040 running (4m) 112s ago 14m 72.1M - 19.0.0-1450-gf78a58c0 f081f847070a 79213f1b751f 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.qovvde smithi163 running (3m) 2m ago 14m 13.9M - 19.0.0-1450-gf78a58c0 f081f847070a 8358b866ac44 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:mds.cephfs.smithi163.rggdhh smithi163 running (3m) 2m ago 14m 12.7M - 19.0.0-1450-gf78a58c0 f081f847070a 33502c65114c 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi040.jzygzo smithi040 *:8443,9283,8765 running (10m) 112s ago 21m 511M - 19.0.0-1450-gf78a58c0 f081f847070a 8a47cbd25a47 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.smithi163.wjfclz smithi163 *:8443,9283,8765 running (9m) 2m ago 18m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5341cf372d7c 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi040 smithi040 running (9m) 112s ago 21m 55.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 1c1011f60db6 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:mon.smithi163 smithi163 running (8m) 2m ago 18m 40.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 4840eea014c3 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi040 smithi040 *:9100 running (2m) 112s ago 19m 8799k - 1.5.0 0da6a335fe13 e7da1cc993ec 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:node-exporter.smithi163 smithi163 *:9100 running (2m) 2m ago 18m 7994k - 1.5.0 0da6a335fe13 b6ef06e45eb0 2024-02-16T19:36:13.538 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (7m) 112s ago 17m 108M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 607e375ba25b 2024-02-16T19:36:13.539 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi040 running (7m) 112s ago 17m 115M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 2ba47d6ce1c9 2024-02-16T19:36:13.539 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi040 running (6m) 112s ago 16m 90.0M 1559M 19.0.0-1450-gf78a58c0 f081f847070a 36685277f4cb 2024-02-16T19:36:13.539 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi163 running (6m) 2m ago 16m 111M 2581M 19.0.0-1450-gf78a58c0 f081f847070a d4b9a7b78af1 2024-02-16T19:36:13.539 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi163 running (5m) 2m ago 16m 100M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 0c47887cb850 2024-02-16T19:36:13.539 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi163 running (4m) 2m ago 15m 101M 2581M 19.0.0-1450-gf78a58c0 f081f847070a 807b119a828c 2024-02-16T19:36:13.539 INFO:teuthology.orchestra.run.smithi040.stdout:prometheus.smithi040 smithi040 *:9095 running (2m) 112s ago 19m 47.1M - 2.43.0 a07b618ecd1d 8a7a0a27c8fb 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.538+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8e5406d7e0 msgr2=0x7f8e5406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.538+0000 7f8e6e8e9700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8e5406d7e0 0x7f8e5406fc90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e50006fd0 tx=0x7f8e50006db0 comp rx=0 tx=0).stop 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.538+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 msgr2=0x7f8e68112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.538+0000 7f8e6e8e9700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 0x7f8e68112970 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800b3b0 tx=0x7f8e58009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.538+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 shutdown_connections 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.539+0000 7f8e6e8e9700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f8e5406d7e0 0x7f8e5406fc90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.539+0000 7f8e6e8e9700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e68101e40 0x7f8e68112970 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.539+0000 7f8e6e8e9700 1 --2- 172.21.15.40:0/598352014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e681027e0 0x7f8e68112eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.539+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 >> 172.21.15.40:0/598352014 conn(0x7f8e68071d90 msgr2=0x7f8e680fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.539+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 shutdown_connections 2024-02-16T19:36:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:13.539+0000 7f8e6e8e9700 1 -- 172.21.15.40:0/598352014 wait complete. 2024-02-16T19:36:13.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:13 smithi040 ceph-mon[94210]: pgmap v312: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:13.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:13 smithi163 ceph-mon[81637]: pgmap v312: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:14.228 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-16T19:36:14.498 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:36:14.621 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:14 smithi040 ceph-mon[94210]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:14.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:14 smithi163 ceph-mon[81637]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-16T19:36:15.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:15 smithi163 ceph-mon[81637]: pgmap v313: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:15.769 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:15 smithi040 ceph-mon[94210]: pgmap v313: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:16.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 -- 172.21.15.40:0/3949679623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 msgr2=0x7f9eb40fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 --2- 172.21.15.40:0/3949679623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 0x7f9eb40fcf60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8004940 tx=0x7f9ea8009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:16.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 -- 172.21.15.40:0/3949679623 shutdown_connections 2024-02-16T19:36:16.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 --2- 172.21.15.40:0/3949679623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 0x7f9eb40fcf60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 --2- 172.21.15.40:0/3949679623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb4102f50 0x7f9eb4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 -- 172.21.15.40:0/3949679623 >> 172.21.15.40:0/3949679623 conn(0x7f9eb40f8ae0 msgr2=0x7f9eb40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:16.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 -- 172.21.15.40:0/3949679623 shutdown_connections 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 -- 172.21.15.40:0/3949679623 wait complete. 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 Processor -- start 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.289+0000 7f9eba689700 1 -- start start 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eba689700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb4102f50 0x7f9eb4113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eba689700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 0x7f9eb4113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eba689700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4114620 con 0x7f9eb41038f0 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eba689700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb410d950 con 0x7f9eb4102f50 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 0x7f9eb4113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 0x7f9eb4113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52000/0 (socket says 172.21.15.40:52000) 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eb37fe700 1 -- 172.21.15.40:0/127195023 learned_addr learned my addr 172.21.15.40:0/127195023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:16.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eb3fff700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb4102f50 0x7f9eb4113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eb37fe700 1 -- 172.21.15.40:0/127195023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb4102f50 msgr2=0x7f9eb4113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eb37fe700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb4102f50 0x7f9eb4113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eb37fe700 1 -- 172.21.15.40:0/127195023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8005040 con 0x7f9eb41038f0 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.290+0000 7f9eb37fe700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 0x7f9eb4113fd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800b3b0 tx=0x7f9ea8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.291+0000 7f9eb0ff9700 1 -- 172.21.15.40:0/127195023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea800e040 con 0x7f9eb41038f0 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.291+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb410dbd0 con 0x7f9eb41038f0 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.291+0000 7f9eb0ff9700 1 -- 172.21.15.40:0/127195023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ea8009af0 con 0x7f9eb41038f0 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.291+0000 7f9eb0ff9700 1 -- 172.21.15.40:0/127195023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea801c3f0 con 0x7f9eb41038f0 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.291+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb410e110 con 0x7f9eb41038f0 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.292+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb4105420 con 0x7f9eb41038f0 2024-02-16T19:36:16.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.292+0000 7f9eb0ff9700 1 -- 172.21.15.40:0/127195023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9ea800bbf0 con 0x7f9eb41038f0 2024-02-16T19:36:16.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.293+0000 7f9eb0ff9700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9e9c06d7e0 0x7f9e9c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:16.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.293+0000 7f9eb3fff700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9e9c06d7e0 0x7f9e9c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:16.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.293+0000 7f9eb0ff9700 1 -- 172.21.15.40:0/127195023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9ea808e310 con 0x7f9eb41038f0 2024-02-16T19:36:16.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.293+0000 7f9eb3fff700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9e9c06d7e0 0x7f9e9c06fc90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40041d0 tx=0x7f9ea40043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:16.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.298+0000 7f9eb0ff9700 1 -- 172.21.15.40:0/127195023 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ea805b660 con 0x7f9eb41038f0 2024-02-16T19:36:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.585+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9eb410f070 con 0x7f9eb41038f0 2024-02-16T19:36:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.586+0000 7f9eb0ff9700 1 -- 172.21.15.40:0/127195023 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9ea805b1f0 con 0x7f9eb41038f0 2024-02-16T19:36:16.587 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-02-16T19:36:16.587 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": { 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": { 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": { 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-02-16T19:36:16.588 INFO:teuthology.orchestra.run.smithi040.stdout: "overall": { 2024-02-16T19:36:16.589 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-16T19:36:16.589 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-02-16T19:36:16.589 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9e9c06d7e0 msgr2=0x7f9e9c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9e9c06d7e0 0x7f9e9c06fc90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40041d0 tx=0x7f9ea40043c0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 msgr2=0x7f9eb4113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 0x7f9eb4113fd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800b3b0 tx=0x7f9ea8009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 shutdown_connections 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eb4102f50 0x7f9eb4113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9e9c06d7e0 0x7f9e9c06fc90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 --2- 172.21.15.40:0/127195023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9eb41038f0 0x7f9eb4113fd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 >> 172.21.15.40:0/127195023 conn(0x7f9eb40f8ae0 msgr2=0x7f9eb40fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 shutdown_connections 2024-02-16T19:36:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:16.588+0000 7f9eba689700 1 -- 172.21.15.40:0/127195023 wait complete. 2024-02-16T19:36:17.264 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-16T19:36:17.532 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:36:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:17 smithi040 ceph-mon[94210]: pgmap v314: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:17 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/127195023' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:17.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:17 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:17 smithi163 ceph-mon[81637]: pgmap v314: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:17 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/127195023' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:17.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:17 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:19.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.140+0000 7fec9c9a9700 1 -- 172.21.15.40:0/2638581426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fe8f0 msgr2=0x7fec98107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.140+0000 7fec9c9a9700 1 --2- 172.21.15.40:0/2638581426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fe8f0 0x7fec98107050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fec80004970 tx=0x7fec80009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.140+0000 7fec9c9a9700 1 -- 172.21.15.40:0/2638581426 shutdown_connections 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.140+0000 7fec9c9a9700 1 --2- 172.21.15.40:0/2638581426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fe8f0 0x7fec98107050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.140+0000 7fec9c9a9700 1 --2- 172.21.15.40:0/2638581426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec980fdfe0 0x7fec980fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.140+0000 7fec9c9a9700 1 -- 172.21.15.40:0/2638581426 >> 172.21.15.40:0/2638581426 conn(0x7fec98071d90 msgr2=0x7fec98072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.140+0000 7fec9c9a9700 1 -- 172.21.15.40:0/2638581426 shutdown_connections 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.140+0000 7fec9c9a9700 1 -- 172.21.15.40:0/2638581426 wait complete. 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9c9a9700 1 Processor -- start 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9c9a9700 1 -- start start 2024-02-16T19:36:19.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9c9a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fdfe0 0x7fec980fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9c9a9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec980fe8f0 0x7fec980fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9c9a9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980fc9e0 con 0x7fec980fdfe0 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9c9a9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980fcb50 con 0x7fec980fe8f0 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fdfe0 0x7fec980fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fdfe0 0x7fec980fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59260/0 (socket says 172.21.15.40:59260) 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9659c700 1 -- 172.21.15.40:0/946772013 learned_addr learned my addr 172.21.15.40:0/946772013 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec95d9b700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec980fe8f0 0x7fec980fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9659c700 1 -- 172.21.15.40:0/946772013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec980fe8f0 msgr2=0x7fec980fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9659c700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec980fe8f0 0x7fec980fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.141+0000 7fec9659c700 1 -- 172.21.15.40:0/946772013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80005040 con 0x7fec980fdfe0 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.142+0000 7fec9659c700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fdfe0 0x7fec980fbea0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec90002a50 tx=0x7fec900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.142+0000 7fec8effd700 1 -- 172.21.15.40:0/946772013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec90007cc0 con 0x7fec980fdfe0 2024-02-16T19:36:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.142+0000 7fec8effd700 1 -- 172.21.15.40:0/946772013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec9000f040 con 0x7fec980fdfe0 2024-02-16T19:36:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.142+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec98119c30 con 0x7fec980fdfe0 2024-02-16T19:36:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.142+0000 7fec8effd700 1 -- 172.21.15.40:0/946772013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec9000da30 con 0x7fec980fdfe0 2024-02-16T19:36:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.142+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9811a0e0 con 0x7fec980fdfe0 2024-02-16T19:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.143+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec981046c0 con 0x7fec980fdfe0 2024-02-16T19:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.143+0000 7fec8effd700 1 -- 172.21.15.40:0/946772013 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fec9000dbb0 con 0x7fec980fdfe0 2024-02-16T19:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.144+0000 7fec8effd700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fec8406d7e0 0x7fec8406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.144+0000 7fec8effd700 1 -- 172.21.15.40:0/946772013 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fec9008cd20 con 0x7fec980fdfe0 2024-02-16T19:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.144+0000 7fec95d9b700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fec8406d7e0 0x7fec8406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.144+0000 7fec95d9b700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fec8406d7e0 0x7fec8406fc90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fec80000c00 tx=0x7fec80009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:19.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.149+0000 7fec8effd700 1 -- 172.21.15.40:0/946772013 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec900564c0 con 0x7fec980fdfe0 2024-02-16T19:36:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.437+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec980738b0 con 0x7fec980fdfe0 2024-02-16T19:36:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.437+0000 7fec8effd700 1 -- 172.21.15.40:0/946772013 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fec90059b80 con 0x7fec980fdfe0 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.439+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fec8406d7e0 msgr2=0x7fec8406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.439+0000 7fec9c9a9700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fec8406d7e0 0x7fec8406fc90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fec80000c00 tx=0x7fec80009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.439+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fdfe0 msgr2=0x7fec980fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.439+0000 7fec9c9a9700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fdfe0 0x7fec980fbea0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec90002a50 tx=0x7fec900046b0 comp rx=0 tx=0).stop 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.439+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 shutdown_connections 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.440+0000 7fec9c9a9700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fec8406d7e0 0x7fec8406fc90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.440+0000 7fec9c9a9700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec980fdfe0 0x7fec980fbea0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.440+0000 7fec9c9a9700 1 --2- 172.21.15.40:0/946772013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec980fe8f0 0x7fec980fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.440+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 >> 172.21.15.40:0/946772013 conn(0x7fec98071d90 msgr2=0x7fec98101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.440+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 shutdown_connections 2024-02-16T19:36:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:19.440+0000 7fec9c9a9700 1 -- 172.21.15.40:0/946772013 wait complete. 2024-02-16T19:36:19.451 INFO:teuthology.orchestra.run.smithi040.stdout:true 2024-02-16T19:36:19.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:19 smithi163 ceph-mon[81637]: pgmap v315: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:19.894 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:19 smithi040 ceph-mon[94210]: pgmap v315: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:20.052 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-16T19:36:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:36:20.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:20 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/946772013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:20.738 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:20 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/946772013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:21.711 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:21 smithi040 ceph-mon[94210]: pgmap v316: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:21.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:21 smithi163 ceph-mon[81637]: pgmap v316: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 -- 172.21.15.40:0/3015897678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f641038f0 msgr2=0x7f5f640fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 --2- 172.21.15.40:0/3015897678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f641038f0 0x7f5f640fcf60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5f58004940 tx=0x7f5f58009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 -- 172.21.15.40:0/3015897678 shutdown_connections 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 --2- 172.21.15.40:0/3015897678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f641038f0 0x7f5f640fcf60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 --2- 172.21.15.40:0/3015897678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64102f50 0x7f5f64103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 -- 172.21.15.40:0/3015897678 >> 172.21.15.40:0/3015897678 conn(0x7f5f640f8ae0 msgr2=0x7f5f640faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 -- 172.21.15.40:0/3015897678 shutdown_connections 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 -- 172.21.15.40:0/3015897678 wait complete. 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.182+0000 7f5f6a20f700 1 Processor -- start 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f6a20f700 1 -- start start 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f6a20f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f64102f50 0x7f5f64113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f6a20f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f641038f0 0x7f5f64114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f6a20f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f641146e0 con 0x7f5f64102f50 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f6a20f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6410d980 con 0x7f5f641038f0 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f64102f50 0x7f5f64113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f64102f50 0x7f5f64113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59278/0 (socket says 172.21.15.40:59278) 2024-02-16T19:36:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f637fe700 1 -- 172.21.15.40:0/2488526913 learned_addr learned my addr 172.21.15.40:0/2488526913 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f62ffd700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f641038f0 0x7f5f64114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f637fe700 1 -- 172.21.15.40:0/2488526913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f641038f0 msgr2=0x7f5f64114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f637fe700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f641038f0 0x7f5f64114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f637fe700 1 -- 172.21.15.40:0/2488526913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58005040 con 0x7f5f64102f50 2024-02-16T19:36:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.183+0000 7f5f637fe700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f64102f50 0x7f5f64113ac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b790 tx=0x7f5f54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.184+0000 7f5f4bfff700 1 -- 172.21.15.40:0/2488526913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5400f770 con 0x7f5f64102f50 2024-02-16T19:36:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.184+0000 7f5f4bfff700 1 -- 172.21.15.40:0/2488526913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f5400fda0 con 0x7f5f64102f50 2024-02-16T19:36:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.184+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6410dc60 con 0x7f5f64102f50 2024-02-16T19:36:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.184+0000 7f5f4bfff700 1 -- 172.21.15.40:0/2488526913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5400d310 con 0x7f5f64102f50 2024-02-16T19:36:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.184+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6410e1a0 con 0x7f5f64102f50 2024-02-16T19:36:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.185+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f640fc6e0 con 0x7f5f64102f50 2024-02-16T19:36:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.185+0000 7f5f4bfff700 1 -- 172.21.15.40:0/2488526913 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f54010420 con 0x7f5f64102f50 2024-02-16T19:36:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.186+0000 7f5f4bfff700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f4c06d7e0 0x7f5f4c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.186+0000 7f5f4bfff700 1 -- 172.21.15.40:0/2488526913 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f5408cfd0 con 0x7f5f64102f50 2024-02-16T19:36:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.186+0000 7f5f62ffd700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f4c06d7e0 0x7f5f4c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.186+0000 7f5f62ffd700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f4c06d7e0 0x7f5f4c06fc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f6410f2f0 tx=0x7f5f58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.190+0000 7f5f4bfff700 1 -- 172.21.15.40:0/2488526913 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f5405a320 con 0x7f5f64102f50 2024-02-16T19:36:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.483+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5f6410f0a0 con 0x7f5f64102f50 2024-02-16T19:36:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.483+0000 7f5f4bfff700 1 -- 172.21.15.40:0/2488526913 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5f54059eb0 con 0x7f5f64102f50 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.485+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f4c06d7e0 msgr2=0x7f5f4c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.485+0000 7f5f6a20f700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f4c06d7e0 0x7f5f4c06fc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f6410f2f0 tx=0x7f5f58009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.485+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f64102f50 msgr2=0x7f5f64113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.485+0000 7f5f6a20f700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f64102f50 0x7f5f64113ac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b790 tx=0x7f5f54009250 comp rx=0 tx=0).stop 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.485+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 shutdown_connections 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.485+0000 7f5f6a20f700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5f4c06d7e0 0x7f5f4c06fc90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.485+0000 7f5f6a20f700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f64102f50 0x7f5f64113ac0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.486+0000 7f5f6a20f700 1 --2- 172.21.15.40:0/2488526913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f641038f0 0x7f5f64114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.486+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 >> 172.21.15.40:0/2488526913 conn(0x7f5f640f8ae0 msgr2=0x7f5f640fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.486+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 shutdown_connections 2024-02-16T19:36:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:22.486+0000 7f5f6a20f700 1 -- 172.21.15.40:0/2488526913 wait complete. 2024-02-16T19:36:22.497 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)" 2024-02-16T19:36:23.142 DEBUG:teuthology.parallel:result is None 2024-02-16T19:36:23.143 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-16T19:36:23.155 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:36:23.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-16T19:36:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/mon.smithi040/config 2024-02-16T19:36:23.698 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:23 smithi040 ceph-mon[94210]: pgmap v317: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:23.698 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:23 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2488526913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:23 smithi163 ceph-mon[81637]: pgmap v317: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:23.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:23 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2488526913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/306118051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fe940 msgr2=0x7ff9c4106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 --2- 172.21.15.40:0/306118051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fe940 0x7ff9c4106f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8004940 tx=0x7ff9b8009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/306118051 shutdown_connections 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 --2- 172.21.15.40:0/306118051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fe940 0x7ff9c4106f80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 --2- 172.21.15.40:0/306118051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9c40fdfa0 0x7ff9c40fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/306118051 >> 172.21.15.40:0/306118051 conn(0x7ff9c40f79d0 msgr2=0x7ff9c40f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/306118051 shutdown_connections 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/306118051 wait complete. 2024-02-16T19:36:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 Processor -- start 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.038+0000 7ff9cbfd7700 1 -- start start 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9cbfd7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fdfa0 0x7ff9c40fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9cbfd7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9c40fe940 0x7ff9c40fc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9cbfd7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40fc910 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9cbfd7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40fca80 con 0x7ff9c40fe940 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9d73700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fdfa0 0x7ff9c40fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9d73700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fdfa0 0x7ff9c40fbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59286/0 (socket says 172.21.15.40:59286) 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9d73700 1 -- 172.21.15.40:0/3628026183 learned_addr learned my addr 172.21.15.40:0/3628026183 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9572700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9c40fe940 0x7ff9c40fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9d73700 1 -- 172.21.15.40:0/3628026183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9c40fe940 msgr2=0x7ff9c40fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9d73700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9c40fe940 0x7ff9c40fc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9d73700 1 -- 172.21.15.40:0/3628026183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b8005040 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9572700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9c40fe940 0x7ff9c40fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:36:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c9d73700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fdfa0 0x7ff9c40fbdd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c830 tx=0x7ff9bc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c27fc700 1 -- 172.21.15.40:0/3628026183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9bc00a760 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c4119ce0 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c27fc700 1 -- 172.21.15.40:0/3628026183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9bc00ad90 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9c27fc700 1 -- 172.21.15.40:0/3628026183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9bc0113a0 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.039+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c411a1c0 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.040+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c4104590 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.041+0000 7ff9c27fc700 1 -- 172.21.15.40:0/3628026183 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff9bc012400 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.041+0000 7ff9c27fc700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff9ac06d7e0 0x7ff9ac06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.041+0000 7ff9c27fc700 1 -- 172.21.15.40:0/3628026183 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff9bc08cf40 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.041+0000 7ff9c9572700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff9ac06d7e0 0x7ff9ac06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.042+0000 7ff9c9572700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff9ac06d7e0 0x7ff9ac06fc90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40fd7d0 tx=0x7ff9b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.046+0000 7ff9c27fc700 1 -- 172.21.15.40:0/3628026183 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9bc0c0d20 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.293+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff9c40fd510 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.293+0000 7ff9c27fc700 1 -- 172.21.15.40:0/3628026183 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 41 v41) v1 ==== 76+0+1880 (secure 0 0 0) 0x7ff9bc056760 con 0x7ff9c40fdfa0 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:e41 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 41 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-16T19:36:25.295 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-02-16T19:21:57.272432+0000 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-02-16T19:33:18.695800+0000 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 60 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 83 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 2 2024-02-16T19:36:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=34404,1=34436} 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2024-02-16T19:36:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 1 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.ezkppl{0:34404} state up:active seq 18 join_fscid=1 addr [v2:172.21.15.40:6826/620521629,v1:172.21.15.40:6827/620521629] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi040.bmpjpk{1:34436} state up:active seq 14 join_fscid=1 addr [v2:172.21.15.40:6828/1175631718,v1:172.21.15.40:6829/1175631718] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.rggdhh{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6824/1456075822,v1:172.21.15.163:6825/1456075822] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:36:25.298 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.cephfs.smithi163.qovvde{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.163:6826/2051774557,v1:172.21.15.163:6827/2051774557] compat {c=[1],r=[1],i=[fff]}] 2024-02-16T19:36:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff9ac06d7e0 msgr2=0x7ff9ac06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff9ac06d7e0 0x7ff9ac06fc90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40fd7d0 tx=0x7ff9b8009e30 comp rx=0 tx=0).stop 2024-02-16T19:36:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fdfa0 msgr2=0x7ff9c40fbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fdfa0 0x7ff9c40fbdd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c830 tx=0x7ff9bc004470 comp rx=0 tx=0).stop 2024-02-16T19:36:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 shutdown_connections 2024-02-16T19:36:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff9ac06d7e0 0x7ff9ac06fc90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9c40fdfa0 0x7ff9c40fbdd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 --2- 172.21.15.40:0/3628026183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9c40fe940 0x7ff9c40fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 >> 172.21.15.40:0/3628026183 conn(0x7ff9c40f79d0 msgr2=0x7ff9c4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 shutdown_connections 2024-02-16T19:36:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:25.295+0000 7ff9cbfd7700 1 -- 172.21.15.40:0/3628026183 wait complete. 2024-02-16T19:36:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2024-02-16T19:36:25.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:25 smithi040 ceph-mon[94210]: pgmap v318: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:25.615 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:25 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3628026183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:36:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:25 smithi163 ceph-mon[81637]: pgmap v318: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:25.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:25 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3628026183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-16T19:36:25.942 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-16T19:36:25.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T19:36:26.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 -- 172.21.15.40:0/1509278660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 msgr2=0x7f9c140fb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 --2- 172.21.15.40:0/1509278660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 0x7f9c140fb8d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0048d0 tx=0x7f9bfc009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 -- 172.21.15.40:0/1509278660 shutdown_connections 2024-02-16T19:36:26.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 --2- 172.21.15.40:0/1509278660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 0x7f9c140fb8d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 --2- 172.21.15.40:0/1509278660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c141014a0 0x7f9c14101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 -- 172.21.15.40:0/1509278660 >> 172.21.15.40:0/1509278660 conn(0x7f9c140f6fe0 msgr2=0x7f9c140f93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:26.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 -- 172.21.15.40:0/1509278660 shutdown_connections 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 -- 172.21.15.40:0/1509278660 wait complete. 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 Processor -- start 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 -- start start 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 0x7f9c1410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c141014a0 0x7f9c1410e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1410e940 con 0x7f9c140fb460 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c198ae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1410eab0 con 0x7f9c141014a0 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c12ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 0x7f9c1410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c12ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 0x7f9c1410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59300/0 (socket says 172.21.15.40:59300) 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.058+0000 7f9c12ffd640 1 -- 172.21.15.40:0/2895922114 learned_addr learned my addr 172.21.15.40:0/2895922114 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:26.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c0bfff640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c141014a0 0x7f9c1410e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c12ffd640 1 -- 172.21.15.40:0/2895922114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c141014a0 msgr2=0x7f9c1410e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c12ffd640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c141014a0 0x7f9c1410e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c12ffd640 1 -- 172.21.15.40:0/2895922114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc005040 con 0x7f9c140fb460 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c0bfff640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c141014a0 0x7f9c1410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c12ffd640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 0x7f9c1410dec0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400b790 tx=0x7f9c040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c0b7fe640 1 -- 172.21.15.40:0/2895922114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c0400c700 con 0x7f9c140fb460 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c0b7fe640 1 -- 172.21.15.40:0/2895922114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c0400cd30 con 0x7f9c140fb460 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c14116f30 con 0x7f9c140fb460 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c0b7fe640 1 -- 172.21.15.40:0/2895922114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c04013370 con 0x7f9c140fb460 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.059+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c14117310 con 0x7f9c140fb460 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.060+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c14064480 con 0x7f9c140fb460 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.060+0000 7f9c0b7fe640 1 -- 172.21.15.40:0/2895922114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9c04012070 con 0x7f9c140fb460 2024-02-16T19:36:26.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.061+0000 7f9c0b7fe640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c0006d750 0x7f9c0006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.061+0000 7f9c0b7fe640 1 -- 172.21.15.40:0/2895922114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9c0408ca60 con 0x7f9c140fb460 2024-02-16T19:36:26.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.061+0000 7f9c0bfff640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c0006d750 0x7f9c0006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.061+0000 7f9c0bfff640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c0006d750 0x7f9c0006fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc02bda0 tx=0x7f9bfc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:26.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.065+0000 7f9c0b7fe640 1 -- 172.21.15.40:0/2895922114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c04056280 con 0x7f9c140fb460 2024-02-16T19:36:26.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.247+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c141085f0 con 0x7f9c140fb460 2024-02-16T19:36:26.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.248+0000 7f9c0b7fe640 1 -- 172.21.15.40:0/2895922114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16756 (secure 0 0 0) 0x7f9c04059940 con 0x7f9c140fb460 2024-02-16T19:36:26.250 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:26.250 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","created":"2024-02-16T19:14:55.436141+0000","modified":"2024-02-16T19:32:59.315148+0000","last_up_change":"2024-02-16T19:31:25.855269+0000","last_in_change":"2024-02-16T19:20:17.443690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:19:22.349243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T19:21:55.970810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T19:21:56.515184+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db82e60f-540c-4f54-8f54-758be43e2cf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6803","nonce":3792625162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6805","nonce":3792625162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6809","nonce":3792625162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6807","nonce":3792625162}]},"public_addr":"172.21.15.40:6803/3792625162","cluster_addr":"172.21.15.40:6805/3792625162","heartbeat_back_addr":"172.21.15.40:6809/3792625162","heartbeat_front_addr":"172.21.15.40:6807/3792625162","state":["exists","up"]},{"osd":1,"uuid":"22561508-cdb8-4c76-ab07-bf3efd6ee63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6811","nonce":3029722765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6813","nonce":3029722765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6817","nonce":3029722765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6815","nonce":3029722765}]},"public_addr":"172.21.15.40:6811/3029722765","cluster_addr":"172.21.15.40:6813/3029722765","heartbeat_back_addr":"172.21.15.40:6817/3029722765","heartbeat_front_addr":"172.21.15.40:6815/3029722765","state":["exists","up"]},{"osd":2,"uuid":"58ff6654-72f1-4329-8f6c-4c3180353755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6819","nonce":1990065416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6821","nonce":1990065416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6825","nonce":1990065416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6823","nonce":1990065416}]},"public_addr":"172.21.15.40:6819/1990065416","cluster_addr":"172.21.15.40:6821/1990065416","heartbeat_back_addr":"172.21.15.40:6825/1990065416","heartbeat_front_addr":"172.21.15.40:6823/1990065416","state":["exists","up"]},{"osd":3,"uuid":"b6f2b76f-f1fa-4459-8150-c36e624fb4aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6801","nonce":346064807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6803","nonce":346064807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6807","nonce":346064807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6805","nonce":346064807}]},"public_addr":"172.21.15.163:6801/346064807","cluster_addr":"172.21.15.163:6803/346064807","heartbeat_back_addr":"172.21.15.163:6807/346064807","heartbeat_front_addr":"172.21.15.163:6805/346064807","state":["exists","up"]},{"osd":4,"uuid":"49afe3d8-2685-4c11-9830-77bd72a0b272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6809","nonce":2382516830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6811","nonce":2382516830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6815","nonce":2382516830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6813","nonce":2382516830}]},"public_addr":"172.21.15.163:6809/2382516830","cluster_addr":"172.21.15.163:6811/2382516830","heartbeat_back_addr":"172.21.15.163:6815/2382516830","heartbeat_front_addr":"172.21.15.163:6813/2382516830","state":["exists","up"]},{"osd":5,"uuid":"c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6817","nonce":10307935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6819","nonce":10307935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6823","nonce":10307935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6821","nonce":10307935}]},"public_addr":"172.21.15.163:6817/10307935","cluster_addr":"172.21.15.163:6819/10307935","heartbeat_back_addr":"172.21.15.163:6823/10307935","heartbeat_front_addr":"172.21.15.163:6821/10307935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:22.484106+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:52.190602+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:19.421062+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:43.759349+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:09.243860+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:34.636467+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6826/1325230089":"2024-02-17T19:32:59.315119+0000","172.21.15.40:6829/2154720438":"2024-02-17T19:32:20.151991+0000","172.21.15.40:6828/2154720438":"2024-02-17T19:32:20.151991+0000","172.21.15.40:0/493553370":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/4011363422":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1058311844":"2024-02-17T19:15:23.873919+0000","172.21.15.163:6827/1325230089":"2024-02-17T19:32:59.315119+0000","172.21.15.40:0/250713212":"2024-02-17T19:24:41.534183+0000","172.21.15.40:0/1423773760":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/726664255":"2024-02-17T19:24:41.534183+0000","172.21.15.40:0/2148905528":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6800/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6827/659310888":"2024-02-17T19:22:12.998828+0000","172.21.15.40:6800/1137681815":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6801/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/1306942330":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2673381923":"2024-02-17T19:15:54.760982+0000","172.21.15.163:0/1579052634":"2024-02-17T19:26:17.176557+0000","172.21.15.40:6826/659310888":"2024-02-17T19:22:12.998828+0000","172.21.15.40:0/647211605":"2024-02-17T19:27:02.146968+0000","172.21.15.163:0/1076569647":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2189950867":"2024-02-17T19:15:54.760982+0000","172.21.15.163:0/2090165668":"2024-02-17T19:26:17.176557+0000","172.21.15.40:6800/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2144454600":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/8215223":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/1412951630":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6800/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/1754668438":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6800/1321006798":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6801/1321006798":"2024-02-17T19:24:41.534183+0000","172.21.15.163:0/841594618":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2009141374":"2024-02-17T19:17:10.018957+0000","172.21.15.163:6828/3793859838":"2024-02-17T19:26:17.176557+0000","172.21.15.163:6829/3793859838":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2888910656":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/1137681815":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1738800778":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/1430644152":"2024-02-17T19:27:02.146968+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:36:26.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c0006d750 msgr2=0x7f9c0006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c0006d750 0x7f9c0006fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc02bda0 tx=0x7f9bfc009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 msgr2=0x7f9c1410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 0x7f9c1410dec0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400b790 tx=0x7f9c040091d0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 shutdown_connections 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c0006d750 0x7f9c0006fc00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c141014a0 0x7f9c1410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 --2- 172.21.15.40:0/2895922114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c140fb460 0x7f9c1410dec0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 >> 172.21.15.40:0/2895922114 conn(0x7f9c140f6fe0 msgr2=0x7f9c140f7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 shutdown_connections 2024-02-16T19:36:26.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.250+0000 7f9c198ae640 1 -- 172.21.15.40:0/2895922114 wait complete. 2024-02-16T19:36:26.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:19:22.349243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T19:21:55.970810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T19:21:56.515184+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T19:36:26.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T19:36:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.364+0000 7f4579b9d640 1 -- 172.21.15.40:0/3195936334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 msgr2=0x7f4574101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.364+0000 7f4579b9d640 1 --2- 172.21.15.40:0/3195936334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 0x7f4574101640 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45640048d0 tx=0x7f4564009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.364+0000 7f4579b9d640 1 -- 172.21.15.40:0/3195936334 shutdown_connections 2024-02-16T19:36:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.364+0000 7f4579b9d640 1 --2- 172.21.15.40:0/3195936334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574101c10 0x7f45740fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.364+0000 7f4579b9d640 1 --2- 172.21.15.40:0/3195936334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 0x7f4574101640 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.364+0000 7f4579b9d640 1 -- 172.21.15.40:0/3195936334 >> 172.21.15.40:0/3195936334 conn(0x7f45740f6e80 msgr2=0x7f45740f9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.364+0000 7f4579b9d640 1 -- 172.21.15.40:0/3195936334 shutdown_connections 2024-02-16T19:36:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.364+0000 7f4579b9d640 1 -- 172.21.15.40:0/3195936334 wait complete. 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f4579b9d640 1 Processor -- start 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f4579b9d640 1 -- start start 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f4579b9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 0x7f457410dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f4579b9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574101c10 0x7f457410e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f4579b9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457410e760 con 0x7f4574101270 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f4579b9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457410e8d0 con 0x7f4574101c10 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f45737fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 0x7f457410dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f45737fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 0x7f457410dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59304/0 (socket says 172.21.15.40:59304) 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f45737fe640 1 -- 172.21.15.40:0/2217928616 learned_addr learned my addr 172.21.15.40:0/2217928616 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f45737fe640 1 -- 172.21.15.40:0/2217928616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574101c10 msgr2=0x7f457410e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f45737fe640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574101c10 0x7f457410e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f45737fe640 1 -- 172.21.15.40:0/2217928616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4564005040 con 0x7f4574101270 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.365+0000 7f45737fe640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 0x7f457410dce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4564031720 tx=0x7f4564009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.366+0000 7f455bfff640 1 -- 172.21.15.40:0/2217928616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45640094c0 con 0x7f4574101270 2024-02-16T19:36:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.366+0000 7f455bfff640 1 -- 172.21.15.40:0/2217928616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4564009620 con 0x7f4574101270 2024-02-16T19:36:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.366+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457410eb50 con 0x7f4574101270 2024-02-16T19:36:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.366+0000 7f455bfff640 1 -- 172.21.15.40:0/2217928616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456403b640 con 0x7f4574101270 2024-02-16T19:36:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.366+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4574117050 con 0x7f4574101270 2024-02-16T19:36:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.367+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4574064480 con 0x7f4574101270 2024-02-16T19:36:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.367+0000 7f455bfff640 1 -- 172.21.15.40:0/2217928616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f456402f070 con 0x7f4574101270 2024-02-16T19:36:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.368+0000 7f455bfff640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f455c06d750 0x7f455c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.368+0000 7f455bfff640 1 -- 172.21.15.40:0/2217928616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f45640af340 con 0x7f4574101270 2024-02-16T19:36:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.368+0000 7f4572ffd640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f455c06d750 0x7f455c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.368+0000 7f4572ffd640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f455c06d750 0x7f455c06fc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4574065fa0 tx=0x7f4568009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:26.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.372+0000 7f455bfff640 1 -- 172.21.15.40:0/2217928616 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4564078ae0 con 0x7f4574101270 2024-02-16T19:36:26.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.554+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f457410eaf0 con 0x7f4574101270 2024-02-16T19:36:26.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.554+0000 7f455bfff640 1 -- 172.21.15.40:0/2217928616 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f456407c1a0 con 0x7f4574101270 2024-02-16T19:36:26.556 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f455c06d750 msgr2=0x7f455c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f455c06d750 0x7f455c06fc00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4574065fa0 tx=0x7f4568009c50 comp rx=0 tx=0).stop 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 msgr2=0x7f457410dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 0x7f457410dce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4564031720 tx=0x7f4564009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 shutdown_connections 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f455c06d750 0x7f455c06fc00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4574101c10 0x7f457410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 --2- 172.21.15.40:0/2217928616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574101270 0x7f457410dce0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.556+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 >> 172.21.15.40:0/2217928616 conn(0x7f45740f6e80 msgr2=0x7f45740f8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.557+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 shutdown_connections 2024-02-16T19:36:26.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.557+0000 7f4579b9d640 1 -- 172.21.15.40:0/2217928616 wait complete. 2024-02-16T19:36:26.567 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:26 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2895922114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:36:26.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T19:36:26.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.668+0000 7efd9c79d640 1 -- 172.21.15.40:0/4205730489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101be0 msgr2=0x7efd940fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.668+0000 7efd9c79d640 1 --2- 172.21.15.40:0/4205730489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101be0 0x7efd940fb270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd840048d0 tx=0x7efd84009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.668+0000 7efd9c79d640 1 -- 172.21.15.40:0/4205730489 shutdown_connections 2024-02-16T19:36:26.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.668+0000 7efd9c79d640 1 --2- 172.21.15.40:0/4205730489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101be0 0x7efd940fb270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.668+0000 7efd9c79d640 1 --2- 172.21.15.40:0/4205730489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd94101240 0x7efd94101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.668+0000 7efd9c79d640 1 -- 172.21.15.40:0/4205730489 >> 172.21.15.40:0/4205730489 conn(0x7efd940f6e10 msgr2=0x7efd940f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.668+0000 7efd9c79d640 1 -- 172.21.15.40:0/4205730489 shutdown_connections 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.668+0000 7efd9c79d640 1 -- 172.21.15.40:0/4205730489 wait complete. 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9c79d640 1 Processor -- start 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9c79d640 1 -- start start 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9c79d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101240 0x7efd9410bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9c79d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd94101be0 0x7efd9410c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9c79d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940665e0 con 0x7efd94101240 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9c79d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9410c5c0 con 0x7efd94101be0 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9a539640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101240 0x7efd9410bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9a539640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101240 0x7efd9410bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59320/0 (socket says 172.21.15.40:59320) 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9a539640 1 -- 172.21.15.40:0/3469118562 learned_addr learned my addr 172.21.15.40:0/3469118562 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9a539640 1 -- 172.21.15.40:0/3469118562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd94101be0 msgr2=0x7efd9410c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9a539640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd94101be0 0x7efd9410c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9a539640 1 -- 172.21.15.40:0/3469118562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd84005040 con 0x7efd94101240 2024-02-16T19:36:26.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.669+0000 7efd9a539640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101240 0x7efd9410bb40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efd90002a50 tx=0x7efd90004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:26.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.670+0000 7efd8affd640 1 -- 172.21.15.40:0/3469118562 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd90007c30 con 0x7efd94101240 2024-02-16T19:36:26.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.670+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9410c8a0 con 0x7efd94101240 2024-02-16T19:36:26.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.670+0000 7efd8affd640 1 -- 172.21.15.40:0/3469118562 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd9000f040 con 0x7efd94101240 2024-02-16T19:36:26.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.670+0000 7efd8affd640 1 -- 172.21.15.40:0/3469118562 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9000db50 con 0x7efd94101240 2024-02-16T19:36:26.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.670+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd941193a0 con 0x7efd94101240 2024-02-16T19:36:26.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.670+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd94064480 con 0x7efd94101240 2024-02-16T19:36:26.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.671+0000 7efd8affd640 1 -- 172.21.15.40:0/3469118562 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efd9001c400 con 0x7efd94101240 2024-02-16T19:36:26.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.671+0000 7efd8affd640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8006d750 0x7efd8006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.672+0000 7efd8affd640 1 -- 172.21.15.40:0/3469118562 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efd9008c8d0 con 0x7efd94101240 2024-02-16T19:36:26.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.672+0000 7efd99d38640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8006d750 0x7efd8006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.672+0000 7efd99d38640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8006d750 0x7efd8006fc00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd84034040 tx=0x7efd84009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:26.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.676+0000 7efd8affd640 1 -- 172.21.15.40:0/3469118562 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd90055fd0 con 0x7efd94101240 2024-02-16T19:36:26.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:26 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2895922114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:36:26.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.858+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7efd941083a0 con 0x7efd94101240 2024-02-16T19:36:26.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.859+0000 7efd8affd640 1 -- 172.21.15.40:0/3469118562 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7efd90059690 con 0x7efd94101240 2024-02-16T19:36:26.860 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.860+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8006d750 msgr2=0x7efd8006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.860+0000 7efd9c79d640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8006d750 0x7efd8006fc00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd84034040 tx=0x7efd84009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.860+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101240 msgr2=0x7efd9410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.860+0000 7efd9c79d640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101240 0x7efd9410bb40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efd90002a50 tx=0x7efd90004600 comp rx=0 tx=0).stop 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.860+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 shutdown_connections 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.860+0000 7efd9c79d640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7efd8006d750 0x7efd8006fc00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.860+0000 7efd9c79d640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd94101be0 0x7efd9410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.861+0000 7efd9c79d640 1 --2- 172.21.15.40:0/3469118562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101240 0x7efd9410bb40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.861+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 >> 172.21.15.40:0/3469118562 conn(0x7efd940f6e10 msgr2=0x7efd940f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.861+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 shutdown_connections 2024-02-16T19:36:26.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.861+0000 7efd9c79d640 1 -- 172.21.15.40:0/3469118562 wait complete. 2024-02-16T19:36:26.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T19:36:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.972+0000 7f98ca19d640 1 -- 172.21.15.40:0/2461826678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102390 msgr2=0x7f98c4102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.972+0000 7f98ca19d640 1 --2- 172.21.15.40:0/2461826678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102390 0x7f98c4102760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98b40048d0 tx=0x7f98b4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.972+0000 7f98ca19d640 1 -- 172.21.15.40:0/2461826678 shutdown_connections 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.972+0000 7f98ca19d640 1 --2- 172.21.15.40:0/2461826678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c4102d30 0x7f98c40fc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.972+0000 7f98ca19d640 1 --2- 172.21.15.40:0/2461826678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102390 0x7f98c4102760 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.972+0000 7f98ca19d640 1 -- 172.21.15.40:0/2461826678 >> 172.21.15.40:0/2461826678 conn(0x7f98c40f7fa0 msgr2=0x7f98c40fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98ca19d640 1 -- 172.21.15.40:0/2461826678 shutdown_connections 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98ca19d640 1 -- 172.21.15.40:0/2461826678 wait complete. 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98ca19d640 1 Processor -- start 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98ca19d640 1 -- start start 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98ca19d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c4102390 0x7f98c4112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98ca19d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102d30 0x7f98c4113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98ca19d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4113740 con 0x7f98c4102d30 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98ca19d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c410ccd0 con 0x7f98c4102390 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102d30 0x7f98c4113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.973+0000 7f98c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102d30 0x7f98c4113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59342/0 (socket says 172.21.15.40:59342) 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98c2ffd640 1 -- 172.21.15.40:0/324816467 learned_addr learned my addr 172.21.15.40:0/324816467 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98c37fe640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c4102390 0x7f98c4112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98c2ffd640 1 -- 172.21.15.40:0/324816467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c4102390 msgr2=0x7f98c4112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98c2ffd640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c4102390 0x7f98c4112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98c2ffd640 1 -- 172.21.15.40:0/324816467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b4005040 con 0x7f98c4102d30 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98c2ffd640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102d30 0x7f98c4113140 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98b8002a50 tx=0x7f98b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98abfff640 1 -- 172.21.15.40:0/324816467 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b8007c30 con 0x7f98c4102d30 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c410cfb0 con 0x7f98c4102d30 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98abfff640 1 -- 172.21.15.40:0/324816467 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98b800f040 con 0x7f98c4102d30 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98abfff640 1 -- 172.21.15.40:0/324816467 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b800db50 con 0x7f98c4102d30 2024-02-16T19:36:26.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.974+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c410d4d0 con 0x7f98c4102d30 2024-02-16T19:36:26.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.975+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c4064030 con 0x7f98c4102d30 2024-02-16T19:36:26.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.976+0000 7f98abfff640 1 -- 172.21.15.40:0/324816467 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f98b801c400 con 0x7f98c4102d30 2024-02-16T19:36:26.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.976+0000 7f98abfff640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98ac06d750 0x7f98ac06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:26.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.976+0000 7f98abfff640 1 -- 172.21.15.40:0/324816467 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f98b808c7b0 con 0x7f98c4102d30 2024-02-16T19:36:26.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.976+0000 7f98c37fe640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98ac06d750 0x7f98ac06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:26.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.976+0000 7f98c37fe640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98ac06d750 0x7f98ac06fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98b4000c00 tx=0x7f98b4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:26.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:26.981+0000 7f98abfff640 1 -- 172.21.15.40:0/324816467 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f98b8056000 con 0x7f98c4102d30 2024-02-16T19:36:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.161+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f98c410d6f0 con 0x7f98c4102d30 2024-02-16T19:36:27.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.161+0000 7f98abfff640 1 -- 172.21.15.40:0/324816467 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f98b80596c0 con 0x7f98c4102d30 2024-02-16T19:36:27.162 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-02-16T19:36:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98ac06d750 msgr2=0x7f98ac06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98ac06d750 0x7f98ac06fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98b4000c00 tx=0x7f98b4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102d30 msgr2=0x7f98c4113140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102d30 0x7f98c4113140 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f98b8002a50 tx=0x7f98b8004600 comp rx=0 tx=0).stop 2024-02-16T19:36:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 shutdown_connections 2024-02-16T19:36:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98ac06d750 0x7f98ac06fc00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98c4102d30 0x7f98c4113140 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 --2- 172.21.15.40:0/324816467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98c4102390 0x7f98c4112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 >> 172.21.15.40:0/324816467 conn(0x7f98c40f7fa0 msgr2=0x7f98c40f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 shutdown_connections 2024-02-16T19:36:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.163+0000 7f98ca19d640 1 -- 172.21.15.40:0/324816467 wait complete. 2024-02-16T19:36:27.175 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-16T19:36:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.272+0000 7f885ed9d640 1 -- 172.21.15.40:0/889860371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88581012c0 msgr2=0x7f8858101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.272+0000 7f885ed9d640 1 --2- 172.21.15.40:0/889860371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88581012c0 0x7f8858101690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8850009c50 tx=0x7f8850004d10 comp rx=0 tx=0).stop 2024-02-16T19:36:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.272+0000 7f885ed9d640 1 -- 172.21.15.40:0/889860371 shutdown_connections 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.272+0000 7f885ed9d640 1 --2- 172.21.15.40:0/889860371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8858101c60 0x7f88580fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.272+0000 7f885ed9d640 1 --2- 172.21.15.40:0/889860371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88581012c0 0x7f8858101690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.272+0000 7f885ed9d640 1 -- 172.21.15.40:0/889860371 >> 172.21.15.40:0/889860371 conn(0x7f88580718e0 msgr2=0x7f8858071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.272+0000 7f885ed9d640 1 -- 172.21.15.40:0/889860371 shutdown_connections 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.272+0000 7f885ed9d640 1 -- 172.21.15.40:0/889860371 wait complete. 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f885ed9d640 1 Processor -- start 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f885ed9d640 1 -- start start 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f885ed9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88581012c0 0x7f8858112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f885ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8858101c60 0x7f8858113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f885ed9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8858113740 con 0x7f8858101c60 2024-02-16T19:36:27.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f885ed9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885810ccd0 con 0x7f88581012c0 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f8857fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8858101c60 0x7f8858113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f8857fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8858101c60 0x7f8858113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59362/0 (socket says 172.21.15.40:59362) 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f8857fff640 1 -- 172.21.15.40:0/3607339548 learned_addr learned my addr 172.21.15.40:0/3607339548 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f885cb39640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88581012c0 0x7f8858112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f8857fff640 1 -- 172.21.15.40:0/3607339548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88581012c0 msgr2=0x7f8858112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f8857fff640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88581012c0 0x7f8858112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.273+0000 7f8857fff640 1 -- 172.21.15.40:0/3607339548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88500098b0 con 0x7f8858101c60 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.274+0000 7f885cb39640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88581012c0 0x7f8858112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.274+0000 7f8857fff640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8858101c60 0x7f8858113140 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f884c002870 tx=0x7f884c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.274+0000 7f88557fa640 1 -- 172.21.15.40:0/3607339548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c007c30 con 0x7f8858101c60 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.274+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885810cfb0 con 0x7f8858101c60 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.274+0000 7f88557fa640 1 -- 172.21.15.40:0/3607339548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f884c00f040 con 0x7f8858101c60 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.274+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885810d4d0 con 0x7f8858101c60 2024-02-16T19:36:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.274+0000 7f88557fa640 1 -- 172.21.15.40:0/3607339548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c009590 con 0x7f8858101c60 2024-02-16T19:36:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.275+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8858064480 con 0x7f8858101c60 2024-02-16T19:36:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.275+0000 7f88557fa640 1 -- 172.21.15.40:0/3607339548 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f884c0096f0 con 0x7f8858101c60 2024-02-16T19:36:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.276+0000 7f88557fa640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f884006d7a0 0x7f884006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.276+0000 7f88557fa640 1 -- 172.21.15.40:0/3607339548 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f884c08c8b0 con 0x7f8858101c60 2024-02-16T19:36:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.276+0000 7f885cb39640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f884006d7a0 0x7f884006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.276+0000 7f885cb39640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f884006d7a0 0x7f884006fc50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8850004a30 tx=0x7f885002e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:27.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.280+0000 7f88557fa640 1 -- 172.21.15.40:0/3607339548 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f884c055fb0 con 0x7f8858101c60 2024-02-16T19:36:27.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.519+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88580ff5a0 con 0x7f8858101c60 2024-02-16T19:36:27.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.519+0000 7f88557fa640 1 -- 172.21.15.40:0/3607339548 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5016 (secure 0 0 0) 0x7f884c059670 con 0x7f8858101c60 2024-02-16T19:36:27.521 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:27.521 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44335,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6827/2051774557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2051774557},{"type":"v1","addr":"172.21.15.163:6827","nonce":2051774557}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:33:18.695800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34404,"mds_1":34436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":0,"incarnation":35,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34436":{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":1,"incarnation":40,"state":"up:active","state_seq":14,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:27.521 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f884006d7a0 msgr2=0x7f884006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f884006d7a0 0x7f884006fc50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8850004a30 tx=0x7f885002e040 comp rx=0 tx=0).stop 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8858101c60 msgr2=0x7f8858113140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8858101c60 0x7f8858113140 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f884c002870 tx=0x7f884c004600 comp rx=0 tx=0).stop 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 shutdown_connections 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f884006d7a0 0x7f884006fc50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8858101c60 0x7f8858113140 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 --2- 172.21.15.40:0/3607339548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88581012c0 0x7f8858112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 >> 172.21.15.40:0/3607339548 conn(0x7f88580718e0 msgr2=0x7f8858105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.521+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 shutdown_connections 2024-02-16T19:36:27.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.522+0000 7f885ed9d640 1 -- 172.21.15.40:0/3607339548 wait complete. 2024-02-16T19:36:27.534 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-02-16T19:36:27.534 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-02-16T19:36:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 -- 172.21.15.40:0/616688196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 msgr2=0x7f41901016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 --2- 172.21.15.40:0/616688196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 0x7f41901016a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f41800048d0 tx=0x7f4180009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 -- 172.21.15.40:0/616688196 shutdown_connections 2024-02-16T19:36:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 --2- 172.21.15.40:0/616688196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41900fb2b0 0x7f41900fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 --2- 172.21.15.40:0/616688196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 0x7f41901016a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 -- 172.21.15.40:0/616688196 >> 172.21.15.40:0/616688196 conn(0x7f4190071230 msgr2=0x7f4190071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 -- 172.21.15.40:0/616688196 shutdown_connections 2024-02-16T19:36:27.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 -- 172.21.15.40:0/616688196 wait complete. 2024-02-16T19:36:27.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.634+0000 7f4196465640 1 Processor -- start 2024-02-16T19:36:27.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f4196465640 1 -- start start 2024-02-16T19:36:27.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f4196465640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41900fb2b0 0x7f41901020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f4196465640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 0x7f4190102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f4196465640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190104080 con 0x7f41901012d0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f4196465640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190102b60 con 0x7f41900fb2b0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 0x7f4190102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 0x7f4190102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38218/0 (socket says 172.21.15.40:38218) 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f418f7fe640 1 -- 172.21.15.40:0/1023062459 learned_addr learned my addr 172.21.15.40:0/1023062459 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f418ffff640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41900fb2b0 0x7f41901020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f418ffff640 1 -- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 msgr2=0x7f4190102620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f418ffff640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 0x7f4190102620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f418ffff640 1 -- 172.21.15.40:0/1023062459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4180005040 con 0x7f41900fb2b0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.635+0000 7f418f7fe640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 0x7f4190102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.636+0000 7f418ffff640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41900fb2b0 0x7f41901020e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f418002bda0 tx=0x7f4180009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.636+0000 7f418cff9640 1 -- 172.21.15.40:0/1023062459 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41800092b0 con 0x7f41900fb2b0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.636+0000 7f418cff9640 1 -- 172.21.15.40:0/1023062459 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f418003d450 con 0x7f41900fb2b0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.636+0000 7f418cff9640 1 -- 172.21.15.40:0/1023062459 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f418003b640 con 0x7f41900fb2b0 2024-02-16T19:36:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.636+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4190102db0 con 0x7f41900fb2b0 2024-02-16T19:36:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.636+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4190119260 con 0x7f41900fb2b0 2024-02-16T19:36:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.637+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4190064030 con 0x7f41900fb2b0 2024-02-16T19:36:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.637+0000 7f418cff9640 1 -- 172.21.15.40:0/1023062459 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f418002f070 con 0x7f41900fb2b0 2024-02-16T19:36:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.638+0000 7f418cff9640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f417806d750 0x7f417806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.638+0000 7f418cff9640 1 -- 172.21.15.40:0/1023062459 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f41800af210 con 0x7f41900fb2b0 2024-02-16T19:36:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.638+0000 7f418f7fe640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f417806d750 0x7f417806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.638+0000 7f418f7fe640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f417806d750 0x7f417806fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41901038d0 tx=0x7f418400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.643+0000 7f418cff9640 1 -- 172.21.15.40:0/1023062459 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4180078a70 con 0x7f41900fb2b0 2024-02-16T19:36:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:27 smithi163 ceph-mon[81637]: pgmap v319: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:27 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2217928616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:36:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:27 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3469118562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:36:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:27 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/324816467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:36:27.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:27 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:36:27.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.874+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f41901083c0 con 0x7f41900fb2b0 2024-02-16T19:36:27.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.875+0000 7f418cff9640 1 -- 172.21.15.40:0/1023062459 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v41) v1 ==== 107+0+4847 (secure 0 0 0) 0x7f418007c130 con 0x7f41900fb2b0 2024-02-16T19:36:27.876 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:27.876 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/4253297290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4253297290},{"type":"v1","addr":"172.21.15.40:6827","nonce":4253297290}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:23:33.336776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi163.rggdhh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6825/1432724463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1432724463},{"type":"v1","addr":"172.21.15.163:6825","nonce":1432724463}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:27.877 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 18 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f417806d750 msgr2=0x7f417806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f417806d750 0x7f417806fc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41901038d0 tx=0x7f418400b040 comp rx=0 tx=0).stop 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41900fb2b0 msgr2=0x7f41901020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41900fb2b0 0x7f41901020e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f418002bda0 tx=0x7f4180009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 shutdown_connections 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f417806d750 0x7f417806fc00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41901012d0 0x7f4190102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 --2- 172.21.15.40:0/1023062459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41900fb2b0 0x7f41901020e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.877+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 >> 172.21.15.40:0/1023062459 conn(0x7f4190071230 msgr2=0x7f4190064480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:27.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.878+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 shutdown_connections 2024-02-16T19:36:27.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.878+0000 7f4196465640 1 -- 172.21.15.40:0/1023062459 wait complete. 2024-02-16T19:36:27.889 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:27 smithi040 ceph-mon[94210]: pgmap v319: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:27.889 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:27 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2217928616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:36:27.889 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:27 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3469118562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:36:27.889 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:27 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/324816467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:36:27.889 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:27 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-16T19:36:27.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 -- 172.21.15.40:0/2886056126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 msgr2=0x7f64040fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 --2- 172.21.15.40:0/2886056126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 0x7f64040fb700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f63f40048d0 tx=0x7f63f4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 -- 172.21.15.40:0/2886056126 shutdown_connections 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 --2- 172.21.15.40:0/2886056126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 0x7f64040fb700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 --2- 172.21.15.40:0/2886056126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64041012d0 0x7f64041016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 -- 172.21.15.40:0/2886056126 >> 172.21.15.40:0/2886056126 conn(0x7f6404071230 msgr2=0x7f6404071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 -- 172.21.15.40:0/2886056126 shutdown_connections 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 -- 172.21.15.40:0/2886056126 wait complete. 2024-02-16T19:36:27.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 Processor -- start 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.991+0000 7f640bf9d640 1 -- start start 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f640bf9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 0x7f640410dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f640bf9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64041012d0 0x7f640410e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f640bf9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640410e7a0 con 0x7f64040fb2b0 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f640bf9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640410e8e0 con 0x7f64041012d0 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f6409d39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 0x7f640410dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f6409d39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 0x7f640410dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38232/0 (socket says 172.21.15.40:38232) 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f6409d39640 1 -- 172.21.15.40:0/962005614 learned_addr learned my addr 172.21.15.40:0/962005614 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f6409538640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64041012d0 0x7f640410e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f6409d39640 1 -- 172.21.15.40:0/962005614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64041012d0 msgr2=0x7f640410e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f6409d39640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64041012d0 0x7f640410e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f6409d39640 1 -- 172.21.15.40:0/962005614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4005040 con 0x7f64040fb2b0 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f6409d39640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 0x7f640410dd20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f640000eb70 tx=0x7f640000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:27.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f63fa7fc640 1 -- 172.21.15.40:0/962005614 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6400007c30 con 0x7f64040fb2b0 2024-02-16T19:36:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f63fa7fc640 1 -- 172.21.15.40:0/962005614 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f640000f040 con 0x7f64040fb2b0 2024-02-16T19:36:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640410eb60 con 0x7f64040fb2b0 2024-02-16T19:36:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f63fa7fc640 1 -- 172.21.15.40:0/962005614 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64000095b0 con 0x7f64040fb2b0 2024-02-16T19:36:27.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.992+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64041170f0 con 0x7f64040fb2b0 2024-02-16T19:36:27.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.993+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6404064030 con 0x7f64040fb2b0 2024-02-16T19:36:27.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.994+0000 7f63fa7fc640 1 -- 172.21.15.40:0/962005614 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6400009730 con 0x7f64040fb2b0 2024-02-16T19:36:27.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.994+0000 7f63fa7fc640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f63f006d750 0x7f63f006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:27.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.995+0000 7f6409538640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f63f006d750 0x7f63f006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:27.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.995+0000 7f63fa7fc640 1 -- 172.21.15.40:0/962005614 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f640008cd10 con 0x7f64040fb2b0 2024-02-16T19:36:27.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.995+0000 7f6409538640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f63f006d750 0x7f63f006fc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6404064d30 tx=0x7f63f4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:28.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:27.999+0000 7f63fa7fc640 1 -- 172.21.15.40:0/962005614 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64000564b0 con 0x7f64040fb2b0 2024-02-16T19:36:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.222+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f64041083c0 con 0x7f64040fb2b0 2024-02-16T19:36:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.223+0000 7f63fa7fc640 1 -- 172.21.15.40:0/962005614 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v41) v1 ==== 107+0+4848 (secure 0 0 0) 0x7f6400059b70 con 0x7f64040fb2b0 2024-02-16T19:36:28.224 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:28.224 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/4253297290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4253297290},{"type":"v1","addr":"172.21.15.40:6827","nonce":4253297290}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:24:09.512876+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi163.rggdhh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6825/1432724463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1432724463},{"type":"v1","addr":"172.21.15.163:6825","nonce":1432724463}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 19 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.224+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f63f006d750 msgr2=0x7f63f006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.224+0000 7f640bf9d640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f63f006d750 0x7f63f006fc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6404064d30 tx=0x7f63f4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.224+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 msgr2=0x7f640410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.225+0000 7f640bf9d640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 0x7f640410dd20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f640000eb70 tx=0x7f640000b040 comp rx=0 tx=0).stop 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.225+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 shutdown_connections 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.225+0000 7f640bf9d640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f63f006d750 0x7f63f006fc00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.225+0000 7f640bf9d640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64041012d0 0x7f640410e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.225+0000 7f640bf9d640 1 --2- 172.21.15.40:0/962005614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64040fb2b0 0x7f640410dd20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.225+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 >> 172.21.15.40:0/962005614 conn(0x7f6404071230 msgr2=0x7f64040fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.225+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 shutdown_connections 2024-02-16T19:36:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.225+0000 7f640bf9d640 1 -- 172.21.15.40:0/962005614 wait complete. 2024-02-16T19:36:28.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.337+0000 7f74d086e640 1 -- 172.21.15.40:0/3678905522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 msgr2=0x7f74cc0fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.337+0000 7f74d086e640 1 --2- 172.21.15.40:0/3678905522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 0x7f74cc0fb700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f74c00048d0 tx=0x7f74c0009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.337+0000 7f74d086e640 1 -- 172.21.15.40:0/3678905522 shutdown_connections 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.337+0000 7f74d086e640 1 --2- 172.21.15.40:0/3678905522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 0x7f74cc0fb700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.337+0000 7f74d086e640 1 --2- 172.21.15.40:0/3678905522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74cc1012d0 0x7f74cc1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.337+0000 7f74d086e640 1 -- 172.21.15.40:0/3678905522 >> 172.21.15.40:0/3678905522 conn(0x7f74cc071230 msgr2=0x7f74cc071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74d086e640 1 -- 172.21.15.40:0/3678905522 shutdown_connections 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74d086e640 1 -- 172.21.15.40:0/3678905522 wait complete. 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74d086e640 1 Processor -- start 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74d086e640 1 -- start start 2024-02-16T19:36:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74d086e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 0x7f74cc10dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74d086e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74cc1012d0 0x7f74cc10e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74d086e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc10e7a0 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74d086e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc10e8e0 con 0x7f74cc1012d0 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74ca59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 0x7f74cc10dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74ca59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 0x7f74cc10dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38248/0 (socket says 172.21.15.40:38248) 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.338+0000 7f74ca59c640 1 -- 172.21.15.40:0/19501170 learned_addr learned my addr 172.21.15.40:0/19501170 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74c9d9b640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74cc1012d0 0x7f74cc10e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74ca59c640 1 -- 172.21.15.40:0/19501170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74cc1012d0 msgr2=0x7f74cc10e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74ca59c640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74cc1012d0 0x7f74cc10e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74ca59c640 1 -- 172.21.15.40:0/19501170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0005040 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74c9d9b640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74cc1012d0 0x7f74cc10e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74ca59c640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 0x7f74cc10dd20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f74c400eb70 tx=0x7f74c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74baffd640 1 -- 172.21.15.40:0/19501170 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74c4007c30 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74baffd640 1 -- 172.21.15.40:0/19501170 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74c400f040 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc10eb60 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74baffd640 1 -- 172.21.15.40:0/19501170 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74c40095b0 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.339+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc1170f0 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.340+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc064030 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.340+0000 7f74baffd640 1 -- 172.21.15.40:0/19501170 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f74c4009730 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.341+0000 7f74baffd640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f74b406d750 0x7f74b406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.341+0000 7f74baffd640 1 -- 172.21.15.40:0/19501170 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f74c408cd10 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.341+0000 7f74c9d9b640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f74b406d750 0x7f74b406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.341+0000 7f74c9d9b640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f74b406d750 0x7f74b406fc00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f74c002bda0 tx=0x7f74c0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.345+0000 7f74baffd640 1 -- 172.21.15.40:0/19501170 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74c40564b0 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.570+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f74cc117380 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.570+0000 7f74baffd640 1 -- 172.21.15.40:0/19501170 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v41) v1 ==== 107+0+4846 (secure 0 0 0) 0x7f74c4059b70 con 0x7f74cc0fb2b0 2024-02-16T19:36:28.572 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:28.572 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/4253297290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4253297290},{"type":"v1","addr":"172.21.15.40:6827","nonce":4253297290}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:27:43.925056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi163.rggdhh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6825/1432724463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1432724463},{"type":"v1","addr":"172.21.15.163:6825","nonce":1432724463}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 20 2024-02-16T19:36:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.572+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f74b406d750 msgr2=0x7f74b406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.572+0000 7f74d086e640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f74b406d750 0x7f74b406fc00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f74c002bda0 tx=0x7f74c0009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.572+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 msgr2=0x7f74cc10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.572+0000 7f74d086e640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 0x7f74cc10dd20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f74c400eb70 tx=0x7f74c400b040 comp rx=0 tx=0).stop 2024-02-16T19:36:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.573+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 shutdown_connections 2024-02-16T19:36:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.573+0000 7f74d086e640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f74b406d750 0x7f74b406fc00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.573+0000 7f74d086e640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74cc1012d0 0x7f74cc10e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.573+0000 7f74d086e640 1 --2- 172.21.15.40:0/19501170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74cc0fb2b0 0x7f74cc10dd20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.573+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 >> 172.21.15.40:0/19501170 conn(0x7f74cc071230 msgr2=0x7f74cc0fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.573+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 shutdown_connections 2024-02-16T19:36:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.573+0000 7f74d086e640 1 -- 172.21.15.40:0/19501170 wait complete. 2024-02-16T19:36:28.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:28 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3607339548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:36:28.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:36:28.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:28 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:36:28.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:28 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1023062459' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-16T19:36:28.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:28 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/962005614' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-16T19:36:28.585 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-16T19:36:28.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.683+0000 7f80b446e640 1 -- 172.21.15.40:0/707099085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac0fb2b0 msgr2=0x7f80ac0fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.683+0000 7f80b446e640 1 --2- 172.21.15.40:0/707099085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac0fb2b0 0x7f80ac0fb700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f80a00048d0 tx=0x7f80a0009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.683+0000 7f80b446e640 1 -- 172.21.15.40:0/707099085 shutdown_connections 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.683+0000 7f80b446e640 1 --2- 172.21.15.40:0/707099085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac0fb2b0 0x7f80ac0fb700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.683+0000 7f80b446e640 1 --2- 172.21.15.40:0/707099085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80ac1012d0 0x7f80ac1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.683+0000 7f80b446e640 1 -- 172.21.15.40:0/707099085 >> 172.21.15.40:0/707099085 conn(0x7f80ac071230 msgr2=0x7f80ac071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b446e640 1 -- 172.21.15.40:0/707099085 shutdown_connections 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b446e640 1 -- 172.21.15.40:0/707099085 wait complete. 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b446e640 1 Processor -- start 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b446e640 1 -- start start 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b446e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80ac0fb2b0 0x7f80ac1020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b446e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac1012d0 0x7f80ac102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b446e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac102b60 con 0x7f80ac1012d0 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b446e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac102cd0 con 0x7f80ac0fb2b0 2024-02-16T19:36:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b1a09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac1012d0 0x7f80ac102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b1a09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac1012d0 0x7f80ac102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38266/0 (socket says 172.21.15.40:38266) 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.684+0000 7f80b1a09640 1 -- 172.21.15.40:0/2935102650 learned_addr learned my addr 172.21.15.40:0/2935102650 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80b1a09640 1 -- 172.21.15.40:0/2935102650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80ac0fb2b0 msgr2=0x7f80ac1020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80b1a09640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80ac0fb2b0 0x7f80ac1020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80b1a09640 1 -- 172.21.15.40:0/2935102650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a0005040 con 0x7f80ac1012d0 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80b1a09640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac1012d0 0x7f80ac102620 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f80a0031720 tx=0x7f80a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80aaffd640 1 -- 172.21.15.40:0/2935102650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a00094c0 con 0x7f80ac1012d0 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac102f50 con 0x7f80ac1012d0 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80aaffd640 1 -- 172.21.15.40:0/2935102650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80a0009620 con 0x7f80ac1012d0 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80aaffd640 1 -- 172.21.15.40:0/2935102650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a003b640 con 0x7f80ac1012d0 2024-02-16T19:36:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.685+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac119340 con 0x7f80ac1012d0 2024-02-16T19:36:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.686+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac064030 con 0x7f80ac1012d0 2024-02-16T19:36:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.686+0000 7f80aaffd640 1 -- 172.21.15.40:0/2935102650 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f80a002f070 con 0x7f80ac1012d0 2024-02-16T19:36:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.687+0000 7f80aaffd640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f809406d750 0x7f809406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.687+0000 7f80aaffd640 1 -- 172.21.15.40:0/2935102650 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80a00af370 con 0x7f80ac1012d0 2024-02-16T19:36:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.687+0000 7f80b220a640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f809406d750 0x7f809406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.687+0000 7f80b220a640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f809406d750 0x7f809406fc00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80a40097e0 tx=0x7f80a4006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:28.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.691+0000 7f80aaffd640 1 -- 172.21.15.40:0/2935102650 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80a0078b10 con 0x7f80ac1012d0 2024-02-16T19:36:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:28 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3607339548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-16T19:36:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:36:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:28 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:36:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:28 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1023062459' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-16T19:36:28.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:28 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/962005614' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-16T19:36:28.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.912+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f80ac108420 con 0x7f80ac1012d0 2024-02-16T19:36:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.913+0000 7f80aaffd640 1 -- 172.21.15.40:0/2935102650 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v41) v1 ==== 107+0+4846 (secure 0 0 0) 0x7f80a007c1d0 con 0x7f80ac1012d0 2024-02-16T19:36:28.914 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:28.914 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/4253297290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4253297290},{"type":"v1","addr":"172.21.15.40:6827","nonce":4253297290}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:31:26.873426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14486,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi163.rggdhh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6825/1432724463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1432724463},{"type":"v1","addr":"172.21.15.163:6825","nonce":1432724463}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 21 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f809406d750 msgr2=0x7f809406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f809406d750 0x7f809406fc00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80a40097e0 tx=0x7f80a4006d20 comp rx=0 tx=0).stop 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac1012d0 msgr2=0x7f80ac102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac1012d0 0x7f80ac102620 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f80a0031720 tx=0x7f80a0009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 shutdown_connections 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f809406d750 0x7f809406fc00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80ac1012d0 0x7f80ac102620 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 --2- 172.21.15.40:0/2935102650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80ac0fb2b0 0x7f80ac1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 >> 172.21.15.40:0/2935102650 conn(0x7f80ac071230 msgr2=0x7f80ac064500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 shutdown_connections 2024-02-16T19:36:28.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:28.915+0000 7f80b446e640 1 -- 172.21.15.40:0/2935102650 wait complete. 2024-02-16T19:36:28.928 DEBUG:tasks.fs:max_mds reduced in epoch 21 2024-02-16T19:36:28.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-16T19:36:29.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.027+0000 7f3239d1d640 1 -- 172.21.15.40:0/814339443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101c10 msgr2=0x7f32340fb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.027+0000 7f3239d1d640 1 --2- 172.21.15.40:0/814339443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101c10 0x7f32340fb2a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f321c0048d0 tx=0x7f321c009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.027+0000 7f3239d1d640 1 -- 172.21.15.40:0/814339443 shutdown_connections 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.027+0000 7f3239d1d640 1 --2- 172.21.15.40:0/814339443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101c10 0x7f32340fb2a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.027+0000 7f3239d1d640 1 --2- 172.21.15.40:0/814339443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3234101270 0x7f3234101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.027+0000 7f3239d1d640 1 -- 172.21.15.40:0/814339443 >> 172.21.15.40:0/814339443 conn(0x7f32340f6e80 msgr2=0x7f32340f9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.027+0000 7f3239d1d640 1 -- 172.21.15.40:0/814339443 shutdown_connections 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.027+0000 7f3239d1d640 1 -- 172.21.15.40:0/814339443 wait complete. 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f3239d1d640 1 Processor -- start 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f3239d1d640 1 -- start start 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f3239d1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101270 0x7f323410ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f3239d1d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3234101c10 0x7f323410e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f3239d1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323410e830 con 0x7f3234101270 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f3239d1d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323410e970 con 0x7f3234101c10 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f32337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101270 0x7f323410ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f32337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101270 0x7f323410ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38278/0 (socket says 172.21.15.40:38278) 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f32337fe640 1 -- 172.21.15.40:0/818770468 learned_addr learned my addr 172.21.15.40:0/818770468 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f322bfff640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3234101c10 0x7f323410e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f32337fe640 1 -- 172.21.15.40:0/818770468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3234101c10 msgr2=0x7f323410e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f32337fe640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3234101c10 0x7f323410e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f32337fe640 1 -- 172.21.15.40:0/818770468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c005040 con 0x7f3234101270 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.028+0000 7f32337fe640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101270 0x7f323410ddb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3224002a50 tx=0x7f3224004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:29.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.029+0000 7f3230ff9640 1 -- 172.21.15.40:0/818770468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3224007c30 con 0x7f3234101270 2024-02-16T19:36:29.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.029+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3234116e20 con 0x7f3234101270 2024-02-16T19:36:29.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.029+0000 7f3230ff9640 1 -- 172.21.15.40:0/818770468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f322400f040 con 0x7f3234101270 2024-02-16T19:36:29.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.029+0000 7f3230ff9640 1 -- 172.21.15.40:0/818770468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f322400db50 con 0x7f3234101270 2024-02-16T19:36:29.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.029+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3234117200 con 0x7f3234101270 2024-02-16T19:36:29.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.030+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3234064480 con 0x7f3234101270 2024-02-16T19:36:29.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.031+0000 7f3230ff9640 1 -- 172.21.15.40:0/818770468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f322401c400 con 0x7f3234101270 2024-02-16T19:36:29.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.031+0000 7f3230ff9640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f322006d750 0x7f322006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.031+0000 7f3230ff9640 1 -- 172.21.15.40:0/818770468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f322408c860 con 0x7f3234101270 2024-02-16T19:36:29.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.031+0000 7f322bfff640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f322006d750 0x7f322006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.032+0000 7f322bfff640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f322006d750 0x7f322006fc00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3234065f80 tx=0x7f321c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:29.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.036+0000 7f3230ff9640 1 -- 172.21.15.40:0/818770468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3224056000 con 0x7f3234101270 2024-02-16T19:36:29.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.256+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f3234117420 con 0x7f3234101270 2024-02-16T19:36:29.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.256+0000 7f3230ff9640 1 -- 172.21.15.40:0/818770468 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v41) v1 ==== 107+0+4848 (secure 0 0 0) 0x7f32240596c0 con 0x7f3234101270 2024-02-16T19:36:29.258 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:29.258 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/4253297290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4253297290},{"type":"v1","addr":"172.21.15.40:6827","nonce":4253297290}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:31:27.873379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14486,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi163.rggdhh","rank":1,"incarnation":7,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6825/1432724463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1432724463},{"type":"v1","addr":"172.21.15.163:6825","nonce":1432724463}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:29.258 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 22 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f322006d750 msgr2=0x7f322006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f322006d750 0x7f322006fc00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3234065f80 tx=0x7f321c009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101270 msgr2=0x7f323410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101270 0x7f323410ddb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3224002a50 tx=0x7f3224004600 comp rx=0 tx=0).stop 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 shutdown_connections 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f322006d750 0x7f322006fc00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3234101c10 0x7f323410e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 --2- 172.21.15.40:0/818770468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3234101270 0x7f323410ddb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 >> 172.21.15.40:0/818770468 conn(0x7f32340f6e80 msgr2=0x7f32340f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:29.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 shutdown_connections 2024-02-16T19:36:29.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.258+0000 7f3239d1d640 1 -- 172.21.15.40:0/818770468 wait complete. 2024-02-16T19:36:29.271 DEBUG:tasks.fs:max_mds reduced in epoch 22 2024-02-16T19:36:29.271 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-16T19:36:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 -- 172.21.15.40:0/1104594321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e80fb2b0 msgr2=0x7f39e80fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 --2- 172.21.15.40:0/1104594321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e80fb2b0 0x7f39e80fb700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f39d00048d0 tx=0x7f39d0009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 -- 172.21.15.40:0/1104594321 shutdown_connections 2024-02-16T19:36:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 --2- 172.21.15.40:0/1104594321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e80fb2b0 0x7f39e80fb700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 --2- 172.21.15.40:0/1104594321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39e81012d0 0x7f39e81016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 -- 172.21.15.40:0/1104594321 >> 172.21.15.40:0/1104594321 conn(0x7f39e8071230 msgr2=0x7f39e8071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 -- 172.21.15.40:0/1104594321 shutdown_connections 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 -- 172.21.15.40:0/1104594321 wait complete. 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 Processor -- start 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.369+0000 7f39ecd5d640 1 -- start start 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39ecd5d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39e80fb2b0 0x7f39e8111ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39ecd5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e81012d0 0x7f39e8111ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39ecd5d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e81125f0 con 0x7f39e81012d0 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39ecd5d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e8112760 con 0x7f39e80fb2b0 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e81012d0 0x7f39e8111ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e81012d0 0x7f39e8111ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38290/0 (socket says 172.21.15.40:38290) 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e5d9b640 1 -- 172.21.15.40:0/3854317364 learned_addr learned my addr 172.21.15.40:0/3854317364 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e659c640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39e80fb2b0 0x7f39e8111ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e5d9b640 1 -- 172.21.15.40:0/3854317364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39e80fb2b0 msgr2=0x7f39e8111ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e5d9b640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39e80fb2b0 0x7f39e8111ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e5d9b640 1 -- 172.21.15.40:0/3854317364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d0005040 con 0x7f39e81012d0 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e659c640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39e80fb2b0 0x7f39e8111ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39e5d9b640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e81012d0 0x7f39e8111ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39d0031720 tx=0x7f39d0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39deffd640 1 -- 172.21.15.40:0/3854317364 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39d00094c0 con 0x7f39e81012d0 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39deffd640 1 -- 172.21.15.40:0/3854317364 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39d0009620 con 0x7f39e81012d0 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e810bcd0 con 0x7f39e81012d0 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39deffd640 1 -- 172.21.15.40:0/3854317364 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39d003b640 con 0x7f39e81012d0 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.370+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e810c1f0 con 0x7f39e81012d0 2024-02-16T19:36:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.371+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e8064030 con 0x7f39e81012d0 2024-02-16T19:36:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.372+0000 7f39deffd640 1 -- 172.21.15.40:0/3854317364 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f39d002f070 con 0x7f39e81012d0 2024-02-16T19:36:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.372+0000 7f39deffd640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f39d406d750 0x7f39d406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.372+0000 7f39e659c640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f39d406d750 0x7f39d406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.372+0000 7f39deffd640 1 -- 172.21.15.40:0/3854317364 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f39d00af4f0 con 0x7f39e81012d0 2024-02-16T19:36:29.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.373+0000 7f39e659c640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f39d406d750 0x7f39d406fc00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39e00041d0 tx=0x7f39e0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:29.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.377+0000 7f39deffd640 1 -- 172.21.15.40:0/3854317364 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f39d0078d10 con 0x7f39e81012d0 2024-02-16T19:36:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.597+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f39e810c3d0 con 0x7f39e81012d0 2024-02-16T19:36:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.597+0000 7f39deffd640 1 -- 172.21.15.40:0/3854317364 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v41) v1 ==== 107+0+4050 (secure 0 0 0) 0x7f39d007c3d0 con 0x7f39e81012d0 2024-02-16T19:36:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:29.599 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/4253297290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4253297290},{"type":"v1","addr":"172.21.15.40:6827","nonce":4253297290}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:31:48.931474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14486},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:29.599 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 23 2024-02-16T19:36:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f39d406d750 msgr2=0x7f39d406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f39d406d750 0x7f39d406fc00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39e00041d0 tx=0x7f39e0009040 comp rx=0 tx=0).stop 2024-02-16T19:36:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e81012d0 msgr2=0x7f39e8111ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e81012d0 0x7f39e8111ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39d0031720 tx=0x7f39d0009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 shutdown_connections 2024-02-16T19:36:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f39d406d750 0x7f39d406fc00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39e81012d0 0x7f39e8111ff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 --2- 172.21.15.40:0/3854317364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39e80fb2b0 0x7f39e8111ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 >> 172.21.15.40:0/3854317364 conn(0x7f39e8071230 msgr2=0x7f39e80fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 shutdown_connections 2024-02-16T19:36:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.599+0000 7f39ecd5d640 1 -- 172.21.15.40:0/3854317364 wait complete. 2024-02-16T19:36:29.610 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:29 smithi040 ceph-mon[94210]: pgmap v320: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:29.610 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:29.610 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:36:29.610 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:29 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:36:29.610 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:29 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/19501170' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-16T19:36:29.610 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:29 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2935102650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-16T19:36:29.610 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:29 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/818770468' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-16T19:36:29.611 DEBUG:tasks.fs:max_mds reduced in epoch 23 2024-02-16T19:36:29.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 -- 172.21.15.40:0/2616655352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00faa70 msgr2=0x7f54d00faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 --2- 172.21.15.40:0/2616655352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00faa70 0x7f54d00faee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0048d0 tx=0x7f54bc009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 -- 172.21.15.40:0/2616655352 shutdown_connections 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 --2- 172.21.15.40:0/2616655352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00faa70 0x7f54d00faee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 --2- 172.21.15.40:0/2616655352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54d00fe2f0 0x7f54d00fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 -- 172.21.15.40:0/2616655352 >> 172.21.15.40:0/2616655352 conn(0x7f54d0071950 msgr2=0x7f54d0071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 -- 172.21.15.40:0/2616655352 shutdown_connections 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 -- 172.21.15.40:0/2616655352 wait complete. 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 Processor -- start 2024-02-16T19:36:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.710+0000 7f54d6aae640 1 -- start start 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54d6aae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54d00faa70 0x7f54d0112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54d6aae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00fe2f0 0x7f54d01131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54d6aae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0113860 con 0x7f54d00fe2f0 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54d6aae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d010cd60 con 0x7f54d00faa70 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00fe2f0 0x7f54d01131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00fe2f0 0x7f54d01131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38320/0 (socket says 172.21.15.40:38320) 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cffff640 1 -- 172.21.15.40:0/2168227360 learned_addr learned my addr 172.21.15.40:0/2168227360 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54d484a640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54d00faa70 0x7f54d0112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cffff640 1 -- 172.21.15.40:0/2168227360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54d00faa70 msgr2=0x7f54d0112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cffff640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54d00faa70 0x7f54d0112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cffff640 1 -- 172.21.15.40:0/2168227360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc005040 con 0x7f54d00fe2f0 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54d484a640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54d00faa70 0x7f54d0112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cffff640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00fe2f0 0x7f54d01131d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f54bc031720 tx=0x7f54bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cd7fa640 1 -- 172.21.15.40:0/2168227360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54bc0094c0 con 0x7f54d00fe2f0 2024-02-16T19:36:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d010cfe0 con 0x7f54d00fe2f0 2024-02-16T19:36:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cd7fa640 1 -- 172.21.15.40:0/2168227360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54bc009620 con 0x7f54d00fe2f0 2024-02-16T19:36:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54cd7fa640 1 -- 172.21.15.40:0/2168227360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54bc03b640 con 0x7f54d00fe2f0 2024-02-16T19:36:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.711+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d010d500 con 0x7f54d00fe2f0 2024-02-16T19:36:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.713+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d0064480 con 0x7f54d00fe2f0 2024-02-16T19:36:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.713+0000 7f54cd7fa640 1 -- 172.21.15.40:0/2168227360 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f54bc02f070 con 0x7f54d00fe2f0 2024-02-16T19:36:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.713+0000 7f54cd7fa640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f54c006d7a0 0x7f54c006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.713+0000 7f54cd7fa640 1 -- 172.21.15.40:0/2168227360 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f54bc0af5c0 con 0x7f54d00fe2f0 2024-02-16T19:36:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.713+0000 7f54d484a640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f54c006d7a0 0x7f54c006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.714+0000 7f54d484a640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f54c006d7a0 0x7f54c006fc50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006fd0 tx=0x7f54c4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:29.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.718+0000 7f54cd7fa640 1 -- 172.21.15.40:0/2168227360 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f54bc078cc0 con 0x7f54d00fe2f0 2024-02-16T19:36:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:29 smithi163 ceph-mon[81637]: pgmap v320: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:29.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-16T19:36:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-16T19:36:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:29 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' 2024-02-16T19:36:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:29 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/19501170' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-16T19:36:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:29 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2935102650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-16T19:36:29.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:29 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/818770468' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-16T19:36:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.943+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f54d010d690 con 0x7f54d00fe2f0 2024-02-16T19:36:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.944+0000 7f54cd7fa640 1 -- 172.21.15.40:0/2168227360 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v41) v1 ==== 107+0+4832 (secure 0 0 0) 0x7f54bc07c380 con 0x7f54d00fe2f0 2024-02-16T19:36:29.945 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:29.945 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/4253297290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":4253297290},{"type":"v1","addr":"172.21.15.40:6827","nonce":4253297290}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":44285,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/2272386057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2272386057},{"type":"v1","addr":"172.21.15.163:6825","nonce":2272386057}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:31:48.931474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14486},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:29.946 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2024-02-16T19:36:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f54c006d7a0 msgr2=0x7f54c006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f54c006d7a0 0x7f54c006fc50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006fd0 tx=0x7f54c4008040 comp rx=0 tx=0).stop 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00fe2f0 msgr2=0x7f54d01131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00fe2f0 0x7f54d01131d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f54bc031720 tx=0x7f54bc009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 shutdown_connections 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f54c006d7a0 0x7f54c006fc50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d00fe2f0 0x7f54d01131d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 --2- 172.21.15.40:0/2168227360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54d00faa70 0x7f54d0112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 >> 172.21.15.40:0/2168227360 conn(0x7f54d0071950 msgr2=0x7f54d00f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 shutdown_connections 2024-02-16T19:36:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:29.946+0000 7f54d6aae640 1 -- 172.21.15.40:0/2168227360 wait complete. 2024-02-16T19:36:29.958 DEBUG:tasks.fs:max_mds reduced in epoch 24 2024-02-16T19:36:29.958 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-16T19:36:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.056+0000 7ff2da59d640 1 -- 172.21.15.40:0/361306638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 msgr2=0x7ff2d4101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.056+0000 7ff2da59d640 1 --2- 172.21.15.40:0/361306638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 0x7ff2d4101620 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80048d0 tx=0x7ff2c8009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.056+0000 7ff2da59d640 1 -- 172.21.15.40:0/361306638 shutdown_connections 2024-02-16T19:36:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.056+0000 7ff2da59d640 1 --2- 172.21.15.40:0/361306638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4101bf0 0x7ff2d40fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.056+0000 7ff2da59d640 1 --2- 172.21.15.40:0/361306638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 0x7ff2d4101620 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.056+0000 7ff2da59d640 1 -- 172.21.15.40:0/361306638 >> 172.21.15.40:0/361306638 conn(0x7ff2d40f6e00 msgr2=0x7ff2d40f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.056+0000 7ff2da59d640 1 -- 172.21.15.40:0/361306638 shutdown_connections 2024-02-16T19:36:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.056+0000 7ff2da59d640 1 -- 172.21.15.40:0/361306638 wait complete. 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2da59d640 1 Processor -- start 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2da59d640 1 -- start start 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2da59d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 0x7ff2d4111b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2da59d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4101bf0 0x7ff2d4112080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2da59d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4112710 con 0x7ff2d4101250 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2da59d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d410bc10 con 0x7ff2d4101bf0 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 0x7ff2d4111b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 0x7ff2d4111b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38334/0 (socket says 172.21.15.40:38334) 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2d3fff640 1 -- 172.21.15.40:0/83149985 learned_addr learned my addr 172.21.15.40:0/83149985 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2d3fff640 1 -- 172.21.15.40:0/83149985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4101bf0 msgr2=0x7ff2d4112080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2d3fff640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4101bf0 0x7ff2d4112080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2d3fff640 1 -- 172.21.15.40:0/83149985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8005040 con 0x7ff2d4101250 2024-02-16T19:36:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.057+0000 7ff2d3fff640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 0x7ff2d4111b40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8031720 tx=0x7ff2c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.058+0000 7ff2d0ff9640 1 -- 172.21.15.40:0/83149985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2c80094c0 con 0x7ff2d4101250 2024-02-16T19:36:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.058+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d410be90 con 0x7ff2d4101250 2024-02-16T19:36:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.058+0000 7ff2d0ff9640 1 -- 172.21.15.40:0/83149985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2c8009620 con 0x7ff2d4101250 2024-02-16T19:36:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.058+0000 7ff2d0ff9640 1 -- 172.21.15.40:0/83149985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2c803b640 con 0x7ff2d4101250 2024-02-16T19:36:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.058+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d410c3b0 con 0x7ff2d4101250 2024-02-16T19:36:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.059+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d4064480 con 0x7ff2d4101250 2024-02-16T19:36:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.059+0000 7ff2d0ff9640 1 -- 172.21.15.40:0/83149985 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff2c802f070 con 0x7ff2d4101250 2024-02-16T19:36:30.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.059+0000 7ff2d0ff9640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff2bc06d750 0x7ff2bc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.060+0000 7ff2d0ff9640 1 -- 172.21.15.40:0/83149985 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff2c80af520 con 0x7ff2d4101250 2024-02-16T19:36:30.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.060+0000 7ff2d37fe640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff2bc06d750 0x7ff2bc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:30.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.060+0000 7ff2d37fe640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff2bc06d750 0x7ff2bc06fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2d410d400 tx=0x7ff2c4006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:30.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.064+0000 7ff2d0ff9640 1 -- 172.21.15.40:0/83149985 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2c8078cc0 con 0x7ff2d4101250 2024-02-16T19:36:30.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.283+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7ff2d4108410 con 0x7ff2d4101250 2024-02-16T19:36:30.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.284+0000 7ff2d0ff9640 1 -- 172.21.15.40:0/83149985 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v41) v1 ==== 107+0+4053 (secure 0 0 0) 0x7ff2c807c380 con 0x7ff2d4101250 2024-02-16T19:36:30.285 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:30.285 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":44285,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/2272386057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2272386057},{"type":"v1","addr":"172.21.15.163:6825","nonce":2272386057}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:31:48.931474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14486},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:30.285 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff2bc06d750 msgr2=0x7ff2bc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff2bc06d750 0x7ff2bc06fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2d410d400 tx=0x7ff2c4006d20 comp rx=0 tx=0).stop 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 msgr2=0x7ff2d4111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 0x7ff2d4111b40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8031720 tx=0x7ff2c8009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 shutdown_connections 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff2bc06d750 0x7ff2bc06fc00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4101bf0 0x7ff2d4112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 --2- 172.21.15.40:0/83149985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d4101250 0x7ff2d4111b40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 >> 172.21.15.40:0/83149985 conn(0x7ff2d40f6e00 msgr2=0x7ff2d40f88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 shutdown_connections 2024-02-16T19:36:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.286+0000 7ff2da59d640 1 -- 172.21.15.40:0/83149985 wait complete. 2024-02-16T19:36:30.298 DEBUG:tasks.fs:max_mds reduced in epoch 25 2024-02-16T19:36:30.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.396+0000 7fe41fa65640 1 -- 172.21.15.40:0/230678151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 msgr2=0x7fe4180fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.396+0000 7fe41fa65640 1 --2- 172.21.15.40:0/230678151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 0x7fe4180fb270 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe4080048d0 tx=0x7fe408009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.396+0000 7fe41fa65640 1 -- 172.21.15.40:0/230678151 shutdown_connections 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.396+0000 7fe41fa65640 1 --2- 172.21.15.40:0/230678151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 0x7fe4180fb270 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.396+0000 7fe41fa65640 1 --2- 172.21.15.40:0/230678151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418101240 0x7fe418101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.396+0000 7fe41fa65640 1 -- 172.21.15.40:0/230678151 >> 172.21.15.40:0/230678151 conn(0x7fe4180f6e30 msgr2=0x7fe4180f9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.396+0000 7fe41fa65640 1 -- 172.21.15.40:0/230678151 shutdown_connections 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.396+0000 7fe41fa65640 1 -- 172.21.15.40:0/230678151 wait complete. 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41fa65640 1 Processor -- start 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41fa65640 1 -- start start 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41fa65640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418101240 0x7fe41810dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41fa65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 0x7fe41810e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41fa65640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41810e700 con 0x7fe418101be0 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41fa65640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41810e870 con 0x7fe418101240 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41d000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 0x7fe41810e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41d000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 0x7fe41810e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38354/0 (socket says 172.21.15.40:38354) 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41d000640 1 -- 172.21.15.40:0/2310859040 learned_addr learned my addr 172.21.15.40:0/2310859040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41d801640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418101240 0x7fe41810dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41d000640 1 -- 172.21.15.40:0/2310859040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418101240 msgr2=0x7fe41810dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41d000640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418101240 0x7fe41810dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.397+0000 7fe41d000640 1 -- 172.21.15.40:0/2310859040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408005040 con 0x7fe418101be0 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.398+0000 7fe41d000640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 0x7fe41810e1c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe408031720 tx=0x7fe408009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.398+0000 7fe40dffb640 1 -- 172.21.15.40:0/2310859040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4080094c0 con 0x7fe418101be0 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.398+0000 7fe40dffb640 1 -- 172.21.15.40:0/2310859040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe408009620 con 0x7fe418101be0 2024-02-16T19:36:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.398+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41810eaa0 con 0x7fe418101be0 2024-02-16T19:36:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.398+0000 7fe40dffb640 1 -- 172.21.15.40:0/2310859040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe40803b640 con 0x7fe418101be0 2024-02-16T19:36:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.398+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe418117020 con 0x7fe418101be0 2024-02-16T19:36:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.399+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe418064480 con 0x7fe418101be0 2024-02-16T19:36:30.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.399+0000 7fe40dffb640 1 -- 172.21.15.40:0/2310859040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe40802f070 con 0x7fe418101be0 2024-02-16T19:36:30.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.400+0000 7fe40dffb640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe40406d750 0x7fe40406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.400+0000 7fe41d801640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe40406d750 0x7fe40406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:30.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.400+0000 7fe40dffb640 1 -- 172.21.15.40:0/2310859040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe4080af4f0 con 0x7fe418101be0 2024-02-16T19:36:30.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.400+0000 7fe41d801640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe40406d750 0x7fe40406fc00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4140041d0 tx=0x7fe414009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:30.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.405+0000 7fe40dffb640 1 -- 172.21.15.40:0/2310859040 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe408078d10 con 0x7fe418101be0 2024-02-16T19:36:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.623+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fe418117320 con 0x7fe418101be0 2024-02-16T19:36:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.623+0000 7fe40dffb640 1 -- 172.21.15.40:0/2310859040 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v41) v1 ==== 107+0+4863 (secure 0 0 0) 0x7fe40807c3d0 con 0x7fe418101be0 2024-02-16T19:36:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:30.625 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44285,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/2272386057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2272386057},{"type":"v1","addr":"172.21.15.163:6825","nonce":2272386057}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:31:48.931474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14486},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi040.bmpjpk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.40:6829/2154720438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2154720438},{"type":"v1","addr":"172.21.15.40:6829","nonce":2154720438}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 26 2024-02-16T19:36:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe40406d750 msgr2=0x7fe40406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe40406d750 0x7fe40406fc00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4140041d0 tx=0x7fe414009040 comp rx=0 tx=0).stop 2024-02-16T19:36:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 msgr2=0x7fe41810e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 0x7fe41810e1c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe408031720 tx=0x7fe408009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 shutdown_connections 2024-02-16T19:36:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fe40406d750 0x7fe40406fc00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe418101be0 0x7fe41810e1c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 --2- 172.21.15.40:0/2310859040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe418101240 0x7fe41810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 >> 172.21.15.40:0/2310859040 conn(0x7fe4180f6e30 msgr2=0x7fe4180f88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.625+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 shutdown_connections 2024-02-16T19:36:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.626+0000 7fe41fa65640 1 -- 172.21.15.40:0/2310859040 wait complete. 2024-02-16T19:36:30.636 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:30 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3854317364' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-16T19:36:30.637 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:30 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2168227360' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-16T19:36:30.637 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:30 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/83149985' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-16T19:36:30.638 DEBUG:tasks.fs:max_mds reduced in epoch 26 2024-02-16T19:36:30.639 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-16T19:36:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:30 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3854317364' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-16T19:36:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:30 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2168227360' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-16T19:36:30.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:30 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/83149985' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-16T19:36:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.737+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3160730979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 msgr2=0x7f9c8c1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.737+0000 7f9c93d1d640 1 --2- 172.21.15.40:0/3160730979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 0x7f9c8c1027b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9c880048d0 tx=0x7f9c88009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.737+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3160730979 shutdown_connections 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.737+0000 7f9c93d1d640 1 --2- 172.21.15.40:0/3160730979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8c0644b0 0x7f9c8c0648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.737+0000 7f9c93d1d640 1 --2- 172.21.15.40:0/3160730979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 0x7f9c8c1027b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.737+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3160730979 >> 172.21.15.40:0/3160730979 conn(0x7f9c8c071230 msgr2=0x7f9c8c071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.737+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3160730979 shutdown_connections 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.737+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3160730979 wait complete. 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c93d1d640 1 Processor -- start 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c93d1d640 1 -- start start 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c93d1d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8c0644b0 0x7f9c8c112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c93d1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 0x7f9c8c113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c93d1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c1137b0 con 0x7f9c8c1023e0 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c93d1d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c10ccb0 con 0x7f9c8c0644b0 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c912b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 0x7f9c8c113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c912b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 0x7f9c8c113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38374/0 (socket says 172.21.15.40:38374) 2024-02-16T19:36:30.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c912b8640 1 -- 172.21.15.40:0/3729527371 learned_addr learned my addr 172.21.15.40:0/3729527371 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c91ab9640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8c0644b0 0x7f9c8c112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c912b8640 1 -- 172.21.15.40:0/3729527371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8c0644b0 msgr2=0x7f9c8c112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c912b8640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8c0644b0 0x7f9c8c112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.738+0000 7f9c912b8640 1 -- 172.21.15.40:0/3729527371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88005040 con 0x7f9c8c1023e0 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.739+0000 7f9c912b8640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 0x7f9c8c113120 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00b790 tx=0x7f9c7c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.739+0000 7f9c827fc640 1 -- 172.21.15.40:0/3729527371 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7c00c700 con 0x7f9c8c1023e0 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.739+0000 7f9c827fc640 1 -- 172.21.15.40:0/3729527371 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c7c00cd30 con 0x7f9c8c1023e0 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.739+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c10cf90 con 0x7f9c8c1023e0 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.739+0000 7f9c827fc640 1 -- 172.21.15.40:0/3729527371 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7c013370 con 0x7f9c8c1023e0 2024-02-16T19:36:30.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.739+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c10d4b0 con 0x7f9c8c1023e0 2024-02-16T19:36:30.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.740+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c064030 con 0x7f9c8c1023e0 2024-02-16T19:36:30.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.740+0000 7f9c827fc640 1 -- 172.21.15.40:0/3729527371 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9c7c012070 con 0x7f9c8c1023e0 2024-02-16T19:36:30.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.741+0000 7f9c827fc640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c7806d760 0x7f9c7806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:30.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.741+0000 7f9c827fc640 1 -- 172.21.15.40:0/3729527371 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9c7c08c880 con 0x7f9c8c1023e0 2024-02-16T19:36:30.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.741+0000 7f9c91ab9640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c7806d760 0x7f9c7806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:30.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.741+0000 7f9c91ab9640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c7806d760 0x7f9c7806fc10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9c88034040 tx=0x7f9c88009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:30.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.745+0000 7f9c827fc640 1 -- 172.21.15.40:0/3729527371 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c7c0560a0 con 0x7f9c8c1023e0 2024-02-16T19:36:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.966+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f9c8c109510 con 0x7f9c8c1023e0 2024-02-16T19:36:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.967+0000 7f9c827fc640 1 -- 172.21.15.40:0/3729527371 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v41) v1 ==== 107+0+4073 (secure 0 0 0) 0x7f9c7c059760 con 0x7f9c8c1023e0 2024-02-16T19:36:30.969 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:30.969 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44285,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/2272386057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2272386057},{"type":"v1","addr":"172.21.15.163:6825","nonce":2272386057}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:20.152009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2024-02-16T19:36:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c7806d760 msgr2=0x7f9c7806fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c7806d760 0x7f9c7806fc10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9c88034040 tx=0x7f9c88009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 msgr2=0x7f9c8c113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 0x7f9c8c113120 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00b790 tx=0x7f9c7c0091d0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 shutdown_connections 2024-02-16T19:36:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f9c7806d760 0x7f9c7806fc10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c8c1023e0 0x7f9c8c113120 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 --2- 172.21.15.40:0/3729527371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8c0644b0 0x7f9c8c112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 >> 172.21.15.40:0/3729527371 conn(0x7f9c8c071230 msgr2=0x7f9c8c0fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 shutdown_connections 2024-02-16T19:36:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:30.969+0000 7f9c93d1d640 1 -- 172.21.15.40:0/3729527371 wait complete. 2024-02-16T19:36:30.981 DEBUG:tasks.fs:max_mds reduced in epoch 27 2024-02-16T19:36:30.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-02-16T19:36:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 -- 172.21.15.40:0/1448556648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 msgr2=0x7f03d4106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 --2- 172.21.15.40:0/1448556648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 0x7f03d4106390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f03c40048d0 tx=0x7f03c4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 -- 172.21.15.40:0/1448556648 shutdown_connections 2024-02-16T19:36:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 --2- 172.21.15.40:0/1448556648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 0x7f03d4106390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 --2- 172.21.15.40:0/1448556648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03d40fd3b0 0x7f03d40fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 -- 172.21.15.40:0/1448556648 >> 172.21.15.40:0/1448556648 conn(0x7f03d40f6e00 msgr2=0x7f03d40f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 -- 172.21.15.40:0/1448556648 shutdown_connections 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 -- 172.21.15.40:0/1448556648 wait complete. 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 Processor -- start 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.080+0000 7f03db99d640 1 -- start start 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03db99d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03d40fd3b0 0x7f03d4112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03db99d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 0x7f03d41131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03db99d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41137b0 con 0x7f03d40fdd50 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03db99d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d410cd40 con 0x7f03d40fd3b0 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d8f38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 0x7f03d41131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d8f38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 0x7f03d41131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38386/0 (socket says 172.21.15.40:38386) 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d8f38640 1 -- 172.21.15.40:0/844795621 learned_addr learned my addr 172.21.15.40:0/844795621 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d9739640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03d40fd3b0 0x7f03d4112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d8f38640 1 -- 172.21.15.40:0/844795621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03d40fd3b0 msgr2=0x7f03d4112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d8f38640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03d40fd3b0 0x7f03d4112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d8f38640 1 -- 172.21.15.40:0/844795621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c4005040 con 0x7f03d40fdd50 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d9739640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03d40fd3b0 0x7f03d4112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03d8f38640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 0x7f03d41131b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f03c4031720 tx=0x7f03c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03c9ffb640 1 -- 172.21.15.40:0/844795621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c40094c0 con 0x7f03d40fdd50 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03c9ffb640 1 -- 172.21.15.40:0/844795621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03c4009620 con 0x7f03d40fdd50 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d410cfc0 con 0x7f03d40fdd50 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.081+0000 7f03c9ffb640 1 -- 172.21.15.40:0/844795621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c403b640 con 0x7f03d40fdd50 2024-02-16T19:36:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.082+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d410d4e0 con 0x7f03d40fdd50 2024-02-16T19:36:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.083+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d4064480 con 0x7f03d40fdd50 2024-02-16T19:36:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.083+0000 7f03c9ffb640 1 -- 172.21.15.40:0/844795621 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f03c402f070 con 0x7f03d40fdd50 2024-02-16T19:36:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.083+0000 7f03c9ffb640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f03c006d750 0x7f03c006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.084+0000 7f03c9ffb640 1 -- 172.21.15.40:0/844795621 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f03c40af470 con 0x7f03d40fdd50 2024-02-16T19:36:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.084+0000 7f03d9739640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f03c006d750 0x7f03c006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.084+0000 7f03d9739640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f03c006d750 0x7f03c006fc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03d0007fd0 tx=0x7f03d0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:31.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.088+0000 7f03c9ffb640 1 -- 172.21.15.40:0/844795621 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f03c4078c10 con 0x7f03d40fdd50 2024-02-16T19:36:31.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.307+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f03d41039c0 con 0x7f03d40fdd50 2024-02-16T19:36:31.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.308+0000 7f03c9ffb640 1 -- 172.21.15.40:0/844795621 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v41) v1 ==== 107+0+4084 (secure 0 0 0) 0x7f03c407c2d0 con 0x7f03d40fdd50 2024-02-16T19:36:31.309 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:31.309 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44285,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/2272386057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2272386057},{"type":"v1","addr":"172.21.15.163:6825","nonce":2272386057}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:20.159145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":0,"incarnation":28,"state":"up:replay","state_seq":2,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:31.309 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 28 2024-02-16T19:36:31.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.309+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f03c006d750 msgr2=0x7f03c006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.309+0000 7f03db99d640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f03c006d750 0x7f03c006fc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03d0007fd0 tx=0x7f03d0009e90 comp rx=0 tx=0).stop 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.309+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 msgr2=0x7f03d41131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.309+0000 7f03db99d640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 0x7f03d41131b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f03c4031720 tx=0x7f03c4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.310+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 shutdown_connections 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.310+0000 7f03db99d640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f03c006d750 0x7f03c006fc00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.310+0000 7f03db99d640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03d40fdd50 0x7f03d41131b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.310+0000 7f03db99d640 1 --2- 172.21.15.40:0/844795621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03d40fd3b0 0x7f03d4112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.310+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 >> 172.21.15.40:0/844795621 conn(0x7f03d40f6e00 msgr2=0x7f03d4100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.310+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 shutdown_connections 2024-02-16T19:36:31.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.310+0000 7f03db99d640 1 -- 172.21.15.40:0/844795621 wait complete. 2024-02-16T19:36:31.322 DEBUG:tasks.fs:max_mds reduced in epoch 28 2024-02-16T19:36:31.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-02-16T19:36:31.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 -- 172.21.15.40:0/3720533952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102d10 msgr2=0x7f16580fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 --2- 172.21.15.40:0/3720533952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102d10 0x7f16580fc3a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f164c0048d0 tx=0x7f164c009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 -- 172.21.15.40:0/3720533952 shutdown_connections 2024-02-16T19:36:31.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 --2- 172.21.15.40:0/3720533952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102d10 0x7f16580fc3a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 --2- 172.21.15.40:0/3720533952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1658102370 0x7f1658102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 -- 172.21.15.40:0/3720533952 >> 172.21.15.40:0/3720533952 conn(0x7f16580f7f20 msgr2=0x7f16580fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 -- 172.21.15.40:0/3720533952 shutdown_connections 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 -- 172.21.15.40:0/3720533952 wait complete. 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 Processor -- start 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 -- start start 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102370 0x7f1658112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1658102d10 0x7f1658113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1658113720 con 0x7f1658102370 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.421+0000 7f165f6ae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165810ccb0 con 0x7f1658102d10 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165d44a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102370 0x7f1658112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165d44a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102370 0x7f1658112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38396/0 (socket says 172.21.15.40:38396) 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165d44a640 1 -- 172.21.15.40:0/2634665546 learned_addr learned my addr 172.21.15.40:0/2634665546 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:31.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165cc49640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1658102d10 0x7f1658113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165d44a640 1 -- 172.21.15.40:0/2634665546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1658102d10 msgr2=0x7f1658113120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165d44a640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1658102d10 0x7f1658113120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165d44a640 1 -- 172.21.15.40:0/2634665546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c005040 con 0x7f1658102370 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165cc49640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1658102d10 0x7f1658113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165d44a640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102370 0x7f1658112be0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1650002a50 tx=0x7f1650004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f1655ffb640 1 -- 172.21.15.40:0/2634665546 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1650007c30 con 0x7f1658102370 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165810cf90 con 0x7f1658102370 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f1655ffb640 1 -- 172.21.15.40:0/2634665546 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f165000f040 con 0x7f1658102370 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f1655ffb640 1 -- 172.21.15.40:0/2634665546 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165000dbe0 con 0x7f1658102370 2024-02-16T19:36:31.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.422+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165810d480 con 0x7f1658102370 2024-02-16T19:36:31.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.424+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1658064480 con 0x7f1658102370 2024-02-16T19:36:31.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.424+0000 7f1655ffb640 1 -- 172.21.15.40:0/2634665546 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f165000dd40 con 0x7f1658102370 2024-02-16T19:36:31.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.424+0000 7f1655ffb640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f164006d750 0x7f164006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.424+0000 7f1655ffb640 1 -- 172.21.15.40:0/2634665546 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f165008cf20 con 0x7f1658102370 2024-02-16T19:36:31.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.424+0000 7f165cc49640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f164006d750 0x7f164006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.425+0000 7f165cc49640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f164006d750 0x7f164006fc00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f165810ddf0 tx=0x7f164c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:31.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.429+0000 7f1655ffb640 1 -- 172.21.15.40:0/2634665546 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16500c0d20 con 0x7f1658102370 2024-02-16T19:36:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:31 smithi040 ceph-mon[94210]: pgmap v321: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:31 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2310859040' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-16T19:36:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:31 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3729527371' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-16T19:36:31.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:31 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/844795621' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-16T19:36:31.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.650+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f16581094b0 con 0x7f1658102370 2024-02-16T19:36:31.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.651+0000 7f1655ffb640 1 -- 172.21.15.40:0/2634665546 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v41) v1 ==== 107+0+4089 (secure 0 0 0) 0x7f16500566c0 con 0x7f1658102370 2024-02-16T19:36:31.652 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:31.652 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44285,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/2272386057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2272386057},{"type":"v1","addr":"172.21.15.163:6825","nonce":2272386057}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:26.789396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":155,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:31.652 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 29 2024-02-16T19:36:31.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.652+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f164006d750 msgr2=0x7f164006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.652+0000 7f165f6ae640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f164006d750 0x7f164006fc00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f165810ddf0 tx=0x7f164c009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.652+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102370 msgr2=0x7f1658112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.652+0000 7f165f6ae640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102370 0x7f1658112be0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1650002a50 tx=0x7f1650004600 comp rx=0 tx=0).stop 2024-02-16T19:36:31.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.653+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 shutdown_connections 2024-02-16T19:36:31.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.653+0000 7f165f6ae640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f164006d750 0x7f164006fc00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.653+0000 7f165f6ae640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1658102d10 0x7f1658113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.653+0000 7f165f6ae640 1 --2- 172.21.15.40:0/2634665546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1658102370 0x7f1658112be0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.653+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 >> 172.21.15.40:0/2634665546 conn(0x7f16580f7f20 msgr2=0x7f16580f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:31.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.653+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 shutdown_connections 2024-02-16T19:36:31.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.653+0000 7f165f6ae640 1 -- 172.21.15.40:0/2634665546 wait complete. 2024-02-16T19:36:31.665 DEBUG:tasks.fs:max_mds reduced in epoch 29 2024-02-16T19:36:31.665 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-02-16T19:36:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:31 smithi163 ceph-mon[81637]: pgmap v321: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:31 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2310859040' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-16T19:36:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:31 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3729527371' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-16T19:36:31.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:31 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/844795621' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-16T19:36:31.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 -- 172.21.15.40:0/482504119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0fe2f0 msgr2=0x7fbd0c0fe6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 --2- 172.21.15.40:0/482504119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0fe2f0 0x7fbd0c0fe6c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0048d0 tx=0x7fbcfc009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 -- 172.21.15.40:0/482504119 shutdown_connections 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 --2- 172.21.15.40:0/482504119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c0faa70 0x7fbd0c0faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 --2- 172.21.15.40:0/482504119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0fe2f0 0x7fbd0c0fe6c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 -- 172.21.15.40:0/482504119 >> 172.21.15.40:0/482504119 conn(0x7fbd0c071950 msgr2=0x7fbd0c071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 -- 172.21.15.40:0/482504119 shutdown_connections 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 -- 172.21.15.40:0/482504119 wait complete. 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 Processor -- start 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 -- start start 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.763+0000 7fbd11eae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0faa70 0x7fbd0c112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd11eae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c0fe2f0 0x7fbd0c1131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd11eae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c113830 con 0x7fbd0c0faa70 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd11eae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c10cd30 con 0x7fbd0c0fe2f0 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0faa70 0x7fbd0c112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0faa70 0x7fbd0c112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38402/0 (socket says 172.21.15.40:38402) 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0b7fe640 1 -- 172.21.15.40:0/1592085182 learned_addr learned my addr 172.21.15.40:0/1592085182 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0affd640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c0fe2f0 0x7fbd0c1131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0b7fe640 1 -- 172.21.15.40:0/1592085182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c0fe2f0 msgr2=0x7fbd0c1131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0b7fe640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c0fe2f0 0x7fbd0c1131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0b7fe640 1 -- 172.21.15.40:0/1592085182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc005040 con 0x7fbd0c0faa70 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0affd640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c0fe2f0 0x7fbd0c1131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:31.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd0b7fe640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0faa70 0x7fbd0c112c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc031720 tx=0x7fbcfc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbcf3fff640 1 -- 172.21.15.40:0/1592085182 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcfc0094c0 con 0x7fbd0c0faa70 2024-02-16T19:36:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbcf3fff640 1 -- 172.21.15.40:0/1592085182 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcfc009620 con 0x7fbd0c0faa70 2024-02-16T19:36:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c10cfb0 con 0x7fbd0c0faa70 2024-02-16T19:36:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbcf3fff640 1 -- 172.21.15.40:0/1592085182 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcfc03b640 con 0x7fbd0c0faa70 2024-02-16T19:36:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.764+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c10d4d0 con 0x7fbd0c0faa70 2024-02-16T19:36:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.765+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c064480 con 0x7fbd0c0faa70 2024-02-16T19:36:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.766+0000 7fbcf3fff640 1 -- 172.21.15.40:0/1592085182 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbcfc02f070 con 0x7fbd0c0faa70 2024-02-16T19:36:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.766+0000 7fbcf3fff640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbcf406d750 0x7fbcf406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.766+0000 7fbcf3fff640 1 -- 172.21.15.40:0/1592085182 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbcfc0af520 con 0x7fbd0c0faa70 2024-02-16T19:36:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.766+0000 7fbd0affd640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbcf406d750 0x7fbcf406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.766+0000 7fbd0affd640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbcf406d750 0x7fbcf406fc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbd00006fd0 tx=0x7fbd00008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:31.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.771+0000 7fbcf3fff640 1 -- 172.21.15.40:0/1592085182 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcfc078cc0 con 0x7fbd0c0faa70 2024-02-16T19:36:31.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.991+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fbd0c10d660 con 0x7fbd0c0faa70 2024-02-16T19:36:31.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.991+0000 7fbcf3fff640 1 -- 172.21.15.40:0/1592085182 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v41) v1 ==== 107+0+4086 (secure 0 0 0) 0x7fbcfc07c380 con 0x7fbd0c0faa70 2024-02-16T19:36:31.993 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:31.993 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44285,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/2272386057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2272386057},{"type":"v1","addr":"172.21.15.163:6825","nonce":2272386057}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:27.184688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":156,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:31.993 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 30 2024-02-16T19:36:31.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbcf406d750 msgr2=0x7fbcf406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbcf406d750 0x7fbcf406fc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbd00006fd0 tx=0x7fbd00008040 comp rx=0 tx=0).stop 2024-02-16T19:36:31.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0faa70 msgr2=0x7fbd0c112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:31.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0faa70 0x7fbd0c112c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc031720 tx=0x7fbcfc009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 shutdown_connections 2024-02-16T19:36:31.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fbcf406d750 0x7fbcf406fc00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd0c0fe2f0 0x7fbd0c1131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 --2- 172.21.15.40:0/1592085182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd0c0faa70 0x7fbd0c112c60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:31.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 >> 172.21.15.40:0/1592085182 conn(0x7fbd0c071950 msgr2=0x7fbd0c0f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:31.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 shutdown_connections 2024-02-16T19:36:31.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:31.994+0000 7fbd11eae640 1 -- 172.21.15.40:0/1592085182 wait complete. 2024-02-16T19:36:32.006 DEBUG:tasks.fs:max_mds reduced in epoch 30 2024-02-16T19:36:32.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 -- 172.21.15.40:0/703640364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 msgr2=0x7f28480faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 --2- 172.21.15.40:0/703640364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 0x7f28480faed0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f28380048d0 tx=0x7f2838009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 -- 172.21.15.40:0/703640364 shutdown_connections 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 --2- 172.21.15.40:0/703640364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 0x7f28480faed0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 --2- 172.21.15.40:0/703640364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28481023f0 0x7f28481027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 -- 172.21.15.40:0/703640364 >> 172.21.15.40:0/703640364 conn(0x7f2848071880 msgr2=0x7f2848071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 -- 172.21.15.40:0/703640364 shutdown_connections 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 -- 172.21.15.40:0/703640364 wait complete. 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.104+0000 7f284ef9d640 1 Processor -- start 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284ef9d640 1 -- start start 2024-02-16T19:36:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284ef9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 0x7f2848112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284ef9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28481023f0 0x7f2848113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284ef9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2848113730 con 0x7f28480faa60 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284ef9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28481138a0 con 0x7f28481023f0 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284cd39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 0x7f2848112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284cd39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 0x7f2848112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38434/0 (socket says 172.21.15.40:38434) 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284cd39640 1 -- 172.21.15.40:0/2467893717 learned_addr learned my addr 172.21.15.40:0/2467893717 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284cd39640 1 -- 172.21.15.40:0/2467893717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28481023f0 msgr2=0x7f2848113130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284cd39640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28481023f0 0x7f2848113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284cd39640 1 -- 172.21.15.40:0/2467893717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2838005040 con 0x7f28480faa60 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f284cd39640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 0x7f2848112bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f284000cae0 tx=0x7f2840007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.105+0000 7f28457fa640 1 -- 172.21.15.40:0/2467893717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f284000ab00 con 0x7f28480faa60 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.106+0000 7f28457fa640 1 -- 172.21.15.40:0/2467893717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f284000ac60 con 0x7f28480faa60 2024-02-16T19:36:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.106+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284810cee0 con 0x7f28480faa60 2024-02-16T19:36:32.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.106+0000 7f28457fa640 1 -- 172.21.15.40:0/2467893717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28400116f0 con 0x7f28480faa60 2024-02-16T19:36:32.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.106+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284810d400 con 0x7f28480faa60 2024-02-16T19:36:32.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.106+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2848064480 con 0x7f28480faa60 2024-02-16T19:36:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.107+0000 7f28457fa640 1 -- 172.21.15.40:0/2467893717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2840012440 con 0x7f28480faa60 2024-02-16T19:36:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.107+0000 7f28457fa640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f283406d750 0x7f283406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.107+0000 7f28457fa640 1 -- 172.21.15.40:0/2467893717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f284008d890 con 0x7f28480faa60 2024-02-16T19:36:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.107+0000 7f2847fff640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f283406d750 0x7f283406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.108+0000 7f2847fff640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f283406d750 0x7f283406fc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2838034040 tx=0x7f2838009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:32.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.112+0000 7f28457fa640 1 -- 172.21.15.40:0/2467893717 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2840057030 con 0x7f28480faa60 2024-02-16T19:36:32.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.332+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f28480ff640 con 0x7f28480faa60 2024-02-16T19:36:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.333+0000 7f28457fa640 1 -- 172.21.15.40:0/2467893717 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v41) v1 ==== 107+0+4899 (secure 0 0 0) 0x7f284005a6f0 con 0x7f28480faa60 2024-02-16T19:36:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:32.334 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44285,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/2272386057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2272386057},{"type":"v1","addr":"172.21.15.163:6825","nonce":2272386057}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:28.219148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":0,"incarnation":28,"state":"up:active","state_seq":157,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:32.334 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 31 2024-02-16T19:36:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f283406d750 msgr2=0x7f283406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f283406d750 0x7f283406fc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2838034040 tx=0x7f2838009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 msgr2=0x7f2848112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 0x7f2848112bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f284000cae0 tx=0x7f2840007e50 comp rx=0 tx=0).stop 2024-02-16T19:36:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 shutdown_connections 2024-02-16T19:36:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f283406d750 0x7f283406fc00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28481023f0 0x7f2848113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 --2- 172.21.15.40:0/2467893717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28480faa60 0x7f2848112bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 >> 172.21.15.40:0/2467893717 conn(0x7f2848071880 msgr2=0x7f2848105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 shutdown_connections 2024-02-16T19:36:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.335+0000 7f284ef9d640 1 -- 172.21.15.40:0/2467893717 wait complete. 2024-02-16T19:36:32.347 DEBUG:tasks.fs:max_mds reduced in epoch 31 2024-02-16T19:36:32.347 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-02-16T19:36:32.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.445+0000 7fddca007640 1 -- 172.21.15.40:0/3752872936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fdad0 msgr2=0x7fddc40fbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.445+0000 7fddca007640 1 --2- 172.21.15.40:0/3752872936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fdad0 0x7fddc40fbb80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fddb80048d0 tx=0x7fddb8009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.445+0000 7fddca007640 1 -- 172.21.15.40:0/3752872936 shutdown_connections 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 --2- 172.21.15.40:0/3752872936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fdad0 0x7fddc40fbb80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 --2- 172.21.15.40:0/3752872936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc40fd1c0 0x7fddc40fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 -- 172.21.15.40:0/3752872936 >> 172.21.15.40:0/3752872936 conn(0x7fddc4071950 msgr2=0x7fddc4071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 -- 172.21.15.40:0/3752872936 shutdown_connections 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 -- 172.21.15.40:0/3752872936 wait complete. 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 Processor -- start 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 -- start start 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fd1c0 0x7fddc4112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc40fdad0 0x7fddc41131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41137e0 con 0x7fddc40fd1c0 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddca007640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc410cd70 con 0x7fddc40fdad0 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fd1c0 0x7fddc4112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:32.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fd1c0 0x7fddc4112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38454/0 (socket says 172.21.15.40:38454) 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.446+0000 7fddc37fe640 1 -- 172.21.15.40:0/1972372980 learned_addr learned my addr 172.21.15.40:0/1972372980 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddc2ffd640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc40fdad0 0x7fddc41131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddc37fe640 1 -- 172.21.15.40:0/1972372980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc40fdad0 msgr2=0x7fddc41131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddc37fe640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc40fdad0 0x7fddc41131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddc37fe640 1 -- 172.21.15.40:0/1972372980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8005040 con 0x7fddc40fd1c0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddc37fe640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fd1c0 0x7fddc4112ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fddb400c7e0 tx=0x7fddb40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddabfff640 1 -- 172.21.15.40:0/1972372980 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb400a820 con 0x7fddc40fd1c0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddabfff640 1 -- 172.21.15.40:0/1972372980 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddb400ae50 con 0x7fddc40fd1c0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddabfff640 1 -- 172.21.15.40:0/1972372980 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb40113c0 con 0x7fddc40fd1c0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc410d050 con 0x7fddc40fd1c0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.447+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc410d570 con 0x7fddc40fd1c0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.448+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc4064480 con 0x7fddc40fd1c0 2024-02-16T19:36:32.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.448+0000 7fddabfff640 1 -- 172.21.15.40:0/1972372980 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fddb4012810 con 0x7fddc40fd1c0 2024-02-16T19:36:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.449+0000 7fddabfff640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fddac06d750 0x7fddac06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.449+0000 7fddc2ffd640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fddac06d750 0x7fddac06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.449+0000 7fddabfff640 1 -- 172.21.15.40:0/1972372980 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fddb408d0d0 con 0x7fddc40fd1c0 2024-02-16T19:36:32.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.449+0000 7fddc2ffd640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fddac06d750 0x7fddac06fc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fddc410e560 tx=0x7fddb8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:32.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.454+0000 7fddabfff640 1 -- 172.21.15.40:0/1972372980 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fddb405a420 con 0x7fddc40fd1c0 2024-02-16T19:36:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:32 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2634665546' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-16T19:36:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:32 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1592085182' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-16T19:36:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:32 smithi040 ceph-mon[94210]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:32.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:32 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2467893717' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-16T19:36:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.673+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fddc410e2a0 con 0x7fddc40fd1c0 2024-02-16T19:36:32.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.674+0000 7fddabfff640 1 -- 172.21.15.40:0/1972372980 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v41) v1 ==== 107+0+4117 (secure 0 0 0) 0x7fddb4059fb0 con 0x7fddc40fd1c0 2024-02-16T19:36:32.676 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:32.676 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:28.219148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":0,"incarnation":28,"state":"up:active","state_seq":157,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 32 2024-02-16T19:36:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fddac06d750 msgr2=0x7fddac06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fddac06d750 0x7fddac06fc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fddc410e560 tx=0x7fddb8009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fd1c0 msgr2=0x7fddc4112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fd1c0 0x7fddc4112ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fddb400c7e0 tx=0x7fddb40043c0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 shutdown_connections 2024-02-16T19:36:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fddac06d750 0x7fddac06fc00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc40fdad0 0x7fddc41131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 --2- 172.21.15.40:0/1972372980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc40fd1c0 0x7fddc4112ca0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 >> 172.21.15.40:0/1972372980 conn(0x7fddc4071950 msgr2=0x7fddc40facb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 shutdown_connections 2024-02-16T19:36:32.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.676+0000 7fddca007640 1 -- 172.21.15.40:0/1972372980 wait complete. 2024-02-16T19:36:32.688 DEBUG:tasks.fs:max_mds reduced in epoch 32 2024-02-16T19:36:32.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-02-16T19:36:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:32 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2634665546' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-16T19:36:32.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:32 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1592085182' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-16T19:36:32.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:32 smithi163 ceph-mon[81637]: from='mgr.34104 172.21.15.40:0/3457342499' entity='mgr.smithi040.jzygzo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-16T19:36:32.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:32 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2467893717' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 -- 172.21.15.40:0/3236941792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 msgr2=0x7f98f40faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 --2- 172.21.15.40:0/3236941792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 0x7f98f40faee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f98e80048d0 tx=0x7f98e8009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 -- 172.21.15.40:0/3236941792 shutdown_connections 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 --2- 172.21.15.40:0/3236941792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 0x7f98f40faee0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 --2- 172.21.15.40:0/3236941792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40fe2f0 0x7f98f40fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 -- 172.21.15.40:0/3236941792 >> 172.21.15.40:0/3236941792 conn(0x7f98f4071950 msgr2=0x7f98f4071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 -- 172.21.15.40:0/3236941792 shutdown_connections 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 -- 172.21.15.40:0/3236941792 wait complete. 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 Processor -- start 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 -- start start 2024-02-16T19:36:32.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 0x7f98f40fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40fe2f0 0x7f98f40fc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40fc6d0 con 0x7f98f40faa70 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.787+0000 7f98fbd9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40fc840 con 0x7f98f40fe2f0 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f9b39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 0x7f98f40fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f9b39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 0x7f98f40fbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38478/0 (socket says 172.21.15.40:38478) 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f9b39640 1 -- 172.21.15.40:0/294412056 learned_addr learned my addr 172.21.15.40:0/294412056 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f9338640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40fe2f0 0x7f98f40fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f9b39640 1 -- 172.21.15.40:0/294412056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40fe2f0 msgr2=0x7f98f40fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f9b39640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40fe2f0 0x7f98f40fc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f9b39640 1 -- 172.21.15.40:0/294412056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8005040 con 0x7f98f40faa70 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f9b39640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 0x7f98f40fbc50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00b790 tx=0x7f98ec0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f27fc640 1 -- 172.21.15.40:0/294412056 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98ec00c700 con 0x7f98f40faa70 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f27fc640 1 -- 172.21.15.40:0/294412056 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98ec00cd30 con 0x7f98f40faa70 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f4119100 con 0x7f98f40faa70 2024-02-16T19:36:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98f27fc640 1 -- 172.21.15.40:0/294412056 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98ec013370 con 0x7f98f40faa70 2024-02-16T19:36:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.788+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f4119430 con 0x7f98f40faa70 2024-02-16T19:36:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.789+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f4064480 con 0x7f98f40faa70 2024-02-16T19:36:32.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.790+0000 7f98f27fc640 1 -- 172.21.15.40:0/294412056 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f98ec012070 con 0x7f98f40faa70 2024-02-16T19:36:32.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.790+0000 7f98f27fc640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98dc06d750 0x7f98dc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:32.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.790+0000 7f98f27fc640 1 -- 172.21.15.40:0/294412056 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f98ec08cc70 con 0x7f98f40faa70 2024-02-16T19:36:32.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.790+0000 7f98f9338640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98dc06d750 0x7f98dc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:32.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.791+0000 7f98f9338640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98dc06d750 0x7f98dc06fc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f98f40fd440 tx=0x7f98e8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:32.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:32.795+0000 7f98f27fc640 1 -- 172.21.15.40:0/294412056 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f98ec093080 con 0x7f98f40faa70 2024-02-16T19:36:33.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.013+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f98f40fd180 con 0x7f98f40faa70 2024-02-16T19:36:33.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.014+0000 7f98f27fc640 1 -- 172.21.15.40:0/294412056 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v41) v1 ==== 107+0+4933 (secure 0 0 0) 0x7f98ec056370 con 0x7f98f40faa70 2024-02-16T19:36:33.015 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:33.016 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:28.219148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi163.qovvde","rank":0,"incarnation":28,"state":"up:active","state_seq":157,"addr":"172.21.15.163:6827/1325230089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1325230089},{"type":"v1","addr":"172.21.15.163:6827","nonce":1325230089}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:33.016 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2024-02-16T19:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98dc06d750 msgr2=0x7f98dc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98dc06d750 0x7f98dc06fc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f98f40fd440 tx=0x7f98e8009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 msgr2=0x7f98f40fbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 0x7f98f40fbc50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00b790 tx=0x7f98ec0091d0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 shutdown_connections 2024-02-16T19:36:33.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f98dc06d750 0x7f98dc06fc00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f40fe2f0 0x7f98f40fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 --2- 172.21.15.40:0/294412056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f40faa70 0x7f98f40fbc50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 >> 172.21.15.40:0/294412056 conn(0x7f98f4071950 msgr2=0x7f98f40f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 shutdown_connections 2024-02-16T19:36:33.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.016+0000 7f98fbd9d640 1 -- 172.21.15.40:0/294412056 wait complete. 2024-02-16T19:36:33.028 DEBUG:tasks.fs:max_mds reduced in epoch 33 2024-02-16T19:36:33.028 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-02-16T19:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 -- 172.21.15.40:0/2728452270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102d00 msgr2=0x7fda500fc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 --2- 172.21.15.40:0/2728452270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102d00 0x7fda500fc390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fda440048d0 tx=0x7fda44009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 -- 172.21.15.40:0/2728452270 shutdown_connections 2024-02-16T19:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 --2- 172.21.15.40:0/2728452270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102d00 0x7fda500fc390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 --2- 172.21.15.40:0/2728452270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda50102360 0x7fda50102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 -- 172.21.15.40:0/2728452270 >> 172.21.15.40:0/2728452270 conn(0x7fda500f7ef0 msgr2=0x7fda500fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 -- 172.21.15.40:0/2728452270 shutdown_connections 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 -- 172.21.15.40:0/2728452270 wait complete. 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.126+0000 7fda55c6e640 1 Processor -- start 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda55c6e640 1 -- start start 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda55c6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102360 0x7fda50112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda55c6e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda50102d00 0x7fda50113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda55c6e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda50113730 con 0x7fda50102360 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda55c6e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5010ccc0 con 0x7fda50102d00 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102360 0x7fda50112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102360 0x7fda50112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38496/0 (socket says 172.21.15.40:38496) 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda4f7fe640 1 -- 172.21.15.40:0/23765291 learned_addr learned my addr 172.21.15.40:0/23765291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda4effd640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda50102d00 0x7fda50113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda4f7fe640 1 -- 172.21.15.40:0/23765291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda50102d00 msgr2=0x7fda50113130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda4f7fe640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda50102d00 0x7fda50113130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda4f7fe640 1 -- 172.21.15.40:0/23765291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda44005040 con 0x7fda50102360 2024-02-16T19:36:33.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda4f7fe640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102360 0x7fda50112bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda40002a50 tx=0x7fda40004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:33.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda37fff640 1 -- 172.21.15.40:0/23765291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda40007c30 con 0x7fda50102360 2024-02-16T19:36:33.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda37fff640 1 -- 172.21.15.40:0/23765291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda4000f040 con 0x7fda50102360 2024-02-16T19:36:33.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5010cfa0 con 0x7fda50102360 2024-02-16T19:36:33.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.127+0000 7fda37fff640 1 -- 172.21.15.40:0/23765291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda4000db50 con 0x7fda50102360 2024-02-16T19:36:33.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.128+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5010d490 con 0x7fda50102360 2024-02-16T19:36:33.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.128+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda50064480 con 0x7fda50102360 2024-02-16T19:36:33.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.129+0000 7fda37fff640 1 -- 172.21.15.40:0/23765291 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fda4001c400 con 0x7fda50102360 2024-02-16T19:36:33.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.129+0000 7fda37fff640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fda3806d750 0x7fda3806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.130+0000 7fda4effd640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fda3806d750 0x7fda3806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.130+0000 7fda37fff640 1 -- 172.21.15.40:0/23765291 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fda4008c860 con 0x7fda50102360 2024-02-16T19:36:33.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.130+0000 7fda4effd640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fda3806d750 0x7fda3806fc00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fda5010de00 tx=0x7fda44009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:33.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.134+0000 7fda37fff640 1 -- 172.21.15.40:0/23765291 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda40056000 con 0x7fda50102360 2024-02-16T19:36:33.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.354+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fda5010d6b0 con 0x7fda50102360 2024-02-16T19:36:33.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.354+0000 7fda37fff640 1 -- 172.21.15.40:0/23765291 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v41) v1 ==== 107+0+4138 (secure 0 0 0) 0x7fda400596c0 con 0x7fda50102360 2024-02-16T19:36:33.355 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:33.356 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:59.315130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:33.356 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 34 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fda3806d750 msgr2=0x7fda3806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fda3806d750 0x7fda3806fc00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fda5010de00 tx=0x7fda44009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102360 msgr2=0x7fda50112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102360 0x7fda50112bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda40002a50 tx=0x7fda40004600 comp rx=0 tx=0).stop 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 shutdown_connections 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fda3806d750 0x7fda3806fc00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda50102d00 0x7fda50113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 --2- 172.21.15.40:0/23765291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50102360 0x7fda50112bf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 >> 172.21.15.40:0/23765291 conn(0x7fda500f7ef0 msgr2=0x7fda500f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 shutdown_connections 2024-02-16T19:36:33.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.356+0000 7fda55c6e640 1 -- 172.21.15.40:0/23765291 wait complete. 2024-02-16T19:36:33.369 DEBUG:tasks.fs:max_mds reduced in epoch 34 2024-02-16T19:36:33.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 -- 172.21.15.40:0/2633742795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 msgr2=0x7fb01c106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 --2- 172.21.15.40:0/2633742795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 0x7fb01c106410 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb010004900 tx=0x7fb010009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 -- 172.21.15.40:0/2633742795 shutdown_connections 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 --2- 172.21.15.40:0/2633742795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 0x7fb01c106410 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 --2- 172.21.15.40:0/2633742795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb01c0fd3a0 0x7fb01c0fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 -- 172.21.15.40:0/2633742795 >> 172.21.15.40:0/2633742795 conn(0x7fb01c0f6e00 msgr2=0x7fb01c0f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 -- 172.21.15.40:0/2633742795 shutdown_connections 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 -- 172.21.15.40:0/2633742795 wait complete. 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 Processor -- start 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.467+0000 7fb022825640 1 -- start start 2024-02-16T19:36:33.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb022825640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb01c0fd3a0 0x7fb01c112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb022825640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 0x7fb01c1131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb022825640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1137d0 con 0x7fb01c0fdd40 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb022825640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c10cd60 con 0x7fb01c0fd3a0 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 0x7fb01c1131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 0x7fb01c1131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38510/0 (socket says 172.21.15.40:38510) 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01b7fe640 1 -- 172.21.15.40:0/2050517395 learned_addr learned my addr 172.21.15.40:0/2050517395 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01bfff640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb01c0fd3a0 0x7fb01c112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01b7fe640 1 -- 172.21.15.40:0/2050517395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb01c0fd3a0 msgr2=0x7fb01c112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01b7fe640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb01c0fd3a0 0x7fb01c112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01b7fe640 1 -- 172.21.15.40:0/2050517395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010005040 con 0x7fb01c0fdd40 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01bfff640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb01c0fd3a0 0x7fb01c112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb01b7fe640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 0x7fb01c1131d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb01002bd80 tx=0x7fb010009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb018ff9640 1 -- 172.21.15.40:0/2050517395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01003d450 con 0x7fb01c0fdd40 2024-02-16T19:36:33.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb018ff9640 1 -- 172.21.15.40:0/2050517395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb01003da80 con 0x7fb01c0fdd40 2024-02-16T19:36:33.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb018ff9640 1 -- 172.21.15.40:0/2050517395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01003ba60 con 0x7fb01c0fdd40 2024-02-16T19:36:33.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.468+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c10cfe0 con 0x7fb01c0fdd40 2024-02-16T19:36:33.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.469+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c10d500 con 0x7fb01c0fdd40 2024-02-16T19:36:33.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.470+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c064480 con 0x7fb01c0fdd40 2024-02-16T19:36:33.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.470+0000 7fb018ff9640 1 -- 172.21.15.40:0/2050517395 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb01003bcd0 con 0x7fb01c0fdd40 2024-02-16T19:36:33.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.470+0000 7fb018ff9640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb00406d760 0x7fb00406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.470+0000 7fb018ff9640 1 -- 172.21.15.40:0/2050517395 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb0100afd90 con 0x7fb01c0fdd40 2024-02-16T19:36:33.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.470+0000 7fb01bfff640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb00406d760 0x7fb00406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.471+0000 7fb01bfff640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb00406d760 0x7fb00406fc10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb00c004c60 tx=0x7fb00c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:33.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.475+0000 7fb018ff9640 1 -- 172.21.15.40:0/2050517395 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb010079530 con 0x7fb01c0fdd40 2024-02-16T19:36:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:33 smithi040 ceph-mon[94210]: pgmap v322: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:33 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1972372980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-16T19:36:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:33 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/294412056' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-16T19:36:33.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:33 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/23765291' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-16T19:36:33.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.693+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fb01c10e290 con 0x7fb01c0fdd40 2024-02-16T19:36:33.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.693+0000 7fb018ff9640 1 -- 172.21.15.40:0/2050517395 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v41) v1 ==== 107+0+4183 (secure 0 0 0) 0x7fb01007cbf0 con 0x7fb01c0fdd40 2024-02-16T19:36:33.695 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:33.695 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:32:59.324244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:33.695 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 35 2024-02-16T19:36:33.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb00406d760 msgr2=0x7fb00406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb00406d760 0x7fb00406fc10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb00c004c60 tx=0x7fb00c009040 comp rx=0 tx=0).stop 2024-02-16T19:36:33.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 msgr2=0x7fb01c1131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 0x7fb01c1131d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb01002bd80 tx=0x7fb010009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 shutdown_connections 2024-02-16T19:36:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fb00406d760 0x7fb00406fc10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb01c0fdd40 0x7fb01c1131d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 --2- 172.21.15.40:0/2050517395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb01c0fd3a0 0x7fb01c112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 >> 172.21.15.40:0/2050517395 conn(0x7fb01c0f6e00 msgr2=0x7fb01c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 shutdown_connections 2024-02-16T19:36:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.695+0000 7fb022825640 1 -- 172.21.15.40:0/2050517395 wait complete. 2024-02-16T19:36:33.707 DEBUG:tasks.fs:max_mds reduced in epoch 35 2024-02-16T19:36:33.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-02-16T19:36:33.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:33 smithi163 ceph-mon[81637]: pgmap v322: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:33.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:33 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1972372980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-16T19:36:33.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:33 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/294412056' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-16T19:36:33.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:33 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/23765291' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-16T19:36:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 -- 172.21.15.40:0/82882639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 msgr2=0x7f407c0fd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 --2- 172.21.15.40:0/82882639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 0x7f407c0fd750 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4064004900 tx=0x7f4064009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 -- 172.21.15.40:0/82882639 shutdown_connections 2024-02-16T19:36:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 --2- 172.21.15.40:0/82882639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f407c0fdd20 0x7f407c1063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 --2- 172.21.15.40:0/82882639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 0x7f407c0fd750 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 -- 172.21.15.40:0/82882639 >> 172.21.15.40:0/82882639 conn(0x7f407c0f6de0 msgr2=0x7f407c0f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 -- 172.21.15.40:0/82882639 shutdown_connections 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 -- 172.21.15.40:0/82882639 wait complete. 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 Processor -- start 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.806+0000 7f4081cae640 1 -- start start 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4081cae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 0x7f407c112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4081cae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f407c0fdd20 0x7f407c113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4081cae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c113910 con 0x7f407c0fd380 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4081cae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c10ce10 con 0x7f407c0fdd20 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 0x7f407c112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 0x7f407c112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38518/0 (socket says 172.21.15.40:38518) 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407b7fe640 1 -- 172.21.15.40:0/710857439 learned_addr learned my addr 172.21.15.40:0/710857439 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407affd640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f407c0fdd20 0x7f407c113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407b7fe640 1 -- 172.21.15.40:0/710857439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f407c0fdd20 msgr2=0x7f407c113280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407b7fe640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f407c0fdd20 0x7f407c113280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407b7fe640 1 -- 172.21.15.40:0/710857439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4064005040 con 0x7f407c0fd380 2024-02-16T19:36:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407affd640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f407c0fdd20 0x7f407c113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f407b7fe640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 0x7f407c112d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f406402bd80 tx=0x7f4064009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4073fff640 1 -- 172.21.15.40:0/710857439 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406403d450 con 0x7f407c0fd380 2024-02-16T19:36:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c10d090 con 0x7f407c0fd380 2024-02-16T19:36:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4073fff640 1 -- 172.21.15.40:0/710857439 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f406403da80 con 0x7f407c0fd380 2024-02-16T19:36:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4073fff640 1 -- 172.21.15.40:0/710857439 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406403ba90 con 0x7f407c0fd380 2024-02-16T19:36:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.807+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c10d5b0 con 0x7f407c0fd380 2024-02-16T19:36:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.808+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c064480 con 0x7f407c0fd380 2024-02-16T19:36:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.809+0000 7f4073fff640 1 -- 172.21.15.40:0/710857439 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f406403dbf0 con 0x7f407c0fd380 2024-02-16T19:36:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.809+0000 7f4073fff640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f406806d750 0x7f406806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.809+0000 7f4073fff640 1 -- 172.21.15.40:0/710857439 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f40640afdd0 con 0x7f407c0fd380 2024-02-16T19:36:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.809+0000 7f407affd640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f406806d750 0x7f406806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.810+0000 7f407affd640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f406806d750 0x7f406806fc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f406c006fd0 tx=0x7f406c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:33.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:33.814+0000 7f4073fff640 1 -- 172.21.15.40:0/710857439 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4064079570 con 0x7f407c0fd380 2024-02-16T19:36:34.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.042+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f407c109550 con 0x7f407c0fd380 2024-02-16T19:36:34.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.042+0000 7f4073fff640 1 -- 172.21.15.40:0/710857439 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v41) v1 ==== 107+0+4187 (secure 0 0 0) 0x7f406407cc30 con 0x7f407c0fd380 2024-02-16T19:36:34.044 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:34.044 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:33:05.985216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":0,"incarnation":35,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:34.044 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 36 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f406806d750 msgr2=0x7f406806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f406806d750 0x7f406806fc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f406c006fd0 tx=0x7f406c008040 comp rx=0 tx=0).stop 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 msgr2=0x7f407c112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 0x7f407c112d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f406402bd80 tx=0x7f4064009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 shutdown_connections 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f406806d750 0x7f406806fc00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f407c0fdd20 0x7f407c113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 --2- 172.21.15.40:0/710857439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f407c0fd380 0x7f407c112d40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.044+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 >> 172.21.15.40:0/710857439 conn(0x7f407c0f6de0 msgr2=0x7f407c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.045+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 shutdown_connections 2024-02-16T19:36:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.045+0000 7f4081cae640 1 -- 172.21.15.40:0/710857439 wait complete. 2024-02-16T19:36:34.057 DEBUG:tasks.fs:max_mds reduced in epoch 36 2024-02-16T19:36:34.057 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-02-16T19:36:34.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.155+0000 7f34aa02e640 1 -- 172.21.15.40:0/2983931252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 msgr2=0x7f34a40fb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.155+0000 7f34aa02e640 1 --2- 172.21.15.40:0/2983931252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 0x7f34a40fb270 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f34940048d0 tx=0x7f3494009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.155+0000 7f34aa02e640 1 -- 172.21.15.40:0/2983931252 shutdown_connections 2024-02-16T19:36:34.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.155+0000 7f34aa02e640 1 --2- 172.21.15.40:0/2983931252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 0x7f34a40fb270 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.155+0000 7f34aa02e640 1 --2- 172.21.15.40:0/2983931252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a4101240 0x7f34a4101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.155+0000 7f34aa02e640 1 -- 172.21.15.40:0/2983931252 >> 172.21.15.40:0/2983931252 conn(0x7f34a40f6e10 msgr2=0x7f34a40f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:34.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.155+0000 7f34aa02e640 1 -- 172.21.15.40:0/2983931252 shutdown_connections 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.155+0000 7f34aa02e640 1 -- 172.21.15.40:0/2983931252 wait complete. 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34aa02e640 1 Processor -- start 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34aa02e640 1 -- start start 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34aa02e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a4101240 0x7f34a410dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34aa02e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 0x7f34a410e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34aa02e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a410e7b0 con 0x7f34a4101be0 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34aa02e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a410e920 con 0x7f34a4101240 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34a2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 0x7f34a410e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34a2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 0x7f34a410e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38534/0 (socket says 172.21.15.40:38534) 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34a2ffd640 1 -- 172.21.15.40:0/3883666032 learned_addr learned my addr 172.21.15.40:0/3883666032 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.156+0000 7f34a37fe640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a4101240 0x7f34a410dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f34a2ffd640 1 -- 172.21.15.40:0/3883666032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a4101240 msgr2=0x7f34a410dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f34a2ffd640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a4101240 0x7f34a410dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f34a2ffd640 1 -- 172.21.15.40:0/3883666032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494005040 con 0x7f34a4101be0 2024-02-16T19:36:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f34a2ffd640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 0x7f34a410e270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3494031720 tx=0x7f3494009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f349bfff640 1 -- 172.21.15.40:0/3883666032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34940094c0 con 0x7f34a4101be0 2024-02-16T19:36:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f349bfff640 1 -- 172.21.15.40:0/3883666032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3494009620 con 0x7f34a4101be0 2024-02-16T19:36:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a410eba0 con 0x7f34a4101be0 2024-02-16T19:36:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f349bfff640 1 -- 172.21.15.40:0/3883666032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f349403b640 con 0x7f34a4101be0 2024-02-16T19:36:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.157+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a41170a0 con 0x7f34a4101be0 2024-02-16T19:36:34.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.158+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a4064480 con 0x7f34a4101be0 2024-02-16T19:36:34.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.158+0000 7f349bfff640 1 -- 172.21.15.40:0/3883666032 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f349402f070 con 0x7f34a4101be0 2024-02-16T19:36:34.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.159+0000 7f349bfff640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f349006d750 0x7f349006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.159+0000 7f349bfff640 1 -- 172.21.15.40:0/3883666032 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f34940af2c0 con 0x7f34a4101be0 2024-02-16T19:36:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.159+0000 7f34a37fe640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f349006d750 0x7f349006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.159+0000 7f34a37fe640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f349006d750 0x7f349006fc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f348c007fd0 tx=0x7f348c009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:34.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.163+0000 7f349bfff640 1 -- 172.21.15.40:0/3883666032 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3494078a60 con 0x7f34a4101be0 2024-02-16T19:36:34.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.383+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f34a41083a0 con 0x7f34a4101be0 2024-02-16T19:36:34.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.383+0000 7f349bfff640 1 -- 172.21.15.40:0/3883666032 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v41) v1 ==== 107+0+4184 (secure 0 0 0) 0x7f349407c120 con 0x7f34a4101be0 2024-02-16T19:36:34.385 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:34.385 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:33:06.304060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":0,"incarnation":35,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:34.385 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 37 2024-02-16T19:36:34.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f349006d750 msgr2=0x7f349006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f349006d750 0x7f349006fc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f348c007fd0 tx=0x7f348c009e90 comp rx=0 tx=0).stop 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 msgr2=0x7f34a410e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 0x7f34a410e270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3494031720 tx=0x7f3494009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 shutdown_connections 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f349006d750 0x7f349006fc00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a4101be0 0x7f34a410e270 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 --2- 172.21.15.40:0/3883666032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a4101240 0x7f34a410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 >> 172.21.15.40:0/3883666032 conn(0x7f34a40f6e10 msgr2=0x7f34a40f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 shutdown_connections 2024-02-16T19:36:34.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.385+0000 7f34aa02e640 1 -- 172.21.15.40:0/3883666032 wait complete. 2024-02-16T19:36:34.397 DEBUG:tasks.fs:max_mds reduced in epoch 37 2024-02-16T19:36:34.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-02-16T19:36:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.496+0000 7f811595d640 1 -- 172.21.15.40:0/2758565279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104590 msgr2=0x7f8110104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.496+0000 7f811595d640 1 --2- 172.21.15.40:0/2758565279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104590 0x7f8110104960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8100009c50 tx=0x7f8100004d10 comp rx=0 tx=0).stop 2024-02-16T19:36:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.496+0000 7f811595d640 1 -- 172.21.15.40:0/2758565279 shutdown_connections 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.496+0000 7f811595d640 1 --2- 172.21.15.40:0/2758565279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110104ea0 0x7f81100fe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.496+0000 7f811595d640 1 --2- 172.21.15.40:0/2758565279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104590 0x7f8110104960 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.496+0000 7f811595d640 1 -- 172.21.15.40:0/2758565279 >> 172.21.15.40:0/2758565279 conn(0x7f81100fa190 msgr2=0x7f81100fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.496+0000 7f811595d640 1 -- 172.21.15.40:0/2758565279 shutdown_connections 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.496+0000 7f811595d640 1 -- 172.21.15.40:0/2758565279 wait complete. 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f811595d640 1 Processor -- start 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f811595d640 1 -- start start 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f811595d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110104590 0x7f8110114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f811595d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104ea0 0x7f8110115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f811595d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8110115aa0 con 0x7f8110104ea0 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f811595d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811010efa0 con 0x7f8110104590 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f810e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104ea0 0x7f8110115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f810e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104ea0 0x7f8110115410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38562/0 (socket says 172.21.15.40:38562) 2024-02-16T19:36:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f810e7fc640 1 -- 172.21.15.40:0/3986486035 learned_addr learned my addr 172.21.15.40:0/3986486035 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f810effd640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110104590 0x7f8110114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f810e7fc640 1 -- 172.21.15.40:0/3986486035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110104590 msgr2=0x7f8110114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f810e7fc640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110104590 0x7f8110114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.497+0000 7f810e7fc640 1 -- 172.21.15.40:0/3986486035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81000098b0 con 0x7f8110104ea0 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.498+0000 7f810e7fc640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104ea0 0x7f8110115410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80f800b540 tx=0x7f80f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.498+0000 7f81077fe640 1 -- 172.21.15.40:0/3986486035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80f800f6f0 con 0x7f8110104ea0 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.498+0000 7f81077fe640 1 -- 172.21.15.40:0/3986486035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80f800fd20 con 0x7f8110104ea0 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.498+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811010f280 con 0x7f8110104ea0 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.498+0000 7f81077fe640 1 -- 172.21.15.40:0/3986486035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80f80103a0 con 0x7f8110104ea0 2024-02-16T19:36:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.498+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811010f7a0 con 0x7f8110104ea0 2024-02-16T19:36:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.499+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8110064030 con 0x7f8110104ea0 2024-02-16T19:36:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.499+0000 7f81077fe640 1 -- 172.21.15.40:0/3986486035 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f80f8011400 con 0x7f8110104ea0 2024-02-16T19:36:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.500+0000 7f81077fe640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f80fc06d750 0x7f80fc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.500+0000 7f81077fe640 1 -- 172.21.15.40:0/3986486035 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80f808ce70 con 0x7f8110104ea0 2024-02-16T19:36:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.500+0000 7f810effd640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f80fc06d750 0x7f80fc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.500+0000 7f810effd640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f80fc06d750 0x7f80fc06fc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8100003400 tx=0x7f8100037040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:34.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.504+0000 7f81077fe640 1 -- 172.21.15.40:0/3986486035 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80f8056690 con 0x7f8110104ea0 2024-02-16T19:36:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.722+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f811010fa20 con 0x7f8110104ea0 2024-02-16T19:36:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.723+0000 7f81077fe640 1 -- 172.21.15.40:0/3986486035 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v41) v1 ==== 107+0+4184 (secure 0 0 0) 0x7f80f8059d50 con 0x7f8110104ea0 2024-02-16T19:36:34.724 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:34.725 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:33:07.341718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":0,"incarnation":35,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 38 2024-02-16T19:36:34.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f80fc06d750 msgr2=0x7f80fc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f80fc06d750 0x7f80fc06fc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8100003400 tx=0x7f8100037040 comp rx=0 tx=0).stop 2024-02-16T19:36:34.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104ea0 msgr2=0x7f8110115410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104ea0 0x7f8110115410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80f800b540 tx=0x7f80f80091a0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 shutdown_connections 2024-02-16T19:36:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f80fc06d750 0x7f80fc06fc00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8110104ea0 0x7f8110115410 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 --2- 172.21.15.40:0/3986486035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110104590 0x7f8110114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 >> 172.21.15.40:0/3986486035 conn(0x7f81100fa190 msgr2=0x7f81100fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 shutdown_connections 2024-02-16T19:36:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.725+0000 7f811595d640 1 -- 172.21.15.40:0/3986486035 wait complete. 2024-02-16T19:36:34.736 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:34 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2050517395' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-16T19:36:34.736 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:34 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/710857439' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-16T19:36:34.736 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:34 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3883666032' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-02-16T19:36:34.737 DEBUG:tasks.fs:max_mds reduced in epoch 38 2024-02-16T19:36:34.738 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-02-16T19:36:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:34 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2050517395' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-16T19:36:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:34 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/710857439' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-16T19:36:34.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:34 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3883666032' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-02-16T19:36:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 -- 172.21.15.40:0/3750080259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 msgr2=0x7f695c101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 --2- 172.21.15.40:0/3750080259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 0x7f695c101610 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6954009c50 tx=0x7f6954004d10 comp rx=0 tx=0).stop 2024-02-16T19:36:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 -- 172.21.15.40:0/3750080259 shutdown_connections 2024-02-16T19:36:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 --2- 172.21.15.40:0/3750080259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f695c101be0 0x7f695c0fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 --2- 172.21.15.40:0/3750080259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 0x7f695c101610 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 -- 172.21.15.40:0/3750080259 >> 172.21.15.40:0/3750080259 conn(0x7f695c0f6e10 msgr2=0x7f695c0f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 -- 172.21.15.40:0/3750080259 shutdown_connections 2024-02-16T19:36:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 -- 172.21.15.40:0/3750080259 wait complete. 2024-02-16T19:36:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 Processor -- start 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 -- start start 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 0x7f695c114df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f695c101be0 0x7f695c115330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c115930 con 0x7f695c101240 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f6963b1d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c10eec0 con 0x7f695c101be0 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f69618b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 0x7f695c114df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f69618b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 0x7f695c114df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38572/0 (socket says 172.21.15.40:38572) 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.836+0000 7f69618b9640 1 -- 172.21.15.40:0/1676012514 learned_addr learned my addr 172.21.15.40:0/1676012514 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f69610b8640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f695c101be0 0x7f695c115330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f69618b9640 1 -- 172.21.15.40:0/1676012514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f695c101be0 msgr2=0x7f695c115330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f69618b9640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f695c101be0 0x7f695c115330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f69618b9640 1 -- 172.21.15.40:0/1676012514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69540098b0 con 0x7f695c101240 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f69610b8640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f695c101be0 0x7f695c115330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f69618b9640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 0x7f695c114df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6954004750 tx=0x7f6954004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f695a7fc640 1 -- 172.21.15.40:0/1676012514 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695403a070 con 0x7f695c101240 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f695a7fc640 1 -- 172.21.15.40:0/1676012514 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6954035b20 con 0x7f695c101240 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695c10f140 con 0x7f695c101240 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f695a7fc640 1 -- 172.21.15.40:0/1676012514 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695403ea80 con 0x7f695c101240 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.837+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c10f6e0 con 0x7f695c101240 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.838+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c064480 con 0x7f695c101240 2024-02-16T19:36:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.838+0000 7f695a7fc640 1 -- 172.21.15.40:0/1676012514 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6954032070 con 0x7f695c101240 2024-02-16T19:36:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.839+0000 7f695a7fc640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f694406d750 0x7f694406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.839+0000 7f695a7fc640 1 -- 172.21.15.40:0/1676012514 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f69540af320 con 0x7f695c101240 2024-02-16T19:36:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.839+0000 7f69610b8640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f694406d750 0x7f694406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.839+0000 7f69610b8640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f694406d750 0x7f694406fc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f695c1106b0 tx=0x7f695000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:34.843+0000 7f695a7fc640 1 -- 172.21.15.40:0/1676012514 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6954078ac0 con 0x7f695c101240 2024-02-16T19:36:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.063+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f695c110420 con 0x7f695c101240 2024-02-16T19:36:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.063+0000 7f695a7fc640 1 -- 172.21.15.40:0/1676012514 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v41) v1 ==== 107+0+5000 (secure 0 0 0) 0x7f695407c180 con 0x7f695c101240 2024-02-16T19:36:35.065 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:35.065 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44335,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6827/2051774557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2051774557},{"type":"v1","addr":"172.21.15.163:6827","nonce":2051774557}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:33:07.341718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":0,"incarnation":35,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 39 2024-02-16T19:36:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f694406d750 msgr2=0x7f694406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f694406d750 0x7f694406fc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f695c1106b0 tx=0x7f695000b040 comp rx=0 tx=0).stop 2024-02-16T19:36:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 msgr2=0x7f695c114df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 0x7f695c114df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6954004750 tx=0x7f6954004d10 comp rx=0 tx=0).stop 2024-02-16T19:36:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 shutdown_connections 2024-02-16T19:36:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f694406d750 0x7f694406fc00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f695c101be0 0x7f695c115330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 --2- 172.21.15.40:0/1676012514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f695c101240 0x7f695c114df0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.065+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 >> 172.21.15.40:0/1676012514 conn(0x7f695c0f6e10 msgr2=0x7f695c0656b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.066+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 shutdown_connections 2024-02-16T19:36:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.066+0000 7f6963b1d640 1 -- 172.21.15.40:0/1676012514 wait complete. 2024-02-16T19:36:35.078 DEBUG:tasks.fs:max_mds reduced in epoch 39 2024-02-16T19:36:35.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-02-16T19:36:35.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.176+0000 7faa7b6ae640 1 -- 172.21.15.40:0/768790523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740fb2b0 msgr2=0x7faa740fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.176+0000 7faa7b6ae640 1 --2- 172.21.15.40:0/768790523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740fb2b0 0x7faa740fb700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faa680048d0 tx=0x7faa68009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.176+0000 7faa7b6ae640 1 -- 172.21.15.40:0/768790523 shutdown_connections 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.176+0000 7faa7b6ae640 1 --2- 172.21.15.40:0/768790523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740fb2b0 0x7faa740fb700 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.176+0000 7faa7b6ae640 1 --2- 172.21.15.40:0/768790523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa741012d0 0x7faa741016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.176+0000 7faa7b6ae640 1 -- 172.21.15.40:0/768790523 >> 172.21.15.40:0/768790523 conn(0x7faa74071230 msgr2=0x7faa74071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.176+0000 7faa7b6ae640 1 -- 172.21.15.40:0/768790523 shutdown_connections 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.176+0000 7faa7b6ae640 1 -- 172.21.15.40:0/768790523 wait complete. 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa7b6ae640 1 Processor -- start 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa7b6ae640 1 -- start start 2024-02-16T19:36:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa7b6ae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740fb2b0 0x7faa7410dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa7b6ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa741012d0 0x7faa7410e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa7b6ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7410e7a0 con 0x7faa741012d0 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa7b6ae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7410e8e0 con 0x7faa740fb2b0 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa78c49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa741012d0 0x7faa7410e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa78c49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa741012d0 0x7faa7410e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38588/0 (socket says 172.21.15.40:38588) 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa78c49640 1 -- 172.21.15.40:0/2788016113 learned_addr learned my addr 172.21.15.40:0/2788016113 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa7944a640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740fb2b0 0x7faa7410dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa78c49640 1 -- 172.21.15.40:0/2788016113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740fb2b0 msgr2=0x7faa7410dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa78c49640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740fb2b0 0x7faa7410dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.177+0000 7faa78c49640 1 -- 172.21.15.40:0/2788016113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa68005040 con 0x7faa741012d0 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.178+0000 7faa78c49640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa741012d0 0x7faa7410e260 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa68031720 tx=0x7faa68009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.178+0000 7faa71ffb640 1 -- 172.21.15.40:0/2788016113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa680094c0 con 0x7faa741012d0 2024-02-16T19:36:35.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.178+0000 7faa71ffb640 1 -- 172.21.15.40:0/2788016113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa68009620 con 0x7faa741012d0 2024-02-16T19:36:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.178+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7410eb60 con 0x7faa741012d0 2024-02-16T19:36:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.178+0000 7faa71ffb640 1 -- 172.21.15.40:0/2788016113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6803b640 con 0x7faa741012d0 2024-02-16T19:36:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.178+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa741170f0 con 0x7faa741012d0 2024-02-16T19:36:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.179+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa74064030 con 0x7faa741012d0 2024-02-16T19:36:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.179+0000 7faa71ffb640 1 -- 172.21.15.40:0/2788016113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7faa6802f070 con 0x7faa741012d0 2024-02-16T19:36:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.180+0000 7faa71ffb640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa5c06d750 0x7faa5c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.180+0000 7faa71ffb640 1 -- 172.21.15.40:0/2788016113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faa680af380 con 0x7faa741012d0 2024-02-16T19:36:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.180+0000 7faa7944a640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa5c06d750 0x7faa5c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.180+0000 7faa7944a640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa5c06d750 0x7faa5c06fc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa6c007fd0 tx=0x7faa6c009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.184+0000 7faa71ffb640 1 -- 172.21.15.40:0/2788016113 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa68078ba0 con 0x7faa741012d0 2024-02-16T19:36:35.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.405+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7faa741083c0 con 0x7faa741012d0 2024-02-16T19:36:35.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.405+0000 7faa71ffb640 1 -- 172.21.15.40:0/2788016113 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v41) v1 ==== 107+0+5017 (secure 0 0 0) 0x7faa6807c260 con 0x7faa741012d0 2024-02-16T19:36:35.407 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:35.407 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi163.rggdhh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6825/1456075822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1456075822},{"type":"v1","addr":"172.21.15.163:6825","nonce":1456075822}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44335,"name":"cephfs.smithi163.qovvde","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6827/2051774557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2051774557},{"type":"v1","addr":"172.21.15.163:6827","nonce":2051774557}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-16T19:21:57.272432+0000","modified":"2024-02-16T19:33:18.684980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34404,"mds_1":34436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi040.ezkppl","rank":0,"incarnation":35,"state":"up:active","state_seq":18,"addr":"172.21.15.40:6827/620521629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":620521629},{"type":"v1","addr":"172.21.15.40:6827","nonce":620521629}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34436":{"gid":34436,"name":"cephfs.smithi040.bmpjpk","rank":1,"incarnation":40,"state":"up:starting","state_seq":1,"addr":"172.21.15.40:6829/1175631718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1175631718},{"type":"v1","addr":"172.21.15.40:6829","nonce":1175631718}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-16T19:36:35.407 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 40 2024-02-16T19:36:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa5c06d750 msgr2=0x7faa5c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa5c06d750 0x7faa5c06fc00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa6c007fd0 tx=0x7faa6c009e90 comp rx=0 tx=0).stop 2024-02-16T19:36:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa741012d0 msgr2=0x7faa7410e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa741012d0 0x7faa7410e260 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa68031720 tx=0x7faa68009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 shutdown_connections 2024-02-16T19:36:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7faa5c06d750 0x7faa5c06fc00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa741012d0 0x7faa7410e260 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 --2- 172.21.15.40:0/2788016113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740fb2b0 0x7faa7410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 >> 172.21.15.40:0/2788016113 conn(0x7faa74071230 msgr2=0x7faa740fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 shutdown_connections 2024-02-16T19:36:35.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.408+0000 7faa7b6ae640 1 -- 172.21.15.40:0/2788016113 wait complete. 2024-02-16T19:36:35.420 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-16T19:36:35.432 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-16T19:36:35.432 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:36:35.432 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:36:35.448 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-02-16T19:36:35.448 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:36:35.506 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T19:36:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.644+0000 7ff49f79d640 1 -- 172.21.15.40:0/1793937569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 msgr2=0x7ff498106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.644+0000 7ff49f79d640 1 --2- 172.21.15.40:0/1793937569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 0x7ff498106570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff488004900 tx=0x7ff488009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.644+0000 7ff49f79d640 1 -- 172.21.15.40:0/1793937569 shutdown_connections 2024-02-16T19:36:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.644+0000 7ff49f79d640 1 --2- 172.21.15.40:0/1793937569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 0x7ff498106570 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.644+0000 7ff49f79d640 1 --2- 172.21.15.40:0/1793937569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4980fd500 0x7ff4980fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.644+0000 7ff49f79d640 1 -- 172.21.15.40:0/1793937569 >> 172.21.15.40:0/1793937569 conn(0x7ff4980f6f80 msgr2=0x7ff4980f9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.644+0000 7ff49f79d640 1 -- 172.21.15.40:0/1793937569 shutdown_connections 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.644+0000 7ff49f79d640 1 -- 172.21.15.40:0/1793937569 wait complete. 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49f79d640 1 Processor -- start 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49f79d640 1 -- start start 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49f79d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4980fd500 0x7ff498112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49f79d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 0x7ff498113320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49f79d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498113920 con 0x7ff4980fdea0 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49f79d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49810ceb0 con 0x7ff4980fd500 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49cd38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 0x7ff498113320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49cd38640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 0x7ff498113320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38616/0 (socket says 172.21.15.40:38616) 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.645+0000 7ff49cd38640 1 -- 172.21.15.40:0/1718112806 learned_addr learned my addr 172.21.15.40:0/1718112806 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff49d539640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4980fd500 0x7ff498112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff49cd38640 1 -- 172.21.15.40:0/1718112806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4980fd500 msgr2=0x7ff498112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff49cd38640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4980fd500 0x7ff498112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff49cd38640 1 -- 172.21.15.40:0/1718112806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488005040 con 0x7ff4980fdea0 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff49d539640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4980fd500 0x7ff498112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff49cd38640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 0x7ff498113320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff48802bd80 tx=0x7ff488009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff48dffb640 1 -- 172.21.15.40:0/1718112806 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48803d450 con 0x7ff4980fdea0 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49810d130 con 0x7ff4980fdea0 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff48dffb640 1 -- 172.21.15.40:0/1718112806 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff48803da80 con 0x7ff4980fdea0 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff48dffb640 1 -- 172.21.15.40:0/1718112806 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48803ba90 con 0x7ff4980fdea0 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.646+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49810d650 con 0x7ff4980fdea0 2024-02-16T19:36:35.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.647+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff498064480 con 0x7ff4980fdea0 2024-02-16T19:36:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.648+0000 7ff48dffb640 1 -- 172.21.15.40:0/1718112806 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff48803dbf0 con 0x7ff4980fdea0 2024-02-16T19:36:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.648+0000 7ff48dffb640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff48406d750 0x7ff48406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.648+0000 7ff48dffb640 1 -- 172.21.15.40:0/1718112806 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff4880afe70 con 0x7ff4980fdea0 2024-02-16T19:36:35.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.648+0000 7ff49d539640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff48406d750 0x7ff48406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.648+0000 7ff49d539640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff48406d750 0x7ff48406fc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff494004c80 tx=0x7ff494009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:35.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.653+0000 7ff48dffb640 1 -- 172.21.15.40:0/1718112806 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff488079610 con 0x7ff4980fdea0 2024-02-16T19:36:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:35 smithi163 ceph-mon[81637]: pgmap v323: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:35 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3986486035' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-02-16T19:36:35.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:35 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1676012514' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-02-16T19:36:35.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:35 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2788016113' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-02-16T19:36:35.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.840+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff498103ba0 con 0x7ff4980fdea0 2024-02-16T19:36:35.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.841+0000 7ff48dffb640 1 -- 172.21.15.40:0/1718112806 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16756 (secure 0 0 0) 0x7ff48807ccd0 con 0x7ff4980fdea0 2024-02-16T19:36:35.843 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:35.843 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","created":"2024-02-16T19:14:55.436141+0000","modified":"2024-02-16T19:32:59.315148+0000","last_up_change":"2024-02-16T19:31:25.855269+0000","last_in_change":"2024-02-16T19:20:17.443690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:19:22.349243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T19:21:55.970810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T19:21:56.515184+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db82e60f-540c-4f54-8f54-758be43e2cf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6803","nonce":3792625162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6805","nonce":3792625162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6809","nonce":3792625162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6807","nonce":3792625162}]},"public_addr":"172.21.15.40:6803/3792625162","cluster_addr":"172.21.15.40:6805/3792625162","heartbeat_back_addr":"172.21.15.40:6809/3792625162","heartbeat_front_addr":"172.21.15.40:6807/3792625162","state":["exists","up"]},{"osd":1,"uuid":"22561508-cdb8-4c76-ab07-bf3efd6ee63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6811","nonce":3029722765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6813","nonce":3029722765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6817","nonce":3029722765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6815","nonce":3029722765}]},"public_addr":"172.21.15.40:6811/3029722765","cluster_addr":"172.21.15.40:6813/3029722765","heartbeat_back_addr":"172.21.15.40:6817/3029722765","heartbeat_front_addr":"172.21.15.40:6815/3029722765","state":["exists","up"]},{"osd":2,"uuid":"58ff6654-72f1-4329-8f6c-4c3180353755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6819","nonce":1990065416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6821","nonce":1990065416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6825","nonce":1990065416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6823","nonce":1990065416}]},"public_addr":"172.21.15.40:6819/1990065416","cluster_addr":"172.21.15.40:6821/1990065416","heartbeat_back_addr":"172.21.15.40:6825/1990065416","heartbeat_front_addr":"172.21.15.40:6823/1990065416","state":["exists","up"]},{"osd":3,"uuid":"b6f2b76f-f1fa-4459-8150-c36e624fb4aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6801","nonce":346064807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6803","nonce":346064807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6807","nonce":346064807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6805","nonce":346064807}]},"public_addr":"172.21.15.163:6801/346064807","cluster_addr":"172.21.15.163:6803/346064807","heartbeat_back_addr":"172.21.15.163:6807/346064807","heartbeat_front_addr":"172.21.15.163:6805/346064807","state":["exists","up"]},{"osd":4,"uuid":"49afe3d8-2685-4c11-9830-77bd72a0b272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6809","nonce":2382516830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6811","nonce":2382516830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6815","nonce":2382516830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6813","nonce":2382516830}]},"public_addr":"172.21.15.163:6809/2382516830","cluster_addr":"172.21.15.163:6811/2382516830","heartbeat_back_addr":"172.21.15.163:6815/2382516830","heartbeat_front_addr":"172.21.15.163:6813/2382516830","state":["exists","up"]},{"osd":5,"uuid":"c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6817","nonce":10307935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6819","nonce":10307935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6823","nonce":10307935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6821","nonce":10307935}]},"public_addr":"172.21.15.163:6817/10307935","cluster_addr":"172.21.15.163:6819/10307935","heartbeat_back_addr":"172.21.15.163:6823/10307935","heartbeat_front_addr":"172.21.15.163:6821/10307935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:22.484106+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:52.190602+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:19.421062+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:43.759349+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:09.243860+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:34.636467+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6826/1325230089":"2024-02-17T19:32:59.315119+0000","172.21.15.40:6829/2154720438":"2024-02-17T19:32:20.151991+0000","172.21.15.40:6828/2154720438":"2024-02-17T19:32:20.151991+0000","172.21.15.40:0/493553370":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/4011363422":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1058311844":"2024-02-17T19:15:23.873919+0000","172.21.15.163:6827/1325230089":"2024-02-17T19:32:59.315119+0000","172.21.15.40:0/250713212":"2024-02-17T19:24:41.534183+0000","172.21.15.40:0/1423773760":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/726664255":"2024-02-17T19:24:41.534183+0000","172.21.15.40:0/2148905528":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6800/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6827/659310888":"2024-02-17T19:22:12.998828+0000","172.21.15.40:6800/1137681815":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6801/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/1306942330":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2673381923":"2024-02-17T19:15:54.760982+0000","172.21.15.163:0/1579052634":"2024-02-17T19:26:17.176557+0000","172.21.15.40:6826/659310888":"2024-02-17T19:22:12.998828+0000","172.21.15.40:0/647211605":"2024-02-17T19:27:02.146968+0000","172.21.15.163:0/1076569647":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2189950867":"2024-02-17T19:15:54.760982+0000","172.21.15.163:0/2090165668":"2024-02-17T19:26:17.176557+0000","172.21.15.40:6800/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2144454600":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/8215223":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/1412951630":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6800/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/1754668438":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6800/1321006798":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6801/1321006798":"2024-02-17T19:24:41.534183+0000","172.21.15.163:0/841594618":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2009141374":"2024-02-17T19:17:10.018957+0000","172.21.15.163:6828/3793859838":"2024-02-17T19:26:17.176557+0000","172.21.15.163:6829/3793859838":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2888910656":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/1137681815":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1738800778":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/1430644152":"2024-02-17T19:27:02.146968+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:36:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff48406d750 msgr2=0x7ff48406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff48406d750 0x7ff48406fc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff494004c80 tx=0x7ff494009040 comp rx=0 tx=0).stop 2024-02-16T19:36:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 msgr2=0x7ff498113320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 0x7ff498113320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff48802bd80 tx=0x7ff488009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 shutdown_connections 2024-02-16T19:36:35.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff48406d750 0x7ff48406fc00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4980fdea0 0x7ff498113320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 --2- 172.21.15.40:0/1718112806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4980fd500 0x7ff498112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 >> 172.21.15.40:0/1718112806 conn(0x7ff4980f6f80 msgr2=0x7ff498100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:35.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 shutdown_connections 2024-02-16T19:36:35.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.844+0000 7ff49f79d640 1 -- 172.21.15.40:0/1718112806 wait complete. 2024-02-16T19:36:35.855 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:35 smithi040 ceph-mon[94210]: pgmap v323: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:35.855 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:35 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3986486035' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-02-16T19:36:35.855 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:35 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1676012514' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-02-16T19:36:35.855 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:35 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2788016113' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-02-16T19:36:35.856 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:19:22.349243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T19:21:55.970810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T19:21:56.515184+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T19:36:35.857 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T19:36:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 -- 172.21.15.40:0/1383786257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 msgr2=0x7f5d30101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 --2- 172.21.15.40:0/1383786257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 0x7f5d30101610 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0048d0 tx=0x7f5d2c009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 -- 172.21.15.40:0/1383786257 shutdown_connections 2024-02-16T19:36:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 --2- 172.21.15.40:0/1383786257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d30101be0 0x7f5d300fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 --2- 172.21.15.40:0/1383786257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 0x7f5d30101610 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 -- 172.21.15.40:0/1383786257 >> 172.21.15.40:0/1383786257 conn(0x7f5d300f6e30 msgr2=0x7f5d300f9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 -- 172.21.15.40:0/1383786257 shutdown_connections 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 -- 172.21.15.40:0/1383786257 wait complete. 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 Processor -- start 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 -- start start 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 0x7f5d30111b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d30101be0 0x7f5d301120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30112730 con 0x7f5d30101240 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d37d1d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3010bc30 con 0x7f5d30101be0 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d35ab9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 0x7f5d30111b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d35ab9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 0x7f5d30111b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38628/0 (socket says 172.21.15.40:38628) 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.957+0000 7f5d35ab9640 1 -- 172.21.15.40:0/2128305650 learned_addr learned my addr 172.21.15.40:0/2128305650 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d352b8640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d30101be0 0x7f5d301120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d35ab9640 1 -- 172.21.15.40:0/2128305650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d30101be0 msgr2=0x7f5d301120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d35ab9640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d30101be0 0x7f5d301120a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d35ab9640 1 -- 172.21.15.40:0/2128305650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c005040 con 0x7f5d30101240 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d352b8640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d30101be0 0x7f5d301120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d35ab9640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 0x7f5d30111b60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c031720 tx=0x7f5d2c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d267fc640 1 -- 172.21.15.40:0/2128305650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2c0094c0 con 0x7f5d30101240 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d267fc640 1 -- 172.21.15.40:0/2128305650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d2c009620 con 0x7f5d30101240 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3010beb0 con 0x7f5d30101240 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d267fc640 1 -- 172.21.15.40:0/2128305650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2c03b640 con 0x7f5d30101240 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.958+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3010c3d0 con 0x7f5d30101240 2024-02-16T19:36:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.959+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d30064480 con 0x7f5d30101240 2024-02-16T19:36:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.960+0000 7f5d267fc640 1 -- 172.21.15.40:0/2128305650 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5d2c02f070 con 0x7f5d30101240 2024-02-16T19:36:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.960+0000 7f5d267fc640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5d1c06d750 0x7f5d1c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.960+0000 7f5d267fc640 1 -- 172.21.15.40:0/2128305650 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5d2c0af450 con 0x7f5d30101240 2024-02-16T19:36:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.960+0000 7f5d352b8640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5d1c06d750 0x7f5d1c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:35.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.960+0000 7f5d352b8640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5d1c06d750 0x7f5d1c06fc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006fd0 tx=0x7f5d20008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:35.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:35.965+0000 7f5d267fc640 1 -- 172.21.15.40:0/2128305650 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d2c078c70 con 0x7f5d30101240 2024-02-16T19:36:36.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.145+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d3010d240 con 0x7f5d30101240 2024-02-16T19:36:36.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.146+0000 7f5d267fc640 1 -- 172.21.15.40:0/2128305650 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d2c07c330 con 0x7f5d30101240 2024-02-16T19:36:36.147 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-02-16T19:36:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5d1c06d750 msgr2=0x7f5d1c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5d1c06d750 0x7f5d1c06fc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006fd0 tx=0x7f5d20008040 comp rx=0 tx=0).stop 2024-02-16T19:36:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 msgr2=0x7f5d30111b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 0x7f5d30111b60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c031720 tx=0x7f5d2c009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 shutdown_connections 2024-02-16T19:36:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f5d1c06d750 0x7f5d1c06fc00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d30101be0 0x7f5d301120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 --2- 172.21.15.40:0/2128305650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d30101240 0x7f5d30111b60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 >> 172.21.15.40:0/2128305650 conn(0x7f5d300f6e30 msgr2=0x7f5d300f88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 shutdown_connections 2024-02-16T19:36:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.148+0000 7f5d37d1d640 1 -- 172.21.15.40:0/2128305650 wait complete. 2024-02-16T19:36:36.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T19:36:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.259+0000 7ff94ab1d640 1 -- 172.21.15.40:0/2104664777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102430 msgr2=0x7ff944102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.259+0000 7ff94ab1d640 1 --2- 172.21.15.40:0/2104664777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102430 0x7ff944102800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009c50 tx=0x7ff93c004d10 comp rx=0 tx=0).stop 2024-02-16T19:36:36.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.259+0000 7ff94ab1d640 1 -- 172.21.15.40:0/2104664777 shutdown_connections 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.259+0000 7ff94ab1d640 1 --2- 172.21.15.40:0/2104664777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff944102d40 0x7ff9440faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.259+0000 7ff94ab1d640 1 --2- 172.21.15.40:0/2104664777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102430 0x7ff944102800 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.259+0000 7ff94ab1d640 1 -- 172.21.15.40:0/2104664777 >> 172.21.15.40:0/2104664777 conn(0x7ff9440718c0 msgr2=0x7ff944071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.259+0000 7ff94ab1d640 1 -- 172.21.15.40:0/2104664777 shutdown_connections 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.259+0000 7ff94ab1d640 1 -- 172.21.15.40:0/2104664777 wait complete. 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff94ab1d640 1 Processor -- start 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff94ab1d640 1 -- start start 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff94ab1d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff944102430 0x7ff944112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff94ab1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102d40 0x7ff944113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff94ab1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff944113720 con 0x7ff944102d40 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff94ab1d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94410ccb0 con 0x7ff944102430 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff943fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102d40 0x7ff944113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff943fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102d40 0x7ff944113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38642/0 (socket says 172.21.15.40:38642) 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.260+0000 7ff943fff640 1 -- 172.21.15.40:0/915713042 learned_addr learned my addr 172.21.15.40:0/915713042 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:36.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff9488b9640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff944102430 0x7ff944112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff943fff640 1 -- 172.21.15.40:0/915713042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff944102430 msgr2=0x7ff944112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff943fff640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff944102430 0x7ff944112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff943fff640 1 -- 172.21.15.40:0/915713042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c0098b0 con 0x7ff944102d40 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff9488b9640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff944102430 0x7ff944112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff943fff640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102d40 0x7ff944113120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff93800c760 tx=0x7ff9380043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff9417fa640 1 -- 172.21.15.40:0/915713042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff93800a790 con 0x7ff944102d40 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff9417fa640 1 -- 172.21.15.40:0/915713042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff93800adc0 con 0x7ff944102d40 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94410cf90 con 0x7ff944102d40 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff9417fa640 1 -- 172.21.15.40:0/915713042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9380115e0 con 0x7ff944102d40 2024-02-16T19:36:36.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.261+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94410d530 con 0x7ff944102d40 2024-02-16T19:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.262+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff944064480 con 0x7ff944102d40 2024-02-16T19:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.262+0000 7ff9417fa640 1 -- 172.21.15.40:0/915713042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff938012400 con 0x7ff944102d40 2024-02-16T19:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.263+0000 7ff9417fa640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff92c06d750 0x7ff92c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.263+0000 7ff9488b9640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff92c06d750 0x7ff92c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.263+0000 7ff9417fa640 1 -- 172.21.15.40:0/915713042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff93808d130 con 0x7ff944102d40 2024-02-16T19:36:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.263+0000 7ff9488b9640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff92c06d750 0x7ff92c06fc00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02c1b0 tx=0x7ff93c02e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:36.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.267+0000 7ff9417fa640 1 -- 172.21.15.40:0/915713042 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff938056950 con 0x7ff944102d40 2024-02-16T19:36:36.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.446+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff944109540 con 0x7ff944102d40 2024-02-16T19:36:36.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.447+0000 7ff9417fa640 1 -- 172.21.15.40:0/915713042 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff93805a010 con 0x7ff944102d40 2024-02-16T19:36:36.448 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-02-16T19:36:36.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff92c06d750 msgr2=0x7ff92c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff92c06d750 0x7ff92c06fc00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02c1b0 tx=0x7ff93c02e040 comp rx=0 tx=0).stop 2024-02-16T19:36:36.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102d40 msgr2=0x7ff944113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102d40 0x7ff944113120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff93800c760 tx=0x7ff9380043c0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 shutdown_connections 2024-02-16T19:36:36.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff92c06d750 0x7ff92c06fc00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff944102d40 0x7ff944113120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 --2- 172.21.15.40:0/915713042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff944102430 0x7ff944112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 >> 172.21.15.40:0/915713042 conn(0x7ff9440718c0 msgr2=0x7ff9440f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:36.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 shutdown_connections 2024-02-16T19:36:36.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.449+0000 7ff94ab1d640 1 -- 172.21.15.40:0/915713042 wait complete. 2024-02-16T19:36:36.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T19:36:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.560+0000 7f1e19065640 1 -- 172.21.15.40:0/2114677780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 msgr2=0x7f1e141016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.560+0000 7f1e19065640 1 --2- 172.21.15.40:0/2114677780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 0x7f1e141016a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1e040048d0 tx=0x7f1e04009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.560+0000 7f1e19065640 1 -- 172.21.15.40:0/2114677780 shutdown_connections 2024-02-16T19:36:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.560+0000 7f1e19065640 1 --2- 172.21.15.40:0/2114677780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e140fb2b0 0x7f1e140fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.560+0000 7f1e19065640 1 --2- 172.21.15.40:0/2114677780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 0x7f1e141016a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.560+0000 7f1e19065640 1 -- 172.21.15.40:0/2114677780 >> 172.21.15.40:0/2114677780 conn(0x7f1e14071230 msgr2=0x7f1e14071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.560+0000 7f1e19065640 1 -- 172.21.15.40:0/2114677780 shutdown_connections 2024-02-16T19:36:36.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.560+0000 7f1e19065640 1 -- 172.21.15.40:0/2114677780 wait complete. 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e19065640 1 Processor -- start 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e19065640 1 -- start start 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e19065640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e140fb2b0 0x7f1e1410ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e19065640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 0x7f1e1410bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e19065640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1410c510 con 0x7f1e141012d0 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e19065640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1410c650 con 0x7f1e140fb2b0 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e1259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 0x7f1e1410bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e1259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 0x7f1e1410bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38662/0 (socket says 172.21.15.40:38662) 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e1259c640 1 -- 172.21.15.40:0/2161099418 learned_addr learned my addr 172.21.15.40:0/2161099418 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e1259c640 1 -- 172.21.15.40:0/2161099418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e140fb2b0 msgr2=0x7f1e1410ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e12d9d640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e140fb2b0 0x7f1e1410ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e1259c640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e140fb2b0 0x7f1e1410ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.561+0000 7f1e1259c640 1 -- 172.21.15.40:0/2161099418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04005040 con 0x7f1e141012d0 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.562+0000 7f1e1259c640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 0x7f1e1410bfd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800b790 tx=0x7f1e080091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:36.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.562+0000 7f1dfb7fe640 1 -- 172.21.15.40:0/2161099418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0800c700 con 0x7f1e141012d0 2024-02-16T19:36:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.562+0000 7f1dfb7fe640 1 -- 172.21.15.40:0/2161099418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e0800cd30 con 0x7f1e141012d0 2024-02-16T19:36:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.562+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e14119100 con 0x7f1e141012d0 2024-02-16T19:36:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.562+0000 7f1dfb7fe640 1 -- 172.21.15.40:0/2161099418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e08013370 con 0x7f1e141012d0 2024-02-16T19:36:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.562+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e14119430 con 0x7f1e141012d0 2024-02-16T19:36:36.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.563+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e14064030 con 0x7f1e141012d0 2024-02-16T19:36:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.563+0000 7f1dfb7fe640 1 -- 172.21.15.40:0/2161099418 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1e08012070 con 0x7f1e141012d0 2024-02-16T19:36:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.564+0000 7f1dfb7fe640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1dfc06d750 0x7f1dfc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.564+0000 7f1dfb7fe640 1 -- 172.21.15.40:0/2161099418 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1e0808cb50 con 0x7f1e141012d0 2024-02-16T19:36:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.564+0000 7f1e12d9d640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1dfc06d750 0x7f1dfc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.564+0000 7f1e12d9d640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1dfc06d750 0x7f1dfc06fc00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e0402bda0 tx=0x7f1e04009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:36.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.568+0000 7f1dfb7fe640 1 -- 172.21.15.40:0/2161099418 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e080562f0 con 0x7f1e141012d0 2024-02-16T19:36:36.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:36 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1718112806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:36:36.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:36 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2128305650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:36:36.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:36 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/915713042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:36:36.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.748+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f1e14108420 con 0x7f1e141012d0 2024-02-16T19:36:36.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.748+0000 7f1dfb7fe640 1 -- 172.21.15.40:0/2161099418 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1e080599b0 con 0x7f1e141012d0 2024-02-16T19:36:36.749 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-02-16T19:36:36.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1dfc06d750 msgr2=0x7f1dfc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1dfc06d750 0x7f1dfc06fc00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e0402bda0 tx=0x7f1e04009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 msgr2=0x7f1e1410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 0x7f1e1410bfd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800b790 tx=0x7f1e080091d0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 shutdown_connections 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f1dfc06d750 0x7f1dfc06fc00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e141012d0 0x7f1e1410bfd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 --2- 172.21.15.40:0/2161099418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e140fb2b0 0x7f1e1410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 >> 172.21.15.40:0/2161099418 conn(0x7f1e14071230 msgr2=0x7f1e140fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 shutdown_connections 2024-02-16T19:36:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.750+0000 7f1e19065640 1 -- 172.21.15.40:0/2161099418 wait complete. 2024-02-16T19:36:36.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:36 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1718112806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:36:36.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:36 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2128305650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:36:36.761 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:36 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/915713042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:36:36.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-16T19:36:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.860+0000 7f7d4ccae640 1 -- 172.21.15.40:0/249291327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 msgr2=0x7f7d481016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.860+0000 7f7d4ccae640 1 --2- 172.21.15.40:0/249291327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 0x7f7d481016a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7d380048d0 tx=0x7f7d38009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 -- 172.21.15.40:0/249291327 shutdown_connections 2024-02-16T19:36:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 --2- 172.21.15.40:0/249291327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d480fb2b0 0x7f7d480fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 --2- 172.21.15.40:0/249291327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 0x7f7d481016a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 -- 172.21.15.40:0/249291327 >> 172.21.15.40:0/249291327 conn(0x7f7d48071230 msgr2=0x7f7d48071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 -- 172.21.15.40:0/249291327 shutdown_connections 2024-02-16T19:36:36.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 -- 172.21.15.40:0/249291327 wait complete. 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 Processor -- start 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 -- start start 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d480fb2b0 0x7f7d481020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 0x7f7d48102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48104080 con 0x7f7d481012d0 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d4ccae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48102b60 con 0x7f7d480fb2b0 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d45d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 0x7f7d48102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d45d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 0x7f7d48102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38688/0 (socket says 172.21.15.40:38688) 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.861+0000 7f7d45d9b640 1 -- 172.21.15.40:0/3656428217 learned_addr learned my addr 172.21.15.40:0/3656428217 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d4659c640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d480fb2b0 0x7f7d481020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d45d9b640 1 -- 172.21.15.40:0/3656428217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d480fb2b0 msgr2=0x7f7d481020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d45d9b640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d480fb2b0 0x7f7d481020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d45d9b640 1 -- 172.21.15.40:0/3656428217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38005040 con 0x7f7d481012d0 2024-02-16T19:36:36.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d45d9b640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 0x7f7d48102620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00eb70 tx=0x7f7d3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d36ffd640 1 -- 172.21.15.40:0/3656428217 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d3c007c30 con 0x7f7d481012d0 2024-02-16T19:36:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d48102db0 con 0x7f7d481012d0 2024-02-16T19:36:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d36ffd640 1 -- 172.21.15.40:0/3656428217 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d3c00f040 con 0x7f7d481012d0 2024-02-16T19:36:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d36ffd640 1 -- 172.21.15.40:0/3656428217 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d3c0095b0 con 0x7f7d481012d0 2024-02-16T19:36:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.862+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d481192b0 con 0x7f7d481012d0 2024-02-16T19:36:36.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.863+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d48064030 con 0x7f7d481012d0 2024-02-16T19:36:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.863+0000 7f7d36ffd640 1 -- 172.21.15.40:0/3656428217 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7d3c009730 con 0x7f7d481012d0 2024-02-16T19:36:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.864+0000 7f7d36ffd640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7d3006d750 0x7f7d3006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.864+0000 7f7d36ffd640 1 -- 172.21.15.40:0/3656428217 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7d3c08cd10 con 0x7f7d481012d0 2024-02-16T19:36:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.864+0000 7f7d4659c640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7d3006d750 0x7f7d3006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:36.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.864+0000 7f7d4659c640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7d3006d750 0x7f7d3006fc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000c00 tx=0x7f7d38009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:36.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:36.868+0000 7f7d36ffd640 1 -- 172.21.15.40:0/3656428217 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d3c0564b0 con 0x7f7d481012d0 2024-02-16T19:36:37.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.063+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7d481083c0 con 0x7f7d481012d0 2024-02-16T19:36:37.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.064+0000 7f7d36ffd640 1 -- 172.21.15.40:0/3656428217 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v83) v1 ==== 81+0+2428 (secure 0 0 0) 0x7f7d3c059b70 con 0x7f7d481012d0 2024-02-16T19:36:37.065 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:6826/1325230089 2024-02-17T19:32:59.315119+0000 2024-02-16T19:36:37.065 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6829/2154720438 2024-02-17T19:32:20.151991+0000 2024-02-16T19:36:37.065 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6828/2154720438 2024-02-17T19:32:20.151991+0000 2024-02-16T19:36:37.065 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/493553370 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:37.065 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/4011363422 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:37.065 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1058311844 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:6827/1325230089 2024-02-17T19:32:59.315119+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/250713212 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1423773760 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/726664255 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2148905528 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/2140771334 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6827/659310888 2024-02-17T19:22:12.998828+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1137681815 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/2140771334 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/277905681 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1306942330 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2673381923 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:0/1579052634 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6826/659310888 2024-02-17T19:22:12.998828+0000 2024-02-16T19:36:37.066 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/647211605 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:0/1076569647 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2189950867 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:0/2090165668 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1383032518 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2144454600 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/8215223 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1412951630 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/277905681 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1754668438 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1321006798 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1321006798 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:0/841594618 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2009141374 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:6828/3793859838 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:6829/3793859838 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:37.067 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2888910656 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:37.068 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1137681815 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:37.068 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1383032518 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:37.068 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1738800778 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:37.068 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1430644152 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:37.068 INFO:teuthology.orchestra.run.smithi040.stderr:listed 41 entries 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.065+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7d3006d750 msgr2=0x7f7d3006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.065+0000 7f7d4ccae640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7d3006d750 0x7f7d3006fc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000c00 tx=0x7f7d38009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.065+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 msgr2=0x7f7d48102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.065+0000 7f7d4ccae640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 0x7f7d48102620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00eb70 tx=0x7f7d3c00b040 comp rx=0 tx=0).stop 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.066+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 shutdown_connections 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.066+0000 7f7d4ccae640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f7d3006d750 0x7f7d3006fc00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.066+0000 7f7d4ccae640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d481012d0 0x7f7d48102620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.066+0000 7f7d4ccae640 1 --2- 172.21.15.40:0/3656428217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d480fb2b0 0x7f7d481020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.066+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 >> 172.21.15.40:0/3656428217 conn(0x7f7d48071230 msgr2=0x7f7d480644c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.066+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 shutdown_connections 2024-02-16T19:36:37.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.066+0000 7f7d4ccae640 1 -- 172.21.15.40:0/3656428217 wait complete. 2024-02-16T19:36:37.078 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-16T19:36:37.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:37.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-16T19:36:37.106 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:36:37.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:37.106 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-16T19:36:37.177 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:36:37.177 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:36:37.198 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-02-16T19:36:37.198 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-16T19:36:37.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-16T19:36:37.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.356+0000 7f5606f9d640 1 -- 172.21.15.40:0/1109036130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 msgr2=0x7f5600101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.356+0000 7f5606f9d640 1 --2- 172.21.15.40:0/1109036130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 0x7f5600101570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009c50 tx=0x7f55f8004d10 comp rx=0 tx=0).stop 2024-02-16T19:36:37.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.356+0000 7f5606f9d640 1 -- 172.21.15.40:0/1109036130 shutdown_connections 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.356+0000 7f5606f9d640 1 --2- 172.21.15.40:0/1109036130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5600101b40 0x7f56000fb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.356+0000 7f5606f9d640 1 --2- 172.21.15.40:0/1109036130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 0x7f5600101570 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.356+0000 7f5606f9d640 1 -- 172.21.15.40:0/1109036130 >> 172.21.15.40:0/1109036130 conn(0x7f5600071290 msgr2=0x7f5600071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.356+0000 7f5606f9d640 1 -- 172.21.15.40:0/1109036130 shutdown_connections 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.356+0000 7f5606f9d640 1 -- 172.21.15.40:0/1109036130 wait complete. 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5606f9d640 1 Processor -- start 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5606f9d640 1 -- start start 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5606f9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 0x7f560010dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5606f9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5600101b40 0x7f560010e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5606f9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560010e790 con 0x7f56001011a0 2024-02-16T19:36:37.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5606f9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560010e900 con 0x7f5600101b40 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5604d39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 0x7f560010dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5604d39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 0x7f560010dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38702/0 (socket says 172.21.15.40:38702) 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.357+0000 7f5604d39640 1 -- 172.21.15.40:0/2672348591 learned_addr learned my addr 172.21.15.40:0/2672348591 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f5604d39640 1 -- 172.21.15.40:0/2672348591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5600101b40 msgr2=0x7f560010e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f5604d39640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5600101b40 0x7f560010e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f5604d39640 1 -- 172.21.15.40:0/2672348591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f80098b0 con 0x7f56001011a0 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f5604d39640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 0x7f560010dd10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f55f8003400 tx=0x7f55f8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f55fd7fa640 1 -- 172.21.15.40:0/2672348591 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55f803a070 con 0x7f56001011a0 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f55fd7fa640 1 -- 172.21.15.40:0/2672348591 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55f8035b20 con 0x7f56001011a0 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560010eb80 con 0x7f56001011a0 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f55fd7fa640 1 -- 172.21.15.40:0/2672348591 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55f803eb60 con 0x7f56001011a0 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.358+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5600117110 con 0x7f56001011a0 2024-02-16T19:36:37.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.359+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5600064030 con 0x7f56001011a0 2024-02-16T19:36:37.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.359+0000 7f55fd7fa640 1 -- 172.21.15.40:0/2672348591 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f55f8032070 con 0x7f56001011a0 2024-02-16T19:36:37.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.360+0000 7f55fd7fa640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55e806d750 0x7f55e806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.360+0000 7f55fd7fa640 1 -- 172.21.15.40:0/2672348591 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f55f80af140 con 0x7f56001011a0 2024-02-16T19:36:37.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.360+0000 7f55fffff640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55e806d750 0x7f55e806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:37.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.360+0000 7f55fffff640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55e806d750 0x7f55e806fc00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5600064d30 tx=0x7f55f4006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:37.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.364+0000 7f55fd7fa640 1 -- 172.21.15.40:0/2672348591 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55f80788e0 con 0x7f56001011a0 2024-02-16T19:36:37.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.550+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f560010eb20 con 0x7f56001011a0 2024-02-16T19:36:37.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.551+0000 7f55fd7fa640 1 -- 172.21.15.40:0/2672348591 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16756 (secure 0 0 0) 0x7f55f807bfa0 con 0x7f56001011a0 2024-02-16T19:36:37.552 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-02-16T19:36:37.552 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"8c23bd6c-ccff-11ee-95ba-87774f69a715","created":"2024-02-16T19:14:55.436141+0000","modified":"2024-02-16T19:32:59.315148+0000","last_up_change":"2024-02-16T19:31:25.855269+0000","last_in_change":"2024-02-16T19:20:17.443690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-16T19:19:22.349243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-16T19:21:55.970810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-16T19:21:56.515184+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db82e60f-540c-4f54-8f54-758be43e2cf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6803","nonce":3792625162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6805","nonce":3792625162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6809","nonce":3792625162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3792625162},{"type":"v1","addr":"172.21.15.40:6807","nonce":3792625162}]},"public_addr":"172.21.15.40:6803/3792625162","cluster_addr":"172.21.15.40:6805/3792625162","heartbeat_back_addr":"172.21.15.40:6809/3792625162","heartbeat_front_addr":"172.21.15.40:6807/3792625162","state":["exists","up"]},{"osd":1,"uuid":"22561508-cdb8-4c76-ab07-bf3efd6ee63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6811","nonce":3029722765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6813","nonce":3029722765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6817","nonce":3029722765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3029722765},{"type":"v1","addr":"172.21.15.40:6815","nonce":3029722765}]},"public_addr":"172.21.15.40:6811/3029722765","cluster_addr":"172.21.15.40:6813/3029722765","heartbeat_back_addr":"172.21.15.40:6817/3029722765","heartbeat_front_addr":"172.21.15.40:6815/3029722765","state":["exists","up"]},{"osd":2,"uuid":"58ff6654-72f1-4329-8f6c-4c3180353755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6819","nonce":1990065416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6821","nonce":1990065416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6825","nonce":1990065416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":1990065416},{"type":"v1","addr":"172.21.15.40:6823","nonce":1990065416}]},"public_addr":"172.21.15.40:6819/1990065416","cluster_addr":"172.21.15.40:6821/1990065416","heartbeat_back_addr":"172.21.15.40:6825/1990065416","heartbeat_front_addr":"172.21.15.40:6823/1990065416","state":["exists","up"]},{"osd":3,"uuid":"b6f2b76f-f1fa-4459-8150-c36e624fb4aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6801","nonce":346064807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6803","nonce":346064807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6807","nonce":346064807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":346064807},{"type":"v1","addr":"172.21.15.163:6805","nonce":346064807}]},"public_addr":"172.21.15.163:6801/346064807","cluster_addr":"172.21.15.163:6803/346064807","heartbeat_back_addr":"172.21.15.163:6807/346064807","heartbeat_front_addr":"172.21.15.163:6805/346064807","state":["exists","up"]},{"osd":4,"uuid":"49afe3d8-2685-4c11-9830-77bd72a0b272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6809","nonce":2382516830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6811","nonce":2382516830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6815","nonce":2382516830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":2382516830},{"type":"v1","addr":"172.21.15.163:6813","nonce":2382516830}]},"public_addr":"172.21.15.163:6809/2382516830","cluster_addr":"172.21.15.163:6811/2382516830","heartbeat_back_addr":"172.21.15.163:6815/2382516830","heartbeat_front_addr":"172.21.15.163:6813/2382516830","state":["exists","up"]},{"osd":5,"uuid":"c2cd007f-6c60-4086-aa2d-fa1b5cf91eb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6817","nonce":10307935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6819","nonce":10307935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6823","nonce":10307935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":10307935},{"type":"v1","addr":"172.21.15.163:6821","nonce":10307935}]},"public_addr":"172.21.15.163:6817/10307935","cluster_addr":"172.21.15.163:6819/10307935","heartbeat_back_addr":"172.21.15.163:6823/10307935","heartbeat_front_addr":"172.21.15.163:6821/10307935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:22.484106+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:18:52.190602+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:19.421062+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:19:43.759349+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:09.243860+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-16T19:20:34.636467+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6826/1325230089":"2024-02-17T19:32:59.315119+0000","172.21.15.40:6829/2154720438":"2024-02-17T19:32:20.151991+0000","172.21.15.40:6828/2154720438":"2024-02-17T19:32:20.151991+0000","172.21.15.40:0/493553370":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/4011363422":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1058311844":"2024-02-17T19:15:23.873919+0000","172.21.15.163:6827/1325230089":"2024-02-17T19:32:59.315119+0000","172.21.15.40:0/250713212":"2024-02-17T19:24:41.534183+0000","172.21.15.40:0/1423773760":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/726664255":"2024-02-17T19:24:41.534183+0000","172.21.15.40:0/2148905528":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6800/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6827/659310888":"2024-02-17T19:22:12.998828+0000","172.21.15.40:6800/1137681815":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/2140771334":"2024-02-17T19:15:54.760982+0000","172.21.15.40:6801/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/1306942330":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/2673381923":"2024-02-17T19:15:54.760982+0000","172.21.15.163:0/1579052634":"2024-02-17T19:26:17.176557+0000","172.21.15.40:6826/659310888":"2024-02-17T19:22:12.998828+0000","172.21.15.40:0/647211605":"2024-02-17T19:27:02.146968+0000","172.21.15.163:0/1076569647":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2189950867":"2024-02-17T19:15:54.760982+0000","172.21.15.163:0/2090165668":"2024-02-17T19:26:17.176557+0000","172.21.15.40:6800/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/2144454600":"2024-02-17T19:15:54.760982+0000","172.21.15.40:0/8215223":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/1412951630":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6800/277905681":"2024-02-17T19:17:10.018957+0000","172.21.15.40:0/1754668438":"2024-02-17T19:17:10.018957+0000","172.21.15.40:6800/1321006798":"2024-02-17T19:24:41.534183+0000","172.21.15.40:6801/1321006798":"2024-02-17T19:24:41.534183+0000","172.21.15.163:0/841594618":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2009141374":"2024-02-17T19:17:10.018957+0000","172.21.15.163:6828/3793859838":"2024-02-17T19:26:17.176557+0000","172.21.15.163:6829/3793859838":"2024-02-17T19:26:17.176557+0000","172.21.15.40:0/2888910656":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/1137681815":"2024-02-17T19:27:02.146968+0000","172.21.15.40:6801/1383032518":"2024-02-17T19:15:23.873919+0000","172.21.15.40:0/1738800778":"2024-02-17T19:27:02.146968+0000","172.21.15.40:0/1430644152":"2024-02-17T19:27:02.146968+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-16T19:36:37.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55e806d750 msgr2=0x7f55e806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55e806d750 0x7f55e806fc00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5600064d30 tx=0x7f55f4006d20 comp rx=0 tx=0).stop 2024-02-16T19:36:37.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 msgr2=0x7f560010dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 0x7f560010dd10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f55f8003400 tx=0x7f55f8004d10 comp rx=0 tx=0).stop 2024-02-16T19:36:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 shutdown_connections 2024-02-16T19:36:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f55e806d750 0x7f55e806fc00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5600101b40 0x7f560010e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 --2- 172.21.15.40:0/2672348591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56001011a0 0x7f560010dd10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 >> 172.21.15.40:0/2672348591 conn(0x7f5600071290 msgr2=0x7f56000fa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 shutdown_connections 2024-02-16T19:36:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.553+0000 7f5606f9d640 1 -- 172.21.15.40:0/2672348591 wait complete. 2024-02-16T19:36:37.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-16T19:19:22.349243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-16T19:21:55.970810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-16T19:21:56.515184+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-16T19:36:37.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-16T19:36:37.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.667+0000 7facb3e6e640 1 -- 172.21.15.40:0/2789787168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac1023a0 msgr2=0x7facac102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.667+0000 7facb3e6e640 1 --2- 172.21.15.40:0/2789787168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac1023a0 0x7facac102770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faca40048d0 tx=0x7faca4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.667+0000 7facb3e6e640 1 -- 172.21.15.40:0/2789787168 shutdown_connections 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.667+0000 7facb3e6e640 1 --2- 172.21.15.40:0/2789787168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facac102d40 0x7facac0fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.667+0000 7facb3e6e640 1 --2- 172.21.15.40:0/2789787168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac1023a0 0x7facac102770 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.667+0000 7facb3e6e640 1 -- 172.21.15.40:0/2789787168 >> 172.21.15.40:0/2789787168 conn(0x7facac0f7f90 msgr2=0x7facac0fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.667+0000 7facb3e6e640 1 -- 172.21.15.40:0/2789787168 shutdown_connections 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.667+0000 7facb3e6e640 1 -- 172.21.15.40:0/2789787168 wait complete. 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb3e6e640 1 Processor -- start 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb3e6e640 1 -- start start 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb3e6e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facac1023a0 0x7facac112f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb3e6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac102d40 0x7facac113450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb3e6e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac113a50 con 0x7facac102d40 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb3e6e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac10cfe0 con 0x7facac1023a0 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb1409640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac102d40 0x7facac113450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb1409640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac102d40 0x7facac113450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39420/0 (socket says 172.21.15.40:39420) 2024-02-16T19:36:37.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb1409640 1 -- 172.21.15.40:0/3003422011 learned_addr learned my addr 172.21.15.40:0/3003422011 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb1c0a640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facac1023a0 0x7facac112f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb1409640 1 -- 172.21.15.40:0/3003422011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facac1023a0 msgr2=0x7facac112f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb1409640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facac1023a0 0x7facac112f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.668+0000 7facb1409640 1 -- 172.21.15.40:0/3003422011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca4005040 con 0x7facac102d40 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.669+0000 7facb1409640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac102d40 0x7facac113450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac9c002a50 tx=0x7fac9c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.669+0000 7faca27fc640 1 -- 172.21.15.40:0/3003422011 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9c007c30 con 0x7facac102d40 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.669+0000 7faca27fc640 1 -- 172.21.15.40:0/3003422011 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac9c00f040 con 0x7facac102d40 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.669+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac10d2c0 con 0x7facac102d40 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.669+0000 7faca27fc640 1 -- 172.21.15.40:0/3003422011 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9c00db50 con 0x7facac102d40 2024-02-16T19:36:37.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.669+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac10d7e0 con 0x7facac102d40 2024-02-16T19:36:37.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.670+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac064030 con 0x7facac102d40 2024-02-16T19:36:37.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.670+0000 7faca27fc640 1 -- 172.21.15.40:0/3003422011 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fac9c01c400 con 0x7facac102d40 2024-02-16T19:36:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.670+0000 7faca27fc640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fac9806d750 0x7fac9806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.671+0000 7faca27fc640 1 -- 172.21.15.40:0/3003422011 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fac9c08c860 con 0x7facac102d40 2024-02-16T19:36:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.671+0000 7facb1c0a640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fac9806d750 0x7fac9806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.671+0000 7facb1c0a640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fac9806d750 0x7fac9806fc00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faca4004540 tx=0x7faca4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:37.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.675+0000 7faca27fc640 1 -- 172.21.15.40:0/3003422011 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac9c056000 con 0x7facac102d40 2024-02-16T19:36:37.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:37 smithi163 ceph-mon[81637]: pgmap v324: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:37.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:37 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2161099418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:36:37.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:37 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3656428217' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T19:36:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.856+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facac10e510 con 0x7facac102d40 2024-02-16T19:36:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.856+0000 7faca27fc640 1 -- 172.21.15.40:0/3003422011 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac9c0596c0 con 0x7facac102d40 2024-02-16T19:36:37.857 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-02-16T19:36:37.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fac9806d750 msgr2=0x7fac9806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fac9806d750 0x7fac9806fc00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faca4004540 tx=0x7faca4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac102d40 msgr2=0x7facac113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac102d40 0x7facac113450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac9c002a50 tx=0x7fac9c004600 comp rx=0 tx=0).stop 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 shutdown_connections 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fac9806d750 0x7fac9806fc00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facac102d40 0x7facac113450 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 --2- 172.21.15.40:0/3003422011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facac1023a0 0x7facac112f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 >> 172.21.15.40:0/3003422011 conn(0x7facac0f7f90 msgr2=0x7facac0f9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 shutdown_connections 2024-02-16T19:36:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.858+0000 7facb3e6e640 1 -- 172.21.15.40:0/3003422011 wait complete. 2024-02-16T19:36:37.869 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:37 smithi040 ceph-mon[94210]: pgmap v324: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail 2024-02-16T19:36:37.869 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:37 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2161099418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:36:37.869 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:37 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3656428217' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T19:36:37.870 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-16T19:36:37.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 -- 172.21.15.40:0/4261056945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 msgr2=0x7f50d40fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 --2- 172.21.15.40:0/4261056945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 0x7f50d40fbfe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50c40048d0 tx=0x7f50c4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 -- 172.21.15.40:0/4261056945 shutdown_connections 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 --2- 172.21.15.40:0/4261056945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 0x7f50d40fbfe0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 --2- 172.21.15.40:0/4261056945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d4101330 0x7f50d4101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 -- 172.21.15.40:0/4261056945 >> 172.21.15.40:0/4261056945 conn(0x7f50d40718c0 msgr2=0x7f50d4071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 -- 172.21.15.40:0/4261056945 shutdown_connections 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 -- 172.21.15.40:0/4261056945 wait complete. 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 Processor -- start 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 -- start start 2024-02-16T19:36:37.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 0x7f50d410bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d4101330 0x7f50d410c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40fba40 con 0x7f50d40fbb90 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50dc11d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d410c5b0 con 0x7f50d4101330 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50d9eb9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 0x7f50d410bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.970+0000 7f50d9eb9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 0x7f50d410bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39436/0 (socket says 172.21.15.40:39436) 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50d9eb9640 1 -- 172.21.15.40:0/1660576961 learned_addr learned my addr 172.21.15.40:0/1660576961 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50d96b8640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d4101330 0x7f50d410c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50d9eb9640 1 -- 172.21.15.40:0/1660576961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d4101330 msgr2=0x7f50d410c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50d9eb9640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d4101330 0x7f50d410c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50d9eb9640 1 -- 172.21.15.40:0/1660576961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c4005040 con 0x7f50d40fbb90 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50d9eb9640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 0x7f50d410bb30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50d000b790 tx=0x7f50d00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50ca7fc640 1 -- 172.21.15.40:0/1660576961 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50d000c700 con 0x7f50d40fbb90 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50ca7fc640 1 -- 172.21.15.40:0/1660576961 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50d000cd30 con 0x7f50d40fbb90 2024-02-16T19:36:37.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d410c890 con 0x7f50d40fbb90 2024-02-16T19:36:37.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50ca7fc640 1 -- 172.21.15.40:0/1660576961 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50d0013370 con 0x7f50d40fbb90 2024-02-16T19:36:37.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.971+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d41193f0 con 0x7f50d40fbb90 2024-02-16T19:36:37.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.972+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d4064480 con 0x7f50d40fbb90 2024-02-16T19:36:37.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.973+0000 7f50ca7fc640 1 -- 172.21.15.40:0/1660576961 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f50d0012070 con 0x7f50d40fbb90 2024-02-16T19:36:37.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.973+0000 7f50ca7fc640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f50c006d750 0x7f50c006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:37.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.973+0000 7f50d96b8640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f50c006d750 0x7f50c006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:37.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.973+0000 7f50ca7fc640 1 -- 172.21.15.40:0/1660576961 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f50d008ca60 con 0x7f50d40fbb90 2024-02-16T19:36:37.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.974+0000 7f50d96b8640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f50c006d750 0x7f50c006fc00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000c00 tx=0x7f50c4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:37.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:37.978+0000 7f50ca7fc640 1 -- 172.21.15.40:0/1660576961 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f50d0056280 con 0x7f50d40fbb90 2024-02-16T19:36:38.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.157+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f50d40fb1a0 con 0x7f50d40fbb90 2024-02-16T19:36:38.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.158+0000 7f50ca7fc640 1 -- 172.21.15.40:0/1660576961 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f50d0059940 con 0x7f50d40fbb90 2024-02-16T19:36:38.159 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-02-16T19:36:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f50c006d750 msgr2=0x7f50c006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f50c006d750 0x7f50c006fc00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000c00 tx=0x7f50c4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 msgr2=0x7f50d410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 0x7f50d410bb30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50d000b790 tx=0x7f50d00091d0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 shutdown_connections 2024-02-16T19:36:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7f50c006d750 0x7f50c006fc00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d4101330 0x7f50d410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 --2- 172.21.15.40:0/1660576961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50d40fbb90 0x7f50d410bb30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 >> 172.21.15.40:0/1660576961 conn(0x7f50d40718c0 msgr2=0x7f50d40f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:38.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 shutdown_connections 2024-02-16T19:36:38.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.160+0000 7f50dc11d640 1 -- 172.21.15.40:0/1660576961 wait complete. 2024-02-16T19:36:38.172 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 -- 172.21.15.40:0/4151773847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f00faa80 msgr2=0x7ff1f00faef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 --2- 172.21.15.40:0/4151773847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f00faa80 0x7ff1f00faef0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40048d0 tx=0x7ff1e4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 -- 172.21.15.40:0/4151773847 shutdown_connections 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 --2- 172.21.15.40:0/4151773847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f00faa80 0x7ff1f00faef0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 --2- 172.21.15.40:0/4151773847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f0102410 0x7ff1f01027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 -- 172.21.15.40:0/4151773847 >> 172.21.15.40:0/4151773847 conn(0x7ff1f00718c0 msgr2=0x7ff1f0071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 -- 172.21.15.40:0/4151773847 shutdown_connections 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 -- 172.21.15.40:0/4151773847 wait complete. 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 Processor -- start 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.272+0000 7ff1f58ae640 1 -- start start 2024-02-16T19:36:38.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1f58ae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f00faa80 0x7ff1f0112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1f58ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f0102410 0x7ff1f0113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1f58ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0113760 con 0x7ff1f0102410 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1f58ae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f010ccf0 con 0x7ff1f00faa80 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f0102410 0x7ff1f0113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1ee7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f0102410 0x7ff1f0113160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39454/0 (socket says 172.21.15.40:39454) 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1ee7fc640 1 -- 172.21.15.40:0/2074135318 learned_addr learned my addr 172.21.15.40:0/2074135318 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1eeffd640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f00faa80 0x7ff1f0112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1ee7fc640 1 -- 172.21.15.40:0/2074135318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f00faa80 msgr2=0x7ff1f0112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1ee7fc640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f00faa80 0x7ff1f0112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1ee7fc640 1 -- 172.21.15.40:0/2074135318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e4005040 con 0x7ff1f0102410 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1ee7fc640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f0102410 0x7ff1f0113160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4031720 tx=0x7ff1e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1d77fe640 1 -- 172.21.15.40:0/2074135318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1e40094c0 con 0x7ff1f0102410 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1d77fe640 1 -- 172.21.15.40:0/2074135318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1e4009620 con 0x7ff1f0102410 2024-02-16T19:36:38.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f010cf70 con 0x7ff1f0102410 2024-02-16T19:36:38.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1d77fe640 1 -- 172.21.15.40:0/2074135318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1e403b640 con 0x7ff1f0102410 2024-02-16T19:36:38.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.273+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f010d490 con 0x7ff1f0102410 2024-02-16T19:36:38.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.274+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f0064480 con 0x7ff1f0102410 2024-02-16T19:36:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.275+0000 7ff1d77fe640 1 -- 172.21.15.40:0/2074135318 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff1e402f070 con 0x7ff1f0102410 2024-02-16T19:36:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.275+0000 7ff1d77fe640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff1d806d750 0x7ff1d806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.275+0000 7ff1d77fe640 1 -- 172.21.15.40:0/2074135318 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff1e40af470 con 0x7ff1f0102410 2024-02-16T19:36:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.275+0000 7ff1eeffd640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff1d806d750 0x7ff1d806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:38.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.276+0000 7ff1eeffd640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff1d806d750 0x7ff1d806fc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009470 tx=0x7ff1e0009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:38.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.280+0000 7ff1d77fe640 1 -- 172.21.15.40:0/2074135318 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1e4078b70 con 0x7ff1f0102410 2024-02-16T19:36:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.459+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff1f0109560 con 0x7ff1f0102410 2024-02-16T19:36:38.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.460+0000 7ff1d77fe640 1 -- 172.21.15.40:0/2074135318 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff1e407c230 con 0x7ff1f0102410 2024-02-16T19:36:38.461 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-02-16T19:36:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff1d806d750 msgr2=0x7ff1d806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff1d806d750 0x7ff1d806fc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009470 tx=0x7ff1e0009ed0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f0102410 msgr2=0x7ff1f0113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f0102410 0x7ff1f0113160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4031720 tx=0x7ff1e4009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 shutdown_connections 2024-02-16T19:36:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7ff1d806d750 0x7ff1d806fc00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f0102410 0x7ff1f0113160 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 --2- 172.21.15.40:0/2074135318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1f00faa80 0x7ff1f0112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 >> 172.21.15.40:0/2074135318 conn(0x7ff1f00718c0 msgr2=0x7ff1f01053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:38.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 shutdown_connections 2024-02-16T19:36:38.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.462+0000 7ff1f58ae640 1 -- 172.21.15.40:0/2074135318 wait complete. 2024-02-16T19:36:38.474 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-16T19:36:38.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.572+0000 7fee3ba2e640 1 -- 172.21.15.40:0/3590953508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 msgr2=0x7fee340faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.572+0000 7fee3ba2e640 1 --2- 172.21.15.40:0/3590953508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 0x7fee340faee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fee240048d0 tx=0x7fee24009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 -- 172.21.15.40:0/3590953508 shutdown_connections 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 --2- 172.21.15.40:0/3590953508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 0x7fee340faee0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 --2- 172.21.15.40:0/3590953508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee340fe300 0x7fee340fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 -- 172.21.15.40:0/3590953508 >> 172.21.15.40:0/3590953508 conn(0x7fee34071950 msgr2=0x7fee34071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 -- 172.21.15.40:0/3590953508 shutdown_connections 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 -- 172.21.15.40:0/3590953508 wait complete. 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 Processor -- start 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 -- start start 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 0x7fee34114e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee340fe300 0x7fee341153a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee34115a30 con 0x7fee340faa70 2024-02-16T19:36:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.573+0000 7fee3ba2e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3410ef30 con 0x7fee340fe300 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee397ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 0x7fee34114e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee397ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 0x7fee34114e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39474/0 (socket says 172.21.15.40:39474) 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee397ca640 1 -- 172.21.15.40:0/237270260 learned_addr learned my addr 172.21.15.40:0/237270260 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee38fc9640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee340fe300 0x7fee341153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee397ca640 1 -- 172.21.15.40:0/237270260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee340fe300 msgr2=0x7fee341153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee397ca640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee340fe300 0x7fee341153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee397ca640 1 -- 172.21.15.40:0/237270260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24005040 con 0x7fee340faa70 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee397ca640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 0x7fee34114e60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00b790 tx=0x7fee2c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee29ffb640 1 -- 172.21.15.40:0/237270260 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee2c00c700 con 0x7fee340faa70 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee29ffb640 1 -- 172.21.15.40:0/237270260 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee2c00cd30 con 0x7fee340faa70 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3410f210 con 0x7fee340faa70 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee29ffb640 1 -- 172.21.15.40:0/237270260 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee2c013370 con 0x7fee340faa70 2024-02-16T19:36:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.574+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3410f700 con 0x7fee340faa70 2024-02-16T19:36:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.575+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee34064480 con 0x7fee340faa70 2024-02-16T19:36:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.576+0000 7fee29ffb640 1 -- 172.21.15.40:0/237270260 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fee2c012070 con 0x7fee340faa70 2024-02-16T19:36:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.576+0000 7fee29ffb640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fee2006d750 0x7fee2006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-16T19:36:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.576+0000 7fee29ffb640 1 -- 172.21.15.40:0/237270260 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fee2c08c9e0 con 0x7fee340faa70 2024-02-16T19:36:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.576+0000 7fee38fc9640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fee2006d750 0x7fee2006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-16T19:36:38.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.577+0000 7fee38fc9640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fee2006d750 0x7fee2006fc00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee34110070 tx=0x7fee24009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-16T19:36:38.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.581+0000 7fee29ffb640 1 -- 172.21.15.40:0/237270260 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee2c056200 con 0x7fee340faa70 2024-02-16T19:36:38.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:38 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2672348591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:36:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:38 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/3003422011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:36:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:38 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/1660576961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:36:38.738 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:38 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/2074135318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:36:38.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.773+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fee34110460 con 0x7fee340faa70 2024-02-16T19:36:38.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.774+0000 7fee29ffb640 1 -- 172.21.15.40:0/237270260 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v83) v1 ==== 81+0+2428 (secure 0 0 0) 0x7fee2c0598c0 con 0x7fee340faa70 2024-02-16T19:36:38.775 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:6826/1325230089 2024-02-17T19:32:59.315119+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6829/2154720438 2024-02-17T19:32:20.151991+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6828/2154720438 2024-02-17T19:32:20.151991+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/493553370 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/4011363422 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1058311844 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:6827/1325230089 2024-02-17T19:32:59.315119+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/250713212 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1423773760 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/726664255 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2148905528 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/2140771334 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:38.776 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6827/659310888 2024-02-17T19:22:12.998828+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1137681815 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/2140771334 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/277905681 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1306942330 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2673381923 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:0/1579052634 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6826/659310888 2024-02-17T19:22:12.998828+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/647211605 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:0/1076569647 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2189950867 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:0/2090165668 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1383032518 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2144454600 2024-02-17T19:15:54.760982+0000 2024-02-16T19:36:38.777 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/8215223 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1412951630 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/277905681 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1754668438 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1321006798 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1321006798 2024-02-17T19:24:41.534183+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:0/841594618 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2009141374 2024-02-17T19:17:10.018957+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:6828/3793859838 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.163:6829/3793859838 2024-02-17T19:26:17.176557+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2888910656 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1137681815 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:38.778 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1383032518 2024-02-17T19:15:23.873919+0000 2024-02-16T19:36:38.779 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1738800778 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:38.779 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1430644152 2024-02-17T19:27:02.146968+0000 2024-02-16T19:36:38.779 INFO:teuthology.orchestra.run.smithi040.stderr:listed 41 entries 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fee2006d750 msgr2=0x7fee2006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fee2006d750 0x7fee2006fc00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee34110070 tx=0x7fee24009da0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 msgr2=0x7fee34114e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 0x7fee34114e60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00b790 tx=0x7fee2c0091d0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 shutdown_connections 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:6800/1911827957,v1:172.21.15.40:6801/1911827957] conn(0x7fee2006d750 0x7fee2006fc00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee340fe300 0x7fee341153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 --2- 172.21.15.40:0/237270260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee340faa70 0x7fee34114e60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 >> 172.21.15.40:0/237270260 conn(0x7fee34071950 msgr2=0x7fee340f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 shutdown_connections 2024-02-16T19:36:38.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-02-16T19:36:38.776+0000 7fee3ba2e640 1 -- 172.21.15.40:0/237270260 wait complete. 2024-02-16T19:36:38.787 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:38 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2672348591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-16T19:36:38.787 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:38 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/3003422011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-16T19:36:38.787 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:38 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/1660576961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-16T19:36:38.787 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:38 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/2074135318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-16T19:36:38.788 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-16T19:36:38.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:38.788 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-16T19:36:38.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2024-02-16T19:36:38.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:38.824 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-16T19:36:38.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:38.894 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-02-16T19:36:38.909 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-16T19:36:38.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:38.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-16T19:36:38.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:38.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-02-16T19:36:39.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:39.072 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-02-16T19:36:39.090 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-16T19:36:39.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:39.090 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-16T19:36:39.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-16T19:36:39.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-02-16T19:36:39.259 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-16T19:36:39.270 ERROR:teuthology.run_tasks:Manager failed: full_sequential_finally Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 43, in task ((taskname, confg),) = entry.items() ValueError: too many values to unpack (expected 1) 2024-02-16T19:36:39.271 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-16T19:36:39.300 INFO:tasks.cephadm:Teardown begin 2024-02-16T19:36:39.300 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_ed7ab725e1a43615009dfb74a42bf33d80081bc8/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 43, in task ((taskname, confg),) = entry.items() ValueError: too many values to unpack (expected 1) 2024-02-16T19:36:39.301 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:36:39.331 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:36:39.361 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-16T19:36:39.361 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-16T19:36:39.385 DEBUG:teuthology.orchestra.run.smithi163:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-16T19:36:39.416 INFO:tasks.cephadm:Stopping all daemons... 2024-02-16T19:36:39.416 INFO:tasks.cephadm.mon.smithi040:Stopping mon.smithi040... 2024-02-16T19:36:39.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040 2024-02-16T19:36:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:39 smithi040 systemd[1]: Stopping Ceph mon.smithi040 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:36:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:39 smithi040 ceph-mon[94210]: pgmap v325: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-02-16T19:36:39.644 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:39 smithi040 ceph-mon[94210]: from='client.? 172.21.15.40:0/237270260' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T19:36:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:39 smithi163 ceph-mon[81637]: pgmap v325: 65 pgs: 65 active+clean; 171 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-02-16T19:36:39.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:39 smithi163 ceph-mon[81637]: from='client.? 172.21.15.40:0/237270260' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-16T19:36:40.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:39 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[94205]: 2024-02-16T19:36:39.755+0000 7fd7031bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi040 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:36:40.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:39 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040[94205]: 2024-02-16T19:36:39.755+0000 7fd7031bc700 -1 mon.smithi040@0(leader) e3 *** Got Signal Terminated *** 2024-02-16T19:36:40.144 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:39 smithi040 podman[150539]: 2024-02-16 19:36:39.960602349 +0000 UTC m=+0.342839255 container died 1c1011f60db65020aba4245f7749afcb4a16a678cb3e2f4689b6113ca39fafa3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-02-16T19:36:40.599 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:40 smithi040 podman[150539]: 2024-02-16 19:36:40.331247702 +0000 UTC m=+0.713484607 container cleanup 1c1011f60db65020aba4245f7749afcb4a16a678cb3e2f4689b6113ca39fafa3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-16T19:36:40.599 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Feb 16 19:36:40 smithi040 bash[150539]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi040 2024-02-16T19:36:40.770 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi040.service' 2024-02-16T19:36:40.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:36:40.813 INFO:tasks.cephadm.mon.smithi040:Stopped mon.smithi040 2024-02-16T19:36:40.813 INFO:tasks.cephadm.mon.smithi163:Stopping mon.smithi163... 2024-02-16T19:36:40.813 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi163 2024-02-16T19:36:41.129 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:40 smithi163 systemd[1]: Stopping Ceph mon.smithi163 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:36:41.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:41 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163[81633]: 2024-02-16T19:36:41.126+0000 7f6838a51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi163 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:36:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:41 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163[81633]: 2024-02-16T19:36:41.126+0000 7f6838a51700 -1 mon.smithi163@1(peon) e3 *** Got Signal Terminated *** 2024-02-16T19:36:41.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:41 smithi163 podman[118045]: 2024-02-16 19:36:41.332069009 +0000 UTC m=+0.324025076 container died 4840eea014c332aab5f5fcbb225fe81a40e6edd3c74d6cdd76fc63cc133a6cbf (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214) 2024-02-16T19:36:41.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:41 smithi163 podman[118045]: 2024-02-16 19:36:41.785871662 +0000 UTC m=+0.777827736 container cleanup 4840eea014c332aab5f5fcbb225fe81a40e6edd3c74d6cdd76fc63cc133a6cbf (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-16T19:36:41.988 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Feb 16 19:36:41 smithi163 bash[118045]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-mon-smithi163 2024-02-16T19:36:42.341 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@mon.smithi163.service' 2024-02-16T19:36:42.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:36:42.386 INFO:tasks.cephadm.mon.smithi163:Stopped mon.smithi163 2024-02-16T19:36:42.387 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-16T19:36:42.387 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.0 2024-02-16T19:36:42.702 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:42 smithi040 systemd[1]: Stopping Ceph osd.0 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:36:43.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:42 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[103005]: 2024-02-16T19:36:42.701+0000 7f8ebd097700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:36:43.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:42 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[103005]: 2024-02-16T19:36:42.701+0000 7f8ebd097700 -1 osd.0 83 *** Got signal Terminated *** 2024-02-16T19:36:43.144 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:42 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0[103005]: 2024-02-16T19:36:42.701+0000 7f8ebd097700 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:36:48.462 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:48 smithi040 podman[150671]: 2024-02-16 19:36:48.191583636 +0000 UTC m=+5.609433076 container died 607e375ba25bb58ab7af0699f0e46b134ccdddbd0289f24d2f39c1b8fa542a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-16T19:36:48.846 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:48 smithi040 podman[150671]: 2024-02-16 19:36:48.56045671 +0000 UTC m=+5.978306156 container cleanup 607e375ba25bb58ab7af0699f0e46b134ccdddbd0289f24d2f39c1b8fa542a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-16T19:36:48.846 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:48 smithi040 bash[150671]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0 2024-02-16T19:36:49.104 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:48 smithi040 podman[150688]: 2024-02-16 19:36:48.845450239 +0000 UTC m=+0.654666019 container remove 607e375ba25bb58ab7af0699f0e46b134ccdddbd0289f24d2f39c1b8fa542a9b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-16T19:36:49.380 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:49 smithi040 podman[150788]: 2024-02-16 19:36:49.118771907 +0000 UTC m=+0.021281640 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:36:49.381 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:49 smithi040 podman[150788]: 2024-02-16 19:36:49.238351206 +0000 UTC m=+0.140860918 container create 9c253c6003f93edf2fe1ea195de223846f9a76524fbddcba2e38e04686299057 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:36:49.644 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:49 smithi040 podman[150788]: 2024-02-16 19:36:49.499338072 +0000 UTC m=+0.401847785 container init 9c253c6003f93edf2fe1ea195de223846f9a76524fbddcba2e38e04686299057 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-16T19:36:49.644 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:49 smithi040 podman[150788]: 2024-02-16 19:36:49.503645736 +0000 UTC m=+0.406155456 container start 9c253c6003f93edf2fe1ea195de223846f9a76524fbddcba2e38e04686299057 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T19:36:49.645 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:49 smithi040 podman[150788]: 2024-02-16 19:36:49.574324961 +0000 UTC m=+0.476834680 container attach 9c253c6003f93edf2fe1ea195de223846f9a76524fbddcba2e38e04686299057 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:36:50.078 INFO:journalctl@ceph.osd.0.smithi040.stdout:Feb 16 19:36:49 smithi040 podman[150788]: 2024-02-16 19:36:49.809821076 +0000 UTC m=+0.712330788 container died 9c253c6003f93edf2fe1ea195de223846f9a76524fbddcba2e38e04686299057 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:36:50.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.0.service' 2024-02-16T19:36:50.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:36:50.410 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-16T19:36:50.410 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-16T19:36:50.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.1 2024-02-16T19:36:50.644 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:50 smithi040 systemd[1]: Stopping Ceph osd.1 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:36:51.144 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:50 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[108647]: 2024-02-16T19:36:50.747+0000 7f88f4d24700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:36:51.144 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:50 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[108647]: 2024-02-16T19:36:50.747+0000 7f88f4d24700 -1 osd.1 83 *** Got signal Terminated *** 2024-02-16T19:36:51.144 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:50 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1[108647]: 2024-02-16T19:36:50.747+0000 7f88f4d24700 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:36:56.550 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:56 smithi040 podman[150895]: 2024-02-16 19:36:56.250063791 +0000 UTC m=+5.616124688 container died 2ba47d6ce1c92a64fc3936cde1520f654caddb48ca0c4fb810aae783925cc55e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:36:56.883 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:56 smithi040 podman[150895]: 2024-02-16 19:36:56.639763268 +0000 UTC m=+6.005824167 container cleanup 2ba47d6ce1c92a64fc3936cde1520f654caddb48ca0c4fb810aae783925cc55e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-16T19:36:56.883 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:56 smithi040 bash[150895]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1 2024-02-16T19:36:57.144 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:56 smithi040 podman[150914]: 2024-02-16 19:36:56.882786709 +0000 UTC m=+0.634464566 container remove 2ba47d6ce1c92a64fc3936cde1520f654caddb48ca0c4fb810aae783925cc55e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-16T19:36:57.495 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:57 smithi040 podman[151006]: 2024-02-16 19:36:57.116088358 +0000 UTC m=+0.017862129 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:36:57.495 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:57 smithi040 podman[151006]: 2024-02-16 19:36:57.250546368 +0000 UTC m=+0.152320139 container create 19061cb37f67ee8a007a0397a2d5feb55ad79af8eacd6c6272e745feecce6a37 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True) 2024-02-16T19:36:57.495 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:57 smithi040 podman[151006]: 2024-02-16 19:36:57.494818419 +0000 UTC m=+0.396592193 container init 19061cb37f67ee8a007a0397a2d5feb55ad79af8eacd6c6272e745feecce6a37 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-16T19:36:57.796 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:57 smithi040 podman[151006]: 2024-02-16 19:36:57.498234502 +0000 UTC m=+0.400008275 container start 19061cb37f67ee8a007a0397a2d5feb55ad79af8eacd6c6272e745feecce6a37 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-16T19:36:57.797 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:57 smithi040 podman[151006]: 2024-02-16 19:36:57.577959169 +0000 UTC m=+0.479732946 container attach 19061cb37f67ee8a007a0397a2d5feb55ad79af8eacd6c6272e745feecce6a37 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-16T19:36:58.144 INFO:journalctl@ceph.osd.1.smithi040.stdout:Feb 16 19:36:57 smithi040 podman[151006]: 2024-02-16 19:36:57.796256799 +0000 UTC m=+0.698030573 container died 19061cb37f67ee8a007a0397a2d5feb55ad79af8eacd6c6272e745feecce6a37 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-16T19:36:58.527 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.1.service' 2024-02-16T19:36:58.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:36:58.577 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-16T19:36:58.577 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-16T19:36:58.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.2 2024-02-16T19:36:58.886 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:36:58 smithi040 systemd[1]: Stopping Ceph osd.2 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:36:59.144 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:36:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[114481]: 2024-02-16T19:36:58.884+0000 7f86d6fee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:36:59.144 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:36:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[114481]: 2024-02-16T19:36:58.884+0000 7f86d6fee700 -1 osd.2 83 *** Got signal Terminated *** 2024-02-16T19:36:59.144 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:36:58 smithi040 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2[114481]: 2024-02-16T19:36:58.884+0000 7f86d6fee700 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:37:04.595 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:04 smithi040 podman[151110]: 2024-02-16 19:37:04.29183054 +0000 UTC m=+5.522777254 container died 36685277f4cb24a0668214441427eef646f96680723cdf9085591d7e0ce5cf3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-16T19:37:04.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:04 smithi040 podman[151110]: 2024-02-16 19:37:04.768565605 +0000 UTC m=+5.999512353 container cleanup 36685277f4cb24a0668214441427eef646f96680723cdf9085591d7e0ce5cf3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-02-16T19:37:04.894 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:04 smithi040 bash[151110]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2 2024-02-16T19:37:05.344 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:05 smithi040 podman[151127]: 2024-02-16 19:37:05.011553562 +0000 UTC m=+0.723170904 container remove 36685277f4cb24a0668214441427eef646f96680723cdf9085591d7e0ce5cf3b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:37:05.644 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:05 smithi040 podman[151220]: 2024-02-16 19:37:05.246985768 +0000 UTC m=+0.018266143 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:05.644 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:05 smithi040 podman[151220]: 2024-02-16 19:37:05.35424511 +0000 UTC m=+0.125525477 container create 9fd9322db1eaca1be5c10f7d6b799468848d6fdd991a17daafc7df4397b97232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-16T19:37:05.644 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:05 smithi040 podman[151220]: 2024-02-16 19:37:05.573315865 +0000 UTC m=+0.344596242 container init 9fd9322db1eaca1be5c10f7d6b799468848d6fdd991a17daafc7df4397b97232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-16T19:37:05.644 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:05 smithi040 podman[151220]: 2024-02-16 19:37:05.577649944 +0000 UTC m=+0.348930312 container start 9fd9322db1eaca1be5c10f7d6b799468848d6fdd991a17daafc7df4397b97232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-16T19:37:06.144 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:05 smithi040 podman[151220]: 2024-02-16 19:37:05.648224053 +0000 UTC m=+0.419504441 container attach 9fd9322db1eaca1be5c10f7d6b799468848d6fdd991a17daafc7df4397b97232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-16T19:37:06.144 INFO:journalctl@ceph.osd.2.smithi040.stdout:Feb 16 19:37:05 smithi040 podman[151220]: 2024-02-16 19:37:05.878412276 +0000 UTC m=+0.649692647 container died 9fd9322db1eaca1be5c10f7d6b799468848d6fdd991a17daafc7df4397b97232 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:37:06.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.2.service' 2024-02-16T19:37:06.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:37:06.543 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-16T19:37:06.543 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-16T19:37:06.543 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.3 2024-02-16T19:37:06.855 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:06 smithi163 systemd[1]: Stopping Ceph osd.3 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:37:07.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:06 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:37:06.854+0000 7fec0ab77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:37:07.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:06 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:37:06.854+0000 7fec0ab77700 -1 osd.3 83 *** Got signal Terminated *** 2024-02-16T19:37:07.237 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:06 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:37:06.854+0000 7fec0ab77700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:37:10.729 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:10.360+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:10.729 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:10.380+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:10.987 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:10 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:37:10.728+0000 7fec0796c700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:41.889459+0000 front 2024-02-16T19:36:41.889293+0000 (oldest deadline 2024-02-16T19:37:10.089458+0000) 2024-02-16T19:37:11.683 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:11 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:11.430+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:11.683 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:11 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:11.311+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:11.934 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:11 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3[91087]: 2024-02-16T19:37:11.682+0000 7fec0796c700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:41.889459+0000 front 2024-02-16T19:36:41.889293+0000 (oldest deadline 2024-02-16T19:37:10.089458+0000) 2024-02-16T19:37:12.596 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:12 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:12.454+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:12.596 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:12 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:12.313+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:12.597 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:12 smithi163 podman[118168]: 2024-02-16 19:37:12.291073402 +0000 UTC m=+5.552489255 container died d4b9a7b78af11c512f5d3f8dc6df4bc2cc236ec1653c94dc872dfb9aaad57cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-16T19:37:12.597 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:12 smithi163 podman[118168]: 2024-02-16 19:37:12.596022509 +0000 UTC m=+5.857438363 container cleanup d4b9a7b78af11c512f5d3f8dc6df4bc2cc236ec1653c94dc872dfb9aaad57cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-16T19:37:12.856 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:12 smithi163 bash[118168]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3 2024-02-16T19:37:13.206 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:12 smithi163 podman[118197]: 2024-02-16 19:37:12.855792614 +0000 UTC m=+0.563295162 container remove d4b9a7b78af11c512f5d3f8dc6df4bc2cc236ec1653c94dc872dfb9aaad57cb3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-16T19:37:13.467 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:13 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:13.336+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:13.468 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:13 smithi163 podman[118288]: 2024-02-16 19:37:13.108765892 +0000 UTC m=+0.017684072 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:13.468 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:13 smithi163 podman[118288]: 2024-02-16 19:37:13.248513365 +0000 UTC m=+0.157431535 container create 15e4fcbe361d74c6fcd17371eba979850cef42ea6e0e0a376a85588dc39ba742 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-16T19:37:13.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:13 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:13.466+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:13.738 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:13 smithi163 podman[118288]: 2024-02-16 19:37:13.475852803 +0000 UTC m=+0.384770976 container init 15e4fcbe361d74c6fcd17371eba979850cef42ea6e0e0a376a85588dc39ba742 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:37:13.738 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:13 smithi163 podman[118288]: 2024-02-16 19:37:13.479666248 +0000 UTC m=+0.388584426 container start 15e4fcbe361d74c6fcd17371eba979850cef42ea6e0e0a376a85588dc39ba742 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:37:13.738 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:13 smithi163 podman[118288]: 2024-02-16 19:37:13.592691359 +0000 UTC m=+0.501609538 container attach 15e4fcbe361d74c6fcd17371eba979850cef42ea6e0e0a376a85588dc39ba742 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-16T19:37:14.071 INFO:journalctl@ceph.osd.3.smithi163.stdout:Feb 16 19:37:13 smithi163 podman[118288]: 2024-02-16 19:37:13.776133834 +0000 UTC m=+0.685052019 container died 15e4fcbe361d74c6fcd17371eba979850cef42ea6e0e0a376a85588dc39ba742 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-16T19:37:14.413 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.3.service' 2024-02-16T19:37:14.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:37:14.455 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-16T19:37:14.455 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-16T19:37:14.456 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.4 2024-02-16T19:37:14.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:14 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:14.340+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:14.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:14 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:14.424+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:14.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:14 smithi163 systemd[1]: Stopping Ceph osd.4 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:37:15.237 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:14 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:14.798+0000 7f9ba3ab0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:37:15.237 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:14 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:14.798+0000 7f9ba3ab0700 -1 osd.4 83 *** Got signal Terminated *** 2024-02-16T19:37:15.237 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:14 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:14.798+0000 7f9ba3ab0700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:37:15.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:15 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:15.378+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:15.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:15 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:15.361+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:16.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:16 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:16.380+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:16.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:16 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:16.361+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:17.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:17.385+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:17.738 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:17 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:17.351+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:18.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:18 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:18.371+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:18.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:18 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:18.355+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:19.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:19 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:19.364+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:19.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:19 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:19.335+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:45.611076+0000 front 2024-02-16T19:36:45.611173+0000 (oldest deadline 2024-02-16T19:37:10.311106+0000) 2024-02-16T19:37:19.738 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:19 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4[95954]: 2024-02-16T19:37:19.335+0000 7f9ba08a5700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:53.712366+0000 front 2024-02-16T19:36:53.712408+0000 (oldest deadline 2024-02-16T19:37:19.012396+0000) 2024-02-16T19:37:20.475 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:20 smithi163 podman[118392]: 2024-02-16 19:37:20.184690514 +0000 UTC m=+5.511740624 container died 0c47887cb850ea42f823fce44902fb0f2fbeaddd26137305ca13ad7013d23520 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:37:20.475 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:20 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:20.363+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:20.476 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:20 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:20.363+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:55.717917+0000 front 2024-02-16T19:36:55.717897+0000 (oldest deadline 2024-02-16T19:37:19.817844+0000) 2024-02-16T19:37:20.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:20 smithi163 podman[118392]: 2024-02-16 19:37:20.582600704 +0000 UTC m=+5.909650815 container cleanup 0c47887cb850ea42f823fce44902fb0f2fbeaddd26137305ca13ad7013d23520 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-16T19:37:20.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:20 smithi163 bash[118392]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4 2024-02-16T19:37:21.126 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:20 smithi163 podman[118411]: 2024-02-16 19:37:20.833862365 +0000 UTC m=+0.642517970 container remove 0c47887cb850ea42f823fce44902fb0f2fbeaddd26137305ca13ad7013d23520 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-16T19:37:21.454 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:21 smithi163 podman[118506]: 2024-02-16 19:37:21.028446371 +0000 UTC m=+0.017455376 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:21.454 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:21 smithi163 podman[118506]: 2024-02-16 19:37:21.142887302 +0000 UTC m=+0.131896292 container create 5a59881879507a8f206aaa610b35d91a2821df99996928d8975ff01ae19cc931 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-16T19:37:21.454 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:21 smithi163 podman[118506]: 2024-02-16 19:37:21.354560333 +0000 UTC m=+0.343569325 container init 5a59881879507a8f206aaa610b35d91a2821df99996928d8975ff01ae19cc931 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-16T19:37:21.454 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:21 smithi163 podman[118506]: 2024-02-16 19:37:21.357889805 +0000 UTC m=+0.346898796 container start 5a59881879507a8f206aaa610b35d91a2821df99996928d8975ff01ae19cc931 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-02-16T19:37:21.454 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:21 smithi163 podman[118506]: 2024-02-16 19:37:21.453561484 +0000 UTC m=+0.442570484 container attach 5a59881879507a8f206aaa610b35d91a2821df99996928d8975ff01ae19cc931 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:37:21.454 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:21 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:21.366+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:21.455 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:21 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:21.366+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:55.717917+0000 front 2024-02-16T19:36:55.717897+0000 (oldest deadline 2024-02-16T19:37:19.817844+0000) 2024-02-16T19:37:21.737 INFO:journalctl@ceph.osd.4.smithi163.stdout:Feb 16 19:37:21 smithi163 podman[118506]: 2024-02-16 19:37:21.653796327 +0000 UTC m=+0.642805330 container died 5a59881879507a8f206aaa610b35d91a2821df99996928d8975ff01ae19cc931 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:37:22.277 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.4.service' 2024-02-16T19:37:22.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:37:22.320 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-16T19:37:22.320 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-16T19:37:22.320 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.5 2024-02-16T19:37:22.644 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:22 smithi163 systemd[1]: Stopping Ceph osd.5 for 8c23bd6c-ccff-11ee-95ba-87774f69a715... 2024-02-16T19:37:22.644 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:22 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:22.404+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:22.644 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:22 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:22.404+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:55.717917+0000 front 2024-02-16T19:36:55.717897+0000 (oldest deadline 2024-02-16T19:37:19.817844+0000) 2024-02-16T19:37:22.644 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:22 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:22.642+0000 7f662e0e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-16T19:37:22.644 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:22 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:22.642+0000 7f662e0e2700 -1 osd.5 83 *** Got signal Terminated *** 2024-02-16T19:37:22.644 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:22 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:22.642+0000 7f662e0e2700 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-16T19:37:23.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:23 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:23.442+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:23.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:23 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:23.442+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:55.717917+0000 front 2024-02-16T19:36:55.717897+0000 (oldest deadline 2024-02-16T19:37:19.817844+0000) 2024-02-16T19:37:24.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:24 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:24.438+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:24.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:24 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:24.438+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:55.717917+0000 front 2024-02-16T19:36:55.717897+0000 (oldest deadline 2024-02-16T19:37:19.817844+0000) 2024-02-16T19:37:24.737 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:24 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:24.438+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6822 osd.2 since back 2024-02-16T19:37:02.118619+0000 front 2024-02-16T19:37:02.118595+0000 (oldest deadline 2024-02-16T19:37:24.418457+0000) 2024-02-16T19:37:25.987 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:25 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:25.486+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:25.987 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:25 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:25.486+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:55.717917+0000 front 2024-02-16T19:36:55.717897+0000 (oldest deadline 2024-02-16T19:37:19.817844+0000) 2024-02-16T19:37:25.987 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:25 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:25.486+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6822 osd.2 since back 2024-02-16T19:37:02.118619+0000 front 2024-02-16T19:37:02.118595+0000 (oldest deadline 2024-02-16T19:37:24.418457+0000) 2024-02-16T19:37:26.987 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:26 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:26.499+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:26.988 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:26 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:26.499+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:55.717917+0000 front 2024-02-16T19:36:55.717897+0000 (oldest deadline 2024-02-16T19:37:19.817844+0000) 2024-02-16T19:37:26.988 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:26 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:26.499+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6822 osd.2 since back 2024-02-16T19:37:02.118619+0000 front 2024-02-16T19:37:02.118595+0000 (oldest deadline 2024-02-16T19:37:24.418457+0000) 2024-02-16T19:37:27.973 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:27 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:27.504+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2024-02-16T19:36:47.616777+0000 front 2024-02-16T19:36:47.616793+0000 (oldest deadline 2024-02-16T19:37:09.916564+0000) 2024-02-16T19:37:27.973 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:27 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:27.504+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6814 osd.1 since back 2024-02-16T19:36:55.717917+0000 front 2024-02-16T19:36:55.717897+0000 (oldest deadline 2024-02-16T19:37:19.817844+0000) 2024-02-16T19:37:27.973 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:27 smithi163 ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5[101270]: 2024-02-16T19:37:27.504+0000 7f6626ecf700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.40:6822 osd.2 since back 2024-02-16T19:37:02.118619+0000 front 2024-02-16T19:37:02.118595+0000 (oldest deadline 2024-02-16T19:37:24.418457+0000) 2024-02-16T19:37:28.237 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:27 smithi163 podman[118612]: 2024-02-16 19:37:27.972753775 +0000 UTC m=+5.432020457 container died 807b119a828c7e11884364b2fbee089a31d9f3bc5d4ee2fd9e5e6c80bf8c4d56 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:37:28.611 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:28 smithi163 podman[118612]: 2024-02-16 19:37:28.359655276 +0000 UTC m=+5.818921965 container cleanup 807b119a828c7e11884364b2fbee089a31d9f3bc5d4ee2fd9e5e6c80bf8c4d56 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-16T19:37:28.611 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:28 smithi163 bash[118612]: ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5 2024-02-16T19:37:28.920 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:28 smithi163 podman[118631]: 2024-02-16 19:37:28.610905647 +0000 UTC m=+0.641693743 container remove 807b119a828c7e11884364b2fbee089a31d9f3bc5d4ee2fd9e5e6c80bf8c4d56 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-16T19:37:29.214 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:28 smithi163 podman[118723]: 2024-02-16 19:37:28.91988662 +0000 UTC m=+0.110688560 container create 544d87e904152dcae783160e3521d45daad0b18ed333004a52911d0d46c91e36 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-16T19:37:29.214 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:28 smithi163 podman[118723]: 2024-02-16 19:37:28.82690442 +0000 UTC m=+0.017706361 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-16T19:37:29.215 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:29 smithi163 podman[118723]: 2024-02-16 19:37:29.131008873 +0000 UTC m=+0.321810824 container init 544d87e904152dcae783160e3521d45daad0b18ed333004a52911d0d46c91e36 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-16T19:37:29.215 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:29 smithi163 podman[118723]: 2024-02-16 19:37:29.134229515 +0000 UTC m=+0.325031457 container start 544d87e904152dcae783160e3521d45daad0b18ed333004a52911d0d46c91e36 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-16T19:37:29.215 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:29 smithi163 podman[118723]: 2024-02-16 19:37:29.213987603 +0000 UTC m=+0.404789573 container attach 544d87e904152dcae783160e3521d45daad0b18ed333004a52911d0d46c91e36 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-16T19:37:29.487 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:29 smithi163 podman[118723]: 2024-02-16 19:37:29.431918826 +0000 UTC m=+0.622720768 container died 544d87e904152dcae783160e3521d45daad0b18ed333004a52911d0d46c91e36 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-16T19:37:29.987 INFO:journalctl@ceph.osd.5.smithi163.stdout:Feb 16 19:37:29 smithi163 podman[118749]: 2024-02-16 19:37:29.975606123 +0000 UTC m=+0.535566825 container remove 544d87e904152dcae783160e3521d45daad0b18ed333004a52911d0d46c91e36 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-16T19:37:29.995 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c23bd6c-ccff-11ee-95ba-87774f69a715@osd.5.service' 2024-02-16T19:37:30.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-16T19:37:30.035 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-16T19:37:30.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-16T19:37:57.549 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-16T19:38:12.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:38:12.664 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-16T19:38:12.693 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-16T19:38:12.695 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687/remote/smithi040/crash 2024-02-16T19:38:12.696 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/crash -- . 2024-02-16T19:38:12.734 INFO:teuthology.orchestra.run.smithi040.stderr:tar: /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-16T19:38:12.734 INFO:teuthology.orchestra.run.smithi040.stderr:tar: Error is not recoverable: exiting now 2024-02-16T19:38:12.736 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687/remote/smithi163/crash 2024-02-16T19:38:12.737 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/crash -- . 2024-02-16T19:38:12.763 INFO:teuthology.orchestra.run.smithi163.stderr:tar: /var/lib/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-16T19:38:12.763 INFO:teuthology.orchestra.run.smithi163.stderr:tar: Error is not recoverable: exiting now 2024-02-16T19:38:12.764 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-16T19:38:12.764 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-16T19:38:12.814 INFO:tasks.cephadm:Compressing logs... 2024-02-16T19:38:12.814 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-16T19:38:12.856 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-16T19:38:12.883 INFO:teuthology.orchestra.run.smithi163.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-16T19:38:12.883 INFO:teuthology.orchestra.run.smithi163.stderr:: No such file or directory 2024-02-16T19:38:12.883 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-16T19:38:12.883 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-16T19:38:12.887 INFO:teuthology.orchestra.run.smithi040.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-16T19:38:12.887 INFO:teuthology.orchestra.run.smithi040.stderr:: No such file or directory 2024-02-16T19:38:12.887 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mgr.smithi040.jzygzo.log 2024-02-16T19:38:12.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi040.ezkppl.log 2024-02-16T19:38:12.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mgr.smithi040.jzygzo.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mon.smithi040.log 2024-02-16T19:38:12.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi040.ezkppl.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-16T19:38:12.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mon.smithi040.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-16T19:38:12.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-16T19:38:12.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi040.bmpjpk.log 2024-02-16T19:38:12.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-16T19:38:12.889 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi163.rggdhh.log 2024-02-16T19:38:12.889 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.audit.log: /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-16T19:38:12.889 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-16T19:38:12.889 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi163.rggdhh.log: /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-16T19:38:12.889 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.log 2024-02-16T19:38:12.890 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-16T19:38:12.890 INFO:teuthology.orchestra.run.smithi163.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-16T19:38:12.890 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.log: 85.3% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-16T19:38:12.890 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mgr.smithi163.wjfclz.log 2024-02-16T19:38:12.890 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi163.qovvde.log 2024-02-16T19:38:12.890 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mgr.smithi163.wjfclz.log: 91.3% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-16T19:38:12.890 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-16T19:38:12.890 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi040.bmpjpk.log: 91.1% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-16T19:38:12.897 INFO:teuthology.orchestra.run.smithi040.stderr: 90.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-16T19:38:12.899 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mon.smithi163.log 2024-02-16T19:38:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.log 2024-02-16T19:38:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-16T19:38:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.log: 85.3% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-16T19:38:12.903 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-16T19:38:12.908 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi163.qovvde.log: /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mon.smithi163.log: 91.0% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-16T19:38:12.913 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-16T19:38:12.938 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.2.log: /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-16T19:38:12.941 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mgr.smithi163.wjfclz.log.gz 2024-02-16T19:38:13.255 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mgr.smithi040.jzygzo.log.gz 2024-02-16T19:38:13.270 INFO:teuthology.orchestra.run.smithi163.stderr: 92.6% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mon.smithi163.log.gz 2024-02-16T19:38:13.979 INFO:teuthology.orchestra.run.smithi040.stderr: 90.6% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mon.smithi040.log.gz 2024-02-16T19:38:16.983 INFO:teuthology.orchestra.run.smithi163.stderr: 93.4% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-16T19:38:17.095 INFO:teuthology.orchestra.run.smithi040.stderr: 93.4% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-16T19:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr: 95.1% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi163.qovvde.log.gz 2024-02-16T19:38:17.935 INFO:teuthology.orchestra.run.smithi040.stderr: 93.4% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-16T19:38:18.738 INFO:teuthology.orchestra.run.smithi040.stderr: 95.1% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi040.ezkppl.log.gz 2024-02-16T19:38:18.928 INFO:teuthology.orchestra.run.smithi040.stderr: 93.3% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-16T19:38:19.668 INFO:teuthology.orchestra.run.smithi163.stderr: 93.4% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-16T19:38:19.933 INFO:teuthology.orchestra.run.smithi163.stderr: 93.3% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-16T19:38:52.248 INFO:teuthology.orchestra.run.smithi163.stderr: 93.2% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi163.rggdhh.log.gz 2024-02-16T19:38:52.250 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-02-16T19:38:52.250 INFO:teuthology.orchestra.run.smithi163.stderr:real 0m39.379s 2024-02-16T19:38:52.250 INFO:teuthology.orchestra.run.smithi163.stderr:user 1m0.640s 2024-02-16T19:38:52.250 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m1.981s 2024-02-16T19:39:03.376 INFO:teuthology.orchestra.run.smithi040.stderr: 92.9% -- replaced with /var/log/ceph/8c23bd6c-ccff-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi040.bmpjpk.log.gz 2024-02-16T19:39:03.378 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-02-16T19:39:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m50.508s 2024-02-16T19:39:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:user 1m10.992s 2024-02-16T19:39:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m2.017s 2024-02-16T19:39:03.378 INFO:tasks.cephadm:Archiving logs... 2024-02-16T19:39:03.379 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687/remote/smithi040/log 2024-02-16T19:39:03.380 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-16T19:39:13.126 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687/remote/smithi163/log 2024-02-16T19:39:13.127 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-16T19:39:21.824 INFO:tasks.cephadm:Removing cluster... 2024-02-16T19:39:21.824 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 --force 2024-02-16T19:39:22.391 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c23bd6c-ccff-11ee-95ba-87774f69a715 --force 2024-02-16T19:39:22.935 INFO:tasks.cephadm:Removing cephadm ... 2024-02-16T19:39:22.935 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-16T19:39:22.951 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-16T19:39:22.965 INFO:tasks.cephadm:Teardown complete 2024-02-16T19:39:22.966 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-16T19:39:22.978 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/full_sequential_finally.py", line 43, in task ((taskname, confg),) = entry.items() ValueError: too many values to unpack (expected 1) 2024-02-16T19:39:22.979 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-16T19:39:22.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-16T19:39:22.993 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-16T19:39:23.085 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-16T19:39:23.096 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-16T19:39:23.096 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T19:39:23.098 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-16T19:39:23.110 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-02-16T19:39:23.112 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-02-16T19:39:23.115 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T19:39:23.115 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-02-16T19:39:23.115 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +1802us[+1802us] +/- 116ms 2024-02-16T19:39:23.115 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -1917us[-1916us] +/- 64ms 2024-02-16T19:39:23.115 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +1180us[+1180us] +/- 75ms 2024-02-16T19:39:23.115 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:39:23.116 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-16T19:39:23.116 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-02-16T19:39:23.116 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +1775us[+1775us] +/- 116ms 2024-02-16T19:39:23.116 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -1948us[-1898us] +/- 64ms 2024-02-16T19:39:23.116 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +1148us[+1148us] +/- 75ms 2024-02-16T19:39:23.116 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-16T19:39:23.117 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-16T19:39:23.131 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-16T19:39:23.131 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-16T19:39:23.150 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-16T19:39:23.242 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-16T19:39:23.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T19:39:23.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:39:23.350 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-16T19:39:23.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:39:23.381 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-16T19:39:23.393 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-16T19:39:23.434 INFO:teuthology.task.internal:Duration was 1931.593560 seconds 2024-02-16T19:39:23.434 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-16T19:39:23.467 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-16T19:39:23.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-16T19:39:23.470 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-16T19:39:23.510 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T19:39:23.513 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-16T19:39:23.683 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-16T19:39:23.683 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-02-16T19:39:23.683 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-16T19:39:23.742 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-02-16T19:39:23.742 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-16T19:39:23.768 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-16T19:39:23.769 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-16T19:39:23.784 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-16T19:39:23.838 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-16T19:39:23.838 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-16T19:39:23.852 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-16T19:39:24.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-16T19:39:24.499 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-16T19:39:24.499 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-16T19:39:24.526 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-16T19:39:24.555 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-16T19:39:24.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-16T19:39:24.569 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-16T19:39:24.592 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-02-16T19:39:24.621 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-02-16T19:39:24.637 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-16T19:39:24.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:39:24.661 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-16T19:39:24.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-16T19:39:24.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-16T19:39:24.705 INFO:teuthology.task.internal:Transferring archived files... 2024-02-16T19:39:24.705 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687/remote/smithi040 2024-02-16T19:39:24.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-16T19:39:24.755 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687/remote/smithi163 2024-02-16T19:39:24.755 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-16T19:39:24.795 INFO:teuthology.task.internal:Removing archive directory... 2024-02-16T19:39:24.796 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-16T19:39:24.798 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-16T19:39:24.843 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-16T19:39:24.857 INFO:teuthology.task.internal:Not uploading archives. 2024-02-16T19:39:24.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-16T19:39:24.913 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-16T19:39:24.913 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-16T19:39:24.916 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-16T19:39:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 16 19:39 /home/ubuntu/cephtest 2024-02-16T19:39:24.930 INFO:teuthology.orchestra.run.smithi163.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 16 19:39 /home/ubuntu/cephtest 2024-02-16T19:39:24.931 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-16T19:39:24.942 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-16T19:39:24.977 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: ValueError: too many values to unpack (expected 1) 2024-02-16T19:39:24.977 INFO:teuthology.nuke:Checking targets against current locks 2024-02-16T19:39:25.007 DEBUG:teuthology.nuke:shortname: smithi040 2024-02-16T19:39:25.007 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T19:39:25.029 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 18:57:24.683060', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP5WBLJfQrY+S5hsuuLtCOUUFMdivkRktN3VAK5Zq0pceOWiEUs9klmdsq2JNtA2OBvjWYoz3hgO1eY+3UeTtNg='} 2024-02-16T19:39:25.033 DEBUG:teuthology.nuke:shortname: smithi163 2024-02-16T19:39:25.033 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T19:39:25.055 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-16_18:26:20-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562687', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 18:57:24.685004', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBJoyXrV8epnYq1Lr3iMt/rZ9CyK+g/tfMX75McffgPPBagzOh2+ddwpgeioxNducTFjpJa/iW0MTZujfoViF5o='} 2024-02-16T19:39:25.083 INFO:teuthology.orchestra.console.smithi040:Power off 2024-02-16T19:39:25.084 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T19:39:25.109 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2024-02-16T19:39:25.110 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:39:25.125 INFO:teuthology.orchestra.console.smithi163:Power off 2024-02-16T19:39:25.125 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T19:39:25.135 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-02-16T19:39:25.148 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-02-16T19:39:25.148 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:39:25.172 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-02-16T19:39:29.136 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:39:29.173 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:39:29.246 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-02-16T19:39:29.283 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-02-16T19:39:33.252 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:39:33.285 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:39:33.362 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-02-16T19:39:33.395 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-02-16T19:39:33.496 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-02-16T19:39:37.365 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T19:39:37.475 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2024-02-16T19:39:37.576 INFO:teuthology.orchestra.console.smithi040:Power off completed 2024-02-16T19:39:37.680 INFO:teuthology.run:Summary data: description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 0a 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1931.593560218811 failure_reason: too many values to unpack (expected 1) flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-02-16T19:39:37.681 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T19:39:37.753 INFO:teuthology.run:FAIL